2024-02-22T14:44:18.897 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-22T14:44:18.897 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --archive /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354 -- /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/orig.config.yaml 2024-02-22T14:44:18.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-22T14:44:19.046 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354 branch: wip-yuri10-testing-2024-02-21-1249-reef description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: yweinste@redhat.com first_in_suite: false job_id: '7570354' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri10-testing-2024-02-21-1249-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: c1dc18f444603397768fcb7c34a8488b10284df6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c1dc18f444603397768fcb7c34a8488b10284df6 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri10-testing-2024-02-21-1249-reef sha1: c1dc18f444603397768fcb7c34a8488b10284df6 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 1691 sha1: c1dc18f444603397768fcb7c34a8488b10284df6 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri10-testing-2024-02-21-1249-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: c1dc18f444603397768fcb7c34a8488b10284df6 targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC04T1B4YhQkhNaZ1T/tOtUSBIyA2hQNTTXwityN5MkgzaVGXmL4bFkGiD6D5eDzbSD5MC/oN2jPxoE8zLyVAfU= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInULIZZDrJ/cMctnfpAMqX7AiyMkWr3Csj7NmbMNHgHv0l3vqskz4fj/QycLxwFwvSaU+vr5zdIUwBpg3F8TsQ= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-21_23:06:32 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-22T14:44:19.047 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa; will attempt to use it 2024-02-22T14:44:19.047 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks 2024-02-22T14:44:19.047 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-22T14:44:19.049 INFO:teuthology.task.internal:Checking packages... 2024-02-22T14:44:19.073 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c1dc18f444603397768fcb7c34a8488b10284df6' 2024-02-22T14:44:19.073 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T14:44:19.073 INFO:teuthology.packaging:ref: None 2024-02-22T14:44:19.073 INFO:teuthology.packaging:tag: None 2024-02-22T14:44:19.073 INFO:teuthology.packaging:branch: wip-yuri10-testing-2024-02-21-1249-reef 2024-02-22T14:44:19.074 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T14:44:19.074 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri10-testing-2024-02-21-1249-reef 2024-02-22T14:44:19.235 INFO:teuthology.task.internal:Found packages for ceph version 18.2.1-638.gc1dc18f4 2024-02-22T14:44:19.237 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-22T14:44:19.243 INFO:teuthology.task.internal:no buildpackages task found 2024-02-22T14:44:19.243 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-22T14:44:19.253 INFO:teuthology.task.internal:Saving configuration 2024-02-22T14:44:19.275 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-22T14:44:19.281 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T14:44:19.306 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 14:38:42.272301', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC04T1B4YhQkhNaZ1T/tOtUSBIyA2hQNTTXwityN5MkgzaVGXmL4bFkGiD6D5eDzbSD5MC/oN2jPxoE8zLyVAfU='} 2024-02-22T14:44:19.330 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 14:38:42.274065', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInULIZZDrJ/cMctnfpAMqX7AiyMkWr3Csj7NmbMNHgHv0l3vqskz4fj/QycLxwFwvSaU+vr5zdIUwBpg3F8TsQ='} 2024-02-22T14:44:19.330 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-22T14:44:19.336 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-22T14:44:19.337 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-22T14:44:19.337 INFO:teuthology.run_tasks:Running task console_log... 2024-02-22T14:44:19.419 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d954de160>, signals=[15]) 2024-02-22T14:44:19.419 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-22T14:44:19.426 INFO:teuthology.task.internal:Opening connections... 2024-02-22T14:44:19.426 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T14:44:19.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:44:19.509 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-02-22T14:44:19.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:44:19.585 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-22T14:44:19.591 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-02-22T14:44:19.614 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-02-22T14:44:19.615 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="8" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="8" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el8" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-02-22T14:44:19.677 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-22T14:44:19.678 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-22T14:44:19.678 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-22T14:44:19.678 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-02-22T14:44:19.699 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-02-22T14:44:19.722 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-02-22T14:44:19.723 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-02-22T14:44:19.784 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-22T14:44:19.785 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-22T14:44:19.785 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-22T14:44:19.785 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-02-22T14:44:19.807 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-22T14:44:19.816 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-22T14:44:19.851 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-22T14:44:19.852 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-02-22T14:44:19.854 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-02-22T14:44:19.875 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-22T14:44:19.882 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-22T14:44:19.882 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-02-22T14:44:19.917 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-02-22T14:44:19.957 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-22T14:44:20.016 INFO:teuthology.run_tasks:Running task kernel... 2024-02-22T14:44:20.032 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-22T14:44:20.032 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-22T14:44:20.033 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-22T14:44:20.033 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-22T14:44:20.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T14:44:20.054 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-02-22T14:44:20.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T14:44:20.055 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-02-22T14:44:20.116 INFO:teuthology.orchestra.run.smithi019.stdout:4.18.0-489.el8.x86_64 2024-02-22T14:44:20.117 INFO:teuthology.task.kernel:Running kernel on smithi019: 4.18.0-489.el8.x86_64 2024-02-22T14:44:20.117 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-02-22T14:44:20.118 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-489.el8.x86_64 2024-02-22T14:44:20.118 INFO:teuthology.task.kernel:Running kernel on smithi062: 4.18.0-489.el8.x86_64 2024-02-22T14:44:20.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-02-22T14:44:23.872 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - AppStream 21 MB/s | 27 MB 00:01 2024-02-22T14:44:25.263 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - AppStream 11 MB/s | 27 MB 00:02 2024-02-22T14:52:46.556 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - BaseOS 21 kB/s | 10 MB 08:17 2024-02-22T14:52:48.813 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - Extras 31 kB/s | 18 kB 00:00 2024-02-22T14:52:49.147 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - Extras common packages 33 kB/s | 6.9 kB 00:00 2024-02-22T14:52:49.432 INFO:teuthology.orchestra.run.smithi019.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-02-22T14:52:51.588 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - BaseOS 21 kB/s | 10 MB 08:21 2024-02-22T14:52:52.466 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 16 MB 00:02 2024-02-22T14:52:53.830 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras 32 kB/s | 18 kB 00:00 2024-02-22T14:52:54.281 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 6.9 kB 00:00 2024-02-22T14:52:54.566 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-02-22T14:52:55.433 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2024-02-22T14:52:56.433 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 501 kB/s | 24 kB 00:00 2024-02-22T14:52:59.366 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 403 kB/s | 24 kB 00:00 2024-02-22T14:52:59.588 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-22T14:52:59.588 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-22T14:52:59.691 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 4.18.0-540.el8 baseos 10 M 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 4.18.0-540.el8 baseos 44 M 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 4.18.0-540.el8 baseos 36 M 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T14:52:59.692 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T14:52:59.693 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T14:52:59.693 INFO:teuthology.orchestra.run.smithi019.stdout:Install 3 Packages 2024-02-22T14:52:59.693 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T14:52:59.693 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 90 M 2024-02-22T14:52:59.693 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 97 M 2024-02-22T14:52:59.693 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T14:53:02.597 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-22T14:53:02.597 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-22T14:53:02.707 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 4.18.0-540.el8 baseos 10 M 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T14:53:02.708 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core x86_64 4.18.0-540.el8 baseos 44 M 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules x86_64 4.18.0-540.el8 baseos 36 M 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout:Install 3 Packages 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T14:53:02.709 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 90 M 2024-02-22T14:53:02.710 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 97 M 2024-02-22T14:53:02.710 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T14:57:08.302 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] kernel-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T14:57:08.303 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] kernel-core-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-core-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T14:57:08.303 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] kernel-modules-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-modules-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T14:57:08.364 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] kernel-4.18.0-540.el8.x86_64.rpm: Status code: 403 for http://distro.ibiblio.org/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-4.18.0-540.el8.x86_64.rpm (IP: 152.19.134.43) 2024-02-22T14:57:12.222 INFO:teuthology.orchestra.run.smithi019.stdout:(1/3): kernel-4.18.0-540.el8.x86_64.rpm 42 kB/s | 10 MB 04:12 2024-02-22T14:57:14.522 INFO:teuthology.orchestra.run.smithi019.stdout:(2/3): kernel-modules-4.18.0-540.el8.x86_64.rpm 146 kB/s | 36 MB 04:14 2024-02-22T14:57:17.187 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] kernel-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T14:57:17.187 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] kernel-core-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-core-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T14:57:17.187 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] kernel-modules-4.18.0-540.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/kernel-modules-4.18.0-540.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T14:57:20.180 INFO:teuthology.orchestra.run.smithi062.stdout:(1/3): kernel-modules-4.18.0-540.el8.x86_64.rpm 145 kB/s | 36 MB 04:17 2024-02-22T14:57:20.522 INFO:teuthology.orchestra.run.smithi062.stdout:(2/3): kernel-4.18.0-540.el8.x86_64.rpm 41 kB/s | 10 MB 04:17 2024-02-22T14:57:21.646 INFO:teuthology.orchestra.run.smithi019.stdout:(3/3): kernel-core-4.18.0-540.el8.x86_64.rpm 171 kB/s | 44 MB 04:21 2024-02-22T14:57:21.647 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T14:57:21.647 INFO:teuthology.orchestra.run.smithi019.stdout:Total 353 kB/s | 90 MB 04:21 2024-02-22T14:57:22.366 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T14:57:22.812 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T14:57:22.812 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T14:57:24.127 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T14:57:24.158 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T14:57:27.149 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T14:57:27.682 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T14:57:28.254 INFO:teuthology.orchestra.run.smithi062.stdout:(3/3): kernel-core-4.18.0-540.el8.x86_64.rpm 168 kB/s | 44 MB 04:25 2024-02-22T14:57:28.255 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T14:57:28.255 INFO:teuthology.orchestra.run.smithi062.stdout:Total 348 kB/s | 90 MB 04:25 2024-02-22T14:57:28.970 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T14:57:29.469 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T14:57:29.469 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T14:57:30.781 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T14:57:30.819 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T14:57:31.423 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T14:57:31.789 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T14:57:33.792 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T14:57:34.370 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T14:57:35.466 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T14:57:35.733 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:57:38.151 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 1/3 2024-02-22T14:57:38.552 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T14:57:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 2/3 2024-02-22T14:57:42.529 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:57:58.920 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:57:58.944 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:57:59.306 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:57:59.307 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-4.18.0-540.el8.x86_64 1/3 2024-02-22T14:57:59.307 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-4.18.0-540.el8.x86_64 2/3 2024-02-22T14:57:59.835 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:57:59.835 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T14:57:59.835 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T14:57:59.836 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-4.18.0-540.el8.x86_64 kernel-core-4.18.0-540.el8.x86_64 2024-02-22T14:57:59.836 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-4.18.0-540.el8.x86_64 2024-02-22T14:57:59.836 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T14:57:59.836 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T14:57:59.989 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T14:58:00.220 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T14:58:00.221 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T14:58:00.221 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-540.el8.x86_64 2024-02-22T14:58:00.221 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-22T14:58:00.221 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-22T14:58:00.221 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-22T14:58:00.221 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-02-22T14:58:00.221 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T14:58:00.440 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T14:58:00.441 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T14:58:00.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-02-22T14:58:00.524 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.02 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 129.el8 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Group : System Environment/Base 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 9265586 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-22T14:58:00.525 INFO:teuthology.orchestra.run.smithi019.stdout:Relocations : (not relocatable) 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : CentOS BuildSystem 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-22T14:58:00.526 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-22T14:58:00.527 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-02-22T14:58:00.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T14:58:00.527 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-02-22T14:58:00.528 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-22T14:58:00.528 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 4.18.0-540.el8.x86_64 2024-02-22T14:58:00.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-22T14:58:01.883 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-02-22T14:58:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-02-22T14:58:04.085 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-02-22T14:58:04.111 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.EGBQhndyIk 2024-02-22T14:58:04.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EGBQhndyIk 2024-02-22T14:58:04.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.EGBQhndyIk 2024-02-22T14:58:04.345 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.EGBQhndyIk is 6KB 2024-02-22T14:58:04.395 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.EGBQhndyIk 2024-02-22T14:58:04.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-02-22T14:58:04.485 INFO:teuthology.orchestra.run.smithi019.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-22T14:58:04.486 INFO:teuthology.orchestra.run.smithi019.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-22T14:58:04.486 INFO:teuthology.orchestra.run.smithi019.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-22T14:58:04.486 INFO:teuthology.orchestra.run.smithi019.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64.conf 2024-02-22T14:58:04.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64 2024-02-22T14:58:04.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-02-22T14:58:05.969 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:58:05.992 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:58:06.378 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:58:06.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-4.18.0-540.el8.x86_64 1/3 2024-02-22T14:58:06.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-core-4.18.0-540.el8.x86_64 2/3 2024-02-22T14:58:06.888 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-4.18.0-540.el8.x86_64 3/3 2024-02-22T14:58:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T14:58:06.889 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T14:58:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-4.18.0-540.el8.x86_64 kernel-core-4.18.0-540.el8.x86_64 2024-02-22T14:58:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-4.18.0-540.el8.x86_64 2024-02-22T14:58:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T14:58:06.889 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T14:58:07.043 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T14:58:07.267 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T14:58:07.267 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T14:58:07.267 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-540.el8.x86_64 2024-02-22T14:58:07.267 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-22T14:58:07.267 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-22T14:58:07.267 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-22T14:58:07.267 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2024-02-22T14:58:07.267 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-02-22T14:58:07.488 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-4.18.0-540.el8.x86_64 2024-02-22T14:58:07.488 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-540.el8.x86_64 2024-02-22T14:58:07.488 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.02 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 129.el8 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Group : System Environment/Base 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 9265586 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-22T14:58:07.523 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:Relocations : (not relocatable) 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : CentOS BuildSystem 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T14:58:07.524 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-02-22T14:58:07.525 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-22T14:58:07.525 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 4.18.0-540.el8.x86_64 2024-02-22T14:58:07.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-22T14:58:09.029 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-02-22T14:58:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-02-22T14:58:11.186 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-02-22T14:58:11.210 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.NWjM3bMjHZ 2024-02-22T14:58:11.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NWjM3bMjHZ 2024-02-22T14:58:11.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.NWjM3bMjHZ 2024-02-22T14:58:11.424 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.NWjM3bMjHZ is 6KB 2024-02-22T14:58:11.472 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.NWjM3bMjHZ 2024-02-22T14:58:11.490 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-02-22T14:58:11.556 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-22T14:58:11.556 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-22T14:58:11.556 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-22T14:58:11.557 INFO:teuthology.orchestra.run.smithi062.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64.conf 2024-02-22T14:58:11.558 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-540.el8.x86_64 2024-02-22T14:58:11.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-02-22T14:58:34.662 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-22T14:58:34.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-02-22T14:58:34.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:58:41.736 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-22T14:58:41.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-02-22T14:58:41.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:58:53.116 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-02-22T14:59:00.284 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-02-22T14:59:02.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-02-22T14:59:02.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:59:09.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-02-22T14:59:09.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:59:12.348 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-02-22T14:59:24.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-02-22T14:59:24.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T14:59:56.668 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-02-22T14:59:58.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-02-22T14:59:58.025 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-540.el8.x86_64"... 2024-02-22T14:59:58.025 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-02-22T14:59:58.118 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-540.el8.x86_64 2024-02-22T14:59:58.118 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-540.el8.x86_64 vs 4.18.0-540.el8.x86_64 2024-02-22T14:59:58.118 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-22T14:59:58.118 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-22T14:59:59.119 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-22T14:59:59.119 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-22T14:59:59.186 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2024-02-22T14:59:59.200 DEBUG:teuthology.parallel:result is None 2024-02-22T15:00:02.123 DEBUG:teuthology.orchestra.remote:timed out 2024-02-22T15:00:14.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-02-22T15:00:14.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T15:00:14.388 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-02-22T15:00:14.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-02-22T15:00:14.703 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-540.el8.x86_64"... 2024-02-22T15:00:14.703 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-02-22T15:00:14.727 INFO:teuthology.orchestra.run.smithi019.stdout:4.18.0-540.el8.x86_64 2024-02-22T15:00:14.727 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-540.el8.x86_64 vs 4.18.0-540.el8.x86_64 2024-02-22T15:00:14.727 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-22T15:00:14.727 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-22T15:00:15.728 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-22T15:00:15.728 DEBUG:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-22T15:00:15.812 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2024-02-22T15:00:15.823 DEBUG:teuthology.parallel:result is None 2024-02-22T15:00:15.823 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-22T15:00:15.830 INFO:teuthology.task.internal:Creating test directory... 2024-02-22T15:00:15.830 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-22T15:00:15.832 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-22T15:00:15.855 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-22T15:00:15.862 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-22T15:00:15.873 INFO:teuthology.task.internal:Creating archive directory... 2024-02-22T15:00:15.874 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-22T15:00:15.894 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-22T15:00:15.940 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-22T15:00:15.947 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-22T15:00:15.947 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-22T15:00:15.964 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-22T15:00:16.000 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T15:00:16.011 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T15:00:16.016 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T15:00:16.028 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-22T15:00:16.029 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-22T15:00:16.037 INFO:teuthology.task.internal:Configuring sudo... 2024-02-22T15:00:16.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-22T15:00:16.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-22T15:00:16.101 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-22T15:00:16.110 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-22T15:00:16.111 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-22T15:00:16.127 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-22T15:00:16.161 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T15:00:16.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T15:00:16.280 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T15:00:16.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T15:00:16.411 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:00:16.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-22T15:00:16.479 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T15:00:16.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-22T15:00:16.701 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T15:00:16.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-22T15:00:16.794 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:00:16.794 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-22T15:00:16.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-02-22T15:00:16.866 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-02-22T15:00:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T15:00:16.988 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T15:00:17.373 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-22T15:00:17.381 INFO:teuthology.task.internal:Starting timer... 2024-02-22T15:00:17.381 INFO:teuthology.run_tasks:Running task pcp... 2024-02-22T15:00:17.408 INFO:teuthology.run_tasks:Running task selinux... 2024-02-22T15:00:17.417 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-22T15:00:17.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-02-22T15:00:17.504 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-02-22T15:00:17.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-02-22T15:00:17.588 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-02-22T15:00:17.590 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-22T15:00:17.590 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-02-22T15:00:17.613 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-02-22T15:00:17.613 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-02-22T15:00:17.652 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-02-22T15:00:17.652 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive'} 2024-02-22T15:00:17.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T15:00:17.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T15:00:17.682 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T15:00:17.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T15:00:17.723 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-22T15:00:17.723 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-02-22T15:00:17.751 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-02-22T15:00:17.793 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-22T15:00:17.803 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-22T15:00:17.810 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-22T15:00:17.931 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-22T15:00:17.948 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-22T15:00:17.949 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi062.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-22T15:18:53.235 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com')] 2024-02-22T15:18:53.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-02-22T15:18:53.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T15:18:53.323 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-02-22T15:18:53.391 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-02-22T15:18:53.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-02-22T15:18:53.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-22T15:18:53.467 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-02-22T15:18:53.536 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-02-22T15:18:53.537 INFO:teuthology.run_tasks:Running task clock... 2024-02-22T15:18:53.553 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-22T15:18:53.554 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-22T15:18:53.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T15:18:53.557 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-22T15:18:53.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T15:18:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-22T15:18:53.605 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-22T15:18:53.611 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-22T15:18:53.625 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-22T15:18:53.674 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-02-22T15:18:53.685 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-02-22T15:18:53.699 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-02-22T15:18:53.700 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-22T15:18:53.711 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-02-22T15:18:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-22T15:18:53.725 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-22T15:18:53.738 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-22T15:18:53.775 INFO:teuthology.orchestra.run.smithi019.stderr:bash: ntpq: command not found 2024-02-22T15:18:53.779 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T15:18:53.779 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-02-22T15:18:53.779 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.779 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.779 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.779 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.806 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2024-02-22T15:18:53.810 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T15:18:53.810 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-02-22T15:18:53.810 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.810 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.810 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.810 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T15:18:53.811 INFO:teuthology.run_tasks:Running task pexec... 2024-02-22T15:18:53.820 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-22T15:18:53.820 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:18:53.821 DEBUG:teuthology.orchestra.run.smithi019:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-22T15:18:53.821 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi062.front.sepia.ceph.com 2024-02-22T15:18:53.821 DEBUG:teuthology.orchestra.run.smithi062:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-22T15:18:54.640 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:05:39 ago on Thu 22 Feb 2024 03:13:15 PM UTC. 2024-02-22T15:18:54.668 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:05:40 ago on Thu 22 Feb 2024 03:13:14 PM UTC. 2024-02-22T15:18:56.850 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:18:56.851 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:18:56.851 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-02-22T15:18:56.851 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:18:56.852 INFO:teuthology.orchestra.run.smithi019.stdout:Resetting modules: 2024-02-22T15:18:56.852 INFO:teuthology.orchestra.run.smithi019.stdout: container-tools 2024-02-22T15:18:56.852 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:18:56.852 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:18:56.852 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:18:56.852 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:18:56.880 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:18:56.881 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout:Resetting modules: 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:18:56.882 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:18:57.115 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:18:57.156 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:18:58.016 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:05:42 ago on Thu 22 Feb 2024 03:13:15 PM UTC. 2024-02-22T15:18:58.048 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:05:43 ago on Thu 22 Feb 2024 03:13:14 PM UTC. 2024-02-22T15:19:00.219 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:19:00.243 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout:Installing group/module packages: 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-22T15:19:00.258 INFO:teuthology.orchestra.run.smithi019.stdout: crun x86_64 1.14.1-1.module_el8+951+32019cde appstream 257 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:Installing module profiles: 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: container-tools/common 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:19:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling module streams: 2024-02-22T15:19:00.260 INFO:teuthology.orchestra.run.smithi019.stdout: container-tools rhel8 2024-02-22T15:19:00.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:19:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:19:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:19:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:Install 13 Packages 2024-02-22T15:19:00.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:19:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 27 M 2024-02-22T15:19:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 91 M 2024-02-22T15:19:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout:Installing group/module packages: 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-22T15:19:00.282 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: crun x86_64 1.14.1-1.module_el8+951+32019cde appstream 257 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout:Installing module profiles: 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools/common 2024-02-22T15:19:00.283 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling module streams: 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout: container-tools rhel8 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout:Install 13 Packages 2024-02-22T15:19:00.284 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:19:00.285 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 27 M 2024-02-22T15:19:00.285 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 91 M 2024-02-22T15:19:00.285 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:19:00.640 INFO:teuthology.orchestra.run.smithi019.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 2.0 MB/s | 688 kB 00:00 2024-02-22T15:19:00.691 INFO:teuthology.orchestra.run.smithi019.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 2.6 MB/s | 1.0 MB 00:00 2024-02-22T15:19:00.716 INFO:teuthology.orchestra.run.smithi019.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 504 kB/s | 38 kB 00:00 2024-02-22T15:19:00.824 INFO:teuthology.orchestra.run.smithi019.stdout:(4/13): crun-1.14.1-1.module_el8+951+32019cde.x 1.9 MB/s | 257 kB 00:00 2024-02-22T15:19:00.900 INFO:teuthology.orchestra.run.smithi019.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 2.4 MB/s | 180 kB 00:00 2024-02-22T15:19:00.914 INFO:teuthology.orchestra.run.smithi062.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 1.5 MB/s | 688 kB 00:00 2024-02-22T15:19:00.958 INFO:teuthology.orchestra.run.smithi019.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 426 kB/s | 25 kB 00:00 2024-02-22T15:19:00.973 INFO:teuthology.orchestra.run.smithi062.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 649 kB/s | 38 kB 00:00 2024-02-22T15:19:01.017 INFO:teuthology.orchestra.run.smithi019.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.6 MB/s | 94 kB 00:00 2024-02-22T15:19:01.040 INFO:teuthology.orchestra.run.smithi062.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 1.8 MB/s | 1.0 MB 00:00 2024-02-22T15:19:01.132 INFO:teuthology.orchestra.run.smithi062.stdout:(4/13): crun-1.14.1-1.module_el8+951+32019cde.x 1.6 MB/s | 257 kB 00:00 2024-02-22T15:19:01.241 INFO:teuthology.orchestra.run.smithi062.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.6 MB/s | 180 kB 00:00 2024-02-22T15:19:01.299 INFO:teuthology.orchestra.run.smithi062.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 424 kB/s | 25 kB 00:00 2024-02-22T15:19:01.391 INFO:teuthology.orchestra.run.smithi062.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.0 MB/s | 94 kB 00:00 2024-02-22T15:19:01.442 INFO:teuthology.orchestra.run.smithi019.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 5.7 MB/s | 4.1 MB 00:00 2024-02-22T15:19:01.751 INFO:teuthology.orchestra.run.smithi019.stdout:(9/13): buildah-1.33.5-1.module_el8+885+7da147f 6.7 MB/s | 9.7 MB 00:01 2024-02-22T15:19:01.818 INFO:teuthology.orchestra.run.smithi019.stdout:(10/13): toolbox-0.0.99.5-1.module_el8+847+7863 6.7 MB/s | 2.5 MB 00:00 2024-02-22T15:19:01.843 INFO:teuthology.orchestra.run.smithi019.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 575 kB/s | 53 kB 00:00 2024-02-22T15:19:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 700 kB/s | 41 kB 00:00 2024-02-22T15:19:01.950 INFO:teuthology.orchestra.run.smithi062.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 4.5 MB/s | 4.1 MB 00:00 2024-02-22T15:19:02.168 INFO:teuthology.orchestra.run.smithi019.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 7.7 MB/s | 8.8 MB 00:01 2024-02-22T15:19:02.169 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:19:02.169 INFO:teuthology.orchestra.run.smithi019.stdout:Total 14 MB/s | 27 MB 00:01 2024-02-22T15:19:02.200 INFO:teuthology.orchestra.run.smithi062.stdout:(9/13): buildah-1.33.5-1.module_el8+885+7da147f 5.6 MB/s | 9.7 MB 00:01 2024-02-22T15:19:02.267 INFO:teuthology.orchestra.run.smithi062.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 789 kB/s | 53 kB 00:00 2024-02-22T15:19:02.353 INFO:teuthology.orchestra.run.smithi062.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 478 kB/s | 41 kB 00:00 2024-02-22T15:19:02.409 INFO:teuthology.orchestra.run.smithi062.stdout:(12/13): toolbox-0.0.99.5-1.module_el8+847+7863 5.5 MB/s | 2.5 MB 00:00 2024-02-22T15:19:02.447 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:19:02.470 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:19:02.470 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:19:02.744 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:19:02.747 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:19:02.760 INFO:teuthology.orchestra.run.smithi062.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 6.4 MB/s | 8.8 MB 00:01 2024-02-22T15:19:02.761 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:19:02.761 INFO:teuthology.orchestra.run.smithi062.stdout:Total 11 MB/s | 27 MB 00:02 2024-02-22T15:19:03.038 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:19:03.061 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:19:03.061 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:19:03.358 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:19:03.362 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:19:03.589 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:19:04.198 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:19:04.567 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-22T15:19:04.754 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T15:19:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-22T15:19:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-22T15:19:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-22T15:19:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-22T15:19:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T15:19:05.588 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-22T15:19:05.769 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-22T15:19:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-22T15:19:06.038 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-22T15:19:06.621 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-22T15:19:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-02-22T15:19:07.203 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-22T15:19:07.418 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : crun-1.14.1-1.module_el8+951+32019cde.x86_64 9/13 2024-02-22T15:19:07.670 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-02-22T15:19:07.907 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-22T15:19:07.934 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-22T15:19:08.110 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T15:19:08.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : crun-1.14.1-1.module_el8+951+32019cde.x86_64 9/13 2024-02-22T15:19:08.355 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T15:19:08.514 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T15:19:08.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-22T15:19:08.974 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T15:19:08.974 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : crun-1.14.1-1.module_el8+951+32019cde.x86_64 5/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T15:19:08.975 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T15:19:09.353 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-22T15:19:09.542 INFO:teuthology.orchestra.run.smithi019.stdout: crun-1.14.1-1.module_el8+951+32019cde.x86_64 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:19:09.543 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:19:09.598 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T15:19:09.789 DEBUG:teuthology.parallel:result is None 2024-02-22T15:19:09.838 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T15:19:10.280 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : crun-1.14.1-1.module_el8+951+32019cde.x86_64 5/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-22T15:19:10.281 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-22T15:19:10.282 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-22T15:19:10.282 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-22T15:19:10.282 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-22T15:19:10.282 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: crun-1.14.1-1.module_el8+951+32019cde.x86_64 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-22T15:19:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-22T15:19:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-22T15:19:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-22T15:19:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-22T15:19:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-22T15:19:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:19:10.814 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:19:11.068 DEBUG:teuthology.parallel:result is None 2024-02-22T15:19:11.068 INFO:teuthology.run_tasks:Running task install... 2024-02-22T15:19:11.079 DEBUG:teuthology.task.install:project ceph 2024-02-22T15:19:11.079 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c1dc18f444603397768fcb7c34a8488b10284df6'}} 2024-02-22T15:19:11.079 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'c1dc18f444603397768fcb7c34a8488b10284df6'} 2024-02-22T15:19:11.079 INFO:teuthology.task.install:Using flavor: default 2024-02-22T15:19:11.091 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-22T15:19:11.092 INFO:teuthology.task.install:extra packages: [] 2024-02-22T15:19:11.096 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c1dc18f444603397768fcb7c34a8488b10284df6', 'tag': None, 'wait_for_package': False} 2024-02-22T15:19:11.096 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T15:19:11.096 INFO:teuthology.packaging:ref: None 2024-02-22T15:19:11.096 INFO:teuthology.packaging:tag: None 2024-02-22T15:19:11.096 INFO:teuthology.packaging:branch: quincy 2024-02-22T15:19:11.097 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:19:11.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-02-22T15:19:11.099 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c1dc18f444603397768fcb7c34a8488b10284df6', 'tag': None, 'wait_for_package': False} 2024-02-22T15:19:11.099 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T15:19:11.099 INFO:teuthology.packaging:ref: None 2024-02-22T15:19:11.099 INFO:teuthology.packaging:tag: None 2024-02-22T15:19:11.099 INFO:teuthology.packaging:branch: quincy 2024-02-22T15:19:11.099 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:19:11.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-02-22T15:19:11.326 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/ 2024-02-22T15:19:11.326 INFO:teuthology.task.install.rpm:Package version is 17.2.7-527.g6ea2a55e 2024-02-22T15:19:11.440 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/ 2024-02-22T15:19:11.440 INFO:teuthology.task.install.rpm:Package version is 17.2.7-527.g6ea2a55e 2024-02-22T15:19:11.446 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-22T15:19:11.447 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:19:11.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-22T15:19:11.484 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-22T15:19:11.484 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T15:19:11.484 INFO:teuthology.packaging:ref: None 2024-02-22T15:19:11.484 INFO:teuthology.packaging:tag: None 2024-02-22T15:19:11.484 INFO:teuthology.packaging:branch: quincy 2024-02-22T15:19:11.484 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:19:11.484 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-22T15:19:11.549 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-22T15:19:11.549 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:19:11.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-22T15:19:11.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-22T15:19:11.579 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-22T15:19:11.579 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T15:19:11.579 INFO:teuthology.packaging:ref: None 2024-02-22T15:19:11.579 INFO:teuthology.packaging:tag: None 2024-02-22T15:19:11.579 INFO:teuthology.packaging:branch: quincy 2024-02-22T15:19:11.579 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:19:11.580 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-22T15:19:11.640 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-22T15:19:11.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-22T15:19:11.710 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-02-22T15:19:11.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-02-22T15:19:11.729 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-22T15:19:11.797 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-02-22T15:19:11.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-02-22T15:19:12.000 INFO:teuthology.orchestra.run.smithi019.stdout:76 files removed 2024-02-22T15:19:12.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw 2024-02-22T15:19:12.073 INFO:teuthology.orchestra.run.smithi062.stdout:76 files removed 2024-02-22T15:19:12.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw 2024-02-22T15:19:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 212 kB/s | 78 kB 00:00 2024-02-22T15:19:12.845 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 247 kB/s | 78 kB 00:00 2024-02-22T15:19:13.145 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 35 kB/s | 9.1 kB 00:00 2024-02-22T15:19:13.195 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 28 kB/s | 9.1 kB 00:00 2024-02-22T15:19:13.454 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.2 kB/s | 1.4 kB 00:00 2024-02-22T15:19:13.496 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 5.4 kB/s | 1.4 kB 00:00 2024-02-22T15:19:15.546 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - AppStream 14 MB/s | 27 MB 00:02 2024-02-22T15:19:15.555 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - AppStream 13 MB/s | 27 MB 00:02 2024-02-22T15:27:38.079 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - BaseOS 21 kB/s | 10 MB 08:17 2024-02-22T15:27:40.178 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - Extras 44 kB/s | 18 kB 00:00 2024-02-22T15:27:40.654 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - Extras common packages 19 kB/s | 6.9 kB 00:00 2024-02-22T15:27:41.004 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - BaseOS 21 kB/s | 10 MB 08:20 2024-02-22T15:27:41.871 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 8 - PowerTools 4.3 MB/s | 4.9 MB 00:01 2024-02-22T15:27:43.106 INFO:teuthology.orchestra.run.smithi019.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-02-22T15:27:43.288 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras 30 kB/s | 18 kB 00:00 2024-02-22T15:27:44.973 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 16 MB 00:01 2024-02-22T15:27:45.063 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - Extras common packages 4.2 kB/s | 6.9 kB 00:01 2024-02-22T15:27:46.314 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 8 - PowerTools 4.2 MB/s | 4.9 MB 00:01 2024-02-22T15:27:47.440 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-02-22T15:27:49.048 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 384 kB/s | 24 kB 00:00 2024-02-22T15:27:49.349 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 16 MB 00:01 2024-02-22T15:27:52.875 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:27:52.877 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:27:52.877 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 17 M 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.5 M 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.6 M 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 5.2 M 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 23 M 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 24 k 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-22T15:27:52.878 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 797 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 506 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 5.6 M 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 45 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 115 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 190 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 371 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 359 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 106 k 2024-02-22T15:27:52.879 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-02-22T15:27:52.880 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:27:52.880 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:27:52.880 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:27:52.880 INFO:teuthology.orchestra.run.smithi019.stdout:Install 21 Packages 2024-02-22T15:27:52.880 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 2 Packages 2024-02-22T15:27:52.880 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:27:52.881 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 64 M 2024-02-22T15:27:52.881 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:27:53.482 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 407 kB/s | 24 kB 00:00 2024-02-22T15:27:55.333 INFO:teuthology.orchestra.run.smithi019.stdout:(1/23): ceph-common-17.2.7-527.g6ea2a55e.el8.x8 12 MB/s | 23 MB 00:01 2024-02-22T15:27:55.524 INFO:teuthology.orchestra.run.smithi019.stdout:(2/23): ceph-radosgw-17.2.7-527.g6ea2a55e.el8.x 7.7 MB/s | 17 MB 00:02 2024-02-22T15:27:55.667 INFO:teuthology.orchestra.run.smithi019.stdout:(3/23): ceph-selinux-17.2.7-527.g6ea2a55e.el8.x 73 kB/s | 24 kB 00:00 2024-02-22T15:27:55.708 INFO:teuthology.orchestra.run.smithi019.stdout:(4/23): libcephfs2-17.2.7-527.g6ea2a55e.el8.x86 4.2 MB/s | 797 kB 00:00 2024-02-22T15:27:55.775 INFO:teuthology.orchestra.run.smithi019.stdout:(5/23): libradosstriper1-17.2.7-527.g6ea2a55e.e 4.6 MB/s | 506 kB 00:00 2024-02-22T15:27:55.892 INFO:teuthology.orchestra.run.smithi019.stdout:(6/23): python3-ceph-argparse-17.2.7-527.g6ea2a 381 kB/s | 45 kB 00:00 2024-02-22T15:27:55.951 INFO:teuthology.orchestra.run.smithi019.stdout:(7/23): python3-ceph-common-17.2.7-527.g6ea2a55 1.9 MB/s | 115 kB 00:00 2024-02-22T15:27:56.018 INFO:teuthology.orchestra.run.smithi019.stdout:(8/23): python3-cephfs-17.2.7-527.g6ea2a55e.el8 2.8 MB/s | 190 kB 00:00 2024-02-22T15:27:56.093 INFO:teuthology.orchestra.run.smithi019.stdout:(9/23): librgw2-17.2.7-527.g6ea2a55e.el8.x86_64 15 MB/s | 5.6 MB 00:00 2024-02-22T15:27:56.119 INFO:teuthology.orchestra.run.smithi019.stdout:(10/23): python3-rados-17.2.7-527.g6ea2a55e.el8 3.6 MB/s | 371 kB 00:00 2024-02-22T15:27:56.160 INFO:teuthology.orchestra.run.smithi019.stdout:(11/23): python3-rbd-17.2.7-527.g6ea2a55e.el8.x 5.2 MB/s | 359 kB 00:00 2024-02-22T15:27:56.186 INFO:teuthology.orchestra.run.smithi019.stdout:(12/23): python3-rgw-17.2.7-527.g6ea2a55e.el8.x 1.6 MB/s | 106 kB 00:00 2024-02-22T15:27:56.511 INFO:teuthology.orchestra.run.smithi019.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 799 kB/s | 259 kB 00:00 2024-02-22T15:27:56.536 INFO:teuthology.orchestra.run.smithi019.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.8 MB/s | 677 kB 00:00 2024-02-22T15:27:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 476 kB/s | 44 kB 00:00 2024-02-22T15:27:56.754 INFO:teuthology.orchestra.run.smithi019.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 2.0 MB/s | 306 kB 00:00 2024-02-22T15:27:56.821 INFO:teuthology.orchestra.run.smithi019.stdout:(17/23): ceph-base-17.2.7-527.g6ea2a55e.el8.x86 1.5 MB/s | 5.2 MB 00:03 2024-02-22T15:27:56.838 INFO:teuthology.orchestra.run.smithi019.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 703 kB/s | 59 kB 00:00 2024-02-22T15:27:56.938 INFO:teuthology.orchestra.run.smithi019.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 642 kB/s | 75 kB 00:00 2024-02-22T15:27:56.980 INFO:teuthology.orchestra.run.smithi019.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2024-02-22T15:27:57.292 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:27:57.294 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:27:57.294 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 17 M 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.5 M 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.6 M 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 5.2 M 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 23 M 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 24 k 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 797 k 2024-02-22T15:27:57.295 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 506 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 5.6 M 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 45 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 115 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 190 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 371 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 359 k 2024-02-22T15:27:57.296 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 106 k 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout:Install 21 Packages 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 2 Packages 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:27:57.297 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 64 M 2024-02-22T15:27:57.298 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:27:57.347 INFO:teuthology.orchestra.run.smithi019.stdout:(21/23): librbd1-17.2.7-527.g6ea2a55e.el8.x86_6 9.8 MB/s | 3.6 MB 00:00 2024-02-22T15:27:57.689 INFO:teuthology.orchestra.run.smithi019.stdout:(22/23): librados2-17.2.7-527.g6ea2a55e.el8.x86 4.6 MB/s | 3.5 MB 00:00 2024-02-22T15:27:58.800 INFO:teuthology.orchestra.run.smithi062.stdout:(1/23): ceph-base-17.2.7-527.g6ea2a55e.el8.x86_ 4.7 MB/s | 5.2 MB 00:01 2024-02-22T15:27:58.900 INFO:teuthology.orchestra.run.smithi062.stdout:(2/23): ceph-selinux-17.2.7-527.g6ea2a55e.el8.x 244 kB/s | 24 kB 00:00 2024-02-22T15:27:59.151 INFO:teuthology.orchestra.run.smithi062.stdout:(3/23): libcephfs2-17.2.7-527.g6ea2a55e.el8.x86 3.1 MB/s | 797 kB 00:00 2024-02-22T15:27:59.309 INFO:teuthology.orchestra.run.smithi062.stdout:(4/23): libradosstriper1-17.2.7-527.g6ea2a55e.e 3.1 MB/s | 506 kB 00:00 2024-02-22T15:27:59.693 INFO:teuthology.orchestra.run.smithi062.stdout:(5/23): ceph-common-17.2.7-527.g6ea2a55e.el8.x8 12 MB/s | 23 MB 00:02 2024-02-22T15:27:59.870 INFO:teuthology.orchestra.run.smithi062.stdout:(6/23): ceph-radosgw-17.2.7-527.g6ea2a55e.el8.x 7.7 MB/s | 17 MB 00:02 2024-02-22T15:27:59.895 INFO:teuthology.orchestra.run.smithi062.stdout:(7/23): python3-ceph-argparse-17.2.7-527.g6ea2a 221 kB/s | 45 kB 00:00 2024-02-22T15:27:59.921 INFO:teuthology.orchestra.run.smithi062.stdout:(8/23): python3-ceph-common-17.2.7-527.g6ea2a55 2.3 MB/s | 115 kB 00:00 2024-02-22T15:27:59.946 INFO:teuthology.orchestra.run.smithi062.stdout:(9/23): python3-cephfs-17.2.7-527.g6ea2a55e.el8 3.7 MB/s | 190 kB 00:00 2024-02-22T15:27:59.980 INFO:teuthology.orchestra.run.smithi062.stdout:(10/23): python3-rados-17.2.7-527.g6ea2a55e.el8 6.2 MB/s | 371 kB 00:00 2024-02-22T15:28:00.047 INFO:teuthology.orchestra.run.smithi062.stdout:(11/23): python3-rbd-17.2.7-527.g6ea2a55e.el8.x 3.5 MB/s | 359 kB 00:00 2024-02-22T15:28:00.072 INFO:teuthology.orchestra.run.smithi062.stdout:(12/23): python3-rgw-17.2.7-527.g6ea2a55e.el8.x 1.1 MB/s | 106 kB 00:00 2024-02-22T15:28:00.231 INFO:teuthology.orchestra.run.smithi062.stdout:(13/23): librgw2-17.2.7-527.g6ea2a55e.el8.x86_6 6.1 MB/s | 5.6 MB 00:00 2024-02-22T15:28:00.289 INFO:teuthology.orchestra.run.smithi062.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-02-22T15:28:00.356 INFO:teuthology.orchestra.run.smithi062.stdout:(15/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.1 MB/s | 677 kB 00:00 2024-02-22T15:28:00.390 INFO:teuthology.orchestra.run.smithi062.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 276 kB/s | 44 kB 00:00 2024-02-22T15:28:00.549 INFO:teuthology.orchestra.run.smithi062.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 371 kB/s | 59 kB 00:00 2024-02-22T15:28:00.607 INFO:teuthology.orchestra.run.smithi062.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2024-02-22T15:28:00.632 INFO:teuthology.orchestra.run.smithi062.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2024-02-22T15:28:00.916 INFO:teuthology.orchestra.run.smithi062.stdout:(20/23): librados2-17.2.7-527.g6ea2a55e.el8.x86 12 MB/s | 3.5 MB 00:00 2024-02-22T15:28:01.116 INFO:teuthology.orchestra.run.smithi062.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 3.4 MB/s | 1.7 MB 00:00 2024-02-22T15:28:01.200 INFO:teuthology.orchestra.run.smithi062.stdout:(22/23): librbd1-17.2.7-527.g6ea2a55e.el8.x86_6 13 MB/s | 3.6 MB 00:00 2024-02-22T15:32:10.715 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] librabbitmq-0.9.0-5.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/librabbitmq-0.9.0-5.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:32:10.875 INFO:teuthology.orchestra.run.smithi062.stdout:(23/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 192 B/s | 47 kB 04:10 2024-02-22T15:32:10.875 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:32:10.875 INFO:teuthology.orchestra.run.smithi062.stdout:Total 257 kB/s | 64 MB 04:13 2024-02-22T15:32:10.918 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:32:10.968 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:32:10.968 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:32:11.369 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:32:11.373 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:32:11.944 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] librabbitmq-0.9.0-5.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/librabbitmq-0.9.0-5.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:32:12.138 INFO:teuthology.orchestra.run.smithi019.stdout:(23/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 188 B/s | 47 kB 04:15 2024-02-22T15:32:12.139 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:32:12.139 INFO:teuthology.orchestra.run.smithi019.stdout:Total 252 kB/s | 64 MB 04:19 2024-02-22T15:32:12.180 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:32:12.230 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:32:12.231 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:32:12.495 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:32:12.619 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:32:12.623 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:32:12.697 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-22T15:32:12.858 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T15:32:13.423 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T15:32:13.649 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/25 2024-02-22T15:32:13.686 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:32:13.743 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/25 2024-02-22T15:32:13.913 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-22T15:32:14.038 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 4/25 2024-02-22T15:32:14.076 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T15:32:14.232 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/25 2024-02-22T15:32:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-22T15:32:14.651 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/25 2024-02-22T15:32:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/25 2024-02-22T15:32:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/25 2024-02-22T15:32:14.849 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/25 2024-02-22T15:32:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 4/25 2024-02-22T15:32:15.222 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/25 2024-02-22T15:32:15.342 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/25 2024-02-22T15:32:15.446 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-02-22T15:32:15.654 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-02-22T15:32:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/25 2024-02-22T15:32:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/25 2024-02-22T15:32:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/25 2024-02-22T15:32:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-02-22T15:32:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-02-22T15:32:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 10/25 2024-02-22T15:32:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 10/25 2024-02-22T15:32:16.630 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-02-22T15:32:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:17.2.7-527.g6ea2a55e.el8.x 11/25 2024-02-22T15:32:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 12/25 2024-02-22T15:32:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 13/25 2024-02-22T15:32:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 14/25 2024-02-22T15:32:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-02-22T15:32:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 15/25 2024-02-22T15:32:17.617 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 10/25 2024-02-22T15:32:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 15/25 2024-02-22T15:32:17.681 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 10/25 2024-02-22T15:32:17.820 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:17.2.7-527.g6ea2a55e.el8.x 11/25 2024-02-22T15:32:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-22T15:32:18.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 12/25 2024-02-22T15:32:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-22T15:32:18.238 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 13/25 2024-02-22T15:32:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-22T15:32:18.449 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 14/25 2024-02-22T15:32:18.499 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86 19/25 2024-02-22T15:32:18.592 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 15/25 2024-02-22T15:32:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 15/25 2024-02-22T15:32:18.921 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-22T15:32:19.085 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-22T15:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-22T15:32:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86 19/25 2024-02-22T15:32:21.354 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 20/25 2024-02-22T15:32:21.741 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 20/25 2024-02-22T15:32:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 20/25 2024-02-22T15:32:22.360 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 20/25 2024-02-22T15:32:22.467 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 20/25 2024-02-22T15:32:22.673 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 21/25 2024-02-22T15:32:22.785 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 21/25 2024-02-22T15:32:23.082 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 20/25 2024-02-22T15:32:23.131 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 22/25 2024-02-22T15:32:23.324 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 21/25 2024-02-22T15:32:23.447 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 21/25 2024-02-22T15:32:23.616 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 22/25 2024-02-22T15:32:43.584 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 22/25 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2024-02-22T15:32:43.585 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 22/25 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /dev 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /run 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /dev 2024-02-22T15:32:43.873 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:32:45.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 23/25 2024-02-22T15:32:45.437 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 23/25 2024-02-22T15:32:45.437 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T15:32:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T15:32:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T15:32:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 23/25 2024-02-22T15:32:45.851 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 23/25 2024-02-22T15:32:45.852 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T15:32:46.082 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-22T15:32:46.082 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 4/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 7/25 2024-02-22T15:32:46.367 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:17.2.7-527.g6ea2a55e.el8.x 8/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86 9/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 10/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 11/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 12/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 13/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-02-22T15:32:46.368 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 22/25 2024-02-22T15:32:46.369 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-22T15:32:46.369 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 24/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 4/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 7/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:17.2.7-527.g6ea2a55e.el8.x 8/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86 9/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 10/25 2024-02-22T15:32:46.799 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 11/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 12/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 13/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 22/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-22T15:32:46.800 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 24/25 2024-02-22T15:32:47.372 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T15:32:47.372 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.373 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2024-02-22T15:32:47.374 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:32:47.375 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:32:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-22T15:32:47.710 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:32:47.711 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.13.0-2.el8.x86_64 2024-02-22T15:32:47.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:32:47.712 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:32:47.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-test 2024-02-22T15:32:48.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test 2024-02-22T15:32:48.538 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:04:55 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:32:48.840 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:05:00 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:32:50.225 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 52 M 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:32:50.226 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 178 k 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout:Install 6 Packages 2024-02-22T15:32:50.227 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:32:50.228 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 53 M 2024-02-22T15:32:50.228 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 234 M 2024-02-22T15:32:50.228 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:32:50.534 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:32:50.535 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:32:50.535 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:32:50.535 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:32:50.535 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:32:50.536 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 52 M 2024-02-22T15:32:50.536 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:32:50.536 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-22T15:32:50.536 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 178 k 2024-02-22T15:32:50.536 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-22T15:32:50.536 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-22T15:32:50.537 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-22T15:32:50.537 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:32:50.537 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:32:50.537 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:32:50.537 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2024-02-22T15:32:50.538 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:32:50.539 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 53 M 2024-02-22T15:32:50.539 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 234 M 2024-02-22T15:32:50.539 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:32:50.678 INFO:teuthology.orchestra.run.smithi062.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 790 kB/s | 203 kB 00:00 2024-02-22T15:32:50.820 INFO:teuthology.orchestra.run.smithi062.stdout:(2/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-02-22T15:32:50.846 INFO:teuthology.orchestra.run.smithi062.stdout:(3/6): libcephsqlite-17.2.7-527.g6ea2a55e.el8.x 420 kB/s | 178 kB 00:00 2024-02-22T15:32:50.975 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 835 kB/s | 203 kB 00:00 2024-02-22T15:32:51.039 INFO:teuthology.orchestra.run.smithi062.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 365 kB/s | 71 kB 00:00 2024-02-22T15:32:51.079 INFO:teuthology.orchestra.run.smithi062.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2024-02-22T15:32:51.109 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-02-22T15:32:51.134 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): libcephsqlite-17.2.7-527.g6ea2a55e.el8.x 444 kB/s | 178 kB 00:00 2024-02-22T15:32:51.293 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-02-22T15:32:51.359 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 314 kB/s | 71 kB 00:00 2024-02-22T15:32:54.221 INFO:teuthology.orchestra.run.smithi062.stdout:(6/6): ceph-test-17.2.7-527.g6ea2a55e.el8.x86_6 14 MB/s | 52 MB 00:03 2024-02-22T15:32:54.223 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:32:54.223 INFO:teuthology.orchestra.run.smithi062.stdout:Total 13 MB/s | 53 MB 00:03 2024-02-22T15:32:54.266 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:32:54.289 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:32:54.289 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:32:54.623 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:32:54.626 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:32:55.226 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:32:55.444 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-22T15:32:55.664 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-22T15:32:55.785 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T15:32:55.868 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T15:32:56.101 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-22T15:32:56.284 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/6 2024-02-22T15:32:56.867 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): ceph-test-17.2.7-527.g6ea2a55e.el8.x86_6 8.5 MB/s | 52 MB 00:06 2024-02-22T15:32:56.869 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:32:56.870 INFO:teuthology.orchestra.run.smithi019.stdout:Total 8.3 MB/s | 53 MB 00:06 2024-02-22T15:32:56.913 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:32:56.936 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:32:56.936 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:32:57.247 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:32:57.251 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:32:57.727 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:32:57.985 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-22T15:32:58.165 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-22T15:32:58.294 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T15:32:58.356 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-22T15:32:58.553 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-22T15:32:58.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/6 2024-02-22T15:33:01.575 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/6 2024-02-22T15:33:01.792 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/6 2024-02-22T15:33:02.082 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/6 2024-02-22T15:33:02.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/6 2024-02-22T15:33:02.082 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/6 2024-02-22T15:33:02.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-22T15:33:02.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-22T15:33:02.083 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-8.el8.x86_64 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-22T15:33:02.586 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-22T15:33:02.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:33:02.587 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:33:02.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph 2024-02-22T15:33:03.605 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:05:10 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:33:04.086 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/6 2024-02-22T15:33:04.309 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/6 2024-02-22T15:33:04.593 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 6/6 2024-02-22T15:33:04.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/6 2024-02-22T15:33:04.593 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/6 2024-02-22T15:33:04.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-22T15:33:04.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-22T15:33:04.594 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-22T15:33:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-22T15:33:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:33:05.085 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:33:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:33:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: jq-1.6-8.el8.x86_64 2024-02-22T15:33:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:33:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-22T15:33:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-22T15:33:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-22T15:33:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:33:05.086 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:33:05.289 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:33:05.292 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:33:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:33:05.292 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:33:05.292 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:33:05.292 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 6.4 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 2.3 M 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 1.6 M 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 239 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.9 M 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 17 M 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-22T15:33:05.293 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-22T15:33:05.294 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout:Install 32 Packages 2024-02-22T15:33:05.295 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:33:05.296 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 28 M 2024-02-22T15:33:05.296 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 110 M 2024-02-22T15:33:05.296 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:33:05.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph 2024-02-22T15:33:06.116 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:05:18 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:33:06.131 INFO:teuthology.orchestra.run.smithi062.stdout:(1/32): ceph-17.2.7-527.g6ea2a55e.el8.x86_64.rp 13 kB/s | 6.4 kB 00:00 2024-02-22T15:33:06.881 INFO:teuthology.orchestra.run.smithi062.stdout:(2/32): ceph-mgr-17.2.7-527.g6ea2a55e.el8.x86_6 1.3 MB/s | 1.6 MB 00:01 2024-02-22T15:33:07.548 INFO:teuthology.orchestra.run.smithi062.stdout:(3/32): ceph-mds-17.2.7-527.g6ea2a55e.el8.x86_6 1.2 MB/s | 2.3 MB 00:01 2024-02-22T15:33:07.800 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:33:07.803 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:33:07.803 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:33:07.803 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:33:07.803 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 6.4 k 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 2.3 M 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 1.6 M 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 239 k 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.9 M 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 17 M 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-22T15:33:07.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-22T15:33:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout:Install 32 Packages 2024-02-22T15:33:07.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:33:07.807 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 28 M 2024-02-22T15:33:07.807 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 110 M 2024-02-22T15:33:07.807 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:33:07.873 INFO:teuthology.orchestra.run.smithi062.stdout:(4/32): ceph-mon-17.2.7-527.g6ea2a55e.el8.x86_6 2.3 MB/s | 3.9 MB 00:01 2024-02-22T15:33:07.974 INFO:teuthology.orchestra.run.smithi062.stdout:(5/32): ceph-mgr-modules-core-17.2.7-527.g6ea2a 562 kB/s | 239 kB 00:00 2024-02-22T15:33:08.057 INFO:teuthology.orchestra.run.smithi062.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 856 kB/s | 157 kB 00:00 2024-02-22T15:33:08.091 INFO:teuthology.orchestra.run.smithi062.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 332 kB/s | 39 kB 00:00 2024-02-22T15:33:08.116 INFO:teuthology.orchestra.run.smithi062.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.7 MB/s | 103 kB 00:00 2024-02-22T15:33:08.174 INFO:teuthology.orchestra.run.smithi062.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 683 kB/s | 57 kB 00:00 2024-02-22T15:33:08.250 INFO:teuthology.orchestra.run.smithi062.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.3 MB/s | 457 kB 00:00 2024-02-22T15:33:08.620 INFO:teuthology.orchestra.run.smithi019.stdout:(1/32): ceph-17.2.7-527.g6ea2a55e.el8.x86_64.rp 24 kB/s | 6.4 kB 00:00 2024-02-22T15:33:10.311 INFO:teuthology.orchestra.run.smithi019.stdout:(2/32): ceph-mds-17.2.7-527.g6ea2a55e.el8.x86_6 1.2 MB/s | 2.3 MB 00:01 2024-02-22T15:33:10.417 INFO:teuthology.orchestra.run.smithi062.stdout:(11/32): ceph-osd-17.2.7-527.g6ea2a55e.el8.x86_ 4.7 MB/s | 17 MB 00:03 2024-02-22T15:33:10.437 INFO:teuthology.orchestra.run.smithi019.stdout:(3/32): ceph-mon-17.2.7-527.g6ea2a55e.el8.x86_6 2.2 MB/s | 3.9 MB 00:01 2024-02-22T15:33:10.478 INFO:teuthology.orchestra.run.smithi019.stdout:(4/32): ceph-mgr-17.2.7-527.g6ea2a55e.el8.x86_6 756 kB/s | 1.6 MB 00:02 2024-02-22T15:33:10.512 INFO:teuthology.orchestra.run.smithi019.stdout:(5/32): ceph-mgr-modules-core-17.2.7-527.g6ea2a 3.1 MB/s | 239 kB 00:00 2024-02-22T15:33:10.688 INFO:teuthology.orchestra.run.smithi019.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 222 kB/s | 39 kB 00:00 2024-02-22T15:33:10.738 INFO:teuthology.orchestra.run.smithi019.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 606 kB/s | 157 kB 00:00 2024-02-22T15:33:10.813 INFO:teuthology.orchestra.run.smithi019.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 825 kB/s | 103 kB 00:00 2024-02-22T15:33:10.839 INFO:teuthology.orchestra.run.smithi019.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 567 kB/s | 57 kB 00:00 2024-02-22T15:33:11.056 INFO:teuthology.orchestra.run.smithi019.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.8 MB/s | 457 kB 00:00 2024-02-22T15:33:15.714 INFO:teuthology.orchestra.run.smithi019.stdout:(11/32): ceph-osd-17.2.7-527.g6ea2a55e.el8.x86_ 3.1 MB/s | 17 MB 00:05 2024-02-22T15:37:17.787 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] python3-cryptography-3.2.1-7.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cryptography-3.2.1-7.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:37:17.789 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] python3-cffi-1.11.5-6.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.11.5-6.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:37:17.983 INFO:teuthology.orchestra.run.smithi062.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 kB/s | 249 kB 04:09 2024-02-22T15:37:18.017 INFO:teuthology.orchestra.run.smithi062.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 2.6 kB/s | 641 kB 04:09 2024-02-22T15:37:18.151 INFO:teuthology.orchestra.run.smithi062.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 261 kB/s | 44 kB 00:00 2024-02-22T15:37:18.192 INFO:teuthology.orchestra.run.smithi062.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.0 MB/s | 185 kB 00:00 2024-02-22T15:37:18.252 INFO:teuthology.orchestra.run.smithi062.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.7 MB/s | 173 kB 00:00 2024-02-22T15:37:18.276 INFO:teuthology.orchestra.run.smithi062.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.5 MB/s | 384 kB 00:00 2024-02-22T15:37:18.302 INFO:teuthology.orchestra.run.smithi062.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2024-02-22T15:37:18.327 INFO:teuthology.orchestra.run.smithi062.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2024-02-22T15:37:18.352 INFO:teuthology.orchestra.run.smithi062.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 972 kB/s | 49 kB 00:00 2024-02-22T15:37:18.377 INFO:teuthology.orchestra.run.smithi062.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2024-02-22T15:37:18.403 INFO:teuthology.orchestra.run.smithi062.stdout:(22/32): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2024-02-22T15:37:18.428 INFO:teuthology.orchestra.run.smithi062.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2024-02-22T15:37:18.453 INFO:teuthology.orchestra.run.smithi062.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 374 kB/s | 19 kB 00:00 2024-02-22T15:37:18.478 INFO:teuthology.orchestra.run.smithi062.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 476 kB/s | 24 kB 00:00 2024-02-22T15:37:18.504 INFO:teuthology.orchestra.run.smithi062.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 581 kB/s | 29 kB 00:00 2024-02-22T15:37:18.529 INFO:teuthology.orchestra.run.smithi062.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 547 kB/s | 27 kB 00:00 2024-02-22T15:37:18.579 INFO:teuthology.orchestra.run.smithi062.stdout:(28/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.3 MB/s | 255 kB 00:00 2024-02-22T15:37:18.605 INFO:teuthology.orchestra.run.smithi062.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.3 MB/s | 251 kB 00:00 2024-02-22T15:37:18.630 INFO:teuthology.orchestra.run.smithi062.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2024-02-22T15:37:18.655 INFO:teuthology.orchestra.run.smithi062.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 458 kB/s | 23 kB 00:00 2024-02-22T15:37:23.149 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] python3-pycparser-2.14-14.el8.noarch.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.14-14.el8.noarch.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:37:23.407 INFO:teuthology.orchestra.run.smithi019.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 451 B/s | 109 kB 04:07 2024-02-22T15:37:23.557 INFO:teuthology.orchestra.run.smithi019.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 291 kB/s | 44 kB 00:00 2024-02-22T15:37:23.732 INFO:teuthology.orchestra.run.smithi019.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.0 MB/s | 185 kB 00:00 2024-02-22T15:37:23.849 INFO:teuthology.orchestra.run.smithi019.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.4 MB/s | 173 kB 00:00 2024-02-22T15:37:24.091 INFO:teuthology.orchestra.run.smithi019.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.6 MB/s | 384 kB 00:00 2024-02-22T15:37:24.141 INFO:teuthology.orchestra.run.smithi019.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2024-02-22T15:37:24.200 INFO:teuthology.orchestra.run.smithi019.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 302 kB/s | 18 kB 00:00 2024-02-22T15:37:24.295 INFO:teuthology.orchestra.run.smithi019.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 511 kB/s | 49 kB 00:00 2024-02-22T15:37:24.375 INFO:teuthology.orchestra.run.smithi019.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 737 kB/s | 59 kB 00:00 2024-02-22T15:37:24.584 INFO:teuthology.orchestra.run.smithi019.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2024-02-22T15:37:24.743 INFO:teuthology.orchestra.run.smithi019.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 99 kB/s | 16 kB 00:00 2024-02-22T15:37:24.818 INFO:teuthology.orchestra.run.smithi019.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 249 kB/s | 19 kB 00:00 2024-02-22T15:37:24.877 INFO:teuthology.orchestra.run.smithi019.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 411 kB/s | 24 kB 00:00 2024-02-22T15:37:24.944 INFO:teuthology.orchestra.run.smithi019.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 439 kB/s | 29 kB 00:00 2024-02-22T15:37:25.011 INFO:teuthology.orchestra.run.smithi019.stdout:(26/32): python3-trustme-0.6.0-4.el8.noarch.rpm 414 kB/s | 27 kB 00:00 2024-02-22T15:37:25.203 INFO:teuthology.orchestra.run.smithi019.stdout:(27/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.3 MB/s | 255 kB 00:00 2024-02-22T15:37:25.320 INFO:teuthology.orchestra.run.smithi019.stdout:(28/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.1 MB/s | 251 kB 00:00 2024-02-22T15:37:25.387 INFO:teuthology.orchestra.run.smithi019.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.3 MB/s | 85 kB 00:00 2024-02-22T15:37:25.445 INFO:teuthology.orchestra.run.smithi019.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 394 kB/s | 23 kB 00:00 2024-02-22T15:37:25.981 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] python3-pycparser-2.14-14.el8.noarch.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.14-14.el8.noarch.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:37:26.138 INFO:teuthology.orchestra.run.smithi062.stdout:(32/32): python3-pycparser-2.14-14.el8.noarch.r 436 B/s | 109 kB 04:15 2024-02-22T15:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:26.140 INFO:teuthology.orchestra.run.smithi062.stdout:Total 112 kB/s | 28 MB 04:20 2024-02-22T15:37:26.216 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:37:26.257 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:37:26.581 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:37:26.584 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:37:27.320 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] python3-cryptography-3.2.1-7.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cryptography-3.2.1-7.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:37:27.629 INFO:teuthology.orchestra.run.smithi019.stdout:(31/32): python3-cryptography-3.2.1-7.el8.x86_6 2.5 kB/s | 641 kB 04:16 2024-02-22T15:37:27.781 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:37:28.030 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-22T15:37:28.282 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-22T15:37:28.509 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-22T15:37:28.835 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-22T15:37:29.033 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-22T15:37:29.215 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-22T15:37:29.415 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-22T15:37:29.609 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-22T15:37:29.847 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-22T15:37:30.074 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-22T15:37:30.312 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-22T15:37:30.579 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-22T15:37:30.944 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-22T15:37:31.182 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-22T15:37:31.325 INFO:teuthology.orchestra.run.smithi019.stdout:[MIRROR] python3-cffi-1.11.5-6.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://mirrors.cmich.edu/centos/8-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.11.5-6.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-22T15:37:31.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-22T15:37:31.503 INFO:teuthology.orchestra.run.smithi019.stdout:(32/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 977 B/s | 249 kB 04:20 2024-02-22T15:37:31.504 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:31.504 INFO:teuthology.orchestra.run.smithi019.stdout:Total 110 kB/s | 28 MB 04:23 2024-02-22T15:37:31.581 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:37:31.621 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:37:31.621 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:37:31.658 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-22T15:37:31.889 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T15:37:31.934 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:37:31.937 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:37:32.208 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-22T15:37:32.469 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-22T15:37:32.666 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-22T15:37:32.851 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-22T15:37:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:37:33.105 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-22T15:37:33.271 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-22T15:37:33.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-22T15:37:33.538 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-22T15:37:33.573 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-22T15:37:33.767 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-22T15:37:33.903 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-22T15:37:34.102 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-22T15:37:34.207 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-22T15:37:34.309 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-22T15:37:34.475 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-22T15:37:34.600 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-527.g6ea2a55e.el8.n 27/32 2024-02-22T15:37:34.649 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-22T15:37:34.818 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-22T15:37:34.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 28/32 2024-02-22T15:37:35.014 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-22T15:37:35.266 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-22T15:37:35.480 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-22T15:37:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-22T15:37:36.071 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-22T15:37:36.308 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-22T15:37:36.408 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 28/32 2024-02-22T15:37:36.470 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-22T15:37:36.586 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 29/32 2024-02-22T15:37:36.733 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-22T15:37:36.956 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T15:37:37.001 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 29/32 2024-02-22T15:37:37.177 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 30/32 2024-02-22T15:37:37.256 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-22T15:37:37.414 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 30/32 2024-02-22T15:37:37.477 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-22T15:37:37.602 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 31/32 2024-02-22T15:37:37.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 31/32 2024-02-22T15:37:37.666 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-22T15:37:37.744 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 32/32 2024-02-22T15:37:37.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-22T15:37:38.079 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-22T15:37:38.166 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 32/32 2024-02-22T15:37:38.166 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/32 2024-02-22T15:37:38.166 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/32 2024-02-22T15:37:38.166 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/32 2024-02-22T15:37:38.166 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 4/32 2024-02-22T15:37:38.166 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-527.g6ea2a55e.el8.n 6/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-22T15:37:38.167 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-22T15:37:38.168 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-22T15:37:38.169 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-22T15:37:38.169 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-22T15:37:38.169 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-22T15:37:38.169 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-22T15:37:38.301 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-22T15:37:38.549 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-22T15:37:38.869 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-22T15:37:39.000 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-22T15:37:39.001 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-22T15:37:39.002 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:39.003 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:37:39.156 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-22T15:37:39.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-base 2024-02-22T15:37:39.525 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-527.g6ea2a55e.el8.n 27/32 2024-02-22T15:37:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 28/32 2024-02-22T15:37:40.066 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:09:47 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:37:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 28/32 2024-02-22T15:37:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 29/32 2024-02-22T15:37:41.693 INFO:teuthology.orchestra.run.smithi062.stdout:Package ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:37:41.744 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:37:41.745 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:37:41.745 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:37:41.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install cephadm 2024-02-22T15:37:41.901 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 29/32 2024-02-22T15:37:42.095 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 30/32 2024-02-22T15:37:42.329 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 30/32 2024-02-22T15:37:42.495 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 31/32 2024-02-22T15:37:42.513 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 31/32 2024-02-22T15:37:42.662 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:09:49 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:37:42.678 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 32/32 2024-02-22T15:37:43.105 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 32/32 2024-02-22T15:37:43.105 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 3/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 4/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 5/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-527.g6ea2a55e.el8.n 6/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-22T15:37:43.106 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-22T15:37:43.107 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-22T15:37:43.108 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-22T15:37:43.956 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-22T15:37:43.956 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:43.956 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:37:43.956 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:43.956 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-22T15:37:43.957 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-2.6-1.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-22T15:37:43.958 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-22T15:37:43.959 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-22T15:37:43.959 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-22T15:37:43.959 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:43.959 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:37:44.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base 2024-02-22T15:37:44.341 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 80 k 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 80 k 2024-02-22T15:37:44.343 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 364 k 2024-02-22T15:37:44.344 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:37:44.511 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm-17.2.7-527.g6ea2a55e.el8.noarch.rpm 479 kB/s | 80 kB 00:00 2024-02-22T15:37:44.512 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:44.512 INFO:teuthology.orchestra.run.smithi062.stdout:Total 475 kB/s | 80 kB 00:00 2024-02-22T15:37:44.512 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:37:44.521 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:37:44.521 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:37:44.523 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:37:44.526 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:37:44.573 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:37:44.988 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:09:56 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:37:45.069 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:45.247 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:45.481 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:45.878 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:45.878 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:45.878 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:37:45.878 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:45.878 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:45.878 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:37:46.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-immutable-object-cache 2024-02-22T15:37:46.619 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-base-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:37:46.670 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:37:46.671 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:37:46.671 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:37:46.792 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install cephadm 2024-02-22T15:37:46.906 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:09:53 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:37:47.577 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:09:59 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:37:48.590 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:37:48.591 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:48.591 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2024-02-22T15:37:48.591 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:48.591 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:37:48.591 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 162 k 2024-02-22T15:37:48.591 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 162 k 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 490 k 2024-02-22T15:37:48.592 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:37:48.783 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-immutable-object-cache-17.2.7-527.g6ea2a55 851 kB/s | 162 kB 00:00 2024-02-22T15:37:48.783 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:48.783 INFO:teuthology.orchestra.run.smithi062.stdout:Total 846 kB/s | 162 kB 00:00 2024-02-22T15:37:48.784 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:37:48.796 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:37:48.796 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:37:48.846 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:37:48.850 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:37:48.967 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:37:49.176 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.e 1/1 2024-02-22T15:37:49.269 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:37:49.270 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:37:49.270 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:37:49.270 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:37:49.271 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:37:49.271 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 80 k 2024-02-22T15:37:49.271 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:49.272 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:37:49.272 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:37:49.272 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:37:49.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:49.273 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 80 k 2024-02-22T15:37:49.273 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 364 k 2024-02-22T15:37:49.273 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:37:49.437 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm-17.2.7-527.g6ea2a55e.el8.noarch.rpm 482 kB/s | 80 kB 00:00 2024-02-22T15:37:49.438 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:49.438 INFO:teuthology.orchestra.run.smithi019.stdout:Total 478 kB/s | 80 kB 00:00 2024-02-22T15:37:49.438 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:37:49.447 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:37:49.447 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:37:49.449 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:37:49.452 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:37:49.498 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:37:49.617 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.e 1/1 2024-02-22T15:37:49.970 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:50.023 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.e 1/1 2024-02-22T15:37:50.024 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:50.024 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:37:50.024 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:50.024 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:50.024 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:37:50.181 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:50.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr 2024-02-22T15:37:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:50.823 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/1 2024-02-22T15:37:50.823 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:50.823 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:37:50.823 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:50.823 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:50.823 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:37:51.020 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:09:58 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:37:51.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-immutable-object-cache 2024-02-22T15:37:51.841 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:03 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:37:52.663 INFO:teuthology.orchestra.run.smithi062.stdout:Package ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:37:52.711 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:37:52.712 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:37:52.712 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:37:52.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-dashboard 2024-02-22T15:37:53.520 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 162 k 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:37:53.521 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:53.522 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 162 k 2024-02-22T15:37:53.522 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 490 k 2024-02-22T15:37:53.522 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:37:53.613 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:00 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:37:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-immutable-object-cache-17.2.7-527.g6ea2a55 549 kB/s | 162 kB 00:00 2024-02-22T15:37:53.817 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:Total 546 kB/s | 162 kB 00:00 2024-02-22T15:37:53.818 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:37:53.830 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:37:53.830 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:37:53.882 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:37:53.885 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:37:54.000 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:37:54.252 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.e 1/1 2024-02-22T15:37:54.682 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.e 1/1 2024-02-22T15:37:55.075 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.e 1/1 2024-02-22T15:37:55.075 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:55.075 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:37:55.075 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:37:55.075 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:37:55.075 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:37:55.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr 2024-02-22T15:37:55.309 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:37:55.310 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:55.310 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:37:55.310 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:55.310 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:37:55.310 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 1.7 M 2024-02-22T15:37:55.310 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 23 k 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 14 k 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout:Install 6 Packages 2024-02-22T15:37:55.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:55.312 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 2.0 M 2024-02-22T15:37:55.312 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 8.0 M 2024-02-22T15:37:55.312 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:37:55.968 INFO:teuthology.orchestra.run.smithi062.stdout:(1/6): ceph-grafana-dashboards-17.2.7-527.g6ea2 177 kB/s | 23 kB 00:00 2024-02-22T15:37:56.069 INFO:teuthology.orchestra.run.smithi062.stdout:(2/6): ceph-prometheus-alerts-17.2.7-527.g6ea2a 62 kB/s | 14 kB 00:00 2024-02-22T15:37:56.083 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:08 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:37:56.202 INFO:teuthology.orchestra.run.smithi062.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 182 kB/s | 43 kB 00:00 2024-02-22T15:37:56.428 INFO:teuthology.orchestra.run.smithi062.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 94 kB/s | 34 kB 00:00 2024-02-22T15:37:56.511 INFO:teuthology.orchestra.run.smithi062.stdout:(5/6): ceph-mgr-dashboard-17.2.7-527.g6ea2a55e. 2.5 MB/s | 1.7 MB 00:00 2024-02-22T15:37:56.545 INFO:teuthology.orchestra.run.smithi062.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 574 kB/s | 196 kB 00:00 2024-02-22T15:37:56.547 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:37:56.547 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-02-22T15:37:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:37:56.570 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:37:56.570 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:37:56.659 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:37:56.663 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:37:56.926 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:37:57.141 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-22T15:37:57.385 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-22T15:37:57.594 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-22T15:37:57.714 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-mgr-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:37:57.714 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-527.g6ea2a55e.el8.no 4/6 2024-02-22T15:37:57.764 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:37:57.765 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:37:57.766 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:37:57.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-dashboard 2024-02-22T15:37:58.092 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-527.g6ea2a55e.el8.n 5/6 2024-02-22T15:37:58.315 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 6/6 2024-02-22T15:37:58.615 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 6/6 2024-02-22T15:37:58.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-527.g6ea2a55e.el8.n 1/6 2024-02-22T15:37:58.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 2/6 2024-02-22T15:37:58.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-527.g6ea2a55e.el8.no 3/6 2024-02-22T15:37:58.615 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-22T15:37:58.616 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-22T15:37:58.660 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:10 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:37:59.019 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-22T15:37:59.019 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:59.019 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:37:59.020 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:37:59.153 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-22T15:37:59.934 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:06 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:00.404 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 1.7 M 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 23 k 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 14 k 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:00.405 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:00.406 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:00.406 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2024-02-22T15:38:00.406 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:00.406 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.0 M 2024-02-22T15:38:00.406 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 8.0 M 2024-02-22T15:38:00.406 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:00.656 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): ceph-prometheus-alerts-17.2.7-527.g6ea2a 97 kB/s | 14 kB 00:00 2024-02-22T15:38:00.746 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): ceph-grafana-dashboards-17.2.7-527.g6ea2 96 kB/s | 23 kB 00:00 2024-02-22T15:38:00.905 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 170 kB/s | 43 kB 00:00 2024-02-22T15:38:00.930 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 183 kB/s | 34 kB 00:00 2024-02-22T15:38:00.980 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): ceph-mgr-dashboard-17.2.7-527.g6ea2a55e. 3.6 MB/s | 1.7 MB 00:00 2024-02-22T15:38:01.122 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 906 kB/s | 196 kB 00:00 2024-02-22T15:38:01.125 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:01.125 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2024-02-22T15:38:01.136 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:01.146 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:01.146 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:01.231 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:01.235 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:01.630 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:01.631 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-02-22T15:38:01.632 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:38:01.632 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-02-22T15:38:01.632 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:38:01.632 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 7.4 M 2024-02-22T15:38:01.632 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:38:01.632 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:01.633 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:38:01.634 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================ 2024-02-22T15:38:01.634 INFO:teuthology.orchestra.run.smithi062.stdout:Install 8 Packages 2024-02-22T15:38:01.634 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:01.634 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 36 M 2024-02-22T15:38:01.635 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 210 M 2024-02-22T15:38:01.635 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:38:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-22T15:38:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-22T15:38:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-22T15:38:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-527.g6ea2a55e.el8.no 4/6 2024-02-22T15:38:02.327 INFO:teuthology.orchestra.run.smithi062.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2024-02-22T15:38:02.394 INFO:teuthology.orchestra.run.smithi062.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 10 MB/s | 4.9 MB 00:00 2024-02-22T15:38:02.461 INFO:teuthology.orchestra.run.smithi062.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.3 MB/s | 225 kB 00:00 2024-02-22T15:38:02.553 INFO:teuthology.orchestra.run.smithi062.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 17 MB/s | 3.7 MB 00:00 2024-02-22T15:38:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-527.g6ea2a55e.el8.n 5/6 2024-02-22T15:38:02.870 INFO:teuthology.orchestra.run.smithi062.stdout:(5/8): python3-scipy-1.0.0-21.module_el8+762+77 35 MB/s | 14 MB 00:00 2024-02-22T15:38:02.873 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 6/6 2024-02-22T15:38:03.175 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 6/6 2024-02-22T15:38:03.175 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-527.g6ea2a55e.el8.n 1/6 2024-02-22T15:38:03.176 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 2/6 2024-02-22T15:38:03.176 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-527.g6ea2a55e.el8.no 3/6 2024-02-22T15:38:03.176 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-22T15:38:03.176 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-22T15:38:03.329 INFO:teuthology.orchestra.run.smithi062.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 376 kB/s | 172 kB 00:00 2024-02-22T15:38:03.555 INFO:teuthology.orchestra.run.smithi062.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 645 kB/s | 645 kB 00:01 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-22T15:38:03.602 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:03.603 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:03.646 INFO:teuthology.orchestra.run.smithi062.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.7-527 4.3 MB/s | 7.4 MB 00:01 2024-02-22T15:38:03.646 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:03.647 INFO:teuthology.orchestra.run.smithi062.stdout:Total 18 MB/s | 36 MB 00:02 2024-02-22T15:38:03.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-22T15:38:03.891 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:38:03.905 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:38:03.905 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:38:04.301 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:38:04.305 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:38:04.535 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:16 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:04.888 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:38:05.039 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T15:38:05.174 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T15:38:05.331 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T15:38:05.948 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T15:38:06.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T15:38:06.261 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:06.262 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-02-22T15:38:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:38:06.262 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-02-22T15:38:06.262 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 7.4 M 2024-02-22T15:38:06.262 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout:Install 8 Packages 2024-02-22T15:38:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 36 M 2024-02-22T15:38:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 210 M 2024-02-22T15:38:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:06.751 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T15:38:06.889 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T15:38:07.626 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T15:38:07.948 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T15:38:08.189 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T15:38:08.288 INFO:teuthology.orchestra.run.smithi019.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2024-02-22T15:38:08.378 INFO:teuthology.orchestra.run.smithi019.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 9.7 MB/s | 4.9 MB 00:00 2024-02-22T15:38:08.447 INFO:teuthology.orchestra.run.smithi019.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.2 MB/s | 225 kB 00:00 2024-02-22T15:38:08.505 INFO:teuthology.orchestra.run.smithi019.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 17 MB/s | 3.7 MB 00:00 2024-02-22T15:38:08.848 INFO:teuthology.orchestra.run.smithi019.stdout:(5/8): python3-scipy-1.0.0-21.module_el8+762+77 35 MB/s | 14 MB 00:00 2024-02-22T15:38:09.089 INFO:teuthology.orchestra.run.smithi019.stdout:(6/8): ceph-mgr-diskprediction-local-17.2.7-527 6.1 MB/s | 7.4 MB 00:01 2024-02-22T15:38:09.148 INFO:teuthology.orchestra.run.smithi019.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.0 MB/s | 645 kB 00:00 2024-02-22T15:38:09.232 INFO:teuthology.orchestra.run.smithi019.stdout:(8/8): libquadmath-8.5.0-21.el8.x86_64.rpm 450 kB/s | 172 kB 00:00 2024-02-22T15:38:09.232 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:09.232 INFO:teuthology.orchestra.run.smithi019.stdout:Total 12 MB/s | 36 MB 00:02 2024-02-22T15:38:09.468 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:09.487 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:09.487 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:09.894 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:09.898 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:10.097 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T15:38:10.501 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:10.632 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T15:38:10.756 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-22T15:38:10.957 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T15:38:11.368 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-22T15:38:11.574 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-22T15:38:11.589 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e 8/8 2024-02-22T15:38:11.757 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T15:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e 8/8 2024-02-22T15:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e 1/8 2024-02-22T15:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-22T15:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-22T15:38:11.896 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-22T15:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T15:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-22T15:38:11.897 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-22T15:38:12.324 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-22T15:38:12.324 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:12.324 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:38:12.324 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:12.324 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-22T15:38:12.324 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:12.325 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:12.395 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-22T15:38:12.463 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-rook 2024-02-22T15:38:13.182 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T15:38:13.250 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:20 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:13.933 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-22T15:38:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T15:38:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T15:38:14.932 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:14.934 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:14.934 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:38:14.934 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:14.934 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:38:14.934 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 47 k 2024-02-22T15:38:14.934 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:14.935 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:38:14.936 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:14.936 INFO:teuthology.orchestra.run.smithi062.stdout:Install 13 Packages 2024-02-22T15:38:14.936 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:14.936 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 1.9 M 2024-02-22T15:38:14.936 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 25 M 2024-02-22T15:38:14.936 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:38:15.313 INFO:teuthology.orchestra.run.smithi062.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 164 kB/s | 20 kB 00:00 2024-02-22T15:38:15.338 INFO:teuthology.orchestra.run.smithi062.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 181 kB/s | 27 kB 00:00 2024-02-22T15:38:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:(3/13): ceph-mgr-rook-17.2.7-527.g6ea2a55e.el8. 262 kB/s | 47 kB 00:00 2024-02-22T15:38:15.439 INFO:teuthology.orchestra.run.smithi062.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2024-02-22T15:38:15.464 INFO:teuthology.orchestra.run.smithi062.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 875 kB/s | 110 kB 00:00 2024-02-22T15:38:15.656 INFO:teuthology.orchestra.run.smithi062.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 196 kB/s | 43 kB 00:00 2024-02-22T15:38:15.706 INFO:teuthology.orchestra.run.smithi062.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 464 kB/s | 155 kB 00:00 2024-02-22T15:38:15.740 INFO:teuthology.orchestra.run.smithi062.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 120 kB/s | 33 kB 00:00 2024-02-22T15:38:15.882 INFO:teuthology.orchestra.run.smithi062.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 72 kB/s | 16 kB 00:00 2024-02-22T15:38:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 283 kB/s | 81 kB 00:00 2024-02-22T15:38:16.041 INFO:teuthology.orchestra.run.smithi062.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 405 kB/s | 64 kB 00:00 2024-02-22T15:38:16.082 INFO:teuthology.orchestra.run.smithi062.stdout:(12/13): python3-kubernetes-11.0.0-6.el8.noarch 3.4 MB/s | 1.2 MB 00:00 2024-02-22T15:38:16.108 INFO:teuthology.orchestra.run.smithi062.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 564 kB/s | 61 kB 00:00 2024-02-22T15:38:16.111 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:16.111 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.6 MB/s | 1.9 MB 00:01 2024-02-22T15:38:16.170 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:38:16.187 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:38:16.187 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:38:16.282 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:38:16.285 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:38:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-22T15:38:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:38:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-22T15:38:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-22T15:38:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-22T15:38:17.685 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-22T15:38:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-22T15:38:17.897 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e 8/8 2024-02-22T15:38:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-22T15:38:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-22T15:38:18.224 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e 8/8 2024-02-22T15:38:18.224 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e 1/8 2024-02-22T15:38:18.224 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-22T15:38:18.224 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-22T15:38:18.224 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-22T15:38:18.225 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-22T15:38:18.225 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-22T15:38:18.225 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-22T15:38:18.502 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-22T15:38:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-22T15:38:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:18.679 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:18.737 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-22T15:38:18.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-rook 2024-02-22T15:38:19.602 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:31 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:19.837 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-22T15:38:20.086 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-22T15:38:20.702 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-22T15:38:20.924 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-22T15:38:21.122 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 13/13 2024-02-22T15:38:21.281 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:21.283 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:21.283 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:38:21.283 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 47 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-22T15:38:21.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout:Install 13 Packages 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:21.285 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 1.9 M 2024-02-22T15:38:21.286 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 25 M 2024-02-22T15:38:21.286 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:21.440 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 13/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 1/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-22T15:38:21.441 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-22T15:38:21.680 INFO:teuthology.orchestra.run.smithi019.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 149 kB/s | 20 kB 00:00 2024-02-22T15:38:21.705 INFO:teuthology.orchestra.run.smithi019.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 167 kB/s | 27 kB 00:00 2024-02-22T15:38:21.731 INFO:teuthology.orchestra.run.smithi019.stdout:(3/13): ceph-mgr-rook-17.2.7-527.g6ea2a55e.el8. 257 kB/s | 47 kB 00:00 2024-02-22T15:38:21.806 INFO:teuthology.orchestra.run.smithi019.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2024-02-22T15:38:21.831 INFO:teuthology.orchestra.run.smithi019.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 875 kB/s | 110 kB 00:00 2024-02-22T15:38:21.914 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-22T15:38:21.914 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:21.914 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:38:21.914 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:21.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:21.915 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:22.032 INFO:teuthology.orchestra.run.smithi019.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 189 kB/s | 43 kB 00:00 2024-02-22T15:38:22.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-cephadm 2024-02-22T15:38:22.065 INFO:teuthology.orchestra.run.smithi019.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 463 kB/s | 155 kB 00:00 2024-02-22T15:38:22.232 INFO:teuthology.orchestra.run.smithi019.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 82 kB/s | 33 kB 00:00 2024-02-22T15:38:22.362 INFO:teuthology.orchestra.run.smithi019.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 50 kB/s | 16 kB 00:00 2024-02-22T15:38:22.525 INFO:teuthology.orchestra.run.smithi019.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 177 kB/s | 81 kB 00:00 2024-02-22T15:38:22.566 INFO:teuthology.orchestra.run.smithi019.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 308 kB/s | 64 kB 00:00 2024-02-22T15:38:22.658 INFO:teuthology.orchestra.run.smithi019.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 459 kB/s | 61 kB 00:00 2024-02-22T15:38:22.742 INFO:teuthology.orchestra.run.smithi019.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.3 MB/s | 1.2 MB 00:00 2024-02-22T15:38:22.745 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:22.745 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.3 MB/s | 1.9 MB 00:01 2024-02-22T15:38:22.804 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:22.821 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:22.822 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:22.829 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:29 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:22.917 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:22.920 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:23.502 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:23.987 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-22T15:38:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-22T15:38:24.428 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-22T15:38:24.523 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 121 k 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-22T15:38:24.524 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout:Install 5 Packages 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:24.525 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 6.0 M 2024-02-22T15:38:24.526 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 26 M 2024-02-22T15:38:24.526 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:38:24.655 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-22T15:38:24.847 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-22T15:38:24.907 INFO:teuthology.orchestra.run.smithi062.stdout:(1/5): ceph-mgr-cephadm-17.2.7-527.g6ea2a55e.el 572 kB/s | 121 kB 00:00 2024-02-22T15:38:25.024 INFO:teuthology.orchestra.run.smithi062.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.7 MB/s | 578 kB 00:00 2024-02-22T15:38:25.074 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-22T15:38:25.133 INFO:teuthology.orchestra.run.smithi062.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-02-22T15:38:25.402 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-22T15:38:25.491 INFO:teuthology.orchestra.run.smithi062.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 129 kB/s | 60 kB 00:00 2024-02-22T15:38:25.612 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-22T15:38:25.641 INFO:teuthology.orchestra.run.smithi062.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 648 kB/s | 476 kB 00:00 2024-02-22T15:38:25.643 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:25.644 INFO:teuthology.orchestra.run.smithi062.stdout:Total 5.4 MB/s | 6.0 MB 00:01 2024-02-22T15:38:25.699 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:38:25.711 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:38:25.711 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:38:25.820 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:38:25.823 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:38:26.089 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:38:26.427 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-02-22T15:38:26.676 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-22T15:38:26.962 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-22T15:38:27.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-22T15:38:27.357 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-22T15:38:27.467 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-02-22T15:38:27.539 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 13/13 2024-02-22T15:38:27.832 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-02-22T15:38:27.835 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 13/13 2024-02-22T15:38:27.835 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 1/13 2024-02-22T15:38:27.835 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-22T15:38:27.836 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-22T15:38:28.098 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-02-22T15:38:28.269 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 5/5 2024-02-22T15:38:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-22T15:38:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:28.311 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-22T15:38:28.311 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:28.312 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:28.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-cephadm 2024-02-22T15:38:28.552 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 5/5 2024-02-22T15:38:28.552 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/5 2024-02-22T15:38:28.552 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-02-22T15:38:28.552 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-02-22T15:38:28.552 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-02-22T15:38:28.954 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:28.955 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:29.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-fuse 2024-02-22T15:38:29.213 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:41 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:29.877 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:36 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:30.900 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:30.901 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:30.901 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:38:30.901 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:30.901 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:17.2.7-527.g6ea2a55e.el8 ceph-noarch 121 k 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout:Install 5 Packages 2024-02-22T15:38:30.902 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:30.903 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 6.0 M 2024-02-22T15:38:30.903 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 26 M 2024-02-22T15:38:30.903 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:31.567 INFO:teuthology.orchestra.run.smithi019.stdout:(1/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.9 MB/s | 578 kB 00:00 2024-02-22T15:38:31.585 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 877 k 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-02-22T15:38:31.586 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-22T15:38:31.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:38:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:Install 2 Packages 2024-02-22T15:38:31.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:31.587 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 960 k 2024-02-22T15:38:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 3.0 M 2024-02-22T15:38:31.588 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:38:31.592 INFO:teuthology.orchestra.run.smithi019.stdout:(2/5): ceph-mgr-cephadm-17.2.7-527.g6ea2a55e.el 369 kB/s | 121 kB 00:00 2024-02-22T15:38:31.873 INFO:teuthology.orchestra.run.smithi019.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 7.9 MB/s | 4.8 MB 00:00 2024-02-22T15:38:32.151 INFO:teuthology.orchestra.run.smithi062.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 233 kB/s | 83 kB 00:00 2024-02-22T15:38:32.359 INFO:teuthology.orchestra.run.smithi019.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 79 kB/s | 60 kB 00:00 2024-02-22T15:38:32.384 INFO:teuthology.orchestra.run.smithi062.stdout:(2/2): ceph-fuse-17.2.7-527.g6ea2a55e.el8.x86_6 1.5 MB/s | 877 kB 00:00 2024-02-22T15:38:32.384 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:32.385 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.2 MB/s | 960 kB 00:00 2024-02-22T15:38:32.397 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:38:32.421 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:38:32.421 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:38:32.522 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:38:32.525 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:38:32.687 INFO:teuthology.orchestra.run.smithi019.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 425 kB/s | 476 kB 00:01 2024-02-22T15:38:32.688 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:32.689 INFO:teuthology.orchestra.run.smithi019.stdout:Total 3.4 MB/s | 6.0 MB 00:01 2024-02-22T15:38:32.704 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:38:32.744 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:32.756 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:32.757 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:32.866 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:32.869 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:33.008 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-22T15:38:33.130 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:33.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/2 2024-02-22T15:38:33.469 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-02-22T15:38:33.704 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/2 2024-02-22T15:38:33.704 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/2 2024-02-22T15:38:34.110 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-22T15:38:34.110 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:34.110 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:38:34.110 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-22T15:38:34.110 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:34.111 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:34.317 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librados-devel 2024-02-22T15:38:34.663 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-02-22T15:38:34.999 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-02-22T15:38:35.104 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:42 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:35.274 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-02-22T15:38:35.453 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 5/5 2024-02-22T15:38:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 5/5 2024-02-22T15:38:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 1/5 2024-02-22T15:38:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-02-22T15:38:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-02-22T15:38:35.753 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-02-22T15:38:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-02-22T15:38:36.164 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:17.2.7-527.g6ea2a55e.el8.noarch 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:36.165 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:36.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse 2024-02-22T15:38:36.797 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:36.797 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:36.797 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2024-02-22T15:38:36.797 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:36.797 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 130 k 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:36.798 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 130 k 2024-02-22T15:38:36.799 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 464 k 2024-02-22T15:38:36.799 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:38:37.091 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:49 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:37.098 INFO:teuthology.orchestra.run.smithi062.stdout:librados-devel-17.2.7-527.g6ea2a55e.el8.x86_64. 434 kB/s | 130 kB 00:00 2024-02-22T15:38:37.099 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:37.099 INFO:teuthology.orchestra.run.smithi062.stdout:Total 432 kB/s | 130 kB 00:00 2024-02-22T15:38:37.099 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:38:37.105 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:38:37.105 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:38:37.159 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:38:37.162 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:38:37.271 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:38:37.519 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:37.811 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:38.236 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:38.236 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:38.236 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:38:38.236 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:38:38.237 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:38.237 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:38.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install libcephfs2 2024-02-22T15:38:38.763 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:38.763 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 877 k 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout:Install 2 Packages 2024-02-22T15:38:38.764 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:38.765 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 960 k 2024-02-22T15:38:38.765 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 3.0 M 2024-02-22T15:38:38.765 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:39.237 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:46 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:39.377 INFO:teuthology.orchestra.run.smithi019.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 146 kB/s | 83 kB 00:00 2024-02-22T15:38:39.402 INFO:teuthology.orchestra.run.smithi019.stdout:(2/2): ceph-fuse-17.2.7-527.g6ea2a55e.el8.x86_6 1.4 MB/s | 877 kB 00:00 2024-02-22T15:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:39.403 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.5 MB/s | 960 kB 00:00 2024-02-22T15:38:39.414 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:39.437 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:39.438 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:39.536 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:39.540 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:39.728 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:40.026 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-22T15:38:40.254 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/2 2024-02-22T15:38:40.643 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2/2 2024-02-22T15:38:40.644 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/2 2024-02-22T15:38:40.880 INFO:teuthology.orchestra.run.smithi062.stdout:Package libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:40.929 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:40.931 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:38:40.931 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:41.032 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install libcephfs-devel 2024-02-22T15:38:41.070 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-22T15:38:41.070 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:41.070 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:41.071 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-22T15:38:41.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:41.071 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:41.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados-devel 2024-02-22T15:38:41.824 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:48 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:42.092 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:54 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:43.502 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 25 k 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:38:43.503 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:38:43.504 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:38:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:43.504 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 25 k 2024-02-22T15:38:43.505 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 109 k 2024-02-22T15:38:43.505 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:38:43.664 INFO:teuthology.orchestra.run.smithi062.stdout:libcephfs-devel-17.2.7-527.g6ea2a55e.el8.x86_64 156 kB/s | 25 kB 00:00 2024-02-22T15:38:43.664 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:43.664 INFO:teuthology.orchestra.run.smithi062.stdout:Total 155 kB/s | 25 kB 00:00 2024-02-22T15:38:43.665 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:38:43.668 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:38:43.668 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:38:43.670 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:38:43.676 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:38:43.726 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:38:43.771 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 130 k 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:38:43.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:43.773 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 130 k 2024-02-22T15:38:43.773 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 464 k 2024-02-22T15:38:43.773 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:43.900 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:44.180 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:44.216 INFO:teuthology.orchestra.run.smithi019.stdout:librados-devel-17.2.7-527.g6ea2a55e.el8.x86_64. 293 kB/s | 130 kB 00:00 2024-02-22T15:38:44.217 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:44.217 INFO:teuthology.orchestra.run.smithi019.stdout:Total 292 kB/s | 130 kB 00:00 2024-02-22T15:38:44.217 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:44.224 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:44.224 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:44.280 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:44.283 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:44.565 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:44.565 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:44.565 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:38:44.565 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:38:44.565 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:38:44.565 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:44.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librados2 2024-02-22T15:38:44.873 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:45.260 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:45.484 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:52 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:45.584 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:46.012 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:38:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:46.012 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:46.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2 2024-02-22T15:38:47.012 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:10:59 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:47.131 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:47.182 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:47.183 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:38:47.183 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:47.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librbd1 2024-02-22T15:38:48.089 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:55 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:48.659 INFO:teuthology.orchestra.run.smithi019.stdout:Package libcephfs2-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:48.708 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:38:48.710 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:48.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs-devel 2024-02-22T15:38:49.601 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:01 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:49.728 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:49.778 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:49.779 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:38:49.779 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:49.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rados 2024-02-22T15:38:50.679 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:10:57 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:51.282 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 25 k 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:51.283 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 25 k 2024-02-22T15:38:51.284 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 109 k 2024-02-22T15:38:51.284 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:38:51.430 INFO:teuthology.orchestra.run.smithi019.stdout:libcephfs-devel-17.2.7-527.g6ea2a55e.el8.x86_64 169 kB/s | 25 kB 00:00 2024-02-22T15:38:51.431 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:38:51.431 INFO:teuthology.orchestra.run.smithi019.stdout:Total 168 kB/s | 25 kB 00:00 2024-02-22T15:38:51.431 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:38:51.435 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:38:51.435 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:38:51.437 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:38:51.440 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:38:51.493 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:38:51.692 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:51.974 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:52.306 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:52.333 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:38:52.333 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:52.333 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:38:52.333 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:38:52.333 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:38:52.334 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:52.356 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:52.357 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:38:52.357 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:52.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2 2024-02-22T15:38:52.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rgw 2024-02-22T15:38:53.249 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:05 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:53.258 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:11:00 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:54.870 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:54.885 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:54.920 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:54.921 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:38:54.922 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:54.935 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:54.936 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:38:54.937 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:55.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1 2024-02-22T15:38:55.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-cephfs 2024-02-22T15:38:55.817 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:07 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:55.832 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:11:02 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:38:57.457 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:57.483 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:38:57.506 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:38:57.508 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:38:57.508 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:38:57.533 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:38:57.534 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:38:57.535 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:38:57.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rados 2024-02-22T15:38:57.641 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rbd 2024-02-22T15:38:58.400 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:10 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:38:58.443 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:11:05 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:39:00.042 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rados-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:39:00.084 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:39:00.092 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:00.093 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:39:00.094 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:00.134 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:39:00.135 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-02-22T15:39:00.136 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:39:00.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rgw 2024-02-22T15:39:00.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-fuse 2024-02-22T15:39:00.986 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:12 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:39:01.035 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:11:08 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:39:02.610 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rgw-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:39:02.660 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:02.661 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:39:02.662 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:02.724 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 90 k 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:39:02.725 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:02.726 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:39:02.726 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:02.726 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 90 k 2024-02-22T15:39:02.726 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 264 k 2024-02-22T15:39:02.726 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:39:02.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-cephfs 2024-02-22T15:39:03.036 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-fuse-17.2.7-527.g6ea2a55e.el8.x86_64.rpm 291 kB/s | 90 kB 00:00 2024-02-22T15:39:03.037 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:03.037 INFO:teuthology.orchestra.run.smithi062.stdout:Total 290 kB/s | 90 kB 00:00 2024-02-22T15:39:03.037 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:39:03.048 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:39:03.048 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:39:03.101 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:39:03.104 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:39:03.320 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:39:03.530 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:03.573 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:15 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:39:03.823 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:04.248 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:04.248 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:39:04.248 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:39:04.248 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:04.248 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:39:04.564 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-mirror 2024-02-22T15:39:05.223 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-cephfs-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:39:05.271 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:05.272 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:39:05.272 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:05.355 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:11:12 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:39:05.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rbd 2024-02-22T15:39:06.188 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:18 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:39:07.028 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.5 M 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:07.029 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:39:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:39:07.030 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 3.5 M 2024-02-22T15:39:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 16 M 2024-02-22T15:39:07.030 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:39:07.813 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 is already installed. 2024-02-22T15:39:07.863 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:07.864 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-02-22T15:39:07.865 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:07.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse 2024-02-22T15:39:08.754 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:20 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:39:09.056 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-mirror-17.2.7-527.g6ea2a55e.el8.x86_64.rpm 1.7 MB/s | 3.5 MB 00:02 2024-02-22T15:39:09.057 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:09.057 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.7 MB/s | 3.5 MB 00:02 2024-02-22T15:39:09.057 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:39:09.072 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:39:09.072 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:39:09.142 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:39:09.145 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:39:09.619 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:39:09.789 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:10.222 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:10.430 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:10.431 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:10.431 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-02-22T15:39:10.431 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:10.431 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:39:10.431 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 90 k 2024-02-22T15:39:10.431 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 90 k 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 264 k 2024-02-22T15:39:10.432 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:39:10.604 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:10.604 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:10.604 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:39:10.604 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:39:10.604 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:10.605 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:39:10.728 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-fuse-17.2.7-527.g6ea2a55e.el8.x86_64.rpm 304 kB/s | 90 kB 00:00 2024-02-22T15:39:10.729 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:10.729 INFO:teuthology.orchestra.run.smithi019.stdout:Total 303 kB/s | 90 kB 00:00 2024-02-22T15:39:10.729 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:39:10.740 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:39:10.740 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:39:10.794 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:39:10.798 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:39:10.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-nbd 2024-02-22T15:39:10.910 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:39:11.148 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:11.451 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:11.618 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:11:18 ago on Thu 22 Feb 2024 03:27:53 PM UTC. 2024-02-22T15:39:11.814 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:11.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:11.814 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:39:11.814 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:39:11.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:11.815 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:12.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-mirror 2024-02-22T15:39:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:24 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:39:13.309 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-02-22T15:39:13.309 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:13.309 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-02-22T15:39:13.309 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:13.309 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 174 k 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 174 k 2024-02-22T15:39:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 556 k 2024-02-22T15:39:13.311 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-02-22T15:39:13.977 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-nbd-17.2.7-527.g6ea2a55e.el8.x86_64.rpm 262 kB/s | 174 kB 00:00 2024-02-22T15:39:13.977 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:13.978 INFO:teuthology.orchestra.run.smithi062.stdout:Total 261 kB/s | 174 kB 00:00 2024-02-22T15:39:13.978 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-02-22T15:39:13.990 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-02-22T15:39:13.990 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-02-22T15:39:14.046 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-02-22T15:39:14.050 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-02-22T15:39:14.162 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-02-22T15:39:14.354 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:14.492 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:14.492 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 3.5 M 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:39:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:14.494 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 3.5 M 2024-02-22T15:39:14.494 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 16 M 2024-02-22T15:39:14.494 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:39:14.664 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:15.054 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:15.054 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:15.054 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-02-22T15:39:15.055 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:39:15.055 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:39:15.055 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-02-22T15:39:15.266 DEBUG:teuthology.parallel:result is None 2024-02-22T15:39:16.075 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-mirror-17.2.7-527.g6ea2a55e.el8.x86_64.rpm 2.2 MB/s | 3.5 MB 00:01 2024-02-22T15:39:16.076 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:16.076 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.2 MB/s | 3.5 MB 00:01 2024-02-22T15:39:16.076 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:39:16.089 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:39:16.089 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:39:16.159 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:39:16.163 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:39:16.626 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:39:16.840 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:17.278 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:17.712 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:17.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:17.712 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:39:17.712 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:39:17.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:17.712 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:17.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-nbd 2024-02-22T15:39:18.712 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:11:30 ago on Thu 22 Feb 2024 03:27:48 PM UTC. 2024-02-22T15:39:20.395 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:17.2.7-527.g6ea2a55e.el8 ceph 174 k 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-02-22T15:39:20.396 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-02-22T15:39:20.397 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:20.397 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 174 k 2024-02-22T15:39:20.397 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 556 k 2024-02-22T15:39:20.397 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-02-22T15:39:20.910 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-nbd-17.2.7-527.g6ea2a55e.el8.x86_64.rpm 340 kB/s | 174 kB 00:00 2024-02-22T15:39:20.910 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:20.911 INFO:teuthology.orchestra.run.smithi019.stdout:Total 339 kB/s | 174 kB 00:00 2024-02-22T15:39:20.911 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-02-22T15:39:20.923 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-02-22T15:39:20.924 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-02-22T15:39:20.979 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-02-22T15:39:20.982 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-02-22T15:39:21.105 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-02-22T15:39:21.388 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:21.678 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:22.088 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 1/1 2024-02-22T15:39:22.088 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:22.088 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-02-22T15:39:22.088 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:17.2.7-527.g6ea2a55e.el8.x86_64 2024-02-22T15:39:22.088 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:22.088 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-02-22T15:39:22.295 DEBUG:teuthology.parallel:result is None 2024-02-22T15:39:22.296 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T15:39:22.296 INFO:teuthology.packaging:ref: None 2024-02-22T15:39:22.296 INFO:teuthology.packaging:tag: None 2024-02-22T15:39:22.296 INFO:teuthology.packaging:branch: quincy 2024-02-22T15:39:22.296 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:39:22.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-02-22T15:39:22.549 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-22T15:39:22.572 INFO:teuthology.orchestra.run.smithi019.stdout:17.2.7-527.g6ea2a55e.el8 2024-02-22T15:39:22.573 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-527.g6ea2a55e.el8 2024-02-22T15:39:22.573 INFO:teuthology.task.install:The correct ceph version 17.2.7-527.g6ea2a55e is installed. 2024-02-22T15:39:22.575 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-22T15:39:22.575 INFO:teuthology.packaging:ref: None 2024-02-22T15:39:22.575 INFO:teuthology.packaging:tag: None 2024-02-22T15:39:22.575 INFO:teuthology.packaging:branch: quincy 2024-02-22T15:39:22.575 INFO:teuthology.packaging:sha1: c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:39:22.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-02-22T15:39:22.975 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-22T15:39:23.004 INFO:teuthology.orchestra.run.smithi062.stdout:17.2.7-527.g6ea2a55e.el8 2024-02-22T15:39:23.005 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-527.g6ea2a55e.el8 2024-02-22T15:39:23.005 INFO:teuthology.task.install:The correct ceph version 17.2.7-527.g6ea2a55e is installed. 2024-02-22T15:39:23.007 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-22T15:39:23.007 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:39:23.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-22T15:39:23.038 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:39:23.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-22T15:39:23.078 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-22T15:39:23.079 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:39:23.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-02-22T15:39:23.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-22T15:39:23.175 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:39:23.176 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-02-22T15:39:23.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-22T15:39:23.283 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-22T15:39:23.283 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:39:23.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-22T15:39:23.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-22T15:39:23.379 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:39:23.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-22T15:39:23.412 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-22T15:39:23.485 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-22T15:39:23.485 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:39:23.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-02-22T15:39:23.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-22T15:39:23.581 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:39:23.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-02-22T15:39:23.612 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-22T15:39:23.683 INFO:teuthology.run_tasks:Running task print... 2024-02-22T15:39:23.693 INFO:teuthology.task.print:**** done install task... 2024-02-22T15:39:23.693 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-22T15:39:23.799 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'c1dc18f444603397768fcb7c34a8488b10284df6'} 2024-02-22T15:39:23.799 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-02-22T15:39:23.800 INFO:tasks.cephadm:Cluster fsid is 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:39:23.800 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-22T15:39:23.800 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-22T15:39:23.800 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi019': '172.21.15.19', 'mon.smithi062': '172.21.15.62'} 2024-02-22T15:39:23.800 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-22T15:39:23.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-02-22T15:39:23.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-02-22T15:39:23.862 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-02-22T15:39:23.862 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-22T15:39:24.133 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-rw-r--. 1 ubuntu ubuntu 367151 Feb 22 15:39 /home/ubuntu/cephtest/cephadm 2024-02-22T15:39:24.133 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-22T15:39:24.281 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-rw-r--. 1 ubuntu ubuntu 367151 Feb 22 15:39 /home/ubuntu/cephtest/cephadm 2024-02-22T15:39:24.282 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-22T15:39:24.303 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-22T15:39:24.361 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-02-22T15:39:24.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-02-22T15:39:24.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-02-22T15:39:25.595 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-22T15:39:25.630 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-22T15:39:51.459 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:39:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)", 2024-02-22T15:39:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "e2e4ad29904cfe88844285791a8c0a2d62257dded9a73cc4981ab809219e6cc7", 2024-02-22T15:39:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-02-22T15:39:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:56ee8a1814e028cce0726268ac2a95b30c041d6b728db5bb791df64e6d93f223" 2024-02-22T15:39:51.460 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-02-22T15:39:51.460 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:39:51.464 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-02-22T15:39:51.464 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)", 2024-02-22T15:39:51.464 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "e2e4ad29904cfe88844285791a8c0a2d62257dded9a73cc4981ab809219e6cc7", 2024-02-22T15:39:51.464 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-02-22T15:39:51.465 INFO:teuthology.orchestra.run.smithi062.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:56ee8a1814e028cce0726268ac2a95b30c041d6b728db5bb791df64e6d93f223" 2024-02-22T15:39:51.465 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-02-22T15:39:51.465 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-02-22T15:39:51.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-02-22T15:39:51.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-02-22T15:39:51.551 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-02-22T15:39:51.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-02-22T15:39:51.619 INFO:tasks.cephadm:Writing seed config... 2024-02-22T15:39:51.620 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-22T15:39:51.620 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-22T15:39:51.620 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-22T15:39:51.620 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-22T15:39:51.620 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-22T15:39:51.620 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-22T15:39:51.621 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-22T15:39:51.622 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-22T15:39:51.623 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-22T15:39:51.624 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:39:51.624 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-22T15:39:51.649 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8db8e6fa-d198-11ee-95bf-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-22T15:39:51.650 DEBUG:teuthology.orchestra.run.smithi019:mon.smithi019> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service 2024-02-22T15:39:51.691 INFO:tasks.cephadm:Bootstrapping... 2024-02-22T15:39:51.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-22T15:39:51.719 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:-- Logs begin at Thu 2024-02-22 14:59:28 UTC. -- 2024-02-22T15:39:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-02-22T15:39:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '8db8e6fa-d198-11ee-95bf-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-02-22T15:39:51.938 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-22T15:39:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-22T15:39:51.938 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-02-22T15:39:51.970 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-22T15:39:51.970 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-02-22T15:39:51.970 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-02-22T15:39:51.977 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-22T15:39:51.977 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-22T15:39:51.983 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-22T15:39:51.983 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-02-22T15:39:51.989 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-02-22T15:39:51.995 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-02-22T15:39:51.995 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-02-22T15:39:51.995 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-02-22T15:39:52.025 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-22T15:39:52.025 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-22T15:39:52.025 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-02-22T15:39:52.025 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-02-22T15:39:52.031 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-22T15:39:52.031 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-22T15:39:52.037 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-22T15:39:52.037 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-02-22T15:39:52.043 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-02-22T15:39:52.049 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-02-22T15:39:52.049 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-02-22T15:39:52.049 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-02-22T15:39:52.049 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:39:52.049 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140244335401448 on /run/cephadm/8db8e6fa-d198-11ee-95bf-87774f69a715.lock 2024-02-22T15:39:52.050 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140244335401448 acquired on /run/cephadm/8db8e6fa-d198-11ee-95bf-87774f69a715.lock 2024-02-22T15:39:52.050 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-02-22T15:39:52.050 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-02-22T15:39:52.050 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-02-22T15:39:52.054 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-02-22T15:39:52.054 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-02-22T15:39:52.057 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-22T15:39:52.057 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-22T15:39:52.057 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-02-22T15:39:52.060 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-22T15:39:52.060 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-22T15:39:52.060 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-22T15:39:52.060 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-22T15:39:52.060 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-02-22T15:39:52.060 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-22T15:39:52.061 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-02-22T15:39:52.061 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-02-22T15:39:52.061 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-22T15:39:52.062 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-22T15:39:52.062 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout e2e4ad29904cfe88844285791a8c0a2d62257dded9a73cc4981ab809219e6cc7 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:be411db89c07b20e574dcbe8009f0a08bd1ef5a5157dfa6ed92253c768f2794f 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:e2e4ad29904cfe88844285791a8c0a2d62257dded9a73cc4981ab809219e6cc7 2024-02-22T15:39:52.423 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-22T15:39:53.636 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable) 2024-02-22T15:39:53.637 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable) 2024-02-22T15:39:53.637 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-02-22T15:39:54.698 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-02-22T15:39:54.699 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-02-22T15:39:55.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDLatdlrAUlCxAA4gsRU+Oqp9QWtFVAEbKiEQ== 2024-02-22T15:39:56.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDMatdlYh7NEBAAefTerwU+HVMOVyEKmHcezw== 2024-02-22T15:39:58.098 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDNatdlldDbGBAAVSkCwLT6wq6d1G7iv7Bc5w== 2024-02-22T15:39:58.099 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for smithi019 [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = octopus 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:39:59.124 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-02-22T15:40:00.320 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.smithi019 on 2024-02-22T15:40:00.432 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-22T15:40:00.522 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-22T15:40:00.621 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715.target → /etc/systemd/system/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715.target. 2024-02-22T15:40:00.621 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715.target → /etc/systemd/system/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715.target. 2024-02-22T15:40:00.759 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019 2024-02-22T15:40:00.759 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service: Unit ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service not loaded. 2024-02-22T15:40:00.854 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715.target.wants/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service → /etc/systemd/system/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@.service. 2024-02-22T15:40:01.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:00 smithi019 systemd[1]: Starting Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:40:02.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:02 smithi019 ceph-mon[66938]: mkfs 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:02.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:02 smithi019 ceph-mon[66938]: mon.smithi019 is new leader, mons smithi019 in quorum (ranks 0) 2024-02-22T15:40:02.960 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-22T15:40:02.960 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-02-22T15:40:02.968 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-22T15:40:02.968 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-02-22T15:40:02.968 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-02-22T15:40:02.968 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-02-22T15:40:02.968 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-02-22T15:40:02.968 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-02-22T15:40:03.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:02 smithi019 bash[66900]: 11308c104fd0cd18773962c1647e6be1e8742fd2c7ae3827ab5f432074c992c2 2024-02-22T15:40:03.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:02 smithi019 systemd[1]: Started Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:40:03.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:03 smithi019 ceph-mon[66938]: mon.smithi019 is new leader, mons smithi019 in quorum (ranks 0) 2024-02-22T15:40:03.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:03 smithi019 ceph-mon[66938]: monmap e1: 1 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:40:03.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:03 smithi019 ceph-mon[66938]: fsmap 2024-02-22T15:40:03.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:03 smithi019 ceph-mon[66938]: osdmap e1: 0 total, 0 up, 0 in 2024-02-22T15:40:03.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:03 smithi019 ceph-mon[66938]: mgrmap e1: no daemons active 2024-02-22T15:40:04.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:04 smithi019 ceph-mon[66938]: from='client.? 172.21.15.19:0/3123049203' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-22T15:40:04.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi019 (age 1.94324s) 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-22T15:40:04.981 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-22T15:40:04.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:04.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-02-22T15:40:04.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-22T15:40:04.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.292+0000 7f1ae4597700 1 Processor -- start 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.293+0000 7f1ae4597700 1 -- start start 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.293+0000 7f1ae4597700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.293+0000 7f1ae4597700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc1045f0 con 0x7f1adc103ca0 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.293+0000 7f1ae2333700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.293+0000 7f1ae2333700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40698/0 (socket says 172.21.15.19:40698) 2024-02-22T15:40:04.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.293+0000 7f1ae2333700 1 -- 172.21.15.19:0/1236218527 learned_addr learned my addr 172.21.15.19:0/1236218527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.294+0000 7f1ae2333700 1 -- 172.21.15.19:0/1236218527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc104730 con 0x7f1adc103ca0 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.294+0000 7f1ae2333700 1 --2- 172.21.15.19:0/1236218527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1040b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80098e0 tx=0x7f1ad8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31deecdd7ddda97 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.294+0000 7f1ae1331700 1 -- 172.21.15.19:0/1236218527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad800b860 con 0x7f1adc103ca0 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.294+0000 7f1ae1331700 1 -- 172.21.15.19:0/1236218527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad800b9c0 con 0x7f1adc103ca0 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.294+0000 7f1ae1331700 1 -- 172.21.15.19:0/1236218527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad800bc90 con 0x7f1adc103ca0 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 -- 172.21.15.19:0/1236218527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 msgr2=0x7f1adc1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 --2- 172.21.15.19:0/1236218527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1040b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80098e0 tx=0x7f1ad8009250 comp rx=0 tx=0).stop 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 -- 172.21.15.19:0/1236218527 shutdown_connections 2024-02-22T15:40:04.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 --2- 172.21.15.19:0/1236218527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1040b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:04.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 -- 172.21.15.19:0/1236218527 >> 172.21.15.19:0/1236218527 conn(0x7f1adc0fc460 msgr2=0x7f1adc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:04.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 -- 172.21.15.19:0/1236218527 shutdown_connections 2024-02-22T15:40:04.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 -- 172.21.15.19:0/1236218527 wait complete. 2024-02-22T15:40:04.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 Processor -- start 2024-02-22T15:40:04.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.295+0000 7f1ae4597700 1 -- start start 2024-02-22T15:40:04.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae4597700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae4597700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc101960 con 0x7f1adc103ca0 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae2333700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae2333700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40700/0 (socket says 172.21.15.19:40700) 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae2333700 1 -- 172.21.15.19:0/3123049203 learned_addr learned my addr 172.21.15.19:0/3123049203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae2333700 1 -- 172.21.15.19:0/3123049203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80094f0 con 0x7f1adc103ca0 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae2333700 1 --2- 172.21.15.19:0/3123049203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1034e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8003080 tx=0x7f1ad8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad801c5c0 con 0x7f1adc103ca0 2024-02-22T15:40:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad801c720 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc101b60 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ad801ca30 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.296+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc101f70 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.297+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1ad801a450 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.297+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1ad802e790 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.297+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc04bed0 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.300+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1ad801a890 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.372+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1adc10a100 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.372+0000 7f1aceffd700 1 -- 172.21.15.19:0/3123049203 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+324 (secure 0 0 0) 0x7f1ad801aa30 con 0x7f1adc103ca0 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 msgr2=0x7f1adc1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 --2- 172.21.15.19:0/3123049203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1034e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8003080 tx=0x7f1ad8016000 comp rx=0 tx=0).stop 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 shutdown_connections 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 --2- 172.21.15.19:0/3123049203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1adc103ca0 0x7f1adc1034e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 >> 172.21.15.19:0/3123049203 conn(0x7f1adc0fc460 msgr2=0x7f1adc105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:04.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 shutdown_connections 2024-02-22T15:40:04.988 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:04.373+0000 7f1ae4597700 1 -- 172.21.15.19:0/3123049203 wait complete. 2024-02-22T15:40:04.988 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-02-22T15:40:04.990 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-02-22T15:40:05.849 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:05 smithi019 ceph-mon[66938]: from='client.? 172.21.15.19:0/4126210331' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-22T15:40:05.850 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:05 smithi019 ceph-mon[66938]: from='client.? 172.21.15.19:0/4126210331' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-22T15:40:06.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:06.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-02-22T15:40:06.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:06.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-02-22T15:40:06.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.500+0000 7fd4b146e700 1 Processor -- start 2024-02-22T15:40:06.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.500+0000 7fd4b146e700 1 -- start start 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.500+0000 7fd4b146e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac100e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4b146e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1013d0 con 0x7fd4ac100a80 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac100e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40706/0 (socket says 172.21.15.19:40706) 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4aaffd700 1 -- 172.21.15.19:0/1518308254 learned_addr learned my addr 172.21.15.19:0/1518308254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4aaffd700 1 -- 172.21.15.19:0/1518308254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac101510 con 0x7fd4ac100a80 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4aaffd700 1 --2- 172.21.15.19:0/1518308254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac100e90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0098e0 tx=0x7fd49c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d059968911f104ce server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4a9ffb700 1 -- 172.21.15.19:0/1518308254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd49c00b860 con 0x7fd4ac100a80 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4a9ffb700 1 -- 172.21.15.19:0/1518308254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c00b9c0 con 0x7fd4ac100a80 2024-02-22T15:40:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.501+0000 7fd4a9ffb700 1 -- 172.21.15.19:0/1518308254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd49c00bc90 con 0x7fd4ac100a80 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 -- 172.21.15.19:0/1518308254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 msgr2=0x7fd4ac100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 --2- 172.21.15.19:0/1518308254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac100e90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0098e0 tx=0x7fd49c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 -- 172.21.15.19:0/1518308254 shutdown_connections 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 --2- 172.21.15.19:0/1518308254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac100e90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 -- 172.21.15.19:0/1518308254 >> 172.21.15.19:0/1518308254 conn(0x7fd4ac0fc460 msgr2=0x7fd4ac0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 -- 172.21.15.19:0/1518308254 shutdown_connections 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.502+0000 7fd4b146e700 1 -- 172.21.15.19:0/1518308254 wait complete. 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4b146e700 1 Processor -- start 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4b146e700 1 -- start start 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4b146e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac1035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4b146e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac101a60 con 0x7fd4ac100a80 2024-02-22T15:40:06.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac1035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4aaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac1035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40714/0 (socket says 172.21.15.19:40714) 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4aaffd700 1 -- 172.21.15.19:0/4126210331 learned_addr learned my addr 172.21.15.19:0/4126210331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4aaffd700 1 -- 172.21.15.19:0/4126210331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c0094f0 con 0x7fd4ac100a80 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4aaffd700 1 --2- 172.21.15.19:0/4126210331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac1035e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd49c003080 tx=0x7fd49c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.503+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd49c01c5c0 con 0x7fd4ac100a80 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.504+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c01c720 con 0x7fd4ac100a80 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.504+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd49c01ca30 con 0x7fd4ac100a80 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.504+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac101c60 con 0x7fd4ac100a80 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.504+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac102070 con 0x7fd4ac100a80 2024-02-22T15:40:06.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.504+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd49c01a450 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.504+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd49c02e790 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.505+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac04bed0 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.507+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd49c01a890 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.576+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd4ac10a6e0 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.579+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 7 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd49c02ee60 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.579+0000 7fd4a37fe700 1 -- 172.21.15.19:0/4126210331 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fd49c024740 con 0x7fd4ac100a80 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 msgr2=0x7fd4ac1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 --2- 172.21.15.19:0/4126210331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac1035e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd49c003080 tx=0x7fd49c016000 comp rx=0 tx=0).stop 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 shutdown_connections 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 --2- 172.21.15.19:0/4126210331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4ac100a80 0x7fd4ac1035e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 >> 172.21.15.19:0/4126210331 conn(0x7fd4ac0fc460 msgr2=0x7fd4ac1056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:06.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 shutdown_connections 2024-02-22T15:40:06.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:05.580+0000 7fd4b146e700 1 -- 172.21.15.19:0/4126210331 wait complete. 2024-02-22T15:40:06.149 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-02-22T15:40:06.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:06 smithi019 ceph-mon[66938]: from='client.? 172.21.15.19:0/2876096143' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:40:07.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.642+0000 7f5200b7a700 1 Processor -- start 2024-02-22T15:40:07.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.642+0000 7f5200b7a700 1 -- start start 2024-02-22T15:40:07.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f5200b7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc101990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:07.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f5200b7a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc101ed0 con 0x7f51fc101580 2024-02-22T15:40:07.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:07.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40720/0 (socket says 172.21.15.19:40720) 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51fa59c700 1 -- 172.21.15.19:0/496774682 learned_addr learned my addr 172.21.15.19:0/496774682 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51fa59c700 1 -- 172.21.15.19:0/496774682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc102010 con 0x7f51fc101580 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51fa59c700 1 --2- 172.21.15.19:0/496774682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc101990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0098e0 tx=0x7f51ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f575b86c3c9a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51f959a700 1 -- 172.21.15.19:0/496774682 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51ec00b860 con 0x7f51fc101580 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51f959a700 1 -- 172.21.15.19:0/496774682 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f51ec00be30 con 0x7f51fc101580 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.643+0000 7f51f959a700 1 -- 172.21.15.19:0/496774682 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51ec010380 con 0x7f51fc101580 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 -- 172.21.15.19:0/496774682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 msgr2=0x7f51fc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 --2- 172.21.15.19:0/496774682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc101990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0098e0 tx=0x7f51ec009250 comp rx=0 tx=0).stop 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 -- 172.21.15.19:0/496774682 shutdown_connections 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 --2- 172.21.15.19:0/496774682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc101990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 -- 172.21.15.19:0/496774682 >> 172.21.15.19:0/496774682 conn(0x7f51fc0fcd00 msgr2=0x7f51fc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 -- 172.21.15.19:0/496774682 shutdown_connections 2024-02-22T15:40:07.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.644+0000 7f5200b7a700 1 -- 172.21.15.19:0/496774682 wait complete. 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f5200b7a700 1 Processor -- start 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f5200b7a700 1 -- start start 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f5200b7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f5200b7a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc076bc0 con 0x7f51fc101580 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f51fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f51fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40726/0 (socket says 172.21.15.19:40726) 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f51fa59c700 1 -- 172.21.15.19:0/2876096143 learned_addr learned my addr 172.21.15.19:0/2876096143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.645+0000 7f51fa59c700 1 -- 172.21.15.19:0/2876096143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0094f0 con 0x7f51fc101580 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f51fa59c700 1 --2- 172.21.15.19:0/2876096143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc114080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f51ec006b00 tx=0x7f51ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51ec0109b0 con 0x7f51fc101580 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f51ec01a430 con 0x7f51fc101580 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc076dc0 con 0x7f51fc101580 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51ec0245b0 con 0x7f51fc101580 2024-02-22T15:40:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc114780 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f51ec024a60 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.646+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f51ec02eb80 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.647+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc04bed0 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.649+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f51ec02ede0 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.719+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f51fc114ac0 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.719+0000 7f51f2ffd700 1 -- 172.21.15.19:0/2876096143 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f51ec01a9c0 con 0x7f51fc101580 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 msgr2=0x7f51fc114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 --2- 172.21.15.19:0/2876096143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc114080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f51ec006b00 tx=0x7f51ec009250 comp rx=0 tx=0).stop 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 shutdown_connections 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 --2- 172.21.15.19:0/2876096143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51fc101580 0x7f51fc114080 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 >> 172.21.15.19:0/2876096143 conn(0x7f51fc0fcd00 msgr2=0x7f51fc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 shutdown_connections 2024-02-22T15:40:07.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:06.720+0000 7f5200b7a700 1 -- 172.21.15.19:0/2876096143 wait complete. 2024-02-22T15:40:07.343 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-02-22T15:40:07.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:07 smithi019 systemd[1]: Stopping Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:40:07.885 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:07 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[66915]: 2024-02-22T15:40:07.610+0000 7f677f5ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi019 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:40:07.885 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:07 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[66915]: 2024-02-22T15:40:07.610+0000 7f677f5ce700 -1 mon.smithi019@0(leader) e1 *** Got Signal Terminated *** 2024-02-22T15:40:08.144 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:07 smithi019 bash[67262]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019 2024-02-22T15:40:08.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service: Succeeded. 2024-02-22T15:40:08.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 systemd[1]: Stopped Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:40:08.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 systemd[1]: Starting Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:40:08.926 INFO:teuthology.orchestra.run.smithi019.stdout:Setting mon public_network to 172.21.0.0/20 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: set uid:gid to 167:167 (ceph:ceph) 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable), process ceph-mon, pid 7 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: pidfile_write: ignore empty --pid-file 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: load: jerasure load: lrc 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: RocksDB version: 6.15.5 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Compile date Feb 21 2024 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: DB SUMMARY 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: DB Session ID: 8RZX0V6MGENGE3SH8R59 2024-02-22T15:40:09.153 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: CURRENT file: CURRENT 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: IDENTITY file: IDENTITY 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi019/store.db dir, Total Num: 1, files: 000008.sst 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi019/store.db: 000010.log size: 97526 ; 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.error_if_exists: 0 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.create_if_missing: 0 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.paranoid_checks: 1 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.env: 0x5574e0bc5a80 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.fs: Posix File System 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.info_log: 0x5574e331bcc0 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_file_opening_threads: 16 2024-02-22T15:40:09.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.statistics: (nil) 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.use_fsync: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_log_file_size: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.keep_log_file_num: 1000 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.recycle_log_file_num: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.allow_fallocate: 1 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.allow_mmap_reads: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.allow_mmap_writes: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.use_direct_reads: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.create_missing_column_families: 0 2024-02-22T15:40:09.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.db_log_dir: 2024-02-22T15:40:09.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi019/store.db 2024-02-22T15:40:09.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-22T15:40:09.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-22T15:40:09.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-22T15:40:09.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-22T15:40:09.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-22T15:40:09.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-22T15:40:09.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.advise_random_on_open: 1 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.db_write_buffer_size: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.write_buffer_manager: 0x5574e4090b40 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.rate_limiter: (nil) 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.wal_recovery_mode: 2 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.enable_thread_tracking: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.enable_pipelined_write: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.unordered_write: 0 2024-02-22T15:40:09.158 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.row_cache: None 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.wal_filter: None 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.allow_ingest_behind: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.preserve_deletes: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.two_write_queues: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.manual_wal_flush: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.atomic_flush: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-22T15:40:09.159 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.log_readahead_size: 0 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.best_efforts_recovery: 0 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.allow_data_in_errors: 0 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.db_host_id: __hostname__ 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_background_jobs: 2 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_background_compactions: -1 2024-02-22T15:40:09.160 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_subcompactions: 1 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_total_wal_size: 0 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_open_files: -1 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bytes_per_sync: 0 2024-02-22T15:40:09.161 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_readahead_size: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_background_flushes: -1 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Compression algorithms supported: 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kZSTD supported: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kXpressCompression supported: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kLZ4HCCompression supported: 1 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kZlibCompression supported: 1 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kSnappyCompression supported: 1 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kLZ4Compression supported: 1 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: kBZip2Compression supported: 0 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-22T15:40:09.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000009 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.merge_operator: 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_filter: None 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_filter_factory: None 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.sst_partitioner_factory: None 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5574e3370f58) 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-22T15:40:09.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_type: 0 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_index_type: 0 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_shortening: 1 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: hash_index_allow_collision: 1 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: checksum: 1 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: no_block_cache: 0 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache: 0x5574e33a3090 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-02-22T15:40:09.164 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_options: 2024-02-22T15:40:09.165 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: capacity : 536870912 2024-02-22T15:40:09.165 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: num_shard_bits : 4 2024-02-22T15:40:09.165 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: strict_capacity_limit : 0 2024-02-22T15:40:09.165 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_compressed: (nil) 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: persistent_cache: (nil) 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size: 4096 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size_deviation: 10 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_restart_interval: 16 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_block_restart_interval: 1 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: metadata_block_size: 4096 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: partition_filters: 0 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: use_delta_encoding: 1 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: whole_key_filtering: 1 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: verify_compression: 0 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: format_version: 4 2024-02-22T15:40:09.166 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_index_compression: 1 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_align: 0 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.write_buffer_size: 33554432 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_write_buffer_number: 2 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression: NoCompression 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression: Disabled 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.prefix_extractor: nullptr 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.num_levels: 7 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-22T15:40:09.167 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.level: 32767 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.strategy: 0 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compression_opts.enabled: false 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-22T15:40:09.168 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.target_file_size_base: 67108864 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-22T15:40:09.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.arena_block_size: 4194304 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.disable_auto_compactions: 0 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-22T15:40:09.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.table_properties_collectors: 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.inplace_update_support: 0 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-22T15:40:09.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.bloom_locality: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.max_successive_merges: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.paranoid_file_checks: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.force_consistency_checks: 1 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.report_bg_io_stats: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.ttl: 2592000 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.enable_blob_files: false 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.min_blob_size: 0 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.blob_file_size: 268435456 2024-02-22T15:40:09.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708616408890252, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708616408891030, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92504, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90599, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13271, "raw_average_key_size": 49, "raw_value_size": 83298, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708616408, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "adc65834-e262-4095-865d-7daf4e8a6abd", "db_session_id": "8RZX0V6MGENGE3SH8R59"}} 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708616408891365, "job": 1, "event": "recovery_finished"} 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi019/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5574e3337180 2024-02-22T15:40:09.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: DB pointer 0x5574e340c000 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** DB Stats ** 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-02-22T15:40:09.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T15:40:09.175 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T15:40:09.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L0 2/0 92.13 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Sum 2/0 92.13 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-02-22T15:40:09.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T15:40:09.177 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T15:40:09.177 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.177 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:40:09.177 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T15:40:09.177 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T15:40:09.177 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T15:40:09.178 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T15:40:09.178 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-22T15:40:09.178 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative compaction: 0.00 GB write, 27.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:40:09.178 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval compaction: 0.00 GB write, 27.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:40:09.178 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T15:40:09.178 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.179 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-22T15:40:09.179 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.179 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-02-22T15:40:09.179 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T15:40:09.179 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T15:40:09.179 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L0 2/0 92.13 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Sum 2/0 92.13 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-22T15:40:09.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-02-22T15:40:09.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-22T15:40:09.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T15:40:09.181 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 181.9 0.00 0.00 1 0.000 0 0 2024-02-22T15:40:09.181 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:40:09.181 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-22T15:40:09.181 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T15:40:09.181 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T15:40:09.182 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T15:40:09.182 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-22T15:40:09.182 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative compaction: 0.00 GB write, 27.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:40:09.182 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:40:09.182 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T15:40:09.182 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: starting mon.smithi019 rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi019 fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???) e1 preinit fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).mds e1 new map 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).mds e1 print_map 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: e1 2024-02-22T15:40:09.183 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:40:09.184 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:40:09.184 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: legacy client fscid: -1 2024-02-22T15:40:09.184 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:40:09.184 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: No filesystems configured 2024-02-22T15:40:09.184 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 bash[67380]: 5b269c418fefd986c3b657fa144d75b7e7301d59984bbea6dfd38c7d9c62e671 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 systemd[1]: Started Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:40:09.185 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mon.smithi019 is new leader, mons smithi019 in quorum (ranks 0) 2024-02-22T15:40:09.186 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: monmap e1: 1 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:40:09.186 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: fsmap 2024-02-22T15:40:09.186 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: osdmap e1: 0 total, 0 up, 0 in 2024-02-22T15:40:09.186 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:08 smithi019 ceph-mon[67418]: mgrmap e1: no daemons active 2024-02-22T15:40:10.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.520+0000 7f12d6ba4700 1 Processor -- start 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.520+0000 7f12d6ba4700 1 -- start start 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.520+0000 7f12d6ba4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d01076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.520+0000 7f12d6ba4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d01030b0 con 0x7f12d00712d0 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.521+0000 7f12d4940700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d01076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.521+0000 7f12d4940700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d01076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48498/0 (socket says 172.21.15.19:48498) 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.521+0000 7f12d4940700 1 -- 172.21.15.19:0/3730436416 learned_addr learned my addr 172.21.15.19:0/3730436416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:10.087 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.521+0000 7f12d4940700 1 -- 172.21.15.19:0/3730436416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0107c00 con 0x7f12d00712d0 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.521+0000 7f12d4940700 1 --2- 172.21.15.19:0/3730436416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d01076c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098e0 tx=0x7f12c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125dee4f6019e5f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.521+0000 7f12cf7fe700 1 -- 172.21.15.19:0/3730436416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12c400b860 con 0x7f12d00712d0 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12cf7fe700 1 -- 172.21.15.19:0/3730436416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f12c400be30 con 0x7f12d00712d0 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12cf7fe700 1 -- 172.21.15.19:0/3730436416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12c4010380 con 0x7f12d00712d0 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3730436416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 msgr2=0x7f12d01076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 --2- 172.21.15.19:0/3730436416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d01076c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12c40098e0 tx=0x7f12c4009250 comp rx=0 tx=0).stop 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3730436416 shutdown_connections 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 --2- 172.21.15.19:0/3730436416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d01076c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3730436416 >> 172.21.15.19:0/3730436416 conn(0x7f12d00fccd0 msgr2=0x7f12d00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3730436416 shutdown_connections 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.522+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3730436416 wait complete. 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d6ba4700 1 Processor -- start 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d6ba4700 1 -- start start 2024-02-22T15:40:10.088 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d6ba4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d0113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d6ba4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0113e60 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d4940700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d0113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d4940700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d0113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48508/0 (socket says 172.21.15.19:48508) 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.523+0000 7f12d4940700 1 -- 172.21.15.19:0/3282545507 learned_addr learned my addr 172.21.15.19:0/3282545507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12d4940700 1 -- 172.21.15.19:0/3282545507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40094f0 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12d4940700 1 --2- 172.21.15.19:0/3282545507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d0113920 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12c40090b0 tx=0x7f12c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12c40109b0 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f12c401a430 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d0114060 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12c40245b0 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d0114470 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f12c4024af0 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.524+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f12c402ebd0 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.525+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d004bed0 con 0x7f12d00712d0 2024-02-22T15:40:10.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.527+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f12c4029070 con 0x7f12d00712d0 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.596+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f12d0064ab0 con 0x7f12d00712d0 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.598+0000 7f12cd7fa700 1 -- 172.21.15.19:0/3282545507 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f12c402ed90 con 0x7f12d00712d0 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 msgr2=0x7f12d0113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 --2- 172.21.15.19:0/3282545507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d0113920 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12c40090b0 tx=0x7f12c4009250 comp rx=0 tx=0).stop 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 shutdown_connections 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 --2- 172.21.15.19:0/3282545507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12d00712d0 0x7f12d0113920 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 >> 172.21.15.19:0/3282545507 conn(0x7f12d00fccd0 msgr2=0x7f12d00ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 shutdown_connections 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:09.599+0000 7f12d6ba4700 1 -- 172.21.15.19:0/3282545507 wait complete. 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-02-22T15:40:10.090 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 9283 ... 2024-02-22T15:40:10.223 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mgr.smithi019.yvjrkp 2024-02-22T15:40:10.223 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mgr.smithi019.yvjrkp.service: Unit ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mgr.smithi019.yvjrkp.service not loaded. 2024-02-22T15:40:10.320 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715.target.wants/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mgr.smithi019.yvjrkp.service → /etc/systemd/system/ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@.service. 2024-02-22T15:40:10.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:10 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3282545507' entity='client.admin' 2024-02-22T15:40:10.947 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-22T15:40:10.947 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-02-22T15:40:10.955 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-22T15:40:10.956 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-02-22T15:40:10.956 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-02-22T15:40:10.956 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-02-22T15:40:10.964 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-22T15:40:10.964 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-02-22T15:40:10.972 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-22T15:40:10.972 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-02-22T15:40:10.972 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-02-22T15:40:10.972 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-22T15:40:10.972 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-02-22T15:40:10.972 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-02-22T15:40:11.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:11 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1606119223' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-22T15:40:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:40:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "8db8e6fa-d198-11ee-95bf-87774f69a715", 2024-02-22T15:40:12.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-22T15:40:12.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "smithi019" 2024-02-22T15:40:12.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:12.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-22T15:40:12.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-22T15:40:12.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:12.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-22T15:40:12.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-22T15:40:12.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-22T15:40:12.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:12.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-22T15:40:12.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-22T15:40:12.307 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-22T15:40:12.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-22T15:40:12.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-22T15:40:12.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-22T15:40:12.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-22T15:40:12.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-22T15:40:12.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-02-22T15:40:02.431710+0000", 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-22T15:40:12.318 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.524+0000 7ffbc9059700 1 Processor -- start 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.524+0000 7ffbc9059700 1 -- start start 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.524+0000 7ffbc9059700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.524+0000 7ffbc9059700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc41045f0 con 0x7ffbc4103ca0 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48538/0 (socket says 172.21.15.19:48538) 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc2d9d700 1 -- 172.21.15.19:0/3123546109 learned_addr learned my addr 172.21.15.19:0/3123546109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc2d9d700 1 -- 172.21.15.19:0/3123546109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4104730 con 0x7ffbc4103ca0 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc2d9d700 1 --2- 172.21.15.19:0/3123546109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41040b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40098e0 tx=0x7ffbb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9746cee7765aac22 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc1d9b700 1 -- 172.21.15.19:0/3123546109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbb400b860 con 0x7ffbc4103ca0 2024-02-22T15:40:12.319 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc1d9b700 1 -- 172.21.15.19:0/3123546109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ffbb400be30 con 0x7ffbc4103ca0 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.525+0000 7ffbc1d9b700 1 -- 172.21.15.19:0/3123546109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbb4010380 con 0x7ffbc4103ca0 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 -- 172.21.15.19:0/3123546109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 msgr2=0x7ffbc41040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 --2- 172.21.15.19:0/3123546109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41040b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40098e0 tx=0x7ffbb4009250 comp rx=0 tx=0).stop 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 -- 172.21.15.19:0/3123546109 shutdown_connections 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 --2- 172.21.15.19:0/3123546109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41040b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 -- 172.21.15.19:0/3123546109 >> 172.21.15.19:0/3123546109 conn(0x7ffbc40fc460 msgr2=0x7ffbc40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 -- 172.21.15.19:0/3123546109 shutdown_connections 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.526+0000 7ffbc9059700 1 -- 172.21.15.19:0/3123546109 wait complete. 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc9059700 1 Processor -- start 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc9059700 1 -- start start 2024-02-22T15:40:12.320 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc9059700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc9059700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4101960 con 0x7ffbc4103ca0 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc2d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48540/0 (socket says 172.21.15.19:48540) 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.527+0000 7ffbc2d9d700 1 -- 172.21.15.19:0/1606119223 learned_addr learned my addr 172.21.15.19:0/1606119223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbc2d9d700 1 -- 172.21.15.19:0/1606119223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40094f0 con 0x7ffbc4103ca0 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbc2d9d700 1 --2- 172.21.15.19:0/1606119223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41034e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006b00 tx=0x7ffbb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbb40109b0 con 0x7ffbc4103ca0 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4101b60 con 0x7ffbc4103ca0 2024-02-22T15:40:12.321 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc4101f70 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ffbb401a430 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbb402d5b0 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ffbb402da90 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.528+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ffbb4019990 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.529+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba40052e0 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.533+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ffbb401a9a0 con 0x7ffbc4103ca0 2024-02-22T15:40:12.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.607+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffba4005590 con 0x7ffbc4103ca0 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.607+0000 7ffbbb7fe700 1 -- 172.21.15.19:0/1606119223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7ffbb4035090 con 0x7ffbc4103ca0 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 msgr2=0x7ffbc41034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 --2- 172.21.15.19:0/1606119223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41034e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006b00 tx=0x7ffbb4009250 comp rx=0 tx=0).stop 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 shutdown_connections 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 --2- 172.21.15.19:0/1606119223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbc4103ca0 0x7ffbc41034e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 >> 172.21.15.19:0/1606119223 conn(0x7ffbc40fc460 msgr2=0x7ffbc40fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:12.323 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 shutdown_connections 2024-02-22T15:40:12.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:11.608+0000 7ffbc9059700 1 -- 172.21.15.19:0/1606119223 wait complete. 2024-02-22T15:40:12.324 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-02-22T15:40:17.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:17 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1584935700' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-22T15:40:18.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: mgrmap e2: smithi019.yvjrkp(active, starting, since 0.00282776s) 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:18.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:18 smithi019 ceph-mon[67418]: from='mgr.14100 172.21.15.19:0/1261834844' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:20.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:19 smithi019 ceph-mon[67418]: mgrmap e3: smithi019.yvjrkp(active, since 1.0068s) 2024-02-22T15:40:21.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:21 smithi019 ceph-mon[67418]: mgrmap e4: smithi019.yvjrkp(active, since 3s) 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "8db8e6fa-d198-11ee-95bf-87774f69a715", 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-22T15:40:23.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "smithi019" 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-22T15:40:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:23.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-22T15:40:23.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-22T15:40:23.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-22T15:40:23.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-22T15:40:23.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-02-22T15:40:02.431710+0000", 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.292+0000 7f3bc239b700 1 Processor -- start 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.292+0000 7f3bc239b700 1 -- start start 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.292+0000 7f3bc239b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc105240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.292+0000 7f3bc239b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc105780 con 0x7f3bbc071350 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc105240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48542/0 (socket says 172.21.15.19:48542) 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbbfff700 1 -- 172.21.15.19:0/1882544551 learned_addr learned my addr 172.21.15.19:0/1882544551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbbfff700 1 -- 172.21.15.19:0/1882544551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc1058c0 con 0x7f3bbc071350 2024-02-22T15:40:23.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbbfff700 1 --2- 172.21.15.19:0/1882544551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc105240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0098e0 tx=0x7f3bac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a012c1f4deaedf8f server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbaffd700 1 -- 172.21.15.19:0/1882544551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bac00b860 con 0x7f3bbc071350 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbaffd700 1 -- 172.21.15.19:0/1882544551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f3bac00be30 con 0x7f3bbc071350 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.293+0000 7f3bbaffd700 1 -- 172.21.15.19:0/1882544551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bac010380 con 0x7f3bbc071350 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 -- 172.21.15.19:0/1882544551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 msgr2=0x7f3bbc105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 --2- 172.21.15.19:0/1882544551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc105240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0098e0 tx=0x7f3bac009250 comp rx=0 tx=0).stop 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 -- 172.21.15.19:0/1882544551 shutdown_connections 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 --2- 172.21.15.19:0/1882544551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc105240 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 -- 172.21.15.19:0/1882544551 >> 172.21.15.19:0/1882544551 conn(0x7f3bbc0fccd0 msgr2=0x7f3bbc0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 -- 172.21.15.19:0/1882544551 shutdown_connections 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 -- 172.21.15.19:0/1882544551 wait complete. 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.294+0000 7f3bc239b700 1 Processor -- start 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bc239b700 1 -- start start 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bc239b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:23.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bc239b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc101140 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48544/0 (socket says 172.21.15.19:48544) 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bbbfff700 1 -- 172.21.15.19:0/1584935700 learned_addr learned my addr 172.21.15.19:0/1584935700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bbbfff700 1 -- 172.21.15.19:0/1584935700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0094f0 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.295+0000 7f3bbbfff700 1 --2- 172.21.15.19:0/1584935700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc102cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0090b0 tx=0x7f3bac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bac0109b0 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f3bac01a430 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc101340 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bac0245b0 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc101750 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3bac024aa0 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.296+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3bac02eb80 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.297+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc10b4e0 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.299+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3bac029070 con 0x7f3bbc071350 2024-02-22T15:40:23.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.373+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3bbc1066a0 con 0x7f3bbc071350 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.373+0000 7f3bb8ff9700 1 -- 172.21.15.19:0/1584935700 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f3bac01a5a0 con 0x7f3bbc071350 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 msgr2=0x7f3bbc102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 --2- 172.21.15.19:0/1584935700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc102cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0090b0 tx=0x7f3bac009250 comp rx=0 tx=0).stop 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 shutdown_connections 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 --2- 172.21.15.19:0/1584935700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc071350 0x7f3bbc102cc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 >> 172.21.15.19:0/1584935700 conn(0x7f3bbc0fccd0 msgr2=0x7f3bbc0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 shutdown_connections 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:17.374+0000 7f3bc239b700 1 -- 172.21.15.19:0/1584935700 wait complete. 2024-02-22T15:40:23.522 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-02-22T15:40:26.700 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:26 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3419271731' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-22T15:40:26.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "8db8e6fa-d198-11ee-95bf-87774f69a715", 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-22T15:40:26.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "smithi019" 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:26.963 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-22T15:40:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-22T15:40:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-22T15:40:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-02-22T15:40:02.431710+0000", 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.110+0000 7f8b737fd700 1 Processor -- start 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.110+0000 7f8b737fd700 1 -- start start 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b737fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c102a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:26.967 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b737fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c102fb0 con 0x7f8b6c102660 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b71599700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b71599700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34932/0 (socket says 172.21.15.19:34932) 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b71599700 1 -- 172.21.15.19:0/3134018531 learned_addr learned my addr 172.21.15.19:0/3134018531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b71599700 1 -- 172.21.15.19:0/3134018531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c1030f0 con 0x7f8b6c102660 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b71599700 1 --2- 172.21.15.19:0/3134018531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c102a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b680098e0 tx=0x7f8b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b72742e20a6048 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.111+0000 7f8b5ffff700 1 -- 172.21.15.19:0/3134018531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b6800b860 con 0x7f8b6c102660 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b5ffff700 1 -- 172.21.15.19:0/3134018531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8b6800be30 con 0x7f8b6c102660 2024-02-22T15:40:26.968 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b5ffff700 1 -- 172.21.15.19:0/3134018531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b68010380 con 0x7f8b6c102660 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 -- 172.21.15.19:0/3134018531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 msgr2=0x7f8b6c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 --2- 172.21.15.19:0/3134018531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c102a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b680098e0 tx=0x7f8b68009250 comp rx=0 tx=0).stop 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 -- 172.21.15.19:0/3134018531 shutdown_connections 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 --2- 172.21.15.19:0/3134018531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c102a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 -- 172.21.15.19:0/3134018531 >> 172.21.15.19:0/3134018531 conn(0x7f8b6c0fddc0 msgr2=0x7f8b6c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 -- 172.21.15.19:0/3134018531 shutdown_connections 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.112+0000 7f8b737fd700 1 -- 172.21.15.19:0/3134018531 wait complete. 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b737fd700 1 Processor -- start 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b737fd700 1 -- start start 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b737fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c111450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b737fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c111990 con 0x7f8b6c102660 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b71599700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c111450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b71599700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c111450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34942/0 (socket says 172.21.15.19:34942) 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b71599700 1 -- 172.21.15.19:0/3419271731 learned_addr learned my addr 172.21.15.19:0/3419271731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:26.969 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b71599700 1 -- 172.21.15.19:0/3419271731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680094f0 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.113+0000 7f8b71599700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c111450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006b00 tx=0x7f8b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.114+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b680109b0 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.114+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8b68024440 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.114+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c111b90 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.114+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b6801a5f0 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.114+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c111fa0 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.115+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f8b6801a860 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.115+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c04bed0 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.115+0000 7f8b5dffb700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f8b580367e0 0x7f8b58038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.115+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8b6804a890 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.115+0000 7f8b70d98700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f8b580367e0 0x7f8b58038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.116+0000 7f8b70d98700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f8b580367e0 0x7f8b58038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009990 tx=0x7f8b60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.120+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8b6801ab10 con 0x7f8b6c102660 2024-02-22T15:40:26.970 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.389+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8b6c112bf0 con 0x7f8b6c102660 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.389+0000 7f8b5dffb700 1 -- 172.21.15.19:0/3419271731 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f8b68010b10 con 0x7f8b6c102660 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f8b580367e0 msgr2=0x7f8b58038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f8b580367e0 0x7f8b58038c90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009990 tx=0x7f8b60006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 msgr2=0x7f8b6c111450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c111450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006b00 tx=0x7f8b68009250 comp rx=0 tx=0).stop 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 shutdown_connections 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f8b580367e0 0x7f8b58038c90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 --2- 172.21.15.19:0/3419271731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b6c102660 0x7f8b6c111450 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 >> 172.21.15.19:0/3419271731 conn(0x7f8b6c0fddc0 msgr2=0x7f8b6c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 shutdown_connections 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:26.391+0000 7f8b737fd700 1 -- 172.21.15.19:0/3419271731 wait complete. 2024-02-22T15:40:26.971 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-02-22T15:40:28.034 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:27 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3930079742' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-22T15:40:28.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:28.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-02-22T15:40:28.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:28.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-22T15:40:28.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-22T15:40:28.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-22T15:40:28.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-22T15:40:28.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:28.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-22T15:40:28.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-22T15:40:28.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.533+0000 7ff402dcc700 1 Processor -- start 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.533+0000 7ff402dcc700 1 -- start start 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.533+0000 7ff402dcc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc1076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.533+0000 7ff402dcc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc1030d0 con 0x7ff3fc0712b0 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff400b68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc1076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff400b68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc1076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38964/0 (socket says 172.21.15.19:38964) 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff400b68700 1 -- 172.21.15.19:0/3693537342 learned_addr learned my addr 172.21.15.19:0/3693537342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:28.290 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff400b68700 1 -- 172.21.15.19:0/3693537342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc107c20 con 0x7ff3fc0712b0 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff400b68700 1 --2- 172.21.15.19:0/3693537342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc1076e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00098e0 tx=0x7ff3f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ec1bff5296f13e server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff3fb7fe700 1 -- 172.21.15.19:0/3693537342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f000b860 con 0x7ff3fc0712b0 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff3fb7fe700 1 -- 172.21.15.19:0/3693537342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff3f000be30 con 0x7ff3fc0712b0 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.534+0000 7ff3fb7fe700 1 -- 172.21.15.19:0/3693537342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f0010380 con 0x7ff3fc0712b0 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 -- 172.21.15.19:0/3693537342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 msgr2=0x7ff3fc1076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 --2- 172.21.15.19:0/3693537342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc1076e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00098e0 tx=0x7ff3f0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 -- 172.21.15.19:0/3693537342 shutdown_connections 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 --2- 172.21.15.19:0/3693537342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc1076e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:28.291 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 -- 172.21.15.19:0/3693537342 >> 172.21.15.19:0/3693537342 conn(0x7ff3fc0fccd0 msgr2=0x7ff3fc0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 -- 172.21.15.19:0/3693537342 shutdown_connections 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.535+0000 7ff402dcc700 1 -- 172.21.15.19:0/3693537342 wait complete. 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff402dcc700 1 Processor -- start 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff402dcc700 1 -- start start 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff402dcc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff402dcc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc113e50 con 0x7ff3fc0712b0 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff400b68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff400b68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38974/0 (socket says 172.21.15.19:38974) 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff400b68700 1 -- 172.21.15.19:0/3930079742 learned_addr learned my addr 172.21.15.19:0/3930079742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.536+0000 7ff400b68700 1 -- 172.21.15.19:0/3930079742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00094f0 con 0x7ff3fc0712b0 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.537+0000 7ff400b68700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc113910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00090b0 tx=0x7ff3f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:28.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.537+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f00109b0 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.537+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff3f001a430 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.537+0000 7ff402dcc700 1 -- 172.21.15.19:0/3930079742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc114050 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.537+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f00245b0 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.537+0000 7ff402dcc700 1 -- 172.21.15.19:0/3930079742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc114460 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.538+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7ff3f0024820 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.538+0000 7ff3f97fa700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff3ec036740 0x7ff3ec038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.538+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff3f004a940 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.538+0000 7ff3fbfff700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff3ec036740 0x7ff3ec038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.538+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3dc0052b0 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.538+0000 7ff3fbfff700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff3ec036740 0x7ff3ec038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009990 tx=0x7ff3e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.543+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff3f002e440 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.738+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff3dc005ae0 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.739+0000 7ff3f97fa700 1 -- 172.21.15.19:0/3930079742 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff3f0048020 con 0x7ff3fc0712b0 2024-02-22T15:40:28.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff3ec036740 msgr2=0x7ff3ec038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff3ec036740 0x7ff3ec038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009990 tx=0x7ff3e8006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 msgr2=0x7ff3fc113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc113910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00090b0 tx=0x7ff3f0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 shutdown_connections 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff3ec036740 0x7ff3ec038bf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 --2- 172.21.15.19:0/3930079742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc0712b0 0x7ff3fc113910 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 >> 172.21.15.19:0/3930079742 conn(0x7ff3fc0fccd0 msgr2=0x7ff3fc0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 shutdown_connections 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:27.741+0000 7ff3e6ffd700 1 -- 172.21.15.19:0/3930079742 wait complete. 2024-02-22T15:40:28.294 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-02-22T15:40:30.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:29 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/4059216256' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-22T15:40:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4597dd4700 1 Processor -- start 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4597dd4700 1 -- start start 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4597dd4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590105280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4597dd4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45901057c0 con 0x7f4590071310 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4595b70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4595b70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38986/0 (socket says 172.21.15.19:38986) 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4595b70700 1 -- 172.21.15.19:0/3572484233 learned_addr learned my addr 172.21.15.19:0/3572484233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.854+0000 7f4595b70700 1 -- 172.21.15.19:0/3572484233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590105900 con 0x7f4590071310 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4595b70700 1 --2- 172.21.15.19:0/3572484233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590105280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f458c0098e0 tx=0x7f458c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815099d52006c6c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4594b6e700 1 -- 172.21.15.19:0/3572484233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f458c00b860 con 0x7f4590071310 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4594b6e700 1 -- 172.21.15.19:0/3572484233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f458c00be30 con 0x7f4590071310 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4594b6e700 1 -- 172.21.15.19:0/3572484233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f458c010380 con 0x7f4590071310 2024-02-22T15:40:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 -- 172.21.15.19:0/3572484233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 msgr2=0x7f4590105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 --2- 172.21.15.19:0/3572484233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590105280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f458c0098e0 tx=0x7f458c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 -- 172.21.15.19:0/3572484233 shutdown_connections 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 --2- 172.21.15.19:0/3572484233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590105280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 -- 172.21.15.19:0/3572484233 >> 172.21.15.19:0/3572484233 conn(0x7f45900fccd0 msgr2=0x7f45900ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 -- 172.21.15.19:0/3572484233 shutdown_connections 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.855+0000 7f4597dd4700 1 -- 172.21.15.19:0/3572484233 wait complete. 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4597dd4700 1 Processor -- start 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4597dd4700 1 -- start start 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4597dd4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4597dd4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4590101180 con 0x7f4590071310 2024-02-22T15:40:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4595b70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4595b70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38994/0 (socket says 172.21.15.19:38994) 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4595b70700 1 -- 172.21.15.19:0/4059216256 learned_addr learned my addr 172.21.15.19:0/4059216256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.856+0000 7f4595b70700 1 -- 172.21.15.19:0/4059216256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0094f0 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.857+0000 7f4595b70700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590102d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f458c0090b0 tx=0x7f458c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.857+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f458c010a80 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.857+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f458c01a430 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.857+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f458c0245b0 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.857+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590101380 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.857+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590101790 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.858+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f458c01a5a0 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.858+0000 7f45867fc700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f457c036790 0x7f457c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.858+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459010b4e0 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.858+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f458c04a2d0 con 0x7f4590071310 2024-02-22T15:40:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.858+0000 7f459536f700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f457c036790 0x7f457c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.858+0000 7f459536f700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f457c036790 0x7f457c038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4580009990 tx=0x7f4580006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:28.863+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f458c02b4d0 con 0x7f4590071310 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.106+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f4590064ab0 con 0x7f4590071310 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.903+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f458c02d3f0 con 0x7f4590071310 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.905+0000 7f45867fc700 1 -- 172.21.15.19:0/4059216256 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f458c00e620 con 0x7f4590071310 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.907+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f457c036790 msgr2=0x7f457c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.907+0000 7f4597dd4700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f457c036790 0x7f457c038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4580009990 tx=0x7f4580006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.907+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 msgr2=0x7f4590102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.907+0000 7f4597dd4700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590102d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f458c0090b0 tx=0x7f458c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.908+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 shutdown_connections 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.908+0000 7f4597dd4700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f457c036790 0x7f457c038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.908+0000 7f4597dd4700 1 --2- 172.21.15.19:0/4059216256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4590071310 0x7f4590102d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.908+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 >> 172.21.15.19:0/4059216256 conn(0x7f45900fccd0 msgr2=0x7f45900ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.908+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 shutdown_connections 2024-02-22T15:40:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:29.908+0000 7f4597dd4700 1 -- 172.21.15.19:0/4059216256 wait complete. 2024-02-22T15:40:31.084 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:30 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/4059216256' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-22T15:40:31.084 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:30 smithi019 ceph-mon[67418]: mgrmap e5: smithi019.yvjrkp(active, since 12s) 2024-02-22T15:40:31.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:31 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1999603087' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-22T15:40:32.136 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:40:32.136 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-22T15:40:32.136 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-22T15:40:32.136 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "smithi019.yvjrkp", 2024-02-22T15:40:32.136 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-22T15:40:32.136 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70da754700 1 Processor -- start 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70da754700 1 -- start start 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70da754700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70da754700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d41057e0 con 0x7f70d40712f0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39030/0 (socket says 172.21.15.19:39030) 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70d3fff700 1 -- 172.21.15.19:0/1681369039 learned_addr learned my addr 172.21.15.19:0/1681369039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.134+0000 7f70d3fff700 1 -- 172.21.15.19:0/1681369039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4105920 con 0x7f70d40712f0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70d3fff700 1 --2- 172.21.15.19:0/1681369039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41052a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098e0 tx=0x7f70c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b80f0f24e9ac32 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70d2ffd700 1 -- 172.21.15.19:0/1681369039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c400b860 con 0x7f70d40712f0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70d2ffd700 1 -- 172.21.15.19:0/1681369039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f70c400be30 con 0x7f70d40712f0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70d2ffd700 1 -- 172.21.15.19:0/1681369039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c4010380 con 0x7f70d40712f0 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 -- 172.21.15.19:0/1681369039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 msgr2=0x7f70d41052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:32.137 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 --2- 172.21.15.19:0/1681369039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41052a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098e0 tx=0x7f70c4009250 comp rx=0 tx=0).stop 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 -- 172.21.15.19:0/1681369039 shutdown_connections 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 --2- 172.21.15.19:0/1681369039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41052a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 -- 172.21.15.19:0/1681369039 >> 172.21.15.19:0/1681369039 conn(0x7f70d40fccd0 msgr2=0x7f70d40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 -- 172.21.15.19:0/1681369039 shutdown_connections 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.135+0000 7f70da754700 1 -- 172.21.15.19:0/1681369039 wait complete. 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.136+0000 7f70da754700 1 Processor -- start 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.136+0000 7f70da754700 1 -- start start 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70da754700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41116f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70da754700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4111c30 con 0x7f70d40712f0 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39040/0 (socket says 172.21.15.19:39040) 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d3fff700 1 -- 172.21.15.19:0/1999603087 learned_addr learned my addr 172.21.15.19:0/1999603087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d3fff700 1 -- 172.21.15.19:0/1999603087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40094f0 con 0x7f70d40712f0 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d3fff700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41116f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f70c40090b0 tx=0x7f70c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:32.138 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40109b0 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f70c401a430 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d4111e30 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70c40245b0 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.137+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d4112240 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.138+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f70c4024820 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.138+0000 7f70d0ff9700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 0x7f70bc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.138+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f70c404a8a0 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.138+0000 7f70d37fe700 1 -- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 msgr2=0x7f70bc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.138+0000 7f70d37fe700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 0x7f70bc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.139+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d404bed0 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.143+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f70c402c340 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.339+0000 7f70d37fe700 1 -- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 msgr2=0x7f70bc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.339+0000 7f70d37fe700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 0x7f70bc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.405+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f70d41067a0 con 0x7f70d40712f0 2024-02-22T15:40:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.405+0000 7f70d0ff9700 1 -- 172.21.15.19:0/1999603087 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f70c4018b40 con 0x7f70d40712f0 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 msgr2=0x7f70bc038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 0x7f70bc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 msgr2=0x7f70d41116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41116f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f70c40090b0 tx=0x7f70c4009250 comp rx=0 tx=0).stop 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 shutdown_connections 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7f70bc036790 0x7f70bc038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 --2- 172.21.15.19:0/1999603087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70d40712f0 0x7f70d41116f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.407+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 >> 172.21.15.19:0/1999603087 conn(0x7f70d40fccd0 msgr2=0x7f70d40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:32.140 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.408+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 shutdown_connections 2024-02-22T15:40:32.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:31.408+0000 7f70da754700 1 -- 172.21.15.19:0/1999603087 wait complete. 2024-02-22T15:40:32.141 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-02-22T15:40:32.141 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-02-22T15:40:36.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: Active manager daemon smithi019.yvjrkp restarted 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: osdmap e2: 0 total, 0 up, 0 in 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: mgrmap e6: smithi019.yvjrkp(active, starting, since 0.00295729s) 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:36.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:40:36.751 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:36 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:40:37.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:37 smithi019 ceph-mon[67418]: Found migration_current of "None". Setting to last migration. 2024-02-22T15:40:37.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:37 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:40:37.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:37 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:40:38.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:37 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:40:38.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:37 smithi019 ceph-mon[67418]: mgrmap e7: smithi019.yvjrkp(active, since 1.00497s) 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.694+0000 7ff735366700 1 Processor -- start 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.694+0000 7ff735366700 1 -- start start 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.694+0000 7ff735366700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff7300760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.694+0000 7ff735366700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730077fd0 con 0x7ff730077bc0 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff7300760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:38.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff7300760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39044/0 (socket says 172.21.15.19:39044) 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72effd700 1 -- 172.21.15.19:0/2668981137 learned_addr learned my addr 172.21.15.19:0/2668981137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72effd700 1 -- 172.21.15.19:0/2668981137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff730076610 con 0x7ff730077bc0 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72effd700 1 --2- 172.21.15.19:0/2668981137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff7300760d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7200098e0 tx=0x7ff720009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450a2831a7824039 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72dffb700 1 -- 172.21.15.19:0/2668981137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff72000b860 con 0x7ff730077bc0 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72dffb700 1 -- 172.21.15.19:0/2668981137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff72000be30 con 0x7ff730077bc0 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.695+0000 7ff72dffb700 1 -- 172.21.15.19:0/2668981137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff720010380 con 0x7ff730077bc0 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 -- 172.21.15.19:0/2668981137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 msgr2=0x7ff7300760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 --2- 172.21.15.19:0/2668981137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff7300760d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7200098e0 tx=0x7ff720009250 comp rx=0 tx=0).stop 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 -- 172.21.15.19:0/2668981137 shutdown_connections 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 --2- 172.21.15.19:0/2668981137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff7300760d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 -- 172.21.15.19:0/2668981137 >> 172.21.15.19:0/2668981137 conn(0x7ff7300fddc0 msgr2=0x7ff7301001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 -- 172.21.15.19:0/2668981137 shutdown_connections 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.696+0000 7ff735366700 1 -- 172.21.15.19:0/2668981137 wait complete. 2024-02-22T15:40:38.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff735366700 1 Processor -- start 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff735366700 1 -- start start 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff735366700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff730113670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff735366700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730113bb0 con 0x7ff730077bc0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff72effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff730113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff72effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff730113670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39060/0 (socket says 172.21.15.19:39060) 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff72effd700 1 -- 172.21.15.19:0/2145649418 learned_addr learned my addr 172.21.15.19:0/2145649418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.697+0000 7ff72effd700 1 -- 172.21.15.19:0/2145649418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200094f0 con 0x7ff730077bc0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.698+0000 7ff72effd700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff730113670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff720006b00 tx=0x7ff720009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.698+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff720010a80 con 0x7ff730077bc0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.698+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff72001a430 con 0x7ff730077bc0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.698+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7200245b0 con 0x7ff730077bc0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.698+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff730113db0 con 0x7ff730077bc0 2024-02-22T15:40:38.408 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.698+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7301141c0 con 0x7ff730077bc0 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.699+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff7200247e0 con 0x7ff730077bc0 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.699+0000 7ff7177fe700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.699+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 --> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7180393a0 con 0x7ff7180367e0 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.699+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff72004b8d0 con 0x7ff730077bc0 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.699+0000 7ff72e7fc700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 msgr2=0x7ff718038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.699+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.900+0000 7ff72e7fc700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 msgr2=0x7ff718038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:32.900+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:33.301+0000 7ff72e7fc700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 msgr2=0x7ff718038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:33.301+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:34.102+0000 7ff72e7fc700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 msgr2=0x7ff718038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:34.102+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:35.703+0000 7ff72e7fc700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 msgr2=0x7ff718038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2711018613 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:35.703+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-22T15:40:38.409 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:36.641+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7ff72002e960 con 0x7ff730077bc0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:36.641+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 msgr2=0x7ff718038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:36.641+0000 7ff7177fe700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.642+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7ff72002c390 con 0x7ff730077bc0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.642+0000 7ff7177fe700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff718039fc0 0x7ff71803c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.642+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7180393a0 con 0x7ff718039fc0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.642+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff718039fc0 0x7ff71803c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.643+0000 7ff72e7fc700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff718039fc0 0x7ff71803c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7240039d0 tx=0x7ff724009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.643+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7ff7180393a0 con 0x7ff718039fc0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.649+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff73004bed0 con 0x7ff718039fc0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff7177fe700 1 -- 172.21.15.19:0/2145649418 <== mgr.14118 v2:172.21.15.19:6800/3545757717 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff73004bed0 con 0x7ff718039fc0 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff718039fc0 msgr2=0x7ff71803c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff718039fc0 0x7ff71803c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff7240039d0 tx=0x7ff724009350 comp rx=0 tx=0).stop 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 msgr2=0x7ff730113670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff730113670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff720006b00 tx=0x7ff720009250 comp rx=0 tx=0).stop 2024-02-22T15:40:38.410 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 shutdown_connections 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff718039fc0 0x7ff71803c3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff730077bc0 0x7ff730113670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 --2- 172.21.15.19:0/2145649418 >> [v2:172.21.15.19:6800/2711018613,v1:172.21.15.19:6801/2711018613] conn(0x7ff7180367e0 0x7ff718038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 >> 172.21.15.19:0/2145649418 conn(0x7ff7300fddc0 msgr2=0x7ff7300ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 shutdown_connections 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:37.650+0000 7ff735366700 1 -- 172.21.15.19:0/2145649418 wait complete. 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-02-22T15:40:38.411 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-02-22T15:40:38.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:38 smithi019 ceph-mon[67418]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-22T15:40:38.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:38 smithi019 ceph-mon[67418]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-22T15:40:38.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:38 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:38.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:38 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.960+0000 7f8cf8b83700 1 Processor -- start 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.960+0000 7f8cf8b83700 1 -- start start 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf8b83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4101980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf8b83700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4101ec0 con 0x7f8cf4101570 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44192/0 (socket says 172.21.15.19:44192) 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf259c700 1 -- 172.21.15.19:0/4058124108 learned_addr learned my addr 172.21.15.19:0/4058124108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:39.812 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf259c700 1 -- 172.21.15.19:0/4058124108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf4102000 con 0x7f8cf4101570 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf259c700 1 --2- 172.21.15.19:0/4058124108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4101980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bb80 tx=0x7f8ce400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281cf760eaf6d17c server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf159a700 1 -- 172.21.15.19:0/4058124108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce400dd00 con 0x7f8cf4101570 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf159a700 1 -- 172.21.15.19:0/4058124108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8ce400de60 con 0x7f8cf4101570 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.961+0000 7f8cf159a700 1 -- 172.21.15.19:0/4058124108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce4018840 con 0x7f8cf4101570 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 -- 172.21.15.19:0/4058124108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 msgr2=0x7f8cf4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 --2- 172.21.15.19:0/4058124108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4101980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bb80 tx=0x7f8ce400b4f0 comp rx=0 tx=0).stop 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 -- 172.21.15.19:0/4058124108 shutdown_connections 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 --2- 172.21.15.19:0/4058124108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4101980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 -- 172.21.15.19:0/4058124108 >> 172.21.15.19:0/4058124108 conn(0x7f8cf40fccd0 msgr2=0x7f8cf40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 -- 172.21.15.19:0/4058124108 shutdown_connections 2024-02-22T15:40:39.813 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.962+0000 7f8cf8b83700 1 -- 172.21.15.19:0/4058124108 wait complete. 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf8b83700 1 Processor -- start 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf8b83700 1 -- start start 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf8b83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4113880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf8b83700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4113dc0 con 0x7f8cf4101570 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4113880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4113880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44202/0 (socket says 172.21.15.19:44202) 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf259c700 1 -- 172.21.15.19:0/2288160906 learned_addr learned my addr 172.21.15.19:0/2288160906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf259c700 1 -- 172.21.15.19:0/2288160906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce400b790 con 0x7f8cf4101570 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf259c700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4113880 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400ddd0 tx=0x7f8ce400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce4010a00 con 0x7f8cf4101570 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf4113fc0 con 0x7f8cf4101570 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8ce4010b60 con 0x7f8cf4101570 2024-02-22T15:40:39.814 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ce4021550 con 0x7f8cf4101570 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.963+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41143d0 con 0x7f8cf4101570 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.964+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f8ce40223f0 con 0x7f8cf4101570 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.964+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf404bed0 con 0x7f8cf4101570 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.965+0000 7f8ce2ffd700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f8cdc036660 0x7f8cdc038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.965+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8ce4014070 con 0x7f8cf4101570 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.965+0000 7f8cf1d9b700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f8cdc036660 0x7f8cdc038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.965+0000 7f8cf1d9b700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f8cdc036660 0x7f8cdc038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009990 tx=0x7f8ce8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:38.970+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8ce4021b40 con 0x7f8cf4101570 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.177+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cf4108bf0 con 0x7f8cdc036660 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.182+0000 7f8ce2ffd700 1 -- 172.21.15.19:0/2288160906 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8cf4108bf0 con 0x7f8cdc036660 2024-02-22T15:40:39.815 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f8cdc036660 msgr2=0x7f8cdc038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f8cdc036660 0x7f8cdc038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009990 tx=0x7f8ce8006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 msgr2=0x7f8cf4113880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4113880 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400ddd0 tx=0x7f8ce400b4f0 comp rx=0 tx=0).stop 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 shutdown_connections 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f8cdc036660 0x7f8cdc038b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 --2- 172.21.15.19:0/2288160906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf4101570 0x7f8cf4113880 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 >> 172.21.15.19:0/2288160906 conn(0x7f8cf40fccd0 msgr2=0x7f8cf41062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 shutdown_connections 2024-02-22T15:40:39.816 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:39.184+0000 7f8cf8b83700 1 -- 172.21.15.19:0/2288160906 wait complete. 2024-02-22T15:40:40.079 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:39 smithi019 ceph-mon[67418]: [22/Feb/2024:15:40:38] ENGINE Bus STARTING 2024-02-22T15:40:40.079 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:39 smithi019 ceph-mon[67418]: [22/Feb/2024:15:40:38] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:40:40.079 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:39 smithi019 ceph-mon[67418]: [22/Feb/2024:15:40:38] ENGINE Bus STARTED 2024-02-22T15:40:40.079 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:39 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:40.079 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:39 smithi019 ceph-mon[67418]: mgrmap e8: smithi019.yvjrkp(active, since 2s) 2024-02-22T15:40:40.079 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:39 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:40:40.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:40 smithi019 ceph-mon[67418]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:41.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-22T15:40:41.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05ebc0700 1 Processor -- start 2024-02-22T15:40:41.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05ebc0700 1 -- start start 2024-02-22T15:40:41.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05ebc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058101970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:41.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05ebc0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058101eb0 con 0x7fc058101560 2024-02-22T15:40:41.310 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05c95c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05c95c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44208/0 (socket says 172.21.15.19:44208) 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.470+0000 7fc05c95c700 1 -- 172.21.15.19:0/4116730475 learned_addr learned my addr 172.21.15.19:0/4116730475 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.471+0000 7fc05c95c700 1 -- 172.21.15.19:0/4116730475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058101ff0 con 0x7fc058101560 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.471+0000 7fc05c95c700 1 --2- 172.21.15.19:0/4116730475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058101970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00bb80 tx=0x7fc04c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09f77d1486beb7c server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.471+0000 7fc0577fe700 1 -- 172.21.15.19:0/4116730475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c00dd00 con 0x7fc058101560 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.471+0000 7fc0577fe700 1 -- 172.21.15.19:0/4116730475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fc04c00de60 con 0x7fc058101560 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.471+0000 7fc0577fe700 1 -- 172.21.15.19:0/4116730475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c018840 con 0x7fc058101560 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.471+0000 7fc05ebc0700 1 -- 172.21.15.19:0/4116730475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 msgr2=0x7fc058101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 --2- 172.21.15.19:0/4116730475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058101970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00bb80 tx=0x7fc04c00b4f0 comp rx=0 tx=0).stop 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 -- 172.21.15.19:0/4116730475 shutdown_connections 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 --2- 172.21.15.19:0/4116730475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058101970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:41.311 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 -- 172.21.15.19:0/4116730475 >> 172.21.15.19:0/4116730475 conn(0x7fc0580fccc0 msgr2=0x7fc0580ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 -- 172.21.15.19:0/4116730475 shutdown_connections 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 -- 172.21.15.19:0/4116730475 wait complete. 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 Processor -- start 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.472+0000 7fc05ebc0700 1 -- start start 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05ebc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058113590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05ebc0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058113ad0 con 0x7fc058101560 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05c95c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05c95c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058113590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44218/0 (socket says 172.21.15.19:44218) 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05c95c700 1 -- 172.21.15.19:0/1376169022 learned_addr learned my addr 172.21.15.19:0/1376169022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05c95c700 1 -- 172.21.15.19:0/1376169022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c00b790 con 0x7fc058101560 2024-02-22T15:40:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05c95c700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058113590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0106d0 tx=0x7fc04c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c00dc70 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fc04c010c20 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc058113cd0 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc04c021550 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.473+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0581140e0 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.474+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc04c0217c0 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.474+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05804bed0 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.474+0000 7fc0557fa700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fc0480367e0 0x7fc048038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.475+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc04c014070 con 0x7fc058101560 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.475+0000 7fc057fff700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fc0480367e0 0x7fc048038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.475+0000 7fc057fff700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fc0480367e0 0x7fc048038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc044009990 tx=0x7fc044006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:41.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.479+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc04c00e540 con 0x7fc058101560 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.674+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc058108890 con 0x7fc0480367e0 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.675+0000 7fc0557fa700 1 -- 172.21.15.19:0/1376169022 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc058108890 con 0x7fc0480367e0 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.676+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fc0480367e0 msgr2=0x7fc048038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fc0480367e0 0x7fc048038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc044009990 tx=0x7fc044006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 msgr2=0x7fc058113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058113590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0106d0 tx=0x7fc04c00b4f0 comp rx=0 tx=0).stop 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 shutdown_connections 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fc0480367e0 0x7fc048038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 --2- 172.21.15.19:0/1376169022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc058101560 0x7fc058113590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 >> 172.21.15.19:0/1376169022 conn(0x7fc0580fccc0 msgr2=0x7fc0580ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 shutdown_connections 2024-02-22T15:40:41.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:40.677+0000 7fc05ebc0700 1 -- 172.21.15.19:0/1376169022 wait complete. 2024-02-22T15:40:41.315 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-02-22T15:40:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:41 smithi019 ceph-mon[67418]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:42.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.929+0000 7ff2635d7700 1 Processor -- start 2024-02-22T15:40:42.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.929+0000 7ff2635d7700 1 -- start start 2024-02-22T15:40:42.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.929+0000 7ff2635d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c102a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:42.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.929+0000 7ff2635d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c102fb0 con 0x7ff25c102660 2024-02-22T15:40:42.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff261373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:42.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff261373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44220/0 (socket says 172.21.15.19:44220) 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff261373700 1 -- 172.21.15.19:0/2294173705 learned_addr learned my addr 172.21.15.19:0/2294173705 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff261373700 1 -- 172.21.15.19:0/2294173705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c1030f0 con 0x7ff25c102660 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff261373700 1 --2- 172.21.15.19:0/2294173705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c102a70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098e0 tx=0x7ff24c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9e5bc3e4be2d6a server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff253fff700 1 -- 172.21.15.19:0/2294173705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff24c00b860 con 0x7ff25c102660 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff253fff700 1 -- 172.21.15.19:0/2294173705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff24c00be30 con 0x7ff25c102660 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.930+0000 7ff253fff700 1 -- 172.21.15.19:0/2294173705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff24c010380 con 0x7ff25c102660 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 -- 172.21.15.19:0/2294173705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 msgr2=0x7ff25c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 --2- 172.21.15.19:0/2294173705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c102a70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098e0 tx=0x7ff24c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 -- 172.21.15.19:0/2294173705 shutdown_connections 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 --2- 172.21.15.19:0/2294173705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c102a70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 -- 172.21.15.19:0/2294173705 >> 172.21.15.19:0/2294173705 conn(0x7ff25c0fde00 msgr2=0x7ff25c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 -- 172.21.15.19:0/2294173705 shutdown_connections 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 -- 172.21.15.19:0/2294173705 wait complete. 2024-02-22T15:40:42.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.931+0000 7ff2635d7700 1 Processor -- start 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff2635d7700 1 -- start start 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff2635d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c1113c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff2635d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c111900 con 0x7ff25c102660 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff261373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c1113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff261373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c1113c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44222/0 (socket says 172.21.15.19:44222) 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff261373700 1 -- 172.21.15.19:0/57310912 learned_addr learned my addr 172.21.15.19:0/57310912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff261373700 1 -- 172.21.15.19:0/57310912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0094f0 con 0x7ff25c102660 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff261373700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c1113c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff24c000c00 tx=0x7ff24c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.932+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff24c010a80 con 0x7ff25c102660 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.933+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff24c01a430 con 0x7ff25c102660 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.933+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff24c0245b0 con 0x7ff25c102660 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.933+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c111b00 con 0x7ff25c102660 2024-02-22T15:40:42.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.933+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c111f10 con 0x7ff25c102660 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.934+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff24c010be0 con 0x7ff25c102660 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.934+0000 7ff251ffb700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff2480367e0 0x7ff248038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.934+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c04bed0 con 0x7ff25c102660 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.934+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff24c04a7f0 con 0x7ff25c102660 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.934+0000 7ff260b72700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff2480367e0 0x7ff248038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.934+0000 7ff260b72700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff2480367e0 0x7ff248038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff258009990 tx=0x7ff258006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:41.939+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff24c024820 con 0x7ff25c102660 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.130+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff25c0ff2a0 con 0x7ff2480367e0 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.358+0000 7ff251ffb700 1 -- 172.21.15.19:0/57310912 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff25c0ff2a0 con 0x7ff2480367e0 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff2480367e0 msgr2=0x7ff248038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff2480367e0 0x7ff248038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff258009990 tx=0x7ff258006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 msgr2=0x7ff25c1113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c1113c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff24c000c00 tx=0x7ff24c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:42.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 shutdown_connections 2024-02-22T15:40:42.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff2480367e0 0x7ff248038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:42.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 --2- 172.21.15.19:0/57310912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c102660 0x7ff25c1113c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:42.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 >> 172.21.15.19:0/57310912 conn(0x7ff25c0fde00 msgr2=0x7ff25c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:42.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 shutdown_connections 2024-02-22T15:40:42.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:42.360+0000 7ff2635d7700 1 -- 172.21.15.19:0/57310912 wait complete. 2024-02-22T15:40:43.692 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:43 smithi019 ceph-mon[67418]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:43.692 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:43 smithi019 ceph-mon[67418]: Generating ssh key... 2024-02-22T15:40:43.692 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:43 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:43.693 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:43 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:44.363 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0tuIQPc1Lg/vCpdDI9qV7uFceJ+ydA6fS3FyH8TgfeSxq4VHzAcoWyeeLYGsisNiv7lfE2L94B960dsOwJyagto9V7qAfsTln9aRzWgaW9avSrVUOugjSauHZ0uj9KfgEuCnDR3euFVnOALOQfS7Bx7YirocGGgTVmz1vaenXXE9x+Cgs6Y1K3AwKZYTeU9BkmYjc96i5CDM0VpE9xUnfP4b9Vs08Qig+nYWpz1GS998D4aGKsLVCup0VqFlqsDdS2FxS2VTNkdqH/d/Xfn0w/3bSITiPedFO5q7C5kaZdpbi4Ux4uzGhzXy1kG7FdAh5iHiW2EEijm4AVaAKHMiPm3NiJ+RT8/acbmxXIREx6B4A/pSlBX409nr+UuZNK9iyaRnGlvQdWQUCfguoVPTBHoAGYVL6EAtqN7uH2PMxgZB6C++bWtkm95nEs4SpYmT9NlV9bKA3DXNlIgmrvczqBWNFh2QRLFkI3f7TzxADb2Xlb4DCETIDGh690oR0cac= ceph-8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.474+0000 7fd1fd90d700 1 Processor -- start 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.474+0000 7fd1fd90d700 1 -- start start 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.474+0000 7fd1fd90d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f81076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.474+0000 7fd1fd90d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8103090 con 0x7fd1f80712f0 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f81076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f81076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44226/0 (socket says 172.21.15.19:44226) 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f6ffd700 1 -- 172.21.15.19:0/1498537328 learned_addr learned my addr 172.21.15.19:0/1498537328 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f6ffd700 1 -- 172.21.15.19:0/1498537328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8107be0 con 0x7fd1f80712f0 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f6ffd700 1 --2- 172.21.15.19:0/1498537328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f81076a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0098e0 tx=0x7fd1ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2318d338d9439f21 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f5ffb700 1 -- 172.21.15.19:0/1498537328 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1ec00b860 con 0x7fd1f80712f0 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f5ffb700 1 -- 172.21.15.19:0/1498537328 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd1ec00be30 con 0x7fd1f80712f0 2024-02-22T15:40:44.364 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.475+0000 7fd1f5ffb700 1 -- 172.21.15.19:0/1498537328 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1ec010380 con 0x7fd1f80712f0 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 -- 172.21.15.19:0/1498537328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 msgr2=0x7fd1f81076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 --2- 172.21.15.19:0/1498537328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f81076a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0098e0 tx=0x7fd1ec009250 comp rx=0 tx=0).stop 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 -- 172.21.15.19:0/1498537328 shutdown_connections 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 --2- 172.21.15.19:0/1498537328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f81076a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 -- 172.21.15.19:0/1498537328 >> 172.21.15.19:0/1498537328 conn(0x7fd1f80fccd0 msgr2=0x7fd1f80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 -- 172.21.15.19:0/1498537328 shutdown_connections 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.476+0000 7fd1fd90d700 1 -- 172.21.15.19:0/1498537328 wait complete. 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1fd90d700 1 Processor -- start 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1fd90d700 1 -- start start 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1fd90d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f8113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1fd90d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8113e80 con 0x7fd1f80712f0 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f8113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f8113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44240/0 (socket says 172.21.15.19:44240) 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1f6ffd700 1 -- 172.21.15.19:0/4270638388 learned_addr learned my addr 172.21.15.19:0/4270638388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:44.365 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1f6ffd700 1 -- 172.21.15.19:0/4270638388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0094f0 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1f6ffd700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f8113940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec006d20 tx=0x7fd1ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1ec0109b0 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd1ec024440 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f8114080 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.477+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1ec01a5f0 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.478+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f8114490 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.478+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd1ec01a820 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.479+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f804bed0 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.479+0000 7fd1df7fe700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd1e0036790 0x7fd1e0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.479+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd1ec04a710 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.479+0000 7fd1f67fc700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd1e0036790 0x7fd1e0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.479+0000 7fd1f67fc700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd1e0036790 0x7fd1e0038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009990 tx=0x7fd1e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.483+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd1ec018b80 con 0x7fd1f80712f0 2024-02-22T15:40:44.366 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.676+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1f8114d80 con 0x7fd1e0036790 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.677+0000 7fd1df7fe700 1 -- 172.21.15.19:0/4270638388 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd1f8114d80 con 0x7fd1e0036790 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd1e0036790 msgr2=0x7fd1e0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd1e0036790 0x7fd1e0038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009990 tx=0x7fd1e8006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 msgr2=0x7fd1f8113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f8113940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec006d20 tx=0x7fd1ec009250 comp rx=0 tx=0).stop 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 shutdown_connections 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd1e0036790 0x7fd1e0038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 --2- 172.21.15.19:0/4270638388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1f80712f0 0x7fd1f8113940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 >> 172.21.15.19:0/4270638388 conn(0x7fd1f80fccd0 msgr2=0x7fd1f80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 shutdown_connections 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:43.679+0000 7fd1fd90d700 1 -- 172.21.15.19:0/4270638388 wait complete. 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-02-22T15:40:44.367 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-02-22T15:40:45.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:44 smithi019 ceph-mon[67418]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:45.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:46 smithi019 ceph-mon[67418]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:45.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:46 smithi019 ceph-mon[67418]: Deploying cephadm binary to smithi019 2024-02-22T15:40:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-02-22T15:40:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.013+0000 7fd628ade700 1 Processor -- start 2024-02-22T15:40:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.013+0000 7fd628ade700 1 -- start start 2024-02-22T15:40:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.014+0000 7fd628ade700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd6240771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.014+0000 7fd628ade700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240790e0 con 0x7fd624078cd0 2024-02-22T15:40:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.014+0000 7fd62259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd6240771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.014+0000 7fd62259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd6240771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44256/0 (socket says 172.21.15.19:44256) 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.014+0000 7fd62259c700 1 -- 172.21.15.19:0/3298297615 learned_addr learned my addr 172.21.15.19:0/3298297615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.014+0000 7fd62259c700 1 -- 172.21.15.19:0/3298297615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624077720 con 0x7fd624078cd0 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd62259c700 1 --2- 172.21.15.19:0/3298297615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd6240771e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6180098e0 tx=0x7fd618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57fb3a50fdac82e server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd62159a700 1 -- 172.21.15.19:0/3298297615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd61800b860 con 0x7fd624078cd0 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd62159a700 1 -- 172.21.15.19:0/3298297615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd61800be30 con 0x7fd624078cd0 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd62159a700 1 -- 172.21.15.19:0/3298297615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd618010380 con 0x7fd624078cd0 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 -- 172.21.15.19:0/3298297615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 msgr2=0x7fd6240771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 --2- 172.21.15.19:0/3298297615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd6240771e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6180098e0 tx=0x7fd618009250 comp rx=0 tx=0).stop 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 -- 172.21.15.19:0/3298297615 shutdown_connections 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 --2- 172.21.15.19:0/3298297615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd6240771e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 -- 172.21.15.19:0/3298297615 >> 172.21.15.19:0/3298297615 conn(0x7fd6240fde10 msgr2=0x7fd624100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 -- 172.21.15.19:0/3298297615 shutdown_connections 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.015+0000 7fd628ade700 1 -- 172.21.15.19:0/3298297615 wait complete. 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd628ade700 1 Processor -- start 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd628ade700 1 -- start start 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd628ade700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd624113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd628ade700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624113b60 con 0x7fd624078cd0 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd62259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd624113620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd62259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd624113620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44266/0 (socket says 172.21.15.19:44266) 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd62259c700 1 -- 172.21.15.19:0/2216937743 learned_addr learned my addr 172.21.15.19:0/2216937743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.016+0000 7fd62259c700 1 -- 172.21.15.19:0/2216937743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180094f0 con 0x7fd624078cd0 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.017+0000 7fd62259c700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd624113620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6180090b0 tx=0x7fd618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.017+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd618010a80 con 0x7fd624078cd0 2024-02-22T15:40:47.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.017+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd61801a430 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.017+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd624113d60 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.017+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6180245b0 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.017+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624114170 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.018+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd61802d3f0 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.018+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62404bed0 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.018+0000 7fd612ffd700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd60c036790 0x7fd60c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.018+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd61804a4b0 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.018+0000 7fd621d9b700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd60c036790 0x7fd60c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.018+0000 7fd621d9b700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd60c036790 0x7fd60c038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd614009990 tx=0x7fd614006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.023+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd61802b6b0 con 0x7fd624078cd0 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:45.217+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) v1 -- 0x7fd624100100 con 0x7fd60c036790 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.007+0000 7fd612ffd700 1 -- 172.21.15.19:0/2216937743 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fd624100100 con 0x7fd60c036790 2024-02-22T15:40:47.684 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.009+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd60c036790 msgr2=0x7fd60c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.009+0000 7fd628ade700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd60c036790 0x7fd60c038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd614009990 tx=0x7fd614006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 msgr2=0x7fd624113620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd624113620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6180090b0 tx=0x7fd618009250 comp rx=0 tx=0).stop 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 shutdown_connections 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fd60c036790 0x7fd60c038c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 --2- 172.21.15.19:0/2216937743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd624078cd0 0x7fd624113620 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 >> 172.21.15.19:0/2216937743 conn(0x7fd6240fde10 msgr2=0x7fd6240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 shutdown_connections 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:47.010+0000 7fd628ade700 1 -- 172.21.15.19:0/2216937743 wait complete. 2024-02-22T15:40:47.685 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying mon service with default placement... 2024-02-22T15:40:48.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:48 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:48.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:48 smithi019 ceph-mon[67418]: Added host smithi019 2024-02-22T15:40:48.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:48 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:40:49.879 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-22T15:40:49.879 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.535+0000 7fb0ae2cc700 1 Processor -- start 2024-02-22T15:40:49.879 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.535+0000 7fb0ae2cc700 1 -- start start 2024-02-22T15:40:49.879 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.535+0000 7fb0ae2cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a81076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:49.879 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.535+0000 7fb0ae2cc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8103090 con 0x7fb0a80712f0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a81076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a81076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59964/0 (socket says 172.21.15.19:59964) 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a7fff700 1 -- 172.21.15.19:0/1938841445 learned_addr learned my addr 172.21.15.19:0/1938841445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a7fff700 1 -- 172.21.15.19:0/1938841445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a8107be0 con 0x7fb0a80712f0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a7fff700 1 --2- 172.21.15.19:0/1938841445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a81076a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0980098e0 tx=0x7fb098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c730f141a2d820 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a6ffd700 1 -- 172.21.15.19:0/1938841445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb09800b860 con 0x7fb0a80712f0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a6ffd700 1 -- 172.21.15.19:0/1938841445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fb09800be30 con 0x7fb0a80712f0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.536+0000 7fb0a6ffd700 1 -- 172.21.15.19:0/1938841445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb098010380 con 0x7fb0a80712f0 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/1938841445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 msgr2=0x7fb0a81076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 --2- 172.21.15.19:0/1938841445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a81076a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0980098e0 tx=0x7fb098009250 comp rx=0 tx=0).stop 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/1938841445 shutdown_connections 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 --2- 172.21.15.19:0/1938841445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a81076a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/1938841445 >> 172.21.15.19:0/1938841445 conn(0x7fb0a80fccd0 msgr2=0x7fb0a80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:49.880 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/1938841445 shutdown_connections 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.537+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/1938841445 wait complete. 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0ae2cc700 1 Processor -- start 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0ae2cc700 1 -- start start 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0ae2cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a8113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0ae2cc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8113e80 con 0x7fb0a80712f0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a8113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0a7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a8113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59974/0 (socket says 172.21.15.19:59974) 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0a7fff700 1 -- 172.21.15.19:0/2530284587 learned_addr learned my addr 172.21.15.19:0/2530284587 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0a7fff700 1 -- 172.21.15.19:0/2530284587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0980094f0 con 0x7fb0a80712f0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.538+0000 7fb0a7fff700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a8113940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0980090b0 tx=0x7fb098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.539+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb098010a80 con 0x7fb0a80712f0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.539+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fb09801a430 con 0x7fb0a80712f0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.539+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a8114080 con 0x7fb0a80712f0 2024-02-22T15:40:49.881 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.539+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0980245b0 con 0x7fb0a80712f0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.539+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8114490 con 0x7fb0a80712f0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.540+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a804bed0 con 0x7fb0a80712f0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.540+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb09801a5a0 con 0x7fb0a80712f0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.540+0000 7fb0a4ff9700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fb0900367e0 0x7fb090038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.540+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb09804a760 con 0x7fb0a80712f0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.540+0000 7fb0a77fe700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fb0900367e0 0x7fb090038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.541+0000 7fb0a77fe700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fb0900367e0 0x7fb090038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009990 tx=0x7fb09c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.545+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb0980189a0 con 0x7fb0a80712f0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.750+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0a810d170 con 0x7fb0900367e0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.754+0000 7fb0a4ff9700 1 -- 172.21.15.19:0/2530284587 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb0a810d170 con 0x7fb0900367e0 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.756+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fb0900367e0 msgr2=0x7fb090038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.756+0000 7fb0ae2cc700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fb0900367e0 0x7fb090038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009990 tx=0x7fb09c006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.756+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 msgr2=0x7fb0a8113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.756+0000 7fb0ae2cc700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a8113940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0980090b0 tx=0x7fb098009250 comp rx=0 tx=0).stop 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.757+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 shutdown_connections 2024-02-22T15:40:49.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.757+0000 7fb0ae2cc700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fb0900367e0 0x7fb090038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.757+0000 7fb0ae2cc700 1 --2- 172.21.15.19:0/2530284587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0a80712f0 0x7fb0a8113940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.757+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 >> 172.21.15.19:0/2530284587 conn(0x7fb0a80fccd0 msgr2=0x7fb0a80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.757+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 shutdown_connections 2024-02-22T15:40:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:48.757+0000 7fb0ae2cc700 1 -- 172.21.15.19:0/2530284587 wait complete. 2024-02-22T15:40:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying mgr service with default placement... 2024-02-22T15:40:49.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:49 smithi019 ceph-mon[67418]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:49.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:49 smithi019 ceph-mon[67418]: Saving service mon spec with placement count:5 2024-02-22T15:40:49.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:49 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:49.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:49 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:51.666 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:51 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:51.666 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:51 smithi019 ceph-mon[67418]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:51.666 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:51 smithi019 ceph-mon[67418]: Saving service mgr spec with placement count:2 2024-02-22T15:40:51.666 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:51 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.424+0000 7f250ea39700 1 Processor -- start 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.424+0000 7f250ea39700 1 -- start start 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f250ea39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25080760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f250ea39700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508077fd0 con 0x7f2508077bc0 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f2507fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25080760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f2507fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25080760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59990/0 (socket says 172.21.15.19:59990) 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f2507fff700 1 -- 172.21.15.19:0/1442896727 learned_addr learned my addr 172.21.15.19:0/1442896727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f2507fff700 1 -- 172.21.15.19:0/1442896727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508076610 con 0x7f2508077bc0 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.425+0000 7f2507fff700 1 --2- 172.21.15.19:0/1442896727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25080760d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24f80098e0 tx=0x7f24f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c364d46093d61e4f server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:51.899 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f2506ffd700 1 -- 172.21.15.19:0/1442896727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f800b860 con 0x7f2508077bc0 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f2506ffd700 1 -- 172.21.15.19:0/1442896727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24f800be70 con 0x7f2508077bc0 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f2506ffd700 1 -- 172.21.15.19:0/1442896727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f8010380 con 0x7f2508077bc0 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 -- 172.21.15.19:0/1442896727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 msgr2=0x7f25080760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 --2- 172.21.15.19:0/1442896727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25080760d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24f80098e0 tx=0x7f24f8009250 comp rx=0 tx=0).stop 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 -- 172.21.15.19:0/1442896727 shutdown_connections 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 --2- 172.21.15.19:0/1442896727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25080760d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 -- 172.21.15.19:0/1442896727 >> 172.21.15.19:0/1442896727 conn(0x7f25080fddc0 msgr2=0x7f25081001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 -- 172.21.15.19:0/1442896727 shutdown_connections 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.426+0000 7f250ea39700 1 -- 172.21.15.19:0/1442896727 wait complete. 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.427+0000 7f250ea39700 1 Processor -- start 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.427+0000 7f250ea39700 1 -- start start 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.427+0000 7f250ea39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25081136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.427+0000 7f250ea39700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508113bf0 con 0x7f2508077bc0 2024-02-22T15:40:51.900 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2507fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25081136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2507fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25081136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60004/0 (socket says 172.21.15.19:60004) 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2507fff700 1 -- 172.21.15.19:0/2766832968 learned_addr learned my addr 172.21.15.19:0/2766832968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2507fff700 1 -- 172.21.15.19:0/2766832968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80094f0 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2507fff700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25081136b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24f8006d40 tx=0x7f24f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f8010960 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24f801a430 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2508113df0 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24f80245b0 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.428+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2508114200 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.429+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250804bed0 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.429+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f24f802d3f0 con 0x7f2508077bc0 2024-02-22T15:40:51.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.430+0000 7f2504ff9700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f24f0036790 0x7f24f0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.430+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f24f804a4f0 con 0x7f2508077bc0 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.430+0000 7f25077fe700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f24f0036790 0x7f24f0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.430+0000 7f25077fe700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f24f0036790 0x7f24f0038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009990 tx=0x7f24fc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.434+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f24f8018b00 con 0x7f2508077bc0 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.648+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f2508100130 con 0x7f24f0036790 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.655+0000 7f2504ff9700 1 -- 172.21.15.19:0/2766832968 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2508100130 con 0x7f24f0036790 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f24f0036790 msgr2=0x7f24f0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:51.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f24f0036790 0x7f24f0038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009990 tx=0x7f24fc006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 msgr2=0x7f25081136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25081136b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24f8006d40 tx=0x7f24f8009250 comp rx=0 tx=0).stop 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 shutdown_connections 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f24f0036790 0x7f24f0038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 --2- 172.21.15.19:0/2766832968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2508077bc0 0x7f25081136b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.657+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 >> 172.21.15.19:0/2766832968 conn(0x7f25080fddc0 msgr2=0x7f25080ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.658+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 shutdown_connections 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:50.658+0000 7f250ea39700 1 -- 172.21.15.19:0/2766832968 wait complete. 2024-02-22T15:40:51.903 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying crash service with default placement... 2024-02-22T15:40:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:52 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:52 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:54.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-22T15:40:54.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.544+0000 7f5a9f743700 1 Processor -- start 2024-02-22T15:40:54.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9f743700 1 -- start start 2024-02-22T15:40:54.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9f743700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98102a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:54.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9f743700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98102fb0 con 0x7f5a98102660 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9d4df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9d4df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60018/0 (socket says 172.21.15.19:60018) 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9d4df700 1 -- 172.21.15.19:0/316691810 learned_addr learned my addr 172.21.15.19:0/316691810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9d4df700 1 -- 172.21.15.19:0/316691810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a981030f0 con 0x7f5a98102660 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.545+0000 7f5a9d4df700 1 --2- 172.21.15.19:0/316691810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98102a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a940098e0 tx=0x7f5a94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5c95630937a4fe server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a8ffff700 1 -- 172.21.15.19:0/316691810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a9400b860 con 0x7f5a98102660 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a8ffff700 1 -- 172.21.15.19:0/316691810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a9400be70 con 0x7f5a98102660 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a8ffff700 1 -- 172.21.15.19:0/316691810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a94010380 con 0x7f5a98102660 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 -- 172.21.15.19:0/316691810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 msgr2=0x7f5a98102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 --2- 172.21.15.19:0/316691810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98102a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5a940098e0 tx=0x7f5a94009250 comp rx=0 tx=0).stop 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 -- 172.21.15.19:0/316691810 shutdown_connections 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 --2- 172.21.15.19:0/316691810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98102a70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 -- 172.21.15.19:0/316691810 >> 172.21.15.19:0/316691810 conn(0x7f5a980fddc0 msgr2=0x7f5a981001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 -- 172.21.15.19:0/316691810 shutdown_connections 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.546+0000 7f5a9f743700 1 -- 172.21.15.19:0/316691810 wait complete. 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.547+0000 7f5a9f743700 1 Processor -- start 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.547+0000 7f5a9f743700 1 -- start start 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.547+0000 7f5a9f743700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98111490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.547+0000 7f5a9f743700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981119d0 con 0x7f5a98102660 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9d4df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9d4df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98111490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60026/0 (socket says 172.21.15.19:60026) 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9d4df700 1 -- 172.21.15.19:0/3908247813 learned_addr learned my addr 172.21.15.19:0/3908247813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9d4df700 1 -- 172.21.15.19:0/3908247813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940094f0 con 0x7f5a98102660 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9d4df700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98111490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a94011b60 tx=0x7f5a94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a94010850 con 0x7f5a98102660 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a94010e60 con 0x7f5a98102660 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a98111bd0 con 0x7f5a98102660 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a9401ae20 con 0x7f5a98102660 2024-02-22T15:40:54.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.548+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a98111fe0 con 0x7f5a98102660 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.549+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5a9402d3f0 con 0x7f5a98102660 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.549+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9804bed0 con 0x7f5a98102660 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.549+0000 7f5a8dffb700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f5a840367e0 0x7f5a84038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.549+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5a9404a630 con 0x7f5a98102660 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.550+0000 7f5a9ccde700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f5a840367e0 0x7f5a84038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.550+0000 7f5a9ccde700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f5a840367e0 0x7f5a84038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009990 tx=0x7f5a88006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.554+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5a9402b8e0 con 0x7f5a98102660 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.761+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a98100130 con 0x7f5a840367e0 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.766+0000 7f5a8dffb700 1 -- 172.21.15.19:0/3908247813 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5a98100130 con 0x7f5a840367e0 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f5a840367e0 msgr2=0x7f5a84038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f5a840367e0 0x7f5a84038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009990 tx=0x7f5a88006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 msgr2=0x7f5a98111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:54.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98111490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a94011b60 tx=0x7f5a94009250 comp rx=0 tx=0).stop 2024-02-22T15:40:54.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 shutdown_connections 2024-02-22T15:40:54.005 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:53 smithi019 ceph-mon[67418]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:54.005 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:53 smithi019 ceph-mon[67418]: Saving service crash spec with placement * 2024-02-22T15:40:54.005 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:53 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:54.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f5a840367e0 0x7f5a84038c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:54.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 --2- 172.21.15.19:0/3908247813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a98102660 0x7f5a98111490 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:54.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 >> 172.21.15.19:0/3908247813 conn(0x7f5a980fddc0 msgr2=0x7f5a980ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:54.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 shutdown_connections 2024-02-22T15:40:54.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:52.768+0000 7f5a9f743700 1 -- 172.21.15.19:0/3908247813 wait complete. 2024-02-22T15:40:54.006 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying prometheus service with default placement... 2024-02-22T15:40:55.610 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-22T15:40:55.610 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bdb931700 1 Processor -- start 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bdb931700 1 -- start start 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bdb931700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4102a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bdb931700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4102fb0 con 0x7f3bd4102660 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bd96cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bd96cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60032/0 (socket says 172.21.15.19:60032) 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.880+0000 7f3bd96cd700 1 -- 172.21.15.19:0/2225591569 learned_addr learned my addr 172.21.15.19:0/2225591569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.881+0000 7f3bd96cd700 1 -- 172.21.15.19:0/2225591569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd41030f0 con 0x7f3bd4102660 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.881+0000 7f3bd96cd700 1 --2- 172.21.15.19:0/2225591569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4102a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098e0 tx=0x7f3bd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cd34858d34dd8d server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.881+0000 7f3bcbfff700 1 -- 172.21.15.19:0/2225591569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bd000b860 con 0x7f3bd4102660 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.881+0000 7f3bcbfff700 1 -- 172.21.15.19:0/2225591569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3bd000be70 con 0x7f3bd4102660 2024-02-22T15:40:55.611 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.881+0000 7f3bcbfff700 1 -- 172.21.15.19:0/2225591569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bd0010380 con 0x7f3bd4102660 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 -- 172.21.15.19:0/2225591569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 msgr2=0x7f3bd4102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 --2- 172.21.15.19:0/2225591569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4102a70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00098e0 tx=0x7f3bd0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 -- 172.21.15.19:0/2225591569 shutdown_connections 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 --2- 172.21.15.19:0/2225591569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4102a70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 -- 172.21.15.19:0/2225591569 >> 172.21.15.19:0/2225591569 conn(0x7f3bd40fde00 msgr2=0x7f3bd4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 -- 172.21.15.19:0/2225591569 shutdown_connections 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 -- 172.21.15.19:0/2225591569 wait complete. 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 Processor -- start 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.882+0000 7f3bdb931700 1 -- start start 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bdb931700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4078ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bdb931700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd4077160 con 0x7f3bd4102660 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bd96cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4078ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bd96cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4078ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60036/0 (socket says 172.21.15.19:60036) 2024-02-22T15:40:55.612 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bd96cd700 1 -- 172.21.15.19:0/2651529623 learned_addr learned my addr 172.21.15.19:0/2651529623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bd96cd700 1 -- 172.21.15.19:0/2651529623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00094f0 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bd96cd700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4078ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000bac0 tx=0x7f3bd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bd00109b0 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3bd001a430 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd4077360 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.883+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bd00245b0 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.884+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd4077770 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.885+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd404bed0 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.885+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f3bd001a5a0 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.885+0000 7f3bc9ffb700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f3bc0036790 0x7f3bc0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.885+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3bd004a9e0 con 0x7f3bd4102660 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.885+0000 7f3bd8ecc700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f3bc0036790 0x7f3bc0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.886+0000 7f3bd8ecc700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f3bc0036790 0x7f3bc0038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009990 tx=0x7f3bc4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:54.890+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3bd0010b10 con 0x7f3bd4102660 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.090+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bd4077ef0 con 0x7f3bc0036790 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.094+0000 7f3bc9ffb700 1 -- 172.21.15.19:0/2651529623 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f3bd4077ef0 con 0x7f3bc0036790 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f3bc0036790 msgr2=0x7f3bc0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f3bc0036790 0x7f3bc0038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009990 tx=0x7f3bc4006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 msgr2=0x7f3bd4078ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4078ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000bac0 tx=0x7f3bd0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 shutdown_connections 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f3bc0036790 0x7f3bc0038c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 --2- 172.21.15.19:0/2651529623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102660 0x7f3bd4078ce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.096+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 >> 172.21.15.19:0/2651529623 conn(0x7f3bd40fde00 msgr2=0x7f3bd40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.097+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 shutdown_connections 2024-02-22T15:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:55.097+0000 7f3bdb931700 1 -- 172.21.15.19:0/2651529623 wait complete. 2024-02-22T15:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying grafana service with default placement... 2024-02-22T15:40:56.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:56 smithi019 ceph-mon[67418]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:56.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:56 smithi019 ceph-mon[67418]: Saving service prometheus spec with placement count:1 2024-02-22T15:40:56.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:56 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.169+0000 7f2e845d7700 1 Processor -- start 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e845d7700 1 -- start start 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e845d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c101970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e845d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c101eb0 con 0x7f2e7c101560 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e82373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e82373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60052/0 (socket says 172.21.15.19:60052) 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e82373700 1 -- 172.21.15.19:0/417697804 learned_addr learned my addr 172.21.15.19:0/417697804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.170+0000 7f2e82373700 1 -- 172.21.15.19:0/417697804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c101ff0 con 0x7f2e7c101560 2024-02-22T15:40:56.958 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e82373700 1 --2- 172.21.15.19:0/417697804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c101970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0098e0 tx=0x7f2e6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbddbf82471cf7b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e81371700 1 -- 172.21.15.19:0/417697804 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e6c00b860 con 0x7f2e7c101560 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e81371700 1 -- 172.21.15.19:0/417697804 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e6c00be70 con 0x7f2e7c101560 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e81371700 1 -- 172.21.15.19:0/417697804 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e6c010380 con 0x7f2e7c101560 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 -- 172.21.15.19:0/417697804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 msgr2=0x7f2e7c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 --2- 172.21.15.19:0/417697804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c101970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0098e0 tx=0x7f2e6c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 -- 172.21.15.19:0/417697804 shutdown_connections 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 --2- 172.21.15.19:0/417697804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c101970 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 -- 172.21.15.19:0/417697804 >> 172.21.15.19:0/417697804 conn(0x7f2e7c0fcd00 msgr2=0x7f2e7c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 -- 172.21.15.19:0/417697804 shutdown_connections 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.171+0000 7f2e845d7700 1 -- 172.21.15.19:0/417697804 wait complete. 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.172+0000 7f2e845d7700 1 Processor -- start 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.172+0000 7f2e845d7700 1 -- start start 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.172+0000 7f2e845d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.172+0000 7f2e845d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c110840 con 0x7f2e7c101560 2024-02-22T15:40:56.959 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e82373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e82373700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c110300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60056/0 (socket says 172.21.15.19:60056) 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e82373700 1 -- 172.21.15.19:0/326830732 learned_addr learned my addr 172.21.15.19:0/326830732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e82373700 1 -- 172.21.15.19:0/326830732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0094f0 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e82373700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c110300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00bac0 tx=0x7f2e6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e6c0109b0 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e6c01a430 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7c110a40 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e6c0245b0 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.173+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7c110e50 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.174+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2e6c010b10 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.174+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c04bed0 con 0x7f2e7c101560 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.174+0000 7f2e72ffd700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f2e68036790 0x7f2e68038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:56.960 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.174+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2e6c04a730 con 0x7f2e7c101560 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.174+0000 7f2e81b72700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f2e68036790 0x7f2e68038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.175+0000 7f2e81b72700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f2e68036790 0x7f2e68038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009990 tx=0x7f2e78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.179+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2e6c01a9f0 con 0x7f2e7c101560 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.381+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e7c0fe1a0 con 0x7f2e68036790 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.385+0000 7f2e72ffd700 1 -- 172.21.15.19:0/326830732 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2e7c0fe1a0 con 0x7f2e68036790 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f2e68036790 msgr2=0x7f2e68038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f2e68036790 0x7f2e68038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009990 tx=0x7f2e78006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 msgr2=0x7f2e7c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c110300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00bac0 tx=0x7f2e6c009250 comp rx=0 tx=0).stop 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 shutdown_connections 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f2e68036790 0x7f2e68038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 --2- 172.21.15.19:0/326830732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c101560 0x7f2e7c110300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 >> 172.21.15.19:0/326830732 conn(0x7f2e7c0fcd00 msgr2=0x7f2e7c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:56.961 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 shutdown_connections 2024-02-22T15:40:56.962 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:56.387+0000 7f2e845d7700 1 -- 172.21.15.19:0/326830732 wait complete. 2024-02-22T15:40:56.962 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying node-exporter service with default placement... 2024-02-22T15:40:57.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:57 smithi019 ceph-mon[67418]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:57.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:57 smithi019 ceph-mon[67418]: Saving service grafana spec with placement count:1 2024-02-22T15:40:57.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:57 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:57.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:57 smithi019 ceph-mon[67418]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:40:58.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-22T15:40:58.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.540+0000 7f1ddc31c700 1 Processor -- start 2024-02-22T15:40:58.389 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1ddc31c700 1 -- start start 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1ddc31c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd41076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1ddc31c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4103090 con 0x7f1dd40712f0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1dda0b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd41076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1dda0b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd41076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53600/0 (socket says 172.21.15.19:53600) 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1dda0b8700 1 -- 172.21.15.19:0/2906124356 learned_addr learned my addr 172.21.15.19:0/2906124356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1dda0b8700 1 -- 172.21.15.19:0/2906124356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4107be0 con 0x7f1dd40712f0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.541+0000 7f1dda0b8700 1 --2- 172.21.15.19:0/2906124356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd41076a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00098e0 tx=0x7f1dd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5823431185bf721 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1dd90b6700 1 -- 172.21.15.19:0/2906124356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dd000b860 con 0x7f1dd40712f0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1dd90b6700 1 -- 172.21.15.19:0/2906124356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1dd000be70 con 0x7f1dd40712f0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1dd90b6700 1 -- 172.21.15.19:0/2906124356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dd0010380 con 0x7f1dd40712f0 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2906124356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 msgr2=0x7f1dd41076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 --2- 172.21.15.19:0/2906124356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd41076a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00098e0 tx=0x7f1dd0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2906124356 shutdown_connections 2024-02-22T15:40:58.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 --2- 172.21.15.19:0/2906124356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd41076a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2906124356 >> 172.21.15.19:0/2906124356 conn(0x7f1dd40fccd0 msgr2=0x7f1dd40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2906124356 shutdown_connections 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.542+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2906124356 wait complete. 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.543+0000 7f1ddc31c700 1 Processor -- start 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.543+0000 7f1ddc31c700 1 -- start start 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.543+0000 7f1ddc31c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.543+0000 7f1ddc31c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4113ec0 con 0x7f1dd40712f0 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dda0b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dda0b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd4113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53608/0 (socket says 172.21.15.19:53608) 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dda0b8700 1 -- 172.21.15.19:0/2190720257 learned_addr learned my addr 172.21.15.19:0/2190720257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dda0b8700 1 -- 172.21.15.19:0/2190720257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00094f0 con 0x7f1dd40712f0 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dda0b8700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd4113980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000ba70 tx=0x7f1dd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dd0010960 con 0x7f1dd40712f0 2024-02-22T15:40:58.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1dd0024430 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd41140c0 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dd001a5f0 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.544+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41144d0 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.545+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1dd00249f0 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.545+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd404bed0 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.545+0000 7f1dcaffd700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f1dc00367e0 0x7f1dc0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.545+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1dd004a450 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.545+0000 7f1dd98b7700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f1dc00367e0 0x7f1dc0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.546+0000 7f1dd98b7700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f1dc00367e0 0x7f1dc0038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009990 tx=0x7f1dc4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.550+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1dd0022920 con 0x7f1dd40712f0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.758+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd410d1b0 con 0x7f1dc00367e0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.762+0000 7f1dcaffd700 1 -- 172.21.15.19:0/2190720257 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1dd410d1b0 con 0x7f1dc00367e0 2024-02-22T15:40:58.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f1dc00367e0 msgr2=0x7f1dc0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f1dc00367e0 0x7f1dc0038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009990 tx=0x7f1dc4006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 msgr2=0x7f1dd4113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd4113980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000ba70 tx=0x7f1dd0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 shutdown_connections 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f1dc00367e0 0x7f1dc0038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 --2- 172.21.15.19:0/2190720257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1dd40712f0 0x7f1dd4113980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 >> 172.21.15.19:0/2190720257 conn(0x7f1dd40fccd0 msgr2=0x7f1dd40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 shutdown_connections 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:57.764+0000 7f1ddc31c700 1 -- 172.21.15.19:0/2190720257 wait complete. 2024-02-22T15:40:58.393 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying alertmanager service with default placement... 2024-02-22T15:40:59.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:58 smithi019 ceph-mon[67418]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:40:59.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:58 smithi019 ceph-mon[67418]: Saving service node-exporter spec with placement * 2024-02-22T15:40:59.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:58 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:40:59.828 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.953+0000 7f0401a57700 1 Processor -- start 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.953+0000 7f0401a57700 1 -- start start 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.953+0000 7f0401a57700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc105290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.953+0000 7f0401a57700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1057d0 con 0x7f03fc076bd0 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.953+0000 7f03faffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc105290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03faffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc105290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53614/0 (socket says 172.21.15.19:53614) 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03faffd700 1 -- 172.21.15.19:0/1673385619 learned_addr learned my addr 172.21.15.19:0/1673385619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03faffd700 1 -- 172.21.15.19:0/1673385619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc105910 con 0x7f03fc076bd0 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03faffd700 1 --2- 172.21.15.19:0/1673385619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc105290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03f00098e0 tx=0x7f03f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2828b321ef6eda52 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03f9ffb700 1 -- 172.21.15.19:0/1673385619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03f000b860 con 0x7f03fc076bd0 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03f9ffb700 1 -- 172.21.15.19:0/1673385619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03f000be70 con 0x7f03fc076bd0 2024-02-22T15:40:59.829 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.954+0000 7f03f9ffb700 1 -- 172.21.15.19:0/1673385619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03f0010380 con 0x7f03fc076bd0 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 -- 172.21.15.19:0/1673385619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 msgr2=0x7f03fc105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 --2- 172.21.15.19:0/1673385619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc105290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03f00098e0 tx=0x7f03f0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 -- 172.21.15.19:0/1673385619 shutdown_connections 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 --2- 172.21.15.19:0/1673385619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc105290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 -- 172.21.15.19:0/1673385619 >> 172.21.15.19:0/1673385619 conn(0x7f03fc0fccc0 msgr2=0x7f03fc0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 -- 172.21.15.19:0/1673385619 shutdown_connections 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 -- 172.21.15.19:0/1673385619 wait complete. 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 Processor -- start 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.955+0000 7f0401a57700 1 -- start start 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f0401a57700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc111460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f0401a57700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1119a0 con 0x7f03fc076bd0 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03faffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc111460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03faffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc111460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53622/0 (socket says 172.21.15.19:53622) 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03faffd700 1 -- 172.21.15.19:0/791077554 learned_addr learned my addr 172.21.15.19:0/791077554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:40:59.830 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03faffd700 1 -- 172.21.15.19:0/791077554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00094f0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03faffd700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc111460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f03f00094c0 tx=0x7f03f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03f0010850 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03f0010e60 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc111ba0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03f00243f0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.956+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc111fb0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.957+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f03f00246b0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.957+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc04bed0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.957+0000 7f03e37fe700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f03e4036790 0x7f03e4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.957+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f03f004a7a0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.957+0000 7f03fa7fc700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f03e4036790 0x7f03e4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.958+0000 7f03fa7fc700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f03e4036790 0x7f03e4038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009990 tx=0x7f03ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:58.962+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f03f00189a0 con 0x7f03fc076bd0 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.164+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f03fc10b360 con 0x7f03e4036790 2024-02-22T15:40:59.831 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.171+0000 7f03e37fe700 1 -- 172.21.15.19:0/791077554 <== mgr.14118 v2:172.21.15.19:6800/3545757717 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f03fc10b360 con 0x7f03e4036790 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f03e4036790 msgr2=0x7f03e4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f03e4036790 0x7f03e4038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009990 tx=0x7f03ec006e40 comp rx=0 tx=0).stop 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 msgr2=0x7f03fc111460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc111460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f03f00094c0 tx=0x7f03f0009250 comp rx=0 tx=0).stop 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 shutdown_connections 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f03e4036790 0x7f03e4038c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 --2- 172.21.15.19:0/791077554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03fc076bd0 0x7f03fc111460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 >> 172.21.15.19:0/791077554 conn(0x7f03fc0fccc0 msgr2=0x7f03fc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 shutdown_connections 2024-02-22T15:40:59.832 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:40:59.173+0000 7f0401a57700 1 -- 172.21.15.19:0/791077554 wait complete. 2024-02-22T15:41:00.089 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:59 smithi019 ceph-mon[67418]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:00.089 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:40:59 smithi019 ceph-mon[67418]: from='mgr.14118 172.21.15.19:0/2275048261' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:01.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff9664a2700 1 Processor -- start 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff9664a2700 1 -- start start 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff9664a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960102f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff9664a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9601034b0 con 0x7ff960102b60 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53626/0 (socket says 172.21.15.19:53626) 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.403+0000 7ff95ffff700 1 -- 172.21.15.19:0/3619234360 learned_addr learned my addr 172.21.15.19:0/3619234360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.404+0000 7ff95ffff700 1 -- 172.21.15.19:0/3619234360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9601035f0 con 0x7ff960102b60 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.404+0000 7ff95ffff700 1 --2- 172.21.15.19:0/3619234360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960102f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098e0 tx=0x7ff950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448f76dabb4b7795 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.404+0000 7ff95effd700 1 -- 172.21.15.19:0/3619234360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff95000b860 con 0x7ff960102b60 2024-02-22T15:41:01.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.404+0000 7ff95effd700 1 -- 172.21.15.19:0/3619234360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff95000be70 con 0x7ff960102b60 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.404+0000 7ff95effd700 1 -- 172.21.15.19:0/3619234360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff950010380 con 0x7ff960102b60 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 -- 172.21.15.19:0/3619234360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 msgr2=0x7ff960102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 --2- 172.21.15.19:0/3619234360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960102f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098e0 tx=0x7ff950009250 comp rx=0 tx=0).stop 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 -- 172.21.15.19:0/3619234360 shutdown_connections 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 --2- 172.21.15.19:0/3619234360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960102f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 -- 172.21.15.19:0/3619234360 >> 172.21.15.19:0/3619234360 conn(0x7ff9600fc460 msgr2=0x7ff9600fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 -- 172.21.15.19:0/3619234360 shutdown_connections 2024-02-22T15:41:01.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 -- 172.21.15.19:0/3619234360 wait complete. 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.405+0000 7ff9664a2700 1 Processor -- start 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff9664a2700 1 -- start start 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff9664a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960111490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff9664a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9601119d0 con 0x7ff960102b60 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960111490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53628/0 (socket says 172.21.15.19:53628) 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff95ffff700 1 -- 172.21.15.19:0/3574945223 learned_addr learned my addr 172.21.15.19:0/3574945223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff95ffff700 1 -- 172.21.15.19:0/3574945223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500094f0 con 0x7ff960102b60 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff95ffff700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960111490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff95000c010 tx=0x7ff950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.406+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff950010960 con 0x7ff960102b60 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.407+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff960111bd0 con 0x7ff960102b60 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.407+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff95001a430 con 0x7ff960102b60 2024-02-22T15:41:01.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.407+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff950019580 con 0x7ff960102b60 2024-02-22T15:41:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.407+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff960111fe0 con 0x7ff960102b60 2024-02-22T15:41:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.407+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff950019800 con 0x7ff960102b60 2024-02-22T15:41:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.408+0000 7ff95cff9700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff948036790 0x7ff948038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.408+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff95004a580 con 0x7ff960102b60 2024-02-22T15:41:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.408+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96004bed0 con 0x7ff960102b60 2024-02-22T15:41:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.408+0000 7ff95f7fe700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff948036790 0x7ff948038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:01.171 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:00 smithi019 ceph-mon[67418]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:41:01.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:00 smithi019 ceph-mon[67418]: Saving service alertmanager spec with placement count:1 2024-02-22T15:41:01.172 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:00 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3574945223' entity='client.admin' 2024-02-22T15:41:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.408+0000 7ff95f7fe700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff948036790 0x7ff948038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff954009990 tx=0x7ff954006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.412+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff95002bd60 con 0x7ff960102b60 2024-02-22T15:41:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.602+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff960102390 con 0x7ff960102b60 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.605+0000 7ff95cff9700 1 -- 172.21.15.19:0/3574945223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff950029e60 con 0x7ff960102b60 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff948036790 msgr2=0x7ff948038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff948036790 0x7ff948038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff954009990 tx=0x7ff954006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 msgr2=0x7ff960111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960111490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff95000c010 tx=0x7ff950009250 comp rx=0 tx=0).stop 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 shutdown_connections 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7ff948036790 0x7ff948038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 --2- 172.21.15.19:0/3574945223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff960102b60 0x7ff960111490 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.607+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 >> 172.21.15.19:0/3574945223 conn(0x7ff9600fc460 msgr2=0x7ff960101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.608+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 shutdown_connections 2024-02-22T15:41:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:00.608+0000 7ff9664a2700 1 -- 172.21.15.19:0/3574945223 wait complete. 2024-02-22T15:41:02.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:01 smithi019 ceph-mon[67418]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe1331e1700 1 Processor -- start 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe1331e1700 1 -- start start 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe1331e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c1040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe1331e1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c1045f0 con 0x7fe12c103ca0 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe130f7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c1040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe130f7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c1040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53638/0 (socket says 172.21.15.19:53638) 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.763+0000 7fe130f7d700 1 -- 172.21.15.19:0/3224874735 learned_addr learned my addr 172.21.15.19:0/3224874735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.764+0000 7fe130f7d700 1 -- 172.21.15.19:0/3224874735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c104730 con 0x7fe12c103ca0 2024-02-22T15:41:02.525 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.764+0000 7fe130f7d700 1 --2- 172.21.15.19:0/3224874735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c1040b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1200098e0 tx=0x7fe120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a26810da3439d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.764+0000 7fe12b7fe700 1 -- 172.21.15.19:0/3224874735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe12000b860 con 0x7fe12c103ca0 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.764+0000 7fe12b7fe700 1 -- 172.21.15.19:0/3224874735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe12000be70 con 0x7fe12c103ca0 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.764+0000 7fe12b7fe700 1 -- 172.21.15.19:0/3224874735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe120010380 con 0x7fe12c103ca0 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 -- 172.21.15.19:0/3224874735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 msgr2=0x7fe12c1040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 --2- 172.21.15.19:0/3224874735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c1040b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1200098e0 tx=0x7fe120009250 comp rx=0 tx=0).stop 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 -- 172.21.15.19:0/3224874735 shutdown_connections 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 --2- 172.21.15.19:0/3224874735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c1040b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 -- 172.21.15.19:0/3224874735 >> 172.21.15.19:0/3224874735 conn(0x7fe12c0fc460 msgr2=0x7fe12c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 -- 172.21.15.19:0/3224874735 shutdown_connections 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 -- 172.21.15.19:0/3224874735 wait complete. 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 Processor -- start 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.765+0000 7fe1331e1700 1 -- start start 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1331e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c111480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1331e1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c1119c0 con 0x7fe12c103ca0 2024-02-22T15:41:02.526 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe130f7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c111480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe130f7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c111480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53652/0 (socket says 172.21.15.19:53652) 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe130f7d700 1 -- 172.21.15.19:0/2482225042 learned_addr learned my addr 172.21.15.19:0/2482225042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe130f7d700 1 -- 172.21.15.19:0/2482225042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200094f0 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe130f7d700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c111480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe120011b60 tx=0x7fe120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe120010850 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c111bc0 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe120010e60 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe12001ae20 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.766+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c111fd0 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.767+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c04bed0 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.768+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe1200109b0 con 0x7fe12c103ca0 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.768+0000 7fe1297fa700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fe11c036790 0x7fe11c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:02.527 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.768+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe12004a650 con 0x7fe12c103ca0 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.768+0000 7fe12bfff700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fe11c036790 0x7fe11c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.769+0000 7fe12bfff700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fe11c036790 0x7fe11c038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe118009990 tx=0x7fe118006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.772+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe12001a6a0 con 0x7fe12c103ca0 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.962+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe12c101150 con 0x7fe12c103ca0 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.967+0000 7fe1297fa700 1 -- 172.21.15.19:0/2482225042 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe12002b2f0 con 0x7fe12c103ca0 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.969+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fe11c036790 msgr2=0x7fe11c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.969+0000 7fe1331e1700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fe11c036790 0x7fe11c038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe118009990 tx=0x7fe118006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.969+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 msgr2=0x7fe12c111480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.969+0000 7fe1331e1700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c111480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe120011b60 tx=0x7fe120009250 comp rx=0 tx=0).stop 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.970+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 shutdown_connections 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.970+0000 7fe1331e1700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7fe11c036790 0x7fe11c038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.970+0000 7fe1331e1700 1 --2- 172.21.15.19:0/2482225042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe12c103ca0 0x7fe12c111480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.970+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 >> 172.21.15.19:0/2482225042 conn(0x7fe12c0fc460 msgr2=0x7fe12c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:02.528 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.970+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 shutdown_connections 2024-02-22T15:41:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:01.970+0000 7fe1331e1700 1 -- 172.21.15.19:0/2482225042 wait complete. 2024-02-22T15:41:02.529 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-02-22T15:41:03.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:02 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2482225042' entity='client.admin' 2024-02-22T15:41:03.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:02 smithi019 ceph-mon[67418]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:04.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:03 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/672718340' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-22T15:41:04.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f8730770700 1 Processor -- start 2024-02-22T15:41:04.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f8730770700 1 -- start start 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f8730770700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f8728105240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f8730770700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8728105780 con 0x7f8728076c30 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f872e50c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f8728105240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f872e50c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f8728105240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53658/0 (socket says 172.21.15.19:53658) 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f872e50c700 1 -- 172.21.15.19:0/468208207 learned_addr learned my addr 172.21.15.19:0/468208207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.127+0000 7f872e50c700 1 -- 172.21.15.19:0/468208207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87281058c0 con 0x7f8728076c30 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f872e50c700 1 --2- 172.21.15.19:0/468208207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f8728105240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87240098e0 tx=0x7f8724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19770a0ac68ce3f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f872d50a700 1 -- 172.21.15.19:0/468208207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f872400b860 con 0x7f8728076c30 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f872d50a700 1 -- 172.21.15.19:0/468208207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f872400be70 con 0x7f8728076c30 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f872d50a700 1 -- 172.21.15.19:0/468208207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8724010380 con 0x7f8728076c30 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f8730770700 1 -- 172.21.15.19:0/468208207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 msgr2=0x7f8728105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f8730770700 1 --2- 172.21.15.19:0/468208207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f8728105240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87240098e0 tx=0x7f8724009250 comp rx=0 tx=0).stop 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f8730770700 1 -- 172.21.15.19:0/468208207 shutdown_connections 2024-02-22T15:41:04.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f8730770700 1 --2- 172.21.15.19:0/468208207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f8728105240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f8730770700 1 -- 172.21.15.19:0/468208207 >> 172.21.15.19:0/468208207 conn(0x7f87280fccd0 msgr2=0x7f87280ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.128+0000 7f8730770700 1 -- 172.21.15.19:0/468208207 shutdown_connections 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f8730770700 1 -- 172.21.15.19:0/468208207 wait complete. 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f8730770700 1 Processor -- start 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f8730770700 1 -- start start 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f8730770700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f87281114a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f8730770700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87281119e0 con 0x7f8728076c30 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f872e50c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f87281114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f872e50c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f87281114a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53674/0 (socket says 172.21.15.19:53674) 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.129+0000 7f872e50c700 1 -- 172.21.15.19:0/672718340 learned_addr learned my addr 172.21.15.19:0/672718340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f872e50c700 1 -- 172.21.15.19:0/672718340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240094f0 con 0x7f8728076c30 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f872e50c700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f87281114a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87240094c0 tx=0x7f8724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8724010850 con 0x7f8728076c30 2024-02-22T15:41:04.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8724010e60 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8728111be0 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f872401a430 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.130+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8728111ff0 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.131+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872804bed0 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.131+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f87240247b0 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.131+0000 7f871effd700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f87140367e0 0x7f8714038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.131+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f872404a5b0 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.131+0000 7f872dd0b700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f87140367e0 0x7f8714038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.132+0000 7f872dd0b700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f87140367e0 0x7f8714038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8718009990 tx=0x7f8718006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.136+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8724024440 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.382+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f872810b300 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.979+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f872402d910 con 0x7f8728076c30 2024-02-22T15:41:04.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.981+0000 7f871effd700 1 -- 172.21.15.19:0/672718340 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f872402c350 con 0x7f8728076c30 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f87140367e0 msgr2=0x7f8714038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f87140367e0 0x7f8714038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8718009990 tx=0x7f8718006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 msgr2=0x7f87281114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f87281114a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87240094c0 tx=0x7f8724009250 comp rx=0 tx=0).stop 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 shutdown_connections 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f87140367e0 0x7f8714038c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 --2- 172.21.15.19:0/672718340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8728076c30 0x7f87281114a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 >> 172.21.15.19:0/672718340 conn(0x7f87280fccd0 msgr2=0x7f87280fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 shutdown_connections 2024-02-22T15:41:04.777 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:03.984+0000 7f8730770700 1 -- 172.21.15.19:0/672718340 wait complete. 2024-02-22T15:41:05.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:04 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/672718340' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-22T15:41:05.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:04 smithi019 ceph-mon[67418]: mgrmap e9: smithi019.yvjrkp(active, since 27s) 2024-02-22T15:41:06.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:05 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2136782700' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-22T15:41:06.257 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "smithi019.yvjrkp", 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.291+0000 7f577f6ff700 1 Processor -- start 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.291+0000 7f577f6ff700 1 -- start start 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.291+0000 7f577f6ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f57781076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.291+0000 7f577f6ff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778103090 con 0x7f57780712f0 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.291+0000 7f577d49b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f57781076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f577d49b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f57781076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53688/0 (socket says 172.21.15.19:53688) 2024-02-22T15:41:06.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f577d49b700 1 -- 172.21.15.19:0/1451473353 learned_addr learned my addr 172.21.15.19:0/1451473353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f577d49b700 1 -- 172.21.15.19:0/1451473353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778107be0 con 0x7f57780712f0 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f577d49b700 1 --2- 172.21.15.19:0/1451473353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f57781076a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57740098e0 tx=0x7f5774009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5901ee47b89e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f576ffff700 1 -- 172.21.15.19:0/1451473353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f577400b860 con 0x7f57780712f0 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f576ffff700 1 -- 172.21.15.19:0/1451473353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f577400be70 con 0x7f57780712f0 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.292+0000 7f576ffff700 1 -- 172.21.15.19:0/1451473353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5774010380 con 0x7f57780712f0 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 -- 172.21.15.19:0/1451473353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 msgr2=0x7f57781076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 --2- 172.21.15.19:0/1451473353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f57781076a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57740098e0 tx=0x7f5774009250 comp rx=0 tx=0).stop 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 -- 172.21.15.19:0/1451473353 shutdown_connections 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 --2- 172.21.15.19:0/1451473353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f57781076a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 -- 172.21.15.19:0/1451473353 >> 172.21.15.19:0/1451473353 conn(0x7f57780fccd0 msgr2=0x7f57780ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 -- 172.21.15.19:0/1451473353 shutdown_connections 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.293+0000 7f577f6ff700 1 -- 172.21.15.19:0/1451473353 wait complete. 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577f6ff700 1 Processor -- start 2024-02-22T15:41:06.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577f6ff700 1 -- start start 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577f6ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f5778113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577f6ff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778113ec0 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577d49b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f5778113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577d49b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f5778113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53700/0 (socket says 172.21.15.19:53700) 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577d49b700 1 -- 172.21.15.19:0/2136782700 learned_addr learned my addr 172.21.15.19:0/2136782700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577d49b700 1 -- 172.21.15.19:0/2136782700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57740094f0 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f577d49b700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f5778113980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57740094c0 tx=0x7f5774009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.294+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5774010850 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.295+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5774010e60 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.295+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781140c0 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.295+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57740243f0 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.295+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781144d0 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.296+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577804bed0 con 0x7f57780712f0 2024-02-22T15:41:06.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.297+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f577401a430 con 0x7f57780712f0 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.297+0000 7f576dffb700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 0x7f5764038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.297+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5774053770 con 0x7f57780712f0 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.297+0000 7f577cc9a700 1 -- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 msgr2=0x7f5764038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.297+0000 7f577cc9a700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 0x7f5764038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.302+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5774050060 con 0x7f57780712f0 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.498+0000 7f577cc9a700 1 -- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 msgr2=0x7f5764038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.498+0000 7f577cc9a700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 0x7f5764038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.549+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f57781088a0 con 0x7f57780712f0 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.549+0000 7f576dffb700 1 -- 172.21.15.19:0/2136782700 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f5774050070 con 0x7f57780712f0 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 msgr2=0x7f5764038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 0x7f5764038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 msgr2=0x7f5778113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f5778113980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57740094c0 tx=0x7f5774009250 comp rx=0 tx=0).stop 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 shutdown_connections 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f57640367e0 0x7f5764038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 --2- 172.21.15.19:0/2136782700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57780712f0 0x7f5778113980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 >> 172.21.15.19:0/2136782700 conn(0x7f57780fccd0 msgr2=0x7f57780ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 shutdown_connections 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:05.551+0000 7f577f6ff700 1 -- 172.21.15.19:0/2136782700 wait complete. 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-02-22T15:41:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-02-22T15:41:10.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: Active manager daemon smithi019.yvjrkp restarted 2024-02-22T15:41:10.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:41:10.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: osdmap e3: 0 total, 0 up, 0 in 2024-02-22T15:41:10.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: mgrmap e10: smithi019.yvjrkp(active, starting, since 0.00370153s) 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:41:10.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:10 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:41:11.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:11 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:41:11.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:11 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:41:11.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:11 smithi019 ceph-mon[67418]: mgrmap e11: smithi019.yvjrkp(active, since 1.00528s) 2024-02-22T15:41:11.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:11 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:12.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-02-22T15:41:12.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-22T15:41:12.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-22T15:41:12.441 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.848+0000 7f1214e26700 1 Processor -- start 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.848+0000 7f1214e26700 1 -- start start 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.848+0000 7f1214e26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210102a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.848+0000 7f1214e26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1210102fb0 con 0x7f1210102660 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53702/0 (socket says 172.21.15.19:53702) 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120e59c700 1 -- 172.21.15.19:0/1094357302 learned_addr learned my addr 172.21.15.19:0/1094357302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120e59c700 1 -- 172.21.15.19:0/1094357302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12101030f0 con 0x7f1210102660 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120e59c700 1 --2- 172.21.15.19:0/1094357302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210102a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f12000098e0 tx=0x7f1200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea27fb8c2586dce server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120d59a700 1 -- 172.21.15.19:0/1094357302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f120000b860 con 0x7f1210102660 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120d59a700 1 -- 172.21.15.19:0/1094357302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f120000be70 con 0x7f1210102660 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.849+0000 7f120d59a700 1 -- 172.21.15.19:0/1094357302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1200010380 con 0x7f1210102660 2024-02-22T15:41:12.442 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 -- 172.21.15.19:0/1094357302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 msgr2=0x7f1210102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 --2- 172.21.15.19:0/1094357302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210102a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f12000098e0 tx=0x7f1200009250 comp rx=0 tx=0).stop 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 -- 172.21.15.19:0/1094357302 shutdown_connections 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 --2- 172.21.15.19:0/1094357302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210102a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 -- 172.21.15.19:0/1094357302 >> 172.21.15.19:0/1094357302 conn(0x7f12100fddc0 msgr2=0x7f12101001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 -- 172.21.15.19:0/1094357302 shutdown_connections 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.850+0000 7f1214e26700 1 -- 172.21.15.19:0/1094357302 wait complete. 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f1214e26700 1 Processor -- start 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f1214e26700 1 -- start start 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f1214e26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210111490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f1214e26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101119d0 con 0x7f1210102660 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f120e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f120e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210111490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53706/0 (socket says 172.21.15.19:53706) 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f120e59c700 1 -- 172.21.15.19:0/945145027 learned_addr learned my addr 172.21.15.19:0/945145027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f120e59c700 1 -- 172.21.15.19:0/945145027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12000094f0 con 0x7f1210102660 2024-02-22T15:41:12.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.851+0000 7f120e59c700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210111490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1200011b60 tx=0x7f1200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.852+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1200010850 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.852+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1200010e60 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.852+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1210111bd0 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.852+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1210111fe0 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.852+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f120001ae20 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.853+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f12000109b0 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.853+0000 7f11feffd700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.853+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 --> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11f80393a0 con 0x7f11f80367e0 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.853+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f120004a470 con 0x7f1210102660 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.853+0000 7f120dd9b700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 msgr2=0x7f11f8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:06.853+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:07.054+0000 7f120dd9b700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 msgr2=0x7f11f8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:12.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:07.054+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:07.455+0000 7f120dd9b700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 msgr2=0x7f11f8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:07.455+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:08.256+0000 7f120dd9b700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 msgr2=0x7f11f8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:08.256+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:09.858+0000 7f120dd9b700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 msgr2=0x7f11f8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3545757717 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:09.858+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:10.805+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f120002d770 con 0x7f1210102660 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:10.805+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 msgr2=0x7f11f8038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:10.805+0000 7f11feffd700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.806+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f120004ba30 con 0x7f1210102660 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.806+0000 7f11feffd700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f11f803a070 0x7f11f803c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.806+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11f80393a0 con 0x7f11f803a070 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.807+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f11f803a070 0x7f11f803c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.807+0000 7f120dd9b700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f11f803a070 0x7f11f803c450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f12040039d0 tx=0x7f12040092f0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:12.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.808+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f11f80393a0 con 0x7f11f803a070 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f121004bed0 con 0x7f11f803a070 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f11feffd700 1 -- 172.21.15.19:0/945145027 <== mgr.14160 v2:172.21.15.19:6800/1788498096 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f121004bed0 con 0x7f11f803a070 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f11f803a070 msgr2=0x7f11f803c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f11f803a070 0x7f11f803c450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f12040039d0 tx=0x7f12040092f0 comp rx=0 tx=0).stop 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 msgr2=0x7f1210111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210111490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1200011b60 tx=0x7f1200009250 comp rx=0 tx=0).stop 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 shutdown_connections 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f11f803a070 0x7f11f803c450 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1210102660 0x7f1210111490 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 --2- 172.21.15.19:0/945145027 >> [v2:172.21.15.19:6800/3545757717,v1:172.21.15.19:6801/3545757717] conn(0x7f11f80367e0 0x7f11f8038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 >> 172.21.15.19:0/945145027 conn(0x7f12100fddc0 msgr2=0x7f12100ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 shutdown_connections 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:11.814+0000 7f1214e26700 1 -- 172.21.15.19:0/945145027 wait complete. 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-02-22T15:41:12.446 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-02-22T15:41:12.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:41:11] ENGINE Bus STARTING 2024-02-22T15:41:13.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:12 smithi019 ceph-mon[67418]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-22T15:41:13.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:12 smithi019 ceph-mon[67418]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-22T15:41:13.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:41:11] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:41:13.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:41:11] ENGINE Bus STARTED 2024-02-22T15:41:13.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:12 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.075+0000 7fb57ce56700 1 Processor -- start 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.075+0000 7fb57ce56700 1 -- start start 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.075+0000 7fb57ce56700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578105280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.075+0000 7fb57ce56700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781057c0 con 0x7fb578071310 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58920/0 (socket says 172.21.15.19:58920) 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57659c700 1 -- 172.21.15.19:0/21407563 learned_addr learned my addr 172.21.15.19:0/21407563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57659c700 1 -- 172.21.15.19:0/21407563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578105900 con 0x7fb578071310 2024-02-22T15:41:13.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57659c700 1 --2- 172.21.15.19:0/21407563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578105280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5680098e0 tx=0x7fb568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc87060e37384cf server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57559a700 1 -- 172.21.15.19:0/21407563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb56800b860 con 0x7fb578071310 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57559a700 1 -- 172.21.15.19:0/21407563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb56800be70 con 0x7fb578071310 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.076+0000 7fb57559a700 1 -- 172.21.15.19:0/21407563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb568010380 con 0x7fb578071310 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 -- 172.21.15.19:0/21407563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 msgr2=0x7fb578105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 --2- 172.21.15.19:0/21407563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578105280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5680098e0 tx=0x7fb568009250 comp rx=0 tx=0).stop 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 -- 172.21.15.19:0/21407563 shutdown_connections 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 --2- 172.21.15.19:0/21407563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578105280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 -- 172.21.15.19:0/21407563 >> 172.21.15.19:0/21407563 conn(0x7fb5780fccd0 msgr2=0x7fb5780ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 -- 172.21.15.19:0/21407563 shutdown_connections 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.077+0000 7fb57ce56700 1 -- 172.21.15.19:0/21407563 wait complete. 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57ce56700 1 Processor -- start 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57ce56700 1 -- start start 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57ce56700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578111780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:13.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57ce56700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578111cc0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578111780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578111780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58932/0 (socket says 172.21.15.19:58932) 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57659c700 1 -- 172.21.15.19:0/4256253012 learned_addr learned my addr 172.21.15.19:0/4256253012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57659c700 1 -- 172.21.15.19:0/4256253012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5680094f0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.078+0000 7fb57659c700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578111780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5680094c0 tx=0x7fb568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.079+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb568010850 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.079+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb568010e60 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.079+0000 7fb57ce56700 1 -- 172.21.15.19:0/4256253012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb578111ec0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.079+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5680243f0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.079+0000 7fb57ce56700 1 -- 172.21.15.19:0/4256253012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781122d0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.079+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fb5680109b0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.080+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c0052b0 con 0x7fb578071310 2024-02-22T15:41:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.080+0000 7fb566ffd700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb5600366b0 0x7fb560038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.080+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb56804a910 con 0x7fb578071310 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.080+0000 7fb575d9b700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb5600366b0 0x7fb560038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.080+0000 7fb575d9b700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb5600366b0 0x7fb560038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009990 tx=0x7fb56c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.085+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb568024bd0 con 0x7fb578071310 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.312+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb55c000bc0 con 0x7fb5600366b0 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.343+0000 7fb566ffd700 1 -- 172.21.15.19:0/4256253012 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb55c000bc0 con 0x7fb5600366b0 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb5600366b0 msgr2=0x7fb560038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb5600366b0 0x7fb560038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009990 tx=0x7fb56c006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 msgr2=0x7fb578111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578111780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5680094c0 tx=0x7fb568009250 comp rx=0 tx=0).stop 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 shutdown_connections 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb5600366b0 0x7fb560038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 --2- 172.21.15.19:0/4256253012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb578071310 0x7fb578111780 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 >> 172.21.15.19:0/4256253012 conn(0x7fb5780fccd0 msgr2=0x7fb5780ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:13.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 shutdown_connections 2024-02-22T15:41:14.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:13.345+0000 7fb564ff9700 1 -- 172.21.15.19:0/4256253012 wait complete. 2024-02-22T15:41:14.000 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-02-22T15:41:14.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:14 smithi019 ceph-mon[67418]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:41:14.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:14 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:14.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:14 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:14.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:14 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:14.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:14 smithi019 ceph-mon[67418]: mgrmap e12: smithi019.yvjrkp(active, since 2s) 2024-02-22T15:41:15.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$NQqaSw/BivvmnMLOqErWjOTyFgWUzJ3kZmu2owZLx8Wpgs47icjWS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708616475, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.553+0000 7f6ed054c700 1 Processor -- start 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.553+0000 7f6ed054c700 1 -- start start 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.554+0000 7f6ed054c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8102a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.554+0000 7f6ed054c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8102fc0 con 0x7f6ec8102670 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.554+0000 7f6ece2e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.554+0000 7f6ece2e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58938/0 (socket says 172.21.15.19:58938) 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.554+0000 7f6ece2e8700 1 -- 172.21.15.19:0/3708899341 learned_addr learned my addr 172.21.15.19:0/3708899341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.554+0000 7f6ece2e8700 1 -- 172.21.15.19:0/3708899341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8103100 con 0x7f6ec8102670 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.555+0000 7f6ece2e8700 1 --2- 172.21.15.19:0/3708899341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8102a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098e0 tx=0x7f6ec4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35413e1ac7a47ebb server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.555+0000 7f6ecd2e6700 1 -- 172.21.15.19:0/3708899341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ec400b860 con 0x7f6ec8102670 2024-02-22T15:41:15.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.555+0000 7f6ecd2e6700 1 -- 172.21.15.19:0/3708899341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ec400be70 con 0x7f6ec8102670 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.555+0000 7f6ecd2e6700 1 -- 172.21.15.19:0/3708899341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ec4010380 con 0x7f6ec8102670 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.555+0000 7f6ed054c700 1 -- 172.21.15.19:0/3708899341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 msgr2=0x7f6ec8102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.555+0000 7f6ed054c700 1 --2- 172.21.15.19:0/3708899341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8102a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098e0 tx=0x7f6ec4009250 comp rx=0 tx=0).stop 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.556+0000 7f6ed054c700 1 -- 172.21.15.19:0/3708899341 shutdown_connections 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.556+0000 7f6ed054c700 1 --2- 172.21.15.19:0/3708899341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8102a80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.556+0000 7f6ed054c700 1 -- 172.21.15.19:0/3708899341 >> 172.21.15.19:0/3708899341 conn(0x7f6ec80fde10 msgr2=0x7f6ec8100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.556+0000 7f6ed054c700 1 -- 172.21.15.19:0/3708899341 shutdown_connections 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.556+0000 7f6ed054c700 1 -- 172.21.15.19:0/3708899341 wait complete. 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.556+0000 7f6ed054c700 1 Processor -- start 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ed054c700 1 -- start start 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ed054c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8078ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ed054c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8077160 con 0x7f6ec8102670 2024-02-22T15:41:15.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ece2e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8078ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ece2e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8078ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58954/0 (socket says 172.21.15.19:58954) 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ece2e8700 1 -- 172.21.15.19:0/1026470447 learned_addr learned my addr 172.21.15.19:0/1026470447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ece2e8700 1 -- 172.21.15.19:0/1026470447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40094f0 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.557+0000 7f6ece2e8700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8078ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4003080 tx=0x7f6ec4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.558+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ec4010800 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.558+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8077360 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.558+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ec4010e10 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.558+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ec401a430 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.558+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec8077770 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.559+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec804bed0 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.559+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f6ec401a6b0 con 0x7f6ec8102670 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.559+0000 7f6ebeffd700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f6eb40367e0 0x7f6eb4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.559+0000 7f6ecdae7700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f6eb40367e0 0x7f6eb4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:15.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.559+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6ec4018d20 con 0x7f6ec8102670 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.560+0000 7f6ecdae7700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f6eb40367e0 0x7f6eb4038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009990 tx=0x7f6eb8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.564+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6ec4018b40 con 0x7f6ec8102670 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:14.777+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6ec805f3e0 con 0x7f6eb40367e0 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.024+0000 7f6ebeffd700 1 -- 172.21.15.19:0/1026470447 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f6ec805f3e0 con 0x7f6eb40367e0 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f6eb40367e0 msgr2=0x7f6eb4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f6eb40367e0 0x7f6eb4038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009990 tx=0x7f6eb8006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 msgr2=0x7f6ec8078ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8078ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4003080 tx=0x7f6ec4009250 comp rx=0 tx=0).stop 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 shutdown_connections 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f6eb40367e0 0x7f6eb4038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 --2- 172.21.15.19:0/1026470447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec8102670 0x7f6ec8078ce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 >> 172.21.15.19:0/1026470447 conn(0x7f6ec80fde10 msgr2=0x7f6ec80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 shutdown_connections 2024-02-22T15:41:15.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:15.026+0000 7f6ed054c700 1 -- 172.21.15.19:0/1026470447 wait complete. 2024-02-22T15:41:15.911 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-02-22T15:41:16.175 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:16 smithi019 ceph-mon[67418]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:41:16.175 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:16 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:17.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-02-22T15:41:17.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.538+0000 7f61a2ffb700 1 Processor -- start 2024-02-22T15:41:17.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.538+0000 7f61a2ffb700 1 -- start start 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.538+0000 7f61a2ffb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.538+0000 7f61a2ffb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c103600 con 0x7f619c102cb0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f61a0d97700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f61a0d97700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58962/0 (socket says 172.21.15.19:58962) 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f61a0d97700 1 -- 172.21.15.19:0/1284733640 learned_addr learned my addr 172.21.15.19:0/1284733640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f61a0d97700 1 -- 172.21.15.19:0/1284733640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c103740 con 0x7f619c102cb0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f61a0d97700 1 --2- 172.21.15.19:0/1284733640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1030c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61900098e0 tx=0x7f6190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f685ad0faa39548 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f619b7fe700 1 -- 172.21.15.19:0/1284733640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f619000b860 con 0x7f619c102cb0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f619b7fe700 1 -- 172.21.15.19:0/1284733640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f619000be70 con 0x7f619c102cb0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.539+0000 7f619b7fe700 1 -- 172.21.15.19:0/1284733640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6190010380 con 0x7f619c102cb0 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1284733640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 msgr2=0x7f619c1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 --2- 172.21.15.19:0/1284733640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1030c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61900098e0 tx=0x7f6190009250 comp rx=0 tx=0).stop 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1284733640 shutdown_connections 2024-02-22T15:41:17.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 --2- 172.21.15.19:0/1284733640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1030c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1284733640 >> 172.21.15.19:0/1284733640 conn(0x7f619c0fc460 msgr2=0x7f619c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1284733640 shutdown_connections 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1284733640 wait complete. 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 Processor -- start 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.540+0000 7f61a2ffb700 1 -- start start 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a2ffb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a2ffb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c113c00 con 0x7f619c102cb0 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a0d97700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a0d97700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58974/0 (socket says 172.21.15.19:58974) 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a0d97700 1 -- 172.21.15.19:0/1858878965 learned_addr learned my addr 172.21.15.19:0/1858878965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a0d97700 1 -- 172.21.15.19:0/1858878965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61900094f0 con 0x7f619c102cb0 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a0d97700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1136c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6190011b60 tx=0x7f6190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:17.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6190010850 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6190010e60 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f619001ae20 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c113e00 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.541+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c114210 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.542+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f619002d3f0 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.542+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c04bed0 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.543+0000 7f61997fa700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f618c036830 0x7f618c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.543+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f619004a660 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.543+0000 7f619bfff700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f618c036830 0x7f618c038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.543+0000 7f619bfff700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f618c036830 0x7f618c038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6188009990 tx=0x7f6188006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.547+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6190018b00 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.741+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f619c1088f0 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.742+0000 7f61997fa700 1 -- 172.21.15.19:0/1858878965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6190018b00 con 0x7f619c102cb0 2024-02-22T15:41:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f618c036830 msgr2=0x7f618c038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:17.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f618c036830 0x7f618c038ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6188009990 tx=0x7f6188006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:17.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 msgr2=0x7f619c1136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:17.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:17 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1858878965' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-22T15:41:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1136c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6190011b60 tx=0x7f6190009250 comp rx=0 tx=0).stop 2024-02-22T15:41:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 shutdown_connections 2024-02-22T15:41:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f618c036830 0x7f618c038ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:17.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 --2- 172.21.15.19:0/1858878965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f619c102cb0 0x7f619c1136c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:17.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 >> 172.21.15.19:0/1858878965 conn(0x7f619c0fc460 msgr2=0x7f619c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:17.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 shutdown_connections 2024-02-22T15:41:17.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:16.744+0000 7f61a2ffb700 1 -- 172.21.15.19:0/1858878965 wait complete. 2024-02-22T15:41:17.493 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-22T15:41:17.493 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-02-22T15:41:17.503 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-22T15:41:17.503 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-02-22T15:41:17.503 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-02-22T15:41:17.503 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-22T15:41:17.505 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-02-22T15:41:17.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:17.505 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-02-22T15:41:17.506 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-02-22T15:41:17.506 INFO:teuthology.orchestra.run.smithi019.stdout: Password: u91c8ent2n 2024-02-22T15:41:17.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:17.506 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config directory 2024-02-22T15:41:17.507 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-02-22T15:41:18.320 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:18 smithi019 ceph-mon[67418]: mgrmap e13: smithi019.yvjrkp(active, since 6s) 2024-02-22T15:41:18.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.108+0000 7f4ecdca0700 1 Processor -- start 2024-02-22T15:41:18.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.108+0000 7f4ecdca0700 1 -- start start 2024-02-22T15:41:18.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.108+0000 7f4ecdca0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec81052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:18.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.108+0000 7f4ecdca0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81057e0 con 0x7f4ec80712f0 2024-02-22T15:41:18.989 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec81052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec81052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32788/0 (socket says 172.21.15.19:32788) 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec77fe700 1 -- 172.21.15.19:0/775120641 learned_addr learned my addr 172.21.15.19:0/775120641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec77fe700 1 -- 172.21.15.19:0/775120641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8105920 con 0x7f4ec80712f0 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec77fe700 1 --2- 172.21.15.19:0/775120641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec81052a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80098e0 tx=0x7f4eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3f7c65a702861d server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec67fc700 1 -- 172.21.15.19:0/775120641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb800b860 con 0x7f4ec80712f0 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec67fc700 1 -- 172.21.15.19:0/775120641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4eb800be70 con 0x7f4ec80712f0 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.109+0000 7f4ec67fc700 1 -- 172.21.15.19:0/775120641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb8010380 con 0x7f4ec80712f0 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 -- 172.21.15.19:0/775120641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 msgr2=0x7f4ec81052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 --2- 172.21.15.19:0/775120641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec81052a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80098e0 tx=0x7f4eb8009250 comp rx=0 tx=0).stop 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 -- 172.21.15.19:0/775120641 shutdown_connections 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 --2- 172.21.15.19:0/775120641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec81052a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 -- 172.21.15.19:0/775120641 >> 172.21.15.19:0/775120641 conn(0x7f4ec80fccd0 msgr2=0x7f4ec80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 -- 172.21.15.19:0/775120641 shutdown_connections 2024-02-22T15:41:18.990 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.110+0000 7f4ecdca0700 1 -- 172.21.15.19:0/775120641 wait complete. 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ecdca0700 1 Processor -- start 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ecdca0700 1 -- start start 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ecdca0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec8102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ecdca0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81011a0 con 0x7f4ec80712f0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ec77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec8102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ec77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec8102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32800/0 (socket says 172.21.15.19:32800) 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.111+0000 7f4ec77fe700 1 -- 172.21.15.19:0/2896409669 learned_addr learned my addr 172.21.15.19:0/2896409669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ec77fe700 1 -- 172.21.15.19:0/2896409669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80094f0 con 0x7f4ec80712f0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ec77fe700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec8102d20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80094c0 tx=0x7f4eb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb8010850 con 0x7f4ec80712f0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4eb8010e60 con 0x7f4ec80712f0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec81013a0 con 0x7f4ec80712f0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb80243f0 con 0x7f4ec80712f0 2024-02-22T15:41:18.991 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.112+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec81017b0 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.113+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec804bed0 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.114+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4eb801a430 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.114+0000 7f4ebffff700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f4eb40368d0 0x7f4eb4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.114+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4eb804acc0 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.114+0000 7f4ec6ffd700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f4eb40368d0 0x7f4eb4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.114+0000 7f4ec6ffd700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f4eb40368d0 0x7f4eb4038d80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009990 tx=0x7f4eb0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.118+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4eb802c990 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.308+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4ec8106750 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.308+0000 7f4ebffff700 1 -- 172.21.15.19:0/2896409669 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4eb802c7b0 con 0x7f4ec80712f0 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f4eb40368d0 msgr2=0x7f4eb4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f4eb40368d0 0x7f4eb4038d80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009990 tx=0x7f4eb0006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 msgr2=0x7f4ec8102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec8102d20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80094c0 tx=0x7f4eb8009250 comp rx=0 tx=0).stop 2024-02-22T15:41:18.992 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 shutdown_connections 2024-02-22T15:41:18.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f4eb40368d0 0x7f4eb4038d80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:18.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 --2- 172.21.15.19:0/2896409669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec80712f0 0x7f4ec8102d20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:18.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.310+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 >> 172.21.15.19:0/2896409669 conn(0x7f4ec80fccd0 msgr2=0x7f4ec80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:18.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.311+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 shutdown_connections 2024-02-22T15:41:18.993 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:18.311+0000 7f4ecdca0700 1 -- 172.21.15.19:0/2896409669 wait complete. 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99fe6a4700 1 Processor -- start 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99fe6a4700 1 -- start start 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99fe6a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f8101980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99fe6a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8101ec0 con 0x7f99f8101570 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f8101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f8101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32812/0 (socket says 172.21.15.19:32812) 2024-02-22T15:41:20.546 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.558+0000 7f99f7fff700 1 -- 172.21.15.19:0/904738427 learned_addr learned my addr 172.21.15.19:0/904738427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.559+0000 7f99f7fff700 1 -- 172.21.15.19:0/904738427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8102000 con 0x7f99f8101570 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.559+0000 7f99f7fff700 1 --2- 172.21.15.19:0/904738427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f8101980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098e0 tx=0x7f99e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcece00ad3213329 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.559+0000 7f99f6ffd700 1 -- 172.21.15.19:0/904738427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99e800b860 con 0x7f99f8101570 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.559+0000 7f99f6ffd700 1 -- 172.21.15.19:0/904738427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99e800be70 con 0x7f99f8101570 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.559+0000 7f99f6ffd700 1 -- 172.21.15.19:0/904738427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99e8010380 con 0x7f99f8101570 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 -- 172.21.15.19:0/904738427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 msgr2=0x7f99f8101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 --2- 172.21.15.19:0/904738427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f8101980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098e0 tx=0x7f99e8009250 comp rx=0 tx=0).stop 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 -- 172.21.15.19:0/904738427 shutdown_connections 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 --2- 172.21.15.19:0/904738427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f8101980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 -- 172.21.15.19:0/904738427 >> 172.21.15.19:0/904738427 conn(0x7f99f80fccd0 msgr2=0x7f99f80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 -- 172.21.15.19:0/904738427 shutdown_connections 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 -- 172.21.15.19:0/904738427 wait complete. 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 Processor -- start 2024-02-22T15:41:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.560+0000 7f99fe6a4700 1 -- start start 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99fe6a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f810f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99fe6a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f810fa40 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f810f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f810f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32824/0 (socket says 172.21.15.19:32824) 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f7fff700 1 -- 172.21.15.19:0/2891211563 learned_addr learned my addr 172.21.15.19:0/2891211563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f7fff700 1 -- 172.21.15.19:0/2891211563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e80094f0 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f7fff700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f810f500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f99e800bac0 tx=0x7f99e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99e80109b0 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99e801a430 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f810fc40 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.561+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99e80245b0 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.562+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f8110050 con 0x7f99f8101570 2024-02-22T15:41:20.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.562+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f99e8024710 con 0x7f99f8101570 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.562+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f804bed0 con 0x7f99f8101570 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.562+0000 7f99f4ff9700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f99e0038b70 0x7f99e003b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.563+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f99e804a9d0 con 0x7f99f8101570 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.563+0000 7f99f77fe700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f99e0038b70 0x7f99e003b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.563+0000 7f99f77fe700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f99e0038b70 0x7f99e003b020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009990 tx=0x7f99ec006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.567+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f99e8018940 con 0x7f99f8101570 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.866+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f99f8064ab0 con 0x7f99f8101570 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.869+0000 7f99f4ff9700 1 -- 172.21.15.19:0/2891211563 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f99e8018350 con 0x7f99f8101570 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f99e0038b70 msgr2=0x7f99e003b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f99e0038b70 0x7f99e003b020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009990 tx=0x7f99ec006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 msgr2=0x7f99f810f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f810f500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f99e800bac0 tx=0x7f99e8009250 comp rx=0 tx=0).stop 2024-02-22T15:41:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 shutdown_connections 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f99e0038b70 0x7f99e003b020 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 --2- 172.21.15.19:0/2891211563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f8101570 0x7f99f810f500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 >> 172.21.15.19:0/2891211563 conn(0x7f99f80fccd0 msgr2=0x7f99f81062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 shutdown_connections 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-02-22T15:41:19.871+0000 7f99fe6a4700 1 -- 172.21.15.19:0/2891211563 wait complete. 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:20.551 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-02-22T15:41:20.586 INFO:tasks.cephadm:Fetching config... 2024-02-22T15:41:20.586 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:41:20.586 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-22T15:41:20.606 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-22T15:41:20.606 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:41:20.606 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-22T15:41:20.668 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-22T15:41:20.669 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:41:20.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/keyring of=/dev/stdout 2024-02-22T15:41:20.737 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-22T15:41:20.738 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:41:20.738 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-22T15:41:20.795 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-22T15:41:20.795 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0tuIQPc1Lg/vCpdDI9qV7uFceJ+ydA6fS3FyH8TgfeSxq4VHzAcoWyeeLYGsisNiv7lfE2L94B960dsOwJyagto9V7qAfsTln9aRzWgaW9avSrVUOugjSauHZ0uj9KfgEuCnDR3euFVnOALOQfS7Bx7YirocGGgTVmz1vaenXXE9x+Cgs6Y1K3AwKZYTeU9BkmYjc96i5CDM0VpE9xUnfP4b9Vs08Qig+nYWpz1GS998D4aGKsLVCup0VqFlqsDdS2FxS2VTNkdqH/d/Xfn0w/3bSITiPedFO5q7C5kaZdpbi4Ux4uzGhzXy1kG7FdAh5iHiW2EEijm4AVaAKHMiPm3NiJ+RT8/acbmxXIREx6B4A/pSlBX409nr+UuZNK9iyaRnGlvQdWQUCfguoVPTBHoAGYVL6EAtqN7uH2PMxgZB6C++bWtkm95nEs4SpYmT9NlV9bKA3DXNlIgmrvczqBWNFh2QRLFkI3f7TzxADb2Xlb4DCETIDGh690oR0cac= ceph-8db8e6fa-d198-11ee-95bf-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-22T15:41:20.874 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0tuIQPc1Lg/vCpdDI9qV7uFceJ+ydA6fS3FyH8TgfeSxq4VHzAcoWyeeLYGsisNiv7lfE2L94B960dsOwJyagto9V7qAfsTln9aRzWgaW9avSrVUOugjSauHZ0uj9KfgEuCnDR3euFVnOALOQfS7Bx7YirocGGgTVmz1vaenXXE9x+Cgs6Y1K3AwKZYTeU9BkmYjc96i5CDM0VpE9xUnfP4b9Vs08Qig+nYWpz1GS998D4aGKsLVCup0VqFlqsDdS2FxS2VTNkdqH/d/Xfn0w/3bSITiPedFO5q7C5kaZdpbi4Ux4uzGhzXy1kG7FdAh5iHiW2EEijm4AVaAKHMiPm3NiJ+RT8/acbmxXIREx6B4A/pSlBX409nr+UuZNK9iyaRnGlvQdWQUCfguoVPTBHoAGYVL6EAtqN7uH2PMxgZB6C++bWtkm95nEs4SpYmT9NlV9bKA3DXNlIgmrvczqBWNFh2QRLFkI3f7TzxADb2Xlb4DCETIDGh690oR0cac= ceph-8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:41:20.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0tuIQPc1Lg/vCpdDI9qV7uFceJ+ydA6fS3FyH8TgfeSxq4VHzAcoWyeeLYGsisNiv7lfE2L94B960dsOwJyagto9V7qAfsTln9aRzWgaW9avSrVUOugjSauHZ0uj9KfgEuCnDR3euFVnOALOQfS7Bx7YirocGGgTVmz1vaenXXE9x+Cgs6Y1K3AwKZYTeU9BkmYjc96i5CDM0VpE9xUnfP4b9Vs08Qig+nYWpz1GS998D4aGKsLVCup0VqFlqsDdS2FxS2VTNkdqH/d/Xfn0w/3bSITiPedFO5q7C5kaZdpbi4Ux4uzGhzXy1kG7FdAh5iHiW2EEijm4AVaAKHMiPm3NiJ+RT8/acbmxXIREx6B4A/pSlBX409nr+UuZNK9iyaRnGlvQdWQUCfguoVPTBHoAGYVL6EAtqN7uH2PMxgZB6C++bWtkm95nEs4SpYmT9NlV9bKA3DXNlIgmrvczqBWNFh2QRLFkI3f7TzxADb2Xlb4DCETIDGh690oR0cac= ceph-8db8e6fa-d198-11ee-95bf-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-22T15:41:20.936 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0tuIQPc1Lg/vCpdDI9qV7uFceJ+ydA6fS3FyH8TgfeSxq4VHzAcoWyeeLYGsisNiv7lfE2L94B960dsOwJyagto9V7qAfsTln9aRzWgaW9avSrVUOugjSauHZ0uj9KfgEuCnDR3euFVnOALOQfS7Bx7YirocGGgTVmz1vaenXXE9x+Cgs6Y1K3AwKZYTeU9BkmYjc96i5CDM0VpE9xUnfP4b9Vs08Qig+nYWpz1GS998D4aGKsLVCup0VqFlqsDdS2FxS2VTNkdqH/d/Xfn0w/3bSITiPedFO5q7C5kaZdpbi4Ux4uzGhzXy1kG7FdAh5iHiW2EEijm4AVaAKHMiPm3NiJ+RT8/acbmxXIREx6B4A/pSlBX409nr+UuZNK9iyaRnGlvQdWQUCfguoVPTBHoAGYVL6EAtqN7uH2PMxgZB6C++bWtkm95nEs4SpYmT9NlV9bKA3DXNlIgmrvczqBWNFh2QRLFkI3f7TzxADb2Xlb4DCETIDGh690oR0cac= ceph-8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:41:20.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-22T15:41:21.224 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:41:21.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:20 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2891211563' entity='client.admin' 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 -- 172.21.15.19:0/2972483973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 msgr2=0x7f13700ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 --2- 172.21.15.19:0/2972483973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700ffaf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f13600048c0 tx=0x7f1360009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 -- 172.21.15.19:0/2972483973 shutdown_connections 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 --2- 172.21.15.19:0/2972483973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700ffaf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 -- 172.21.15.19:0/2972483973 >> 172.21.15.19:0/2972483973 conn(0x7f13700f7190 msgr2=0x7f13700f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 -- 172.21.15.19:0/2972483973 shutdown_connections 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.897+0000 7f13761a8700 1 -- 172.21.15.19:0/2972483973 wait complete. 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f13761a8700 1 Processor -- start 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f13761a8700 1 -- start start 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f13761a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f13761a8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13700fb620 con 0x7f13700fd710 2024-02-22T15:41:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f136f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f136f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700fd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32850/0 (socket says 172.21.15.19:32850) 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.898+0000 7f136f7fe700 1 -- 172.21.15.19:0/2251748904 learned_addr learned my addr 172.21.15.19:0/2251748904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f136f7fe700 1 -- 172.21.15.19:0/2251748904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1360005040 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f136f7fe700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700fd1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1360009040 tx=0x7f1360009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1360009620 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13700fb820 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1360009780 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13700fbd10 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.899+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f136001c5b0 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.900+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1370106860 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.900+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f136001c870 con 0x7f13700fd710 2024-02-22T15:41:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.900+0000 7f1357fff700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1358036870 0x7f1358038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.900+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1360049a00 con 0x7f13700fd710 2024-02-22T15:41:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.900+0000 7f136effd700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1358036870 0x7f1358038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.901+0000 7f136effd700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1358036870 0x7f1358038d20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1364009990 tx=0x7f1364006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:23.909+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1360023300 con 0x7f13700fd710 2024-02-22T15:41:24.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.104+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f137005f5c0 con 0x7f13700fd710 2024-02-22T15:41:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.107+0000 7f1357fff700 1 -- 172.21.15.19:0/2251748904 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1360011b60 con 0x7f13700fd710 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1358036870 msgr2=0x7f1358038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1358036870 0x7f1358038d20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1364009990 tx=0x7f1364006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 msgr2=0x7f13700fd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700fd1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1360009040 tx=0x7f1360009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 shutdown_connections 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1358036870 0x7f1358038d20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 --2- 172.21.15.19:0/2251748904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13700fd710 0x7f13700fd1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 >> 172.21.15.19:0/2251748904 conn(0x7f13700f7190 msgr2=0x7f13700f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:24.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 shutdown_connections 2024-02-22T15:41:24.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:24.110+0000 7f13761a8700 1 -- 172.21.15.19:0/2251748904 wait complete. 2024-02-22T15:41:24.830 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-22T15:41:24.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-22T15:41:25.313 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:25 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2251748904' entity='client.admin' 2024-02-22T15:41:25.436 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:41:26.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:26 smithi019 ceph-mon[67418]: Deploying daemon crash.smithi019 on smithi019 2024-02-22T15:41:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 -- 172.21.15.19:0/3356104529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 msgr2=0x7fe1340fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 --2- 172.21.15.19:0/3356104529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe1340fcfe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe1240048c0 tx=0x7fe124009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 -- 172.21.15.19:0/3356104529 shutdown_connections 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 --2- 172.21.15.19:0/3356104529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe1340fcfe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 -- 172.21.15.19:0/3356104529 >> 172.21.15.19:0/3356104529 conn(0x7fe1340f8290 msgr2=0x7fe1340fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 -- 172.21.15.19:0/3356104529 shutdown_connections 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.301+0000 7fe139778700 1 -- 172.21.15.19:0/3356104529 wait complete. 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.302+0000 7fe139778700 1 Processor -- start 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.302+0000 7fe139778700 1 -- start start 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.302+0000 7fe139778700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe134071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.302+0000 7fe139778700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340702e0 con 0x7fe1340fcbd0 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe132ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe134071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe132ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe134071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57410/0 (socket says 172.21.15.19:57410) 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe132ffd700 1 -- 172.21.15.19:0/2003245652 learned_addr learned my addr 172.21.15.19:0/2003245652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe132ffd700 1 -- 172.21.15.19:0/2003245652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124005040 con 0x7fe1340fcbd0 2024-02-22T15:41:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe132ffd700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe134071e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe124009330 tx=0x7fe124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1240096b0 con 0x7fe1340fcbd0 2024-02-22T15:41:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe124009810 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1340704e0 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe12401c5b0 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.303+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1340709d0 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.304+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe12401c870 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.304+0000 7fe11b7fe700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fe11c036870 0x7fe11c038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.304+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe124049c10 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.304+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134106860 con 0x7fe1340fcbd0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.304+0000 7fe1327fc700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fe11c036870 0x7fe11c038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.305+0000 7fe1327fc700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fe11c036870 0x7fe11c038d20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe128009990 tx=0x7fe128006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.309+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe124011db0 con 0x7fe1340fcbd0 2024-02-22T15:41:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.511+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe13402b410 con 0x7fe11c036870 2024-02-22T15:41:28.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.517+0000 7fe11b7fe700 1 -- 172.21.15.19:0/2003245652 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe13402b410 con 0x7fe11c036870 2024-02-22T15:41:28.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.519+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fe11c036870 msgr2=0x7fe11c038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:28.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.519+0000 7fe139778700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fe11c036870 0x7fe11c038d20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe128009990 tx=0x7fe128006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.519+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 msgr2=0x7fe134071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.519+0000 7fe139778700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe134071e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe124009330 tx=0x7fe124009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.519+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 shutdown_connections 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.519+0000 7fe139778700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fe11c036870 0x7fe11c038d20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.520+0000 7fe139778700 1 --2- 172.21.15.19:0/2003245652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1340fcbd0 0x7fe134071e60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.520+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 >> 172.21.15.19:0/2003245652 conn(0x7fe1340f8290 msgr2=0x7fe1340f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.520+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 shutdown_connections 2024-02-22T15:41:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:28.520+0000 7fe139778700 1 -- 172.21.15.19:0/2003245652 wait complete. 2024-02-22T15:41:29.164 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi062 2024-02-22T15:41:29.164 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:41:29.165 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.conf 2024-02-22T15:41:29.187 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:41:29.187 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:41:29.249 INFO:tasks.cephadm:Adding host smithi062 to orchestrator... 2024-02-22T15:41:29.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch host add smithi062 2024-02-22T15:41:29.625 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:41:29.698 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:29.699 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:29.699 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:29.699 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:29.699 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: Deploying daemon node-exporter.smithi019 on smithi019 2024-02-22T15:41:29.699 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:41:29.699 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:29 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.146+0000 7fcbdef20700 1 -- 172.21.15.19:0/1810824035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 msgr2=0x7fcbd80fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.146+0000 7fcbdef20700 1 --2- 172.21.15.19:0/1810824035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd80fbee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0048c0 tx=0x7fcbcc009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.146+0000 7fcbdef20700 1 -- 172.21.15.19:0/1810824035 shutdown_connections 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.146+0000 7fcbdef20700 1 --2- 172.21.15.19:0/1810824035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd80fbee0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.146+0000 7fcbdef20700 1 -- 172.21.15.19:0/1810824035 >> 172.21.15.19:0/1810824035 conn(0x7fcbd80f7190 msgr2=0x7fcbd80f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdef20700 1 -- 172.21.15.19:0/1810824035 shutdown_connections 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdef20700 1 -- 172.21.15.19:0/1810824035 wait complete. 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdef20700 1 Processor -- start 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdef20700 1 -- start start 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdef20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdef20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8065d60 con 0x7fcbd80fbad0 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdccbc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdccbc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd810fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57440/0 (socket says 172.21.15.19:57440) 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.147+0000 7fcbdccbc700 1 -- 172.21.15.19:0/2395406070 learned_addr learned my addr 172.21.15.19:0/2395406070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbdccbc700 1 -- 172.21.15.19:0/2395406070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc005040 con 0x7fcbd80fbad0 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbdccbc700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd810fb10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b690 tx=0x7fcbcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbcc009650 con 0x7fcbd80fbad0 2024-02-22T15:41:32.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcbcc0097b0 con 0x7fcbd80fbad0 2024-02-22T15:41:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8065f60 con 0x7fcbd80fbad0 2024-02-22T15:41:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbcc01c5d0 con 0x7fcbd80fbad0 2024-02-22T15:41:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.148+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd8110290 con 0x7fcbd80fbad0 2024-02-22T15:41:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.149+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fcbcc01c840 con 0x7fcbd80fbad0 2024-02-22T15:41:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.149+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd8105760 con 0x7fcbd80fbad0 2024-02-22T15:41:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.149+0000 7fcbd57fa700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fcbc8036870 0x7fcbc8038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:32.152 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:31 smithi019 ceph-mon[67418]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:32.152 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:31 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:32.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.149+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcbcc049d00 con 0x7fcbd80fbad0 2024-02-22T15:41:32.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.149+0000 7fcbd7fff700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fcbc8036870 0x7fcbc8038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:32.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.149+0000 7fcbd7fff700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fcbc8036870 0x7fcbc8038d20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009990 tx=0x7fcbc4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:32.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.154+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcbcc0118d0 con 0x7fcbd80fbad0 2024-02-22T15:41:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:32.350+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbd802b540 con 0x7fcbc8036870 2024-02-22T15:41:34.123 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi062' with addr '172.21.15.62' 2024-02-22T15:41:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.137+0000 7fcbd57fa700 1 -- 172.21.15.19:0/2395406070 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fcbd802b540 con 0x7fcbc8036870 2024-02-22T15:41:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fcbc8036870 msgr2=0x7fcbc8038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fcbc8036870 0x7fcbc8038d20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009990 tx=0x7fcbc4006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 msgr2=0x7fcbd810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd810fb10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b690 tx=0x7fcbcc009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:34.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 shutdown_connections 2024-02-22T15:41:34.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fcbc8036870 0x7fcbc8038d20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:34.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 --2- 172.21.15.19:0/2395406070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fbad0 0x7fcbd810fb10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:34.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 >> 172.21.15.19:0/2395406070 conn(0x7fcbd80f7190 msgr2=0x7fcbd80f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:34.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 shutdown_connections 2024-02-22T15:41:34.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:34.140+0000 7fcbdef20700 1 -- 172.21.15.19:0/2395406070 wait complete. 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: Deploying cephadm binary to smithi062 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:34.149 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:33 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:35.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch host ls --format=json 2024-02-22T15:41:35.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:34 smithi019 ceph-mon[67418]: Deploying daemon alertmanager.smithi019 on smithi019 2024-02-22T15:41:35.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:34 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:41:35.954 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:35 smithi019 ceph-mon[67418]: Added host smithi062 2024-02-22T15:41:35.955 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:35 smithi019 ceph-mon[67418]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:35.955 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:35 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.743+0000 7f36b5303700 1 -- 172.21.15.19:0/3194477453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 msgr2=0x7f36b00fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.743+0000 7f36b5303700 1 --2- 172.21.15.19:0/3194477453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b00fcfd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36a00048c0 tx=0x7f36a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.743+0000 7f36b5303700 1 -- 172.21.15.19:0/3194477453 shutdown_connections 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.743+0000 7f36b5303700 1 --2- 172.21.15.19:0/3194477453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b00fcfd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.743+0000 7f36b5303700 1 -- 172.21.15.19:0/3194477453 >> 172.21.15.19:0/3194477453 conn(0x7f36b00f82a0 msgr2=0x7f36b00fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.743+0000 7f36b5303700 1 -- 172.21.15.19:0/3194477453 shutdown_connections 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36b5303700 1 -- 172.21.15.19:0/3194477453 wait complete. 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36b5303700 1 Processor -- start 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36b5303700 1 -- start start 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36b5303700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b010efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36b5303700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b010f520 con 0x7f36b00fcbc0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b010efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b010efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48238/0 (socket says 172.21.15.19:48238) 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.744+0000 7f36aeffd700 1 -- 172.21.15.19:0/1685246985 learned_addr learned my addr 172.21.15.19:0/1685246985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36aeffd700 1 -- 172.21.15.19:0/1685246985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a0005040 con 0x7f36b00fcbc0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36aeffd700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b010efe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009330 tx=0x7f36a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36a00096b0 con 0x7f36b00fcbc0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36a0009810 con 0x7f36b00fcbc0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b010f720 con 0x7f36b00fcbc0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36a001c5b0 con 0x7f36b00fcbc0 2024-02-22T15:41:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.745+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b01103c0 con 0x7f36b00fcbc0 2024-02-22T15:41:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.746+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f36a001c870 con 0x7f36b00fcbc0 2024-02-22T15:41:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.746+0000 7f36977fe700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f3698036870 0x7f3698038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.746+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b005f5c0 con 0x7f36b00fcbc0 2024-02-22T15:41:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.746+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f36a0049e20 con 0x7f36b00fcbc0 2024-02-22T15:41:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.746+0000 7f36ae7fc700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f3698036870 0x7f3698038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.747+0000 7f36ae7fc700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f3698036870 0x7f3698038d20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009990 tx=0x7f36a4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:37.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.751+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f36a0011db0 con 0x7f36b00fcbc0 2024-02-22T15:41:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.953+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36b00708f0 con 0x7f3698036870 2024-02-22T15:41:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.954+0000 7f36977fe700 1 -- 172.21.15.19:0/1685246985 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f36b00708f0 con 0x7f3698036870 2024-02-22T15:41:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:41:37.939 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}] 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f3698036870 msgr2=0x7f3698038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f3698036870 0x7f3698038d20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009990 tx=0x7f36a4006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 msgr2=0x7f36b010efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b010efe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009330 tx=0x7f36a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 shutdown_connections 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f3698036870 0x7f3698038d20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 --2- 172.21.15.19:0/1685246985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b00fcbc0 0x7f36b010efe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 >> 172.21.15.19:0/1685246985 conn(0x7f36b00f82a0 msgr2=0x7f36b00701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:37.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 shutdown_connections 2024-02-22T15:41:37.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:37.956+0000 7f36b5303700 1 -- 172.21.15.19:0/1685246985 wait complete. 2024-02-22T15:41:38.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:37 smithi019 ceph-mon[67418]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:39.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:38 smithi019 ceph-mon[67418]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T15:41:39.305 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-22T15:41:39.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd crush tunables default 2024-02-22T15:41:39.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:39 smithi019 ceph-mon[67418]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:40.682 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:41:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:41 smithi019 ceph-mon[67418]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:44.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:43 smithi019 ceph-mon[67418]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:44.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 -- 172.21.15.19:0/1426850342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 msgr2=0x7f54240fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:44.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 --2- 172.21.15.19:0/1426850342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f54240fbee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54140048c0 tx=0x7f5414009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:44.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 -- 172.21.15.19:0/1426850342 shutdown_connections 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 --2- 172.21.15.19:0/1426850342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f54240fbee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 -- 172.21.15.19:0/1426850342 >> 172.21.15.19:0/1426850342 conn(0x7f54240f7190 msgr2=0x7f54240f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 -- 172.21.15.19:0/1426850342 shutdown_connections 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.355+0000 7f5428a52700 1 -- 172.21.15.19:0/1426850342 wait complete. 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f5428a52700 1 Processor -- start 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f5428a52700 1 -- start start 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f5428a52700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f542410fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:44.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f5428a52700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424065d60 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f542259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f542410fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f542259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f542410fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48252/0 (socket says 172.21.15.19:48252) 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.356+0000 7f542259c700 1 -- 172.21.15.19:0/2812445821 learned_addr learned my addr 172.21.15.19:0/2812445821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f542259c700 1 -- 172.21.15.19:0/2812445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414005040 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f542259c700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f542410fb10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f541400b690 tx=0x7f5414009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5414009650 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5424065f60 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5424110290 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f54140097b0 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.357+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f541401c3f0 con 0x7f54240fbad0 2024-02-22T15:41:44.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.358+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f541401c660 con 0x7f54240fbad0 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.358+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54040052e0 con 0x7f54240fbad0 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.358+0000 7f5412ffd700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f540c036870 0x7f540c038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.358+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5414049d10 con 0x7f54240fbad0 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.358+0000 7f5421d9b700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f540c036870 0x7f540c038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.359+0000 7f5421d9b700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f540c036870 0x7f540c038d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5418009990 tx=0x7f5418006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.364+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5414011dd0 con 0x7f54240fbad0 2024-02-22T15:41:44.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:44.576+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5404005590 con 0x7f54240fbad0 2024-02-22T15:41:45.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.547+0000 7f5412ffd700 1 -- 172.21.15.19:0/2812445821 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5414011370 con 0x7f54240fbad0 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f540c036870 msgr2=0x7f540c038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f540c036870 0x7f540c038d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5418009990 tx=0x7f5418006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 msgr2=0x7f542410fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f542410fb10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f541400b690 tx=0x7f5414009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 shutdown_connections 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f540c036870 0x7f540c038d20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 --2- 172.21.15.19:0/2812445821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54240fbad0 0x7f542410fb10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 >> 172.21.15.19:0/2812445821 conn(0x7f54240f7190 msgr2=0x7f54240f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 shutdown_connections 2024-02-22T15:41:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:41:45.549+0000 7f5428a52700 1 -- 172.21.15.19:0/2812445821 wait complete. 2024-02-22T15:41:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2812445821' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T15:41:45.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T15:41:45.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:45.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: Deploying daemon grafana.smithi019 on smithi019 2024-02-22T15:41:45.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:45 smithi019 ceph-mon[67418]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:46.587 INFO:tasks.cephadm:Adding mon.smithi019 on smithi019 2024-02-22T15:41:46.587 INFO:tasks.cephadm:Adding mon.smithi062 on smithi062 2024-02-22T15:41:46.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch apply mon '2;smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062' 2024-02-22T15:41:46.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:46 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2812445821' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-22T15:41:46.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:46 smithi019 ceph-mon[67418]: osdmap e4: 0 total, 0 up, 0 in 2024-02-22T15:41:46.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:46 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:46.894 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:46.955 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:47.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:47 smithi019 ceph-mon[67418]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:48.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 -- 172.21.15.62:0/4204658417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 msgr2=0x7f05240fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 --2- 172.21.15.62:0/4204658417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f05240fcfd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f05180048c0 tx=0x7f0518009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 -- 172.21.15.62:0/4204658417 shutdown_connections 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 --2- 172.21.15.62:0/4204658417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f05240fcfd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 -- 172.21.15.62:0/4204658417 >> 172.21.15.62:0/4204658417 conn(0x7f05240f82a0 msgr2=0x7f05240fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 -- 172.21.15.62:0/4204658417 shutdown_connections 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.662+0000 7f052a083700 1 -- 172.21.15.62:0/4204658417 wait complete. 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f052a083700 1 Processor -- start 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f052a083700 1 -- start start 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f052a083700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f052410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f052a083700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052410d390 con 0x7f05240fcbc0 2024-02-22T15:41:48.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f05237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f052410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f05237fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f052410ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:41290/0 (socket says 172.21.15.62:41290) 2024-02-22T15:41:48.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f05237fe700 1 -- 172.21.15.62:0/3405232249 learned_addr learned my addr 172.21.15.62:0/3405232249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:41:48.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.663+0000 7f05237fe700 1 -- 172.21.15.62:0/3405232249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518005040 con 0x7f05240fcbc0 2024-02-22T15:41:48.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.664+0000 7f05237fe700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f052410ce50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f051800b690 tx=0x7f0518009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:48.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.664+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0518009650 con 0x7f05240fcbc0 2024-02-22T15:41:48.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.664+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05180097b0 con 0x7f05240fcbc0 2024-02-22T15:41:48.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.664+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052410d590 con 0x7f05240fcbc0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.664+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f051801c5d0 con 0x7f05240fcbc0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.664+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052410e150 con 0x7f05240fcbc0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.665+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f0518013420 con 0x7f05240fcbc0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.665+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052405f5c0 con 0x7f05240fcbc0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.665+0000 7f050bfff700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f050c0368c0 0x7f050c038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.665+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0518049b70 con 0x7f05240fcbc0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.665+0000 7f0522ffd700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f050c0368c0 0x7f050c038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:48.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.666+0000 7f0522ffd700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f050c0368c0 0x7f050c038d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0514009990 tx=0x7f0514006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:48.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.670+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f05180123c0 con 0x7f05240fcbc0 2024-02-22T15:41:48.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.878+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 --> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062", "target": ["mon-mgr", ""]}) v1 -- 0x7f052410ee00 con 0x7f050c0368c0 2024-02-22T15:41:48.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.885+0000 7f050bfff700 1 -- 172.21.15.62:0/3405232249 <== mgr.14160 v2:172.21.15.19:6800/1788498096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f052410ee00 con 0x7f050c0368c0 2024-02-22T15:41:48.871 INFO:teuthology.orchestra.run.smithi062.stdout:Scheduled mon update... 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f050c0368c0 msgr2=0x7f050c038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f050c0368c0 0x7f050c038d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0514009990 tx=0x7f0514006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 msgr2=0x7f052410ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f052410ce50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f051800b690 tx=0x7f0518009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 shutdown_connections 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f050c0368c0 0x7f050c038d70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 --2- 172.21.15.62:0/3405232249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05240fcbc0 0x7f052410ce50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 >> 172.21.15.62:0/3405232249 conn(0x7f05240f82a0 msgr2=0x7f05240f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 shutdown_connections 2024-02-22T15:41:48.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:48.887+0000 7f052a083700 1 -- 172.21.15.62:0/3405232249 wait complete. 2024-02-22T15:41:49.443 DEBUG:teuthology.orchestra.run.smithi062:mon.smithi062> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi062.service 2024-02-22T15:41:49.445 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:41:49.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:41:49.476 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:-- Logs begin at Thu 2024-02-22 14:59:35 UTC. -- 2024-02-22T15:41:49.751 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:49.820 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:50.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:49 smithi019 ceph-mon[67418]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:50.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:49 smithi019 ceph-mon[67418]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:41:50.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:49 smithi019 ceph-mon[67418]: Saving service mon spec with placement smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062;count:2 2024-02-22T15:41:50.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:49 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 -- 172.21.15.62:0/1215197969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 msgr2=0x7ffaf40fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 --2- 172.21.15.62:0/1215197969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf40fbee0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00048c0 tx=0x7ffaf0009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 -- 172.21.15.62:0/1215197969 shutdown_connections 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 --2- 172.21.15.62:0/1215197969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf40fbee0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 -- 172.21.15.62:0/1215197969 >> 172.21.15.62:0/1215197969 conn(0x7ffaf40f7190 msgr2=0x7ffaf40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 -- 172.21.15.62:0/1215197969 shutdown_connections 2024-02-22T15:41:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.378+0000 7ffafb59c700 1 -- 172.21.15.62:0/1215197969 wait complete. 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffafb59c700 1 Processor -- start 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffafb59c700 1 -- start start 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffafb59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf410bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffafb59c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf410c2a0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffaf9338700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf410bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffaf9338700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf410bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:41298/0 (socket says 172.21.15.62:41298) 2024-02-22T15:41:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.379+0000 7ffaf9338700 1 -- 172.21.15.62:0/2927799900 learned_addr learned my addr 172.21.15.62:0/2927799900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffaf9338700 1 -- 172.21.15.62:0/2927799900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0005040 con 0x7ffaf40fbad0 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffaf9338700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf410bd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b690 tx=0x7ffaf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaf0009650 con 0x7ffaf40fbad0 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf410c4a0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffaf00097b0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaf001c5d0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.380+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf410d060 con 0x7ffaf40fbad0 2024-02-22T15:41:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.381+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ffaf0013420 con 0x7ffaf40fbad0 2024-02-22T15:41:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.381+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf405f5c0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.381+0000 7ffae9ffb700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ffae00368c0 0x7ffae0038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.381+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffaf0049c20 con 0x7ffaf40fbad0 2024-02-22T15:41:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.381+0000 7ffaf8b37700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ffae00368c0 0x7ffae0038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.382+0000 7ffaf8b37700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ffae00368c0 0x7ffae0038d70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009990 tx=0x7ffae4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:51.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.389+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ffaf0011db0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.658+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffaf410dca0 con 0x7ffaf40fbad0 2024-02-22T15:41:51.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.659+0000 7ffae9ffb700 1 -- 172.21.15.62:0/2927799900 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ffaf0011370 con 0x7ffaf40fbad0 2024-02-22T15:41:51.645 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:41:51.645 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ffae00368c0 msgr2=0x7ffae0038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ffae00368c0 0x7ffae0038d70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009990 tx=0x7ffae4006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 msgr2=0x7ffaf410bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf410bd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b690 tx=0x7ffaf0009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 shutdown_connections 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ffae00368c0 0x7ffae0038d70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 --2- 172.21.15.62:0/2927799900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaf40fbad0 0x7ffaf410bd60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 >> 172.21.15.62:0/2927799900 conn(0x7ffaf40f7190 msgr2=0x7ffaf40f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 shutdown_connections 2024-02-22T15:41:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:51.661+0000 7ffafb59c700 1 -- 172.21.15.62:0/2927799900 wait complete. 2024-02-22T15:41:51.648 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:41:52.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:51 smithi019 ceph-mon[67418]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:52.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:51 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2927799900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:41:53.239 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:41:53.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:41:53.542 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:53.600 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:54.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:53 smithi019 ceph-mon[67418]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:55.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 -- 172.21.15.62:0/221471378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 msgr2=0x7fb4240fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:55.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 --2- 172.21.15.62:0/221471378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb4240fbee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb4200048c0 tx=0x7fb420009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:55.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 -- 172.21.15.62:0/221471378 shutdown_connections 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 --2- 172.21.15.62:0/221471378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb4240fbee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 -- 172.21.15.62:0/221471378 >> 172.21.15.62:0/221471378 conn(0x7fb4240f7190 msgr2=0x7fb4240f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 -- 172.21.15.62:0/221471378 shutdown_connections 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 -- 172.21.15.62:0/221471378 wait complete. 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 Processor -- start 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.169+0000 7fb42be51700 1 -- start start 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb42be51700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb42410bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb42be51700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410c2a0 con 0x7fb4240fbad0 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb429bed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb42410bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb429bed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb42410bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:49126/0 (socket says 172.21.15.62:49126) 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb429bed700 1 -- 172.21.15.62:0/852628452 learned_addr learned my addr 172.21.15.62:0/852628452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb429bed700 1 -- 172.21.15.62:0/852628452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420005040 con 0x7fb4240fbad0 2024-02-22T15:41:55.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb429bed700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb42410bd60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b690 tx=0x7fb420009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:55.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.170+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb420009650 con 0x7fb4240fbad0 2024-02-22T15:41:55.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.171+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42410c4a0 con 0x7fb4240fbad0 2024-02-22T15:41:55.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.171+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4200097b0 con 0x7fb4240fbad0 2024-02-22T15:41:55.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.171+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb42001c5d0 con 0x7fb4240fbad0 2024-02-22T15:41:55.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.171+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42410d060 con 0x7fb4240fbad0 2024-02-22T15:41:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.171+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb4200139d0 con 0x7fb4240fbad0 2024-02-22T15:41:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.171+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42405f5c0 con 0x7fb4240fbad0 2024-02-22T15:41:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.172+0000 7fb41a7fc700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb410036870 0x7fb410038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.172+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb420049bb0 con 0x7fb4240fbad0 2024-02-22T15:41:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.172+0000 7fb4293ec700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb410036870 0x7fb410038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:55.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.172+0000 7fb4293ec700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb410036870 0x7fb410038d20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb414009990 tx=0x7fb414006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:55.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.177+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb4200123c0 con 0x7fb4240fbad0 2024-02-22T15:41:55.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.448+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb42410dca0 con 0x7fb4240fbad0 2024-02-22T15:41:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.448+0000 7fb41a7fc700 1 -- 172.21.15.62:0/852628452 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb420023300 con 0x7fb4240fbad0 2024-02-22T15:41:55.434 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:41:55.434 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb410036870 msgr2=0x7fb410038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb410036870 0x7fb410038d20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb414009990 tx=0x7fb414006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 msgr2=0x7fb42410bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb42410bd60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b690 tx=0x7fb420009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 shutdown_connections 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb410036870 0x7fb410038d20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 --2- 172.21.15.62:0/852628452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4240fbad0 0x7fb42410bd60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 >> 172.21.15.62:0/852628452 conn(0x7fb4240f7190 msgr2=0x7fb4240f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 shutdown_connections 2024-02-22T15:41:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:55.450+0000 7fb42be51700 1 -- 172.21.15.62:0/852628452 wait complete. 2024-02-22T15:41:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:41:56.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:55 smithi019 ceph-mon[67418]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:56.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:55 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/852628452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:41:57.080 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:41:57.080 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:41:57.376 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:57.435 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:41:57.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:57 smithi019 ceph-mon[67418]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 -- 172.21.15.62:0/2497471059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 msgr2=0x7fa00c0fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 --2- 172.21.15.62:0/2497471059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c0fb290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0048c0 tx=0x7f9ffc009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 -- 172.21.15.62:0/2497471059 shutdown_connections 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 --2- 172.21.15.62:0/2497471059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c0fb290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 -- 172.21.15.62:0/2497471059 >> 172.21.15.62:0/2497471059 conn(0x7fa00c0f6920 msgr2=0x7fa00c0f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 -- 172.21.15.62:0/2497471059 shutdown_connections 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 -- 172.21.15.62:0/2497471059 wait complete. 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.877+0000 7fa01167a700 1 Processor -- start 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa01167a700 1 -- start start 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa01167a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c10cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:58.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa01167a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00c10d3e0 con 0x7fa00c0fae80 2024-02-22T15:41:58.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa00affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c10cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:58.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa00affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c10cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:49142/0 (socket says 172.21.15.62:49142) 2024-02-22T15:41:58.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa00affd700 1 -- 172.21.15.62:0/4025213305 learned_addr learned my addr 172.21.15.62:0/4025213305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:41:58.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa00affd700 1 -- 172.21.15.62:0/4025213305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc005040 con 0x7fa00c0fae80 2024-02-22T15:41:58.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7fa00affd700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c10cea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009040 tx=0x7f9ffc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:58.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.878+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ffc009650 con 0x7fa00c0fae80 2024-02-22T15:41:58.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.879+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ffc0097b0 con 0x7fa00c0fae80 2024-02-22T15:41:58.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.879+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c10d5e0 con 0x7fa00c0fae80 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.879+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ffc01c5b0 con 0x7fa00c0fae80 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.879+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00c10e140 con 0x7fa00c0fae80 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.880+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f9ffc013420 con 0x7fa00c0fae80 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.880+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c05f5c0 con 0x7fa00c0fae80 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.880+0000 7f9ff37fe700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f9ff4036870 0x7f9ff4038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.880+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9ffc049a80 con 0x7fa00c0fae80 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.880+0000 7fa00a7fc700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f9ff4036870 0x7f9ff4038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:41:58.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.880+0000 7fa00a7fc700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f9ff4036870 0x7f9ff4038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa000009990 tx=0x7fa000006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:41:58.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:58.885+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9ffc0139d0 con 0x7fa00c0fae80 2024-02-22T15:41:59.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.152+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa00c04bed0 con 0x7fa00c0fae80 2024-02-22T15:41:59.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.153+0000 7f9ff37fe700 1 -- 172.21.15.62:0/4025213305 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9ffc011370 con 0x7fa00c0fae80 2024-02-22T15:41:59.139 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:41:59.139 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f9ff4036870 msgr2=0x7f9ff4038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f9ff4036870 0x7f9ff4038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa000009990 tx=0x7fa000006e40 comp rx=0 tx=0).stop 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 msgr2=0x7fa00c10cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c10cea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009040 tx=0x7f9ffc009e30 comp rx=0 tx=0).stop 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 shutdown_connections 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f9ff4036870 0x7f9ff4038d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 --2- 172.21.15.62:0/4025213305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa00c0fae80 0x7fa00c10cea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 >> 172.21.15.62:0/4025213305 conn(0x7fa00c0f6920 msgr2=0x7fa00c0f7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 shutdown_connections 2024-02-22T15:41:59.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:41:59.155+0000 7fa01167a700 1 -- 172.21.15.62:0/4025213305 wait complete. 2024-02-22T15:41:59.142 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/4025213305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:41:59.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:41:59 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:00.804 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:00.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:01.179 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:01.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:00 smithi019 ceph-mon[67418]: Deploying daemon prometheus.smithi019 on smithi019 2024-02-22T15:42:01.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:00 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:02.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:01 smithi019 ceph-mon[67418]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 -- 172.21.15.62:0/3076991673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 msgr2=0x7f5ff40fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 --2- 172.21.15.62:0/3076991673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff40fbee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40048c0 tx=0x7f5fe4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 -- 172.21.15.62:0/3076991673 shutdown_connections 2024-02-22T15:42:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 --2- 172.21.15.62:0/3076991673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff40fbee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 -- 172.21.15.62:0/3076991673 >> 172.21.15.62:0/3076991673 conn(0x7f5ff40f7190 msgr2=0x7f5ff40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 -- 172.21.15.62:0/3076991673 shutdown_connections 2024-02-22T15:42:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.722+0000 7f5ff8f53700 1 -- 172.21.15.62:0/3076991673 wait complete. 2024-02-22T15:42:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff8f53700 1 Processor -- start 2024-02-22T15:42:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff8f53700 1 -- start start 2024-02-22T15:42:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff8f53700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff410fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff8f53700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4065d60 con 0x7f5ff40fbad0 2024-02-22T15:42:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff410fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff410fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:49150/0 (socket says 172.21.15.62:49150) 2024-02-22T15:42:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.723+0000 7f5ff259c700 1 -- 172.21.15.62:0/2359921939 learned_addr learned my addr 172.21.15.62:0/2359921939 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5ff259c700 1 -- 172.21.15.62:0/2359921939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4005040 con 0x7f5ff40fbad0 2024-02-22T15:42:02.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5ff259c700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff410fb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009330 tx=0x7f5fe4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fe40096b0 con 0x7f5ff40fbad0 2024-02-22T15:42:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff4065f60 con 0x7f5ff40fbad0 2024-02-22T15:42:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5fe4009810 con 0x7f5ff40fbad0 2024-02-22T15:42:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fe401c5b0 con 0x7f5ff40fbad0 2024-02-22T15:42:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.724+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff4110290 con 0x7f5ff40fbad0 2024-02-22T15:42:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.725+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f5fe4013420 con 0x7f5ff40fbad0 2024-02-22T15:42:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.725+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff4105760 con 0x7f5ff40fbad0 2024-02-22T15:42:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.725+0000 7f5feaffd700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f5fe0036870 0x7f5fe0038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.725+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5fe4049c50 con 0x7f5ff40fbad0 2024-02-22T15:42:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.725+0000 7f5ff1d9b700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f5fe0036870 0x7f5fe0038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:02.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.726+0000 7f5ff1d9b700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f5fe0036870 0x7f5fe0038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009990 tx=0x7f5fdc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:02.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.730+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5fe40119c0 con 0x7f5ff40fbad0 2024-02-22T15:42:02.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:02.999+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5ff404bed0 con 0x7f5ff40fbad0 2024-02-22T15:42:02.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.000+0000 7f5feaffd700 1 -- 172.21.15.62:0/2359921939 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5fe4011370 con 0x7f5ff40fbad0 2024-02-22T15:42:02.985 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:02.985 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.001+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f5fe0036870 msgr2=0x7f5fe0038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.001+0000 7f5ff8f53700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f5fe0036870 0x7f5fe0038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009990 tx=0x7f5fdc006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.001+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 msgr2=0x7f5ff410fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.001+0000 7f5ff8f53700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff410fb10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009330 tx=0x7f5fe4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.002+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 shutdown_connections 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.002+0000 7f5ff8f53700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f5fe0036870 0x7f5fe0038d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.002+0000 7f5ff8f53700 1 --2- 172.21.15.62:0/2359921939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff40fbad0 0x7f5ff410fb10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.002+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 >> 172.21.15.62:0/2359921939 conn(0x7f5ff40f7190 msgr2=0x7f5ff40f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:02.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.002+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 shutdown_connections 2024-02-22T15:42:02.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:03.002+0000 7f5ff8f53700 1 -- 172.21.15.62:0/2359921939 wait complete. 2024-02-22T15:42:02.988 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:04.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:03 smithi019 ceph-mon[67418]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:04.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:03 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2359921939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:04.552 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:04.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:04.853 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:06.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:05 smithi019 ceph-mon[67418]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 -- 172.21.15.62:0/1692486459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 msgr2=0x7ff0b00fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 --2- 172.21.15.62:0/1692486459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b00fcfe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40048c0 tx=0x7ff0a4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 -- 172.21.15.62:0/1692486459 shutdown_connections 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 --2- 172.21.15.62:0/1692486459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b00fcfe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 -- 172.21.15.62:0/1692486459 >> 172.21.15.62:0/1692486459 conn(0x7ff0b00f8270 msgr2=0x7ff0b00fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 -- 172.21.15.62:0/1692486459 shutdown_connections 2024-02-22T15:42:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 -- 172.21.15.62:0/1692486459 wait complete. 2024-02-22T15:42:06.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 Processor -- start 2024-02-22T15:42:06.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.532+0000 7ff0b6f10700 1 -- start start 2024-02-22T15:42:06.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b6f10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b010ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:06.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b6f10700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b010d410 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b4cac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b010ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b4cac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b010ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:35690/0 (socket says 172.21.15.62:35690) 2024-02-22T15:42:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b4cac700 1 -- 172.21.15.62:0/2702597438 learned_addr learned my addr 172.21.15.62:0/2702597438 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b4cac700 1 -- 172.21.15.62:0/2702597438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4005040 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.533+0000 7ff0b4cac700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b010ced0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009330 tx=0x7ff0a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.534+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0a40096b0 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.534+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0a4009810 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.534+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0a401c5b0 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.534+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b010d610 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.534+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b010e1d0 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.534+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff0a4013420 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.535+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b005f5c0 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.535+0000 7ff0ad7fa700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ff0a0036910 0x7ff0a0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.535+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff0a4049c40 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.535+0000 7ff0affff700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ff0a0036910 0x7ff0a0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.535+0000 7ff0affff700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ff0a0036910 0x7ff0a0038dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009990 tx=0x7ff09c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:06.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.540+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff0a40119c0 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.809+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff0b010ee10 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.810+0000 7ff0ad7fa700 1 -- 172.21.15.62:0/2702597438 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff0a4011370 con 0x7ff0b00fcbd0 2024-02-22T15:42:06.795 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:06.796 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ff0a0036910 msgr2=0x7ff0a0038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ff0a0036910 0x7ff0a0038dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009990 tx=0x7ff09c006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 msgr2=0x7ff0b010ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b010ced0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009330 tx=0x7ff0a4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 shutdown_connections 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7ff0a0036910 0x7ff0a0038dc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 --2- 172.21.15.62:0/2702597438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0b00fcbd0 0x7ff0b010ced0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 >> 172.21.15.62:0/2702597438 conn(0x7ff0b00f8270 msgr2=0x7ff0b00f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 shutdown_connections 2024-02-22T15:42:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:06.812+0000 7ff0b6f10700 1 -- 172.21.15.62:0/2702597438 wait complete. 2024-02-22T15:42:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:06 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2702597438' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:08.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:07 smithi019 ceph-mon[67418]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:08.458 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:08.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:08.754 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:08.816 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:10.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:09 smithi019 ceph-mon[67418]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 -- 172.21.15.62:0/445734180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 msgr2=0x7fb9e80fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 --2- 172.21.15.62:0/445734180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e80fbee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40048c0 tx=0x7fb9e4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 -- 172.21.15.62:0/445734180 shutdown_connections 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 --2- 172.21.15.62:0/445734180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e80fbee0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 -- 172.21.15.62:0/445734180 >> 172.21.15.62:0/445734180 conn(0x7fb9e80f7190 msgr2=0x7fb9e80f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 -- 172.21.15.62:0/445734180 shutdown_connections 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.404+0000 7fb9f00ba700 1 -- 172.21.15.62:0/445734180 wait complete. 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9f00ba700 1 Processor -- start 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9f00ba700 1 -- start start 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9f00ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e810fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9f00ba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8065dc0 con 0x7fb9e80fbad0 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9ede56700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e810fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9ede56700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e810fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:35718/0 (socket says 172.21.15.62:35718) 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9ede56700 1 -- 172.21.15.62:0/1963256032 learned_addr learned my addr 172.21.15.62:0/1963256032 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.405+0000 7fb9ede56700 1 -- 172.21.15.62:0/1963256032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4005040 con 0x7fb9e80fbad0 2024-02-22T15:42:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.406+0000 7fb9ede56700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e810fab0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009330 tx=0x7fb9e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.406+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9e40096b0 con 0x7fb9e80fbad0 2024-02-22T15:42:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.406+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e8065fc0 con 0x7fb9e80fbad0 2024-02-22T15:42:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.406+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9e4009810 con 0x7fb9e80fbad0 2024-02-22T15:42:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.406+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9e401c5b0 con 0x7fb9e80fbad0 2024-02-22T15:42:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.406+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e8110230 con 0x7fb9e80fbad0 2024-02-22T15:42:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.407+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e8105760 con 0x7fb9e80fbad0 2024-02-22T15:42:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.407+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb9e4013420 con 0x7fb9e80fbad0 2024-02-22T15:42:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.407+0000 7fb9da7fc700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb9d40368c0 0x7fb9d4038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.407+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb9e4049c40 con 0x7fb9e80fbad0 2024-02-22T15:42:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.407+0000 7fb9ed655700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb9d40368c0 0x7fb9d4038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.408+0000 7fb9ed655700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb9d40368c0 0x7fb9d4038d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009990 tx=0x7fb9dc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.412+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb9e40119c0 con 0x7fb9e80fbad0 2024-02-22T15:42:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.684+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb9e804bed0 con 0x7fb9e80fbad0 2024-02-22T15:42:10.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.685+0000 7fb9da7fc700 1 -- 172.21.15.62:0/1963256032 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb9e4011370 con 0x7fb9e80fbad0 2024-02-22T15:42:10.671 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:10.671 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb9d40368c0 msgr2=0x7fb9d4038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb9d40368c0 0x7fb9d4038d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009990 tx=0x7fb9dc006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 msgr2=0x7fb9e810fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e810fab0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009330 tx=0x7fb9e4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 shutdown_connections 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fb9d40368c0 0x7fb9d4038d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 --2- 172.21.15.62:0/1963256032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9e80fbad0 0x7fb9e810fab0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 >> 172.21.15.62:0/1963256032 conn(0x7fb9e80f7190 msgr2=0x7fb9e80f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 shutdown_connections 2024-02-22T15:42:10.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:10.687+0000 7fb9f00ba700 1 -- 172.21.15.62:0/1963256032 wait complete. 2024-02-22T15:42:10.674 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:11.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:10 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/1963256032' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:12.052 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:11 smithi019 ceph-mon[67418]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:12.317 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:12.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:12.613 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 -- 172.21.15.62:0/3544758761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 msgr2=0x7fd084101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 --2- 172.21.15.62:0/3544758761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd084101b80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd0740048c0 tx=0x7fd074009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 -- 172.21.15.62:0/3544758761 shutdown_connections 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 --2- 172.21.15.62:0/3544758761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd084101b80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 -- 172.21.15.62:0/3544758761 >> 172.21.15.62:0/3544758761 conn(0x7fd0840f7190 msgr2=0x7fd0840f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 -- 172.21.15.62:0/3544758761 shutdown_connections 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.235+0000 7fd08a849700 1 -- 172.21.15.62:0/3544758761 wait complete. 2024-02-22T15:42:14.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd08a849700 1 Processor -- start 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd08a849700 1 -- start start 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd08a849700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd08410efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd08a849700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08410f4e0 con 0x7fd084065df0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd083fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd08410efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd083fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd08410efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:35734/0 (socket says 172.21.15.62:35734) 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd083fff700 1 -- 172.21.15.62:0/3000922094 learned_addr learned my addr 172.21.15.62:0/3000922094 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.236+0000 7fd083fff700 1 -- 172.21.15.62:0/3000922094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074005040 con 0x7fd084065df0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.237+0000 7fd083fff700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd08410efa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd07400b690 tx=0x7fd074009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.237+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd074009650 con 0x7fd084065df0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.237+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0740097b0 con 0x7fd084065df0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.237+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd07401c5b0 con 0x7fd084065df0 2024-02-22T15:42:14.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.237+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08410f6e0 con 0x7fd084065df0 2024-02-22T15:42:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.237+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd084110380 con 0x7fd084065df0 2024-02-22T15:42:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.238+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7fd07401c870 con 0x7fd084065df0 2024-02-22T15:42:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.238+0000 7fd080ff9700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 0x7fd06c038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.238+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08405f5c0 con 0x7fd084065df0 2024-02-22T15:42:14.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.238+0000 7fd0837fe700 1 -- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 msgr2=0x7fd06c038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1788498096 2024-02-22T15:42:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.238+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd074049bb0 con 0x7fd084065df0 2024-02-22T15:42:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.238+0000 7fd0837fe700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 0x7fd06c038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:42:14.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.243+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd0740119c0 con 0x7fd084065df0 2024-02-22T15:42:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:13 smithi019 ceph-mon[67418]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:13 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:14.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:13 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:14.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:13 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:14.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:13 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-22T15:42:14.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.439+0000 7fd0837fe700 1 -- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 msgr2=0x7fd06c038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1788498096 2024-02-22T15:42:14.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.439+0000 7fd0837fe700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 0x7fd06c038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-22T15:42:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.513+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd084110ea0 con 0x7fd084065df0 2024-02-22T15:42:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.514+0000 7fd080ff9700 1 -- 172.21.15.62:0/3000922094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd074011370 con 0x7fd084065df0 2024-02-22T15:42:14.499 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:14.499 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 msgr2=0x7fd06c038d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:42:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 0x7fd06c038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 msgr2=0x7fd08410efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd08410efa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd07400b690 tx=0x7fd074009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 shutdown_connections 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7fd06c0368c0 0x7fd06c038d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 --2- 172.21.15.62:0/3000922094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd084065df0 0x7fd08410efa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 >> 172.21.15.62:0/3000922094 conn(0x7fd0840f7190 msgr2=0x7fd0840f7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 shutdown_connections 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:14.516+0000 7fd08a849700 1 -- 172.21.15.62:0/3000922094 wait complete. 2024-02-22T15:42:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:15.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:15 smithi019 ceph-mon[67418]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-22T15:42:15.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:15 smithi019 ceph-mon[67418]: mgrmap e14: smithi019.yvjrkp(active, since 63s) 2024-02-22T15:42:15.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:15 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/3000922094' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:16.152 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:16.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:16.449 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:18.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.246+0000 7f1eb0b32700 1 -- 172.21.15.62:0/817280651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 msgr2=0x7f1eac0fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:18.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.246+0000 7f1eb0b32700 1 --2- 172.21.15.62:0/817280651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac0fbee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0048c0 tx=0x7f1e9c009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.246+0000 7f1eb0b32700 1 -- 172.21.15.62:0/817280651 shutdown_connections 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.246+0000 7f1eb0b32700 1 --2- 172.21.15.62:0/817280651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac0fbee0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.246+0000 7f1eb0b32700 1 -- 172.21.15.62:0/817280651 >> 172.21.15.62:0/817280651 conn(0x7f1eac0f7190 msgr2=0x7f1eac0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.246+0000 7f1eb0b32700 1 -- 172.21.15.62:0/817280651 shutdown_connections 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eb0b32700 1 -- 172.21.15.62:0/817280651 wait complete. 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eb0b32700 1 Processor -- start 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eb0b32700 1 -- start start 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eb0b32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac10bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eb0b32700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac10c2a0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eaa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac10bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eaa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac10bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:58744/0 (socket says 172.21.15.62:58744) 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.247+0000 7f1eaa59c700 1 -- 172.21.15.62:0/3913088031 learned_addr learned my addr 172.21.15.62:0/3913088031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1eaa59c700 1 -- 172.21.15.62:0/3913088031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c005040 con 0x7f1eac0fbad0 2024-02-22T15:42:18.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1eaa59c700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac10bd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000c00 tx=0x7f1e9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:18.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e9c0097f0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1e9c01e8d0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac10c4a0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e9c0137e0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.248+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac10d060 con 0x7f1eac0fbad0 2024-02-22T15:42:18.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.249+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f1e9c01e420 con 0x7f1eac0fbad0 2024-02-22T15:42:18.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.249+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eac05f5c0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.249+0000 7f1ea2ffd700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 0x7f1e98038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:18.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.249+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1e9c04c5f0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.249+0000 7f1ea9d9b700 1 -- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 msgr2=0x7f1e98038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1788498096 2024-02-22T15:42:18.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.249+0000 7f1ea9d9b700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 0x7f1e98038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:42:18.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.254+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1e9c01ea40 con 0x7f1eac0fbad0 2024-02-22T15:42:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.450+0000 7f1ea9d9b700 1 -- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 msgr2=0x7f1e98038d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1788498096 2024-02-22T15:42:18.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.450+0000 7f1ea9d9b700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 0x7f1e98038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-22T15:42:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.525+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1eac04bed0 con 0x7f1eac0fbad0 2024-02-22T15:42:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.526+0000 7f1ea2ffd700 1 -- 172.21.15.62:0/3913088031 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1e9c011370 con 0x7f1eac0fbad0 2024-02-22T15:42:18.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:18.512 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:18.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 msgr2=0x7f1e98038d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:42:18.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 0x7f1e98038d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:18.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 msgr2=0x7f1eac10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac10bd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000c00 tx=0x7f1e9c009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 shutdown_connections 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:6800/1788498096,v1:172.21.15.19:6801/1788498096] conn(0x7f1e980368c0 0x7f1e98038d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 --2- 172.21.15.62:0/3913088031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1eac0fbad0 0x7f1eac10bd60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 >> 172.21.15.62:0/3913088031 conn(0x7f1eac0f7190 msgr2=0x7f1eac0f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 shutdown_connections 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:18.528+0000 7f1eb0b32700 1 -- 172.21.15.62:0/3913088031 wait complete. 2024-02-22T15:42:18.514 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:18.595 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:18 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/3913088031' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:20.159 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:20.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:20.462 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:20.531 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:20.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: Active manager daemon smithi019.yvjrkp restarted 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: osdmap e5: 0 total, 0 up, 0 in 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: mgrmap e15: smithi019.yvjrkp(active, starting, since 0.00382641s) 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:42:20.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:42:21.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:42:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3529044989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 msgr2=0x7fc4e40ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 --2- 172.21.15.62:0/3529044989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40ffaf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40048c0 tx=0x7fc4d4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3529044989 shutdown_connections 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 --2- 172.21.15.62:0/3529044989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40ffaf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3529044989 >> 172.21.15.62:0/3529044989 conn(0x7fc4e40f7190 msgr2=0x7fc4e40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3529044989 shutdown_connections 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.124+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3529044989 wait complete. 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.125+0000 7fc4e9b63700 1 Processor -- start 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.125+0000 7fc4e9b63700 1 -- start start 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.125+0000 7fc4e9b63700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.125+0000 7fc4e9b63700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40fb620 con 0x7fc4e40fd710 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40fd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:58756/0 (socket says 172.21.15.62:58756) 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e37fe700 1 -- 172.21.15.62:0/3623441700 learned_addr learned my addr 172.21.15.62:0/3623441700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e37fe700 1 -- 172.21.15.62:0/3623441700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4005040 con 0x7fc4e40fd710 2024-02-22T15:42:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e37fe700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40fd1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400b690 tx=0x7fc4d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4d4009650 con 0x7fc4e40fd710 2024-02-22T15:42:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40fb820 con 0x7fc4e40fd710 2024-02-22T15:42:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4d40097b0 con 0x7fc4e40fd710 2024-02-22T15:42:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4d401c5b0 con 0x7fc4e40fd710 2024-02-22T15:42:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.126+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e40fbd10 con 0x7fc4e40fd710 2024-02-22T15:42:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.127+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7fc4d401c820 con 0x7fc4e40fd710 2024-02-22T15:42:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.127+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e4106860 con 0x7fc4e40fd710 2024-02-22T15:42:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.127+0000 7fc4cbfff700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc4cc036750 0x7fc4cc038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.127+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc4d4049aa0 con 0x7fc4e40fd710 2024-02-22T15:42:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.127+0000 7fc4e2ffd700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc4cc036750 0x7fc4cc038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.128+0000 7fc4e2ffd700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc4cc036750 0x7fc4cc038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009990 tx=0x7fc4d8006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:22.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.132+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc4d4023300 con 0x7fc4e40fd710 2024-02-22T15:42:22.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:22.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:42:22.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:42:22.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:42:22.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:22.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:21 smithi019 ceph-mon[67418]: mgrmap e16: smithi019.yvjrkp(active, since 1.00542s) 2024-02-22T15:42:22.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.407+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc4e404bed0 con 0x7fc4e40fd710 2024-02-22T15:42:22.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.407+0000 7fc4cbfff700 1 -- 172.21.15.62:0/3623441700 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc4d4011b60 con 0x7fc4e40fd710 2024-02-22T15:42:22.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:22.393 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc4cc036750 msgr2=0x7fc4cc038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc4cc036750 0x7fc4cc038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009990 tx=0x7fc4d8006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 msgr2=0x7fc4e40fd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40fd1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400b690 tx=0x7fc4d4009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 shutdown_connections 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc4cc036750 0x7fc4cc038c00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 --2- 172.21.15.62:0/3623441700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4e40fd710 0x7fc4e40fd1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 >> 172.21.15.62:0/3623441700 conn(0x7fc4e40f7190 msgr2=0x7fc4e40f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 shutdown_connections 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:22.411+0000 7fc4e9b63700 1 -- 172.21.15.62:0/3623441700 wait complete. 2024-02-22T15:42:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:23.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:23 smithi019 ceph-mon[67418]: [22/Feb/2024:15:42:22] ENGINE Bus STARTING 2024-02-22T15:42:23.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:23 smithi019 ceph-mon[67418]: [22/Feb/2024:15:42:22] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:42:23.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:23 smithi019 ceph-mon[67418]: [22/Feb/2024:15:42:22] ENGINE Bus STARTED 2024-02-22T15:42:23.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:23.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:23.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:23 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/3623441700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:24.120 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:24.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:24.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:24 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:24.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:24 smithi019 ceph-mon[67418]: mgrmap e17: smithi019.yvjrkp(active, since 2s) 2024-02-22T15:42:25.112 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:25.171 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:27.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2790714978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 msgr2=0x7fd4d80ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 --2- 172.21.15.62:0/2790714978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d80ff7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80048c0 tx=0x7fd4c8009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2790714978 shutdown_connections 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 --2- 172.21.15.62:0/2790714978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d80ff7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2790714978 >> 172.21.15.62:0/2790714978 conn(0x7fd4d80f7190 msgr2=0x7fd4d80f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2790714978 shutdown_connections 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2790714978 wait complete. 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 Processor -- start 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.334+0000 7fd4de5dc700 1 -- start start 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4de5dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d810ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:27.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4de5dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d810d360 con 0x7fd4d8065dd0 2024-02-22T15:42:27.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d810ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d810ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:37326/0 (socket says 172.21.15.62:37326) 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4d7fff700 1 -- 172.21.15.62:0/2231087764 learned_addr learned my addr 172.21.15.62:0/2231087764 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4d7fff700 1 -- 172.21.15.62:0/2231087764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c8005040 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4d7fff700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d810ce20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800b690 tx=0x7fd4c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.335+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4c8009650 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.336+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d810d560 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.336+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4c80097b0 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.336+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4c801c5b0 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.336+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d810e120 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.337+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fd4c8013a60 con 0x7fd4d8065dd0 2024-02-22T15:42:27.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.337+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d805f5c0 con 0x7fd4d8065dd0 2024-02-22T15:42:27.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.337+0000 7fd4d4ff9700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd4c00369b0 0x7fd4c0038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:27.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.337+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd4c8049c10 con 0x7fd4d8065dd0 2024-02-22T15:42:27.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.337+0000 7fd4d77fe700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd4c00369b0 0x7fd4c0038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:27.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.338+0000 7fd4d77fe700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd4c00369b0 0x7fd4c0038e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009990 tx=0x7fd4cc006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:27.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.342+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd4c8011550 con 0x7fd4d8065dd0 2024-02-22T15:42:27.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.610+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd4d810ed60 con 0x7fd4d8065dd0 2024-02-22T15:42:27.596 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:27.597 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:27.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.611+0000 7fd4d4ff9700 1 -- 172.21.15.62:0/2231087764 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd4c8011370 con 0x7fd4d8065dd0 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.614+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd4c00369b0 msgr2=0x7fd4c0038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.614+0000 7fd4de5dc700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd4c00369b0 0x7fd4c0038e60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009990 tx=0x7fd4cc006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 msgr2=0x7fd4d810ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d810ce20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800b690 tx=0x7fd4c8009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 shutdown_connections 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd4c00369b0 0x7fd4c0038e60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 --2- 172.21.15.62:0/2231087764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d8065dd0 0x7fd4d810ce20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 >> 172.21.15.62:0/2231087764 conn(0x7fd4d80f7190 msgr2=0x7fd4d80f7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 shutdown_connections 2024-02-22T15:42:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:27.615+0000 7fd4de5dc700 1 -- 172.21.15.62:0/2231087764 wait complete. 2024-02-22T15:42:27.601 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:28.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:28.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:28.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:28.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:28.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:42:28.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:27 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2231087764' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:29.159 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:29.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:29.460 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 -- 172.21.15.62:0/155828884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 msgr2=0x7fba2c0fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 --2- 172.21.15.62:0/155828884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c0fbee0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0048c0 tx=0x7fba1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 -- 172.21.15.62:0/155828884 shutdown_connections 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 --2- 172.21.15.62:0/155828884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c0fbee0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 -- 172.21.15.62:0/155828884 >> 172.21.15.62:0/155828884 conn(0x7fba2c0f7190 msgr2=0x7fba2c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 -- 172.21.15.62:0/155828884 shutdown_connections 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.150+0000 7fba327b2700 1 -- 172.21.15.62:0/155828884 wait complete. 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba327b2700 1 Processor -- start 2024-02-22T15:42:31.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba327b2700 1 -- start start 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba327b2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c10faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba327b2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c065d80 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c10faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c10faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:37332/0 (socket says 172.21.15.62:37332) 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba2bfff700 1 -- 172.21.15.62:0/2104366841 learned_addr learned my addr 172.21.15.62:0/2104366841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.151+0000 7fba2bfff700 1 -- 172.21.15.62:0/2104366841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c005040 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.152+0000 7fba2bfff700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c10faf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009330 tx=0x7fba1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.152+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba1c0096b0 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.152+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba1c009810 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.152+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c065f80 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.152+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba1c01c5b0 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.152+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c110270 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.153+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fba1c013420 con 0x7fba2c0fbad0 2024-02-22T15:42:31.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.153+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c105760 con 0x7fba2c0fbad0 2024-02-22T15:42:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.153+0000 7fba28ff9700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fba14036960 0x7fba14038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.153+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fba1c049ec0 con 0x7fba2c0fbad0 2024-02-22T15:42:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.153+0000 7fba2b7fe700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fba14036960 0x7fba14038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.154+0000 7fba2b7fe700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fba14036960 0x7fba14038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba20009990 tx=0x7fba20006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:31.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.158+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fba1c0119c0 con 0x7fba2c0fbad0 2024-02-22T15:42:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.426+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fba2c04bed0 con 0x7fba2c0fbad0 2024-02-22T15:42:31.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.427+0000 7fba28ff9700 1 -- 172.21.15.62:0/2104366841 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fba1c011370 con 0x7fba2c0fbad0 2024-02-22T15:42:31.413 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:31.413 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fba14036960 msgr2=0x7fba14038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fba14036960 0x7fba14038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba20009990 tx=0x7fba20006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 msgr2=0x7fba2c10faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:31.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c10faf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009330 tx=0x7fba1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 shutdown_connections 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fba14036960 0x7fba14038e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 --2- 172.21.15.62:0/2104366841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c0fbad0 0x7fba2c10faf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 >> 172.21.15.62:0/2104366841 conn(0x7fba2c0f7190 msgr2=0x7fba2c0f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 shutdown_connections 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:31.429+0000 7fba327b2700 1 -- 172.21.15.62:0/2104366841 wait complete. 2024-02-22T15:42:31.415 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:32.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:32.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:32.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2104366841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:32.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:32.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:32.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:42:32.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:32.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:42:33.032 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:33.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:33.344 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:33.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-22T15:42:33.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:34 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:42:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:34 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:42:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:34 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:42:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:34 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:42:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:34 smithi019 ceph-mon[67418]: Deploying daemon crash.smithi062 on smithi062 2024-02-22T15:42:36.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.081+0000 7f70a413e700 1 -- 172.21.15.62:0/3367302022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 msgr2=0x7f709c0fe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.081+0000 7f70a413e700 1 --2- 172.21.15.62:0/3367302022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c0fe020 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70980048c0 tx=0x7f7098009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.081+0000 7f70a413e700 1 -- 172.21.15.62:0/3367302022 shutdown_connections 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.081+0000 7f70a413e700 1 --2- 172.21.15.62:0/3367302022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c0fe020 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.081+0000 7f70a413e700 1 -- 172.21.15.62:0/3367302022 >> 172.21.15.62:0/3367302022 conn(0x7f709c072850 msgr2=0x7f709c072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a413e700 1 -- 172.21.15.62:0/3367302022 shutdown_connections 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a413e700 1 -- 172.21.15.62:0/3367302022 wait complete. 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a413e700 1 Processor -- start 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a413e700 1 -- start start 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a413e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c1150e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a413e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c115620 con 0x7f709c0fdc50 2024-02-22T15:42:36.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.082+0000 7f70a1eda700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c1150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70a1eda700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c1150e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:56872/0 (socket says 172.21.15.62:56872) 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70a1eda700 1 -- 172.21.15.62:0/1562648547 learned_addr learned my addr 172.21.15.62:0/1562648547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70a1eda700 1 -- 172.21.15.62:0/1562648547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098005040 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70a1eda700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c1150e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7098009300 tx=0x7f7098009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7098004e60 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7098013420 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f709801c5b0 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c115820 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.083+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c115c90 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.084+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f709801c7a0 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.084+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c05f5c0 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.084+0000 7f70927fc700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f70880369b0 0x7f7088038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.084+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7098049c00 con 0x7f709c0fdc50 2024-02-22T15:42:36.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.085+0000 7f70a16d9700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f70880369b0 0x7f7088038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:36.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.085+0000 7f70a16d9700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f70880369b0 0x7f7088038e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f708c009990 tx=0x7f708c006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:36.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.090+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7098025400 con 0x7f709c0fdc50 2024-02-22T15:42:36.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:42:36.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.383+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f709c106df0 con 0x7f709c0fdc50 2024-02-22T15:42:36.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.384+0000 7f70927fc700 1 -- 172.21.15.62:0/1562648547 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7098049e60 con 0x7f709c0fdc50 2024-02-22T15:42:36.369 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:36.369 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f70880369b0 msgr2=0x7f7088038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f70880369b0 0x7f7088038e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f708c009990 tx=0x7f708c006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 msgr2=0x7f709c1150e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c1150e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7098009300 tx=0x7f7098009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 shutdown_connections 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f70880369b0 0x7f7088038e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 --2- 172.21.15.62:0/1562648547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f709c0fdc50 0x7f709c1150e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:36.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 >> 172.21.15.62:0/1562648547 conn(0x7f709c072850 msgr2=0x7f709c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:36.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 shutdown_connections 2024-02-22T15:42:36.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:36.386+0000 7f70a413e700 1 -- 172.21.15.62:0/1562648547 wait complete. 2024-02-22T15:42:36.372 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:37.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:37.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:37.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:37.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:37.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:37 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/1562648547' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:37.924 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:37.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:38.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:38 smithi019 ceph-mon[67418]: Deploying daemon node-exporter.smithi062 on smithi062 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.071+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/1336062898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 msgr2=0x7f2e14101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.071+0000 7f2e1bb6f700 1 --2- 172.21.15.62:0/1336062898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14101f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e100048c0 tx=0x7f2e10009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.071+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/1336062898 shutdown_connections 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.071+0000 7f2e1bb6f700 1 --2- 172.21.15.62:0/1336062898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14101f10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.071+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/1336062898 >> 172.21.15.62:0/1336062898 conn(0x7f2e140f7640 msgr2=0x7f2e140f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.071+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/1336062898 shutdown_connections 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.072+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/1336062898 wait complete. 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.072+0000 7f2e1bb6f700 1 Processor -- start 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.072+0000 7f2e1bb6f700 1 -- start start 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1bb6f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14111d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1bb6f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14112270 con 0x7f2e14101b40 2024-02-22T15:42:41.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1990b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14111d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1990b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14111d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:56906/0 (socket says 172.21.15.62:56906) 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1990b700 1 -- 172.21.15.62:0/2586560897 learned_addr learned my addr 172.21.15.62:0/2586560897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1990b700 1 -- 172.21.15.62:0/2586560897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10005040 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1990b700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14111d30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009b80 tx=0x7f2e10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e10004e60 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e10009620 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.073+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14112470 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.074+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e1001c3f0 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.074+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e14112960 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.074+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f2e1001c620 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.074+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1405f5c0 con 0x7f2e14101b40 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.075+0000 7f2e0a7fc700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f2e000369b0 0x7f2e00038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.075+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2e10049d80 con 0x7f2e14101b40 2024-02-22T15:42:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.075+0000 7f2e1910a700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f2e000369b0 0x7f2e00038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.075+0000 7f2e1910a700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f2e000369b0 0x7f2e00038e60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009990 tx=0x7f2e04006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.080+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2e10013940 con 0x7f2e14101b40 2024-02-22T15:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.361+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2e1410d4a0 con 0x7f2e14101b40 2024-02-22T15:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.361+0000 7f2e0a7fc700 1 -- 172.21.15.62:0/2586560897 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2e10023300 con 0x7f2e14101b40 2024-02-22T15:42:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:41.346 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f2e000369b0 msgr2=0x7f2e00038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f2e000369b0 0x7f2e00038e60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009990 tx=0x7f2e04006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 msgr2=0x7f2e14111d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14111d30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009b80 tx=0x7f2e10009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 shutdown_connections 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f2e000369b0 0x7f2e00038e60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 --2- 172.21.15.62:0/2586560897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e14101b40 0x7f2e14111d30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 >> 172.21.15.62:0/2586560897 conn(0x7f2e140f7640 msgr2=0x7f2e14103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 shutdown_connections 2024-02-22T15:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:41.363+0000 7f2e1bb6f700 1 -- 172.21.15.62:0/2586560897 wait complete. 2024-02-22T15:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:42.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:42.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: Deploying daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:42:42.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:41 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2586560897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:42.978 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:42.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:43.277 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:44.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:43 smithi019 ceph-mon[67418]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:44.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:43 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:44.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:43 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:45.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:45.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:45.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:42:45.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:45.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:44 smithi019 ceph-mon[67418]: Deploying daemon mon.smithi062 on smithi062 2024-02-22T15:42:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.408+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/2675655858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 msgr2=0x7f0f28102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.408+0000 7f0f2fdbc700 1 --2- 172.21.15.62:0/2675655858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28102fa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f240048c0 tx=0x7f0f24009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.408+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/2675655858 shutdown_connections 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.408+0000 7f0f2fdbc700 1 --2- 172.21.15.62:0/2675655858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28102fa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.408+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/2675655858 >> 172.21.15.62:0/2675655858 conn(0x7f0f280f8760 msgr2=0x7f0f280fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.408+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/2675655858 shutdown_connections 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.409+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/2675655858 wait complete. 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.409+0000 7f0f2fdbc700 1 Processor -- start 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.409+0000 7f0f2fdbc700 1 -- start start 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.409+0000 7f0f2fdbc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28112e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.409+0000 7f0f2fdbc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281133b0 con 0x7f0f28102bd0 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2db58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28112e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2db58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28112e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.62:43358/0 (socket says 172.21.15.62:43358) 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2db58700 1 -- 172.21.15.62:0/859642240 learned_addr learned my addr 172.21.15.62:0/859642240 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2db58700 1 -- 172.21.15.62:0/859642240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24005040 con 0x7f0f28102bd0 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2db58700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28112e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009300 tx=0x7f0f24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f24004e60 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0f24013420 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f281135b0 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f2401c5b0 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.410+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28113aa0 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.411+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f0f2401c7a0 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.411+0000 7f0f1e7fc700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0f140369b0 0x7f0f14038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.411+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0f24049c00 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.411+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2805f5c0 con 0x7f0f28102bd0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.412+0000 7f0f2d357700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0f140369b0 0x7f0f14038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.412+0000 7f0f2d357700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0f140369b0 0x7f0f14038e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009990 tx=0x7f0f18006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:45.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.417+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0f24023300 con 0x7f0f28102bd0 2024-02-22T15:42:45.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.690+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0f280008d0 con 0x7f0f28102bd0 2024-02-22T15:42:45.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.691+0000 7f0f1e7fc700 1 -- 172.21.15.62:0/859642240 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0f24011b60 con 0x7f0f28102bd0 2024-02-22T15:42:45.677 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:45.677 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:39:58.578235Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0f140369b0 msgr2=0x7f0f14038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0f140369b0 0x7f0f14038e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009990 tx=0x7f0f18006e40 comp rx=0 tx=0).stop 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 msgr2=0x7f0f28112e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28112e70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009300 tx=0x7f0f24009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 shutdown_connections 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0f140369b0 0x7f0f14038e60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:45.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 --2- 172.21.15.62:0/859642240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f28102bd0 0x7f0f28112e70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:45.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 >> 172.21.15.62:0/859642240 conn(0x7f0f280f8760 msgr2=0x7f0f280f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:45.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 shutdown_connections 2024-02-22T15:42:45.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:45.693+0000 7f0f2fdbc700 1 -- 172.21.15.62:0/859642240 wait complete. 2024-02-22T15:42:45.680 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 1 2024-02-22T15:42:46.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:45 smithi019 ceph-mon[67418]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:46.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:45 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/859642240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:46.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:47 smithi019 ceph-mon[67418]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.536 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mon.smithi062@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T15:42:48.537 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mon.smithi062@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T15:42:48.838 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-22T15:42:48.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mon dump -f json 2024-02-22T15:42:48.867 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mon.smithi062@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-22T15:42:48.867 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.867 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon crash.smithi019 on smithi019 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon node-exporter.smithi019 on smithi019 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:42:48.868 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying cephadm binary to smithi062 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon alertmanager.smithi019 on smithi019 2024-02-22T15:42:48.869 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Added host smithi062 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.870 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2812445821' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon grafana.smithi019 on smithi019 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2812445821' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: osdmap e4: 0 total, 0 up, 0 in 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.871 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Saving service mon spec with placement smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062;count:2 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2927799900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/852628452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/4025213305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.872 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon prometheus.smithi019 on smithi019 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2359921939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2702597438' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.873 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/1963256032' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14160 172.21.15.19:0/814360645' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mgrmap e14: smithi019.yvjrkp(active, since 63s) 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/3000922094' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/3913088031' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Active manager daemon smithi019.yvjrkp restarted 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: osdmap e5: 0 total, 0 up, 0 in 2024-02-22T15:42:48.874 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mgrmap e15: smithi019.yvjrkp(active, starting, since 0.00382641s) 2024-02-22T15:42:48.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:42:48.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:42:48.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:42:48.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mgrmap e16: smithi019.yvjrkp(active, since 1.00542s) 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: [22/Feb/2024:15:42:22] ENGINE Bus STARTING 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: [22/Feb/2024:15:42:22] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:42:48.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: [22/Feb/2024:15:42:22] ENGINE Bus STARTED 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/3623441700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mgrmap e17: smithi019.yvjrkp(active, since 2s) 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2231087764' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.877 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2104366841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.878 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon crash.smithi062 on smithi062 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.879 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/1562648547' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon node-exporter.smithi062 on smithi062 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-22T15:42:48.880 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2586560897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: Deploying daemon mon.smithi062 on smithi062 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/859642240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:48.881 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:48.882 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:48.882 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mon.smithi062@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-02-22T15:42:48.882 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 ceph-mon[48130]: mon.smithi062@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2024-02-22T15:42:48.882 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 bash[48087]: 8dce36aa88eb6de2dc0711336dc60319068751bbcc28ebbc2d9c914e24380304 2024-02-22T15:42:48.882 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:48 smithi062 systemd[1]: Started Ceph mon.smithi062 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:42:49.115 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:42:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 -- 172.21.15.62:0/4197751510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 msgr2=0x7f8eb0005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/4197751510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8eb0005a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0065630 tx=0x7f8eb400b040 comp rx=0 tx=0).stop 2024-02-22T15:42:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 -- 172.21.15.62:0/4197751510 shutdown_connections 2024-02-22T15:42:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/4197751510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8eb0005a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 -- 172.21.15.62:0/4197751510 >> 172.21.15.62:0/4197751510 conn(0x7f8ec00729f0 msgr2=0x7f8ec0072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 -- 172.21.15.62:0/4197751510 shutdown_connections 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 -- 172.21.15.62:0/4197751510 wait complete. 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.728+0000 7f8ec6e55700 1 Processor -- start 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ec6e55700 1 -- start start 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ec6e55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8ec01130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ec6e55700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ec6e55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec0113c40 con 0x7f8eb0003580 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ec6e55700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec010d4d0 con 0x7f8ec0113630 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ebffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ebffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33400/0 (socket says 172.21.15.62:33400) 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ebffff700 1 -- 172.21.15.62:0/1973537457 learned_addr learned my addr 172.21.15.62:0/1973537457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ec4bf1700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8ec01130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ebffff700 1 -- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 msgr2=0x7f8ec010cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-22T15:42:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ebffff700 1 -- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 msgr2=0x7f8ec010cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.729+0000 7f8ebffff700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ebffff700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ec4bf1700 1 -- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 msgr2=0x7f8ec010cf90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ec4bf1700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ec4bf1700 1 -- 172.21.15.62:0/1973537457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb4005040 con 0x7f8eb0003580 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ec4bf1700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8ec01130f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400b010 tx=0x7f8eb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eb401b070 con 0x7f8eb0003580 2024-02-22T15:42:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec010d750 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8eb40208f0 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec010dc60 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.730+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eb401f850 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.731+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec005f5c0 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.732+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f8eb4020440 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.732+0000 7f8ebd7fa700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8ea4069360 0x7f8ea406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.732+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8eb4087fa0 con 0x7f8eb0003580 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.732+0000 7f8ebffff700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8ea4069360 0x7f8ea406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:53.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.733+0000 7f8ebffff700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8ea4069360 0x7f8ea406b810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006fd0 tx=0x7f8eac008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:53.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:53.737+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8eb4058da0 con 0x7f8eb0003580 2024-02-22T15:42:53.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: mon.smithi019 calling monitor election 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: mon.smithi062 calling monitor election 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: mon.smithi019 is new leader, mons smithi019,smithi062 in quorum (ranks 0,1) 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: monmap e2: 2 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],smithi062=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: fsmap 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: osdmap e5: 0 total, 0 up, 0 in 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: mgrmap e17: smithi019.yvjrkp(active, since 32s) 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: overall HEALTH_OK 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:42:53.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:53.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:42:53.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: mon.smithi019 calling monitor election 2024-02-22T15:42:53.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: mon.smithi062 calling monitor election 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: mon.smithi019 is new leader, mons smithi019,smithi062 in quorum (ranks 0,1) 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: monmap e2: 2 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],smithi062=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: fsmap 2024-02-22T15:42:53.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: osdmap e5: 0 total, 0 up, 0 in 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: mgrmap e17: smithi019.yvjrkp(active, since 32s) 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: overall HEALTH_OK 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/3181285664' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:42:54.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.019+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8ec004bed0 con 0x7f8eb0003580 2024-02-22T15:42:54.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.020+0000 7f8ebd7fa700 1 -- 172.21.15.62:0/1973537457 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7f8eb4024090 con 0x7f8eb0003580 2024-02-22T15:42:54.005 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-02-22T15:42:54.005 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","modified":"2024-02-22T15:42:48.558101Z","created":"2024-02-22T15:39:58.578235Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi062","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-22T15:42:54.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8ea4069360 msgr2=0x7f8ea406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8ea4069360 0x7f8ea406b810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006fd0 tx=0x7f8eac008040 comp rx=0 tx=0).stop 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 msgr2=0x7f8ec01130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8ec01130f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400b010 tx=0x7f8eb400b040 comp rx=0 tx=0).stop 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 shutdown_connections 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8ea4069360 0x7f8ea406b810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eb0003580 0x7f8ec01130f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 --2- 172.21.15.62:0/1973537457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0113630 0x7f8ec010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 >> 172.21.15.62:0/1973537457 conn(0x7f8ec00729f0 msgr2=0x7f8ec00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 shutdown_connections 2024-02-22T15:42:54.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:42:54.022+0000 7f8ec6e55700 1 -- 172.21.15.62:0/1973537457 wait complete. 2024-02-22T15:42:54.009 INFO:teuthology.orchestra.run.smithi062.stderr:dumped monmap epoch 2 2024-02-22T15:42:54.674 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-22T15:42:54.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph config generate-minimal-conf 2024-02-22T15:42:54.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: mgrmap e18: smithi019.yvjrkp(active, since 32s), standbys: smithi062.ecubmv 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/1973537457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:54.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: mgrmap e18: smithi019.yvjrkp(active, since 32s), standbys: smithi062.ecubmv 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/1973537457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-22T15:42:54.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:42:55.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:55 smithi062 ceph-mon[48130]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:55.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:55 smithi019 ceph-mon[67418]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:56.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 -- 172.21.15.19:0/3008197155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c101b60 msgr2=0x7fcd4c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:56.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 --2- 172.21.15.19:0/3008197155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c101b60 0x7fcd4c101f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0048c0 tx=0x7fcd3c009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:56.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 -- 172.21.15.19:0/3008197155 shutdown_connections 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 --2- 172.21.15.19:0/3008197155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c101b60 0x7fcd4c101f30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 -- 172.21.15.19:0/3008197155 >> 172.21.15.19:0/3008197155 conn(0x7fcd4c072850 msgr2=0x7fcd4c072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 -- 172.21.15.19:0/3008197155 shutdown_connections 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 -- 172.21.15.19:0/3008197155 wait complete. 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 Processor -- start 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 -- start start 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 0x7fcd4c0fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.815+0000 7fcd52207700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 0x7fcd4c0fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd52207700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c119250 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd52207700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c119390 con 0x7fcd4c101b60 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 0x7fcd4c0fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 0x7fcd4c0fe080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33212/0 (socket says 172.21.15.19:33212) 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 -- 172.21.15.19:0/2664426036 learned_addr learned my addr 172.21.15.19:0/2664426036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd4b7fe700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 0x7fcd4c0fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd4b7fe700 1 -- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 msgr2=0x7fcd4c0fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd4b7fe700 1 -- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 msgr2=0x7fcd4c0fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd4b7fe700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 0x7fcd4c0fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd4b7fe700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 0x7fcd4c0fd6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 -- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 msgr2=0x7fcd4c0fd6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 0x7fcd4c0fd6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 -- 172.21.15.19:0/2664426036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c005040 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd43fff700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 0x7fcd4c0fe080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b790 tx=0x7fcd34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd34010870 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd34010e80 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c1195f0 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd3400d3b0 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.816+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c119b80 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.817+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4c05f5c0 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.818+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fcd3400f3e0 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.818+0000 7fcd48ff9700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcd38069370 0x7fcd3806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:42:56.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.819+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fcd340865b0 con 0x7fcd4c0fdc10 2024-02-22T15:42:56.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.819+0000 7fcd4b7fe700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcd38069370 0x7fcd3806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:42:56.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.819+0000 7fcd4b7fe700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcd38069370 0x7fcd3806b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b780 tx=0x7fcd3c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:42:56.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:56.826+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcd340539c0 con 0x7fcd4c0fdc10 2024-02-22T15:42:57.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.020+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fcd4c105c80 con 0x7fcd4c0fdc10 2024-02-22T15:42:57.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.021+0000 7fcd48ff9700 1 -- 172.21.15.19:0/2664426036 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7fcd340140f0 con 0x7fcd4c0fdc10 2024-02-22T15:42:57.006 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:42:57.006 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-02-22T15:42:57.006 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:42:57.006 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcd38069370 msgr2=0x7fcd3806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcd38069370 0x7fcd3806b820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b780 tx=0x7fcd3c009e30 comp rx=0 tx=0).stop 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 msgr2=0x7fcd4c0fe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 0x7fcd4c0fe080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b790 tx=0x7fcd34009250 comp rx=0 tx=0).stop 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 shutdown_connections 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcd38069370 0x7fcd3806b820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd4c101b60 0x7fcd4c0fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 --2- 172.21.15.19:0/2664426036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd4c0fdc10 0x7fcd4c0fe080 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 >> 172.21.15.19:0/2664426036 conn(0x7fcd4c072850 msgr2=0x7fcd4c065960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:42:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 shutdown_connections 2024-02-22T15:42:57.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:42:57.023+0000 7fcd52207700 1 -- 172.21.15.19:0/2664426036 wait complete. 2024-02-22T15:42:57.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:42:57.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:56 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:57.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:57.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:42:57.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:56 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:58.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: Reconfiguring mon.smithi019 (unknown last config time)... 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: Reconfiguring daemon mon.smithi019 on smithi019 2024-02-22T15:42:58.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:57 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2664426036' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:58.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:42:58.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:42:58.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:58.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:42:58.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:42:58.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: Reconfiguring mon.smithi019 (unknown last config time)... 2024-02-22T15:42:58.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: Reconfiguring daemon mon.smithi019 on smithi019 2024-02-22T15:42:58.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:57 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2664426036' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:42:58.425 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-22T15:42:58.426 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:42:58.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-02-22T15:42:58.461 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:42:58.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:42:58.530 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:42:58.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-02-22T15:42:58.567 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:42:58.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:42:58.640 INFO:tasks.cephadm:Deploying OSDs... 2024-02-22T15:42:58.641 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:42:58.641 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-02-22T15:42:58.661 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-22T15:42:58.661 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 196886 Links: 1 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-02-22 15:42:12.015187317 +0000 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-02-22 15:18:20.103411029 +0000 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-02-22 15:18:20.103411029 +0000 2024-02-22T15:42:58.721 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2024-02-22T15:42:58.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-22T15:42:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-02-22T15:42:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-02-22T15:42:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000201521 s, 2.5 MB/s 2024-02-22T15:42:58.792 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-22T15:42:58.852 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-02-22T15:42:58.910 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 196970 Links: 1 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-02-22 15:42:12.015187317 +0000 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-02-22 15:18:20.447407383 +0000 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-02-22 15:18:20.447407383 +0000 2024-02-22T15:42:58.911 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2024-02-22T15:42:58.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-22T15:42:58.978 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-02-22T15:42:58.979 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-02-22T15:42:58.979 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000207135 s, 2.5 MB/s 2024-02-22T15:42:58.980 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-22T15:42:59.041 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-02-22T15:42:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-22T15:42:59.099 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:59.099 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 197903 Links: 1 2024-02-22T15:42:59.100 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:59.100 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:59.100 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-02-22 15:42:12.016187306 +0000 2024-02-22T15:42:59.100 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-02-22 15:18:20.809403545 +0000 2024-02-22T15:42:59.100 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-02-22 15:18:20.809403545 +0000 2024-02-22T15:42:59.100 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2024-02-22T15:42:59.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-22T15:42:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-02-22T15:42:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-02-22T15:42:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000168431 s, 3.0 MB/s 2024-02-22T15:42:59.167 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-22T15:42:59.229 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-02-22T15:42:59.289 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 197037 Links: 1 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-02-22 15:42:12.016187306 +0000 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-02-22 15:18:21.178399634 +0000 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-02-22 15:18:21.178399634 +0000 2024-02-22T15:42:59.290 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2024-02-22T15:42:59.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-22T15:42:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-02-22T15:42:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-02-22T15:42:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000227542 s, 2.3 MB/s 2024-02-22T15:42:59.358 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-22T15:42:59.418 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:42:59.418 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-02-22T15:42:59.441 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-22T15:42:59.441 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 163682 Links: 1 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-02-22 15:42:47.676946183 +0000 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-02-22 15:18:13.235835464 +0000 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-02-22 15:18:13.235835464 +0000 2024-02-22T15:42:59.504 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-02-22T15:42:59.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-22T15:42:59.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-02-22T15:42:59.576 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-02-22T15:42:59.576 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000261156 s, 2.0 MB/s 2024-02-22T15:42:59.577 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-22T15:42:59.639 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 161633 Links: 1 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-02-22 15:42:47.676946183 +0000 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-02-22 15:18:13.594829418 +0000 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-02-22 15:18:13.594829418 +0000 2024-02-22T15:42:59.699 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-02-22T15:42:59.699 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-22T15:42:59.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-02-22T15:42:59.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-02-22T15:42:59.768 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000243116 s, 2.1 MB/s 2024-02-22T15:42:59.769 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-22T15:42:59.832 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-02-22T15:42:59.892 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-22T15:42:59.892 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:42:59.892 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 165528 Links: 1 2024-02-22T15:42:59.892 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:42:59.892 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:42:59.892 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-02-22 15:42:47.676946183 +0000 2024-02-22T15:42:59.893 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-02-22 15:18:14.000822580 +0000 2024-02-22T15:42:59.893 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-02-22 15:18:14.000822580 +0000 2024-02-22T15:42:59.893 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-02-22T15:42:59.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-22T15:42:59.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-02-22T15:42:59.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-02-22T15:42:59.961 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000169288 s, 3.0 MB/s 2024-02-22T15:42:59.962 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-22T15:43:00.023 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 170188 Links: 1 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-02-22 15:42:47.676946183 +0000 2024-02-22T15:43:00.082 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-02-22 15:18:14.341816837 +0000 2024-02-22T15:43:00.083 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-02-22 15:18:14.341816837 +0000 2024-02-22T15:43:00.083 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-02-22T15:43:00.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-22T15:43:00.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-02-22T15:43:00.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-02-22T15:43:00.150 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000232948 s, 2.2 MB/s 2024-02-22T15:43:00.151 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-22T15:43:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: Reconfiguring mgr.smithi019.yvjrkp (unknown last config time)... 2024-02-22T15:43:00.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:43:00.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:43:00.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:00.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:42:59 smithi062 ceph-mon[48130]: Reconfiguring daemon mgr.smithi019.yvjrkp on smithi019 2024-02-22T15:43:00.212 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-02-22T15:43:00.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-22T15:43:00.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:00.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:00.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:00.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: Reconfiguring mgr.smithi019.yvjrkp (unknown last config time)... 2024-02-22T15:43:00.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:43:00.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:43:00.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:00.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:42:59 smithi019 ceph-mon[67418]: Reconfiguring daemon mgr.smithi019.yvjrkp on smithi019 2024-02-22T15:43:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:43:01.910 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:01.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:01.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: Reconfiguring crash.smithi019 (monmap changed)... 2024-02-22T15:43:01.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:43:01.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:01.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: Reconfiguring daemon crash.smithi019 on smithi019 2024-02-22T15:43:01.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:01 smithi019 ceph-mon[67418]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:01.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:01.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:01.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: Reconfiguring crash.smithi019 (monmap changed)... 2024-02-22T15:43:01.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:43:01.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:01.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: Reconfiguring daemon crash.smithi019 on smithi019 2024-02-22T15:43:01.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:01 smithi062 ceph-mon[48130]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:03.445 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-22T15:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-22T15:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-02-22T15:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-02-22T15:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124907 s, 839 MB/s 2024-02-22T15:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-02-22T15:43:04.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:03 smithi062 ceph-mon[48130]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:04.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:04.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:04.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:03 smithi019 ceph-mon[67418]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:04.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:04.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:04.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-02-22T15:43:05.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:04 smithi062 ceph-mon[48130]: Reconfiguring alertmanager.smithi019 (dependencies changed)... 2024-02-22T15:43:05.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:04 smithi062 ceph-mon[48130]: Reconfiguring daemon alertmanager.smithi019 on smithi019 2024-02-22T15:43:05.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:04 smithi019 ceph-mon[67418]: Reconfiguring alertmanager.smithi019 (dependencies changed)... 2024-02-22T15:43:05.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:04 smithi019 ceph-mon[67418]: Reconfiguring daemon alertmanager.smithi019 on smithi019 2024-02-22T15:43:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:43:06.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:05 smithi062 ceph-mon[48130]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:06.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:05 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:06.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:05 smithi019 ceph-mon[67418]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:06.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:05 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:08.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:07 smithi062 ceph-mon[48130]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:08.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:07 smithi019 ceph-mon[67418]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 -- 172.21.15.19:0/118188087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310071e90 msgr2=0x7fa310070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 --2- 172.21.15.19:0/118188087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310071e90 0x7fa310070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa30c004940 tx=0x7fa30c009e30 comp rx=0 tx=0).stop 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 -- 172.21.15.19:0/118188087 shutdown_connections 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 --2- 172.21.15.19:0/118188087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310070840 0x7fa310070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 --2- 172.21.15.19:0/118188087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310071e90 0x7fa310070300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 -- 172.21.15.19:0/118188087 >> 172.21.15.19:0/118188087 conn(0x7fa3100f8620 msgr2=0x7fa3100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 -- 172.21.15.19:0/118188087 shutdown_connections 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.363+0000 7fa317f6e700 1 -- 172.21.15.19:0/118188087 wait complete. 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa317f6e700 1 Processor -- start 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa317f6e700 1 -- start start 2024-02-22T15:43:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa317f6e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 0x7fa31010faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa317f6e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310071e90 0x7fa31010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa317f6e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3101105e0 con 0x7fa310070840 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa317f6e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310110e80 con 0x7fa310071e90 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa315d0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 0x7fa31010faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa315d0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 0x7fa31010faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34762/0 (socket says 172.21.15.19:34762) 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.364+0000 7fa315d0a700 1 -- 172.21.15.19:0/1673368676 learned_addr learned my addr 172.21.15.19:0/1673368676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa315509700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310071e90 0x7fa31010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa315d0a700 1 -- 172.21.15.19:0/1673368676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310071e90 msgr2=0x7fa31010ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa315d0a700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310071e90 0x7fa31010ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa315d0a700 1 -- 172.21.15.19:0/1673368676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c005040 con 0x7fa310070840 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa315d0a700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 0x7fa31010faa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009fd0 tx=0x7fa30c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30c00e040 con 0x7fa310070840 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa30c004290 con 0x7fa310070840 2024-02-22T15:43:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30c01c3f0 con 0x7fa310070840 2024-02-22T15:43:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa310111100 con 0x7fa310070840 2024-02-22T15:43:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.365+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa310111640 con 0x7fa310070840 2024-02-22T15:43:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.367+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31004bed0 con 0x7fa310070840 2024-02-22T15:43:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.367+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fa30c013420 con 0x7fa310070840 2024-02-22T15:43:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.367+0000 7fa3067fc700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa2fc069210 0x7fa2fc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:43:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.367+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa30c087410 con 0x7fa310070840 2024-02-22T15:43:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.367+0000 7fa315509700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa2fc069210 0x7fa2fc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:43:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.368+0000 7fa315509700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa2fc069210 0x7fa2fc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa3000041d0 tx=0x7fa300009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:43:09.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.376+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa30c0582b0 con 0x7fa310070840 2024-02-22T15:43:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:09.589+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa310104570 con 0x7fa2fc069210 2024-02-22T15:43:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:09 smithi062 ceph-mon[48130]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:09 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:43:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:09 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:43:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:09 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:09 smithi019 ceph-mon[67418]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:09 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:43:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:09 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:43:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:09 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:10 smithi062 ceph-mon[48130]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:43:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:10 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:10 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:11.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:10 smithi019 ceph-mon[67418]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:43:11.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:10 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:11.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:10 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:11 smithi062 ceph-mon[48130]: Reconfiguring grafana.smithi019 (dependencies changed)... 2024-02-22T15:43:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:11 smithi062 ceph-mon[48130]: Reconfiguring daemon grafana.smithi019 on smithi019 2024-02-22T15:43:12.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:11 smithi062 ceph-mon[48130]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:12.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:11 smithi019 ceph-mon[67418]: Reconfiguring grafana.smithi019 (dependencies changed)... 2024-02-22T15:43:12.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:11 smithi019 ceph-mon[67418]: Reconfiguring daemon grafana.smithi019 on smithi019 2024-02-22T15:43:12.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:11 smithi019 ceph-mon[67418]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:13 smithi062 ceph-mon[48130]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:13 smithi019 ceph-mon[67418]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:16.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:15 smithi062 ceph-mon[48130]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:16.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:15 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:16.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:15 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:16.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:15 smithi019 ceph-mon[67418]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:16.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:15 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:16.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:15 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:17.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:16 smithi019 ceph-mon[67418]: Reconfiguring prometheus.smithi019 (dependencies changed)... 2024-02-22T15:43:17.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:16 smithi019 ceph-mon[67418]: Reconfiguring daemon prometheus.smithi019 on smithi019 2024-02-22T15:43:17.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:16 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1491444926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8dce22ce-c824-43c8-b665-33df9393d8fa"}]: dispatch 2024-02-22T15:43:17.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:16 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1491444926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8dce22ce-c824-43c8-b665-33df9393d8fa"}]': finished 2024-02-22T15:43:17.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:16 smithi019 ceph-mon[67418]: osdmap e6: 1 total, 0 up, 1 in 2024-02-22T15:43:17.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:16 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:17.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:16 smithi062 ceph-mon[48130]: Reconfiguring prometheus.smithi019 (dependencies changed)... 2024-02-22T15:43:17.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:16 smithi062 ceph-mon[48130]: Reconfiguring daemon prometheus.smithi019 on smithi019 2024-02-22T15:43:17.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:16 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1491444926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8dce22ce-c824-43c8-b665-33df9393d8fa"}]: dispatch 2024-02-22T15:43:17.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:16 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1491444926' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8dce22ce-c824-43c8-b665-33df9393d8fa"}]': finished 2024-02-22T15:43:17.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:16 smithi062 ceph-mon[48130]: osdmap e6: 1 total, 0 up, 1 in 2024-02-22T15:43:17.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:16 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:18.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:17 smithi019 ceph-mon[67418]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:18.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:17 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1290120438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:43:18.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:17 smithi062 ceph-mon[48130]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:18.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:17 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1290120438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:43:20.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:19 smithi019 ceph-mon[67418]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:20.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:19 smithi062 ceph-mon[48130]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:21.121 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:21.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:22.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:21 smithi019 ceph-mon[67418]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:22.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:21 smithi062 ceph-mon[48130]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:24.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:23 smithi019 ceph-mon[67418]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:24.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:23 smithi062 ceph-mon[48130]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:26.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:25 smithi019 ceph-mon[67418]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:26.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:25 smithi062 ceph-mon[48130]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:27.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:27.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:27.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: Reconfiguring crash.smithi062 (monmap changed)... 2024-02-22T15:43:27.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:43:27.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:27.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: Reconfiguring daemon crash.smithi062 on smithi062 2024-02-22T15:43:27.750 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:27 smithi019 ceph-mon[67418]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:27.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:27.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:27.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: Reconfiguring crash.smithi062 (monmap changed)... 2024-02-22T15:43:27.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:43:27.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:27.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: Reconfiguring daemon crash.smithi062 on smithi062 2024-02-22T15:43:27.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:27 smithi062 ceph-mon[48130]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:28.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T15:43:28.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:28.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: Deploying daemon osd.0 on smithi019 2024-02-22T15:43:28.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:28.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:28.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: Reconfiguring mgr.smithi062.ecubmv (monmap changed)... 2024-02-22T15:43:28.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:43:28.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:43:28.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:28.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:28 smithi062 ceph-mon[48130]: Reconfiguring daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:43:28.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T15:43:28.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:28.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: Deploying daemon osd.0 on smithi019 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: Reconfiguring mgr.smithi062.ecubmv (monmap changed)... 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:28.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:28 smithi019 ceph-mon[67418]: Reconfiguring daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:43:29.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:29 smithi019 ceph-mon[67418]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:29.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:29.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:29.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:43:29.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:43:29.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:29 smithi062 ceph-mon[48130]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:43:30.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:43:30.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: Reconfiguring mon.smithi062 (monmap changed)... 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: Reconfiguring daemon mon.smithi062 on smithi062 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi019.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T15:43:31.091 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi019.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi019.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.092 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:43:31.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: Reconfiguring mon.smithi062 (monmap changed)... 2024-02-22T15:43:31.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: Reconfiguring daemon mon.smithi062 on smithi062 2024-02-22T15:43:31.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi019.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi019.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi019.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:31.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:43:32.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:43:32.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi019.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T15:43:32.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:43:32.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:32.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi019.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T15:43:32.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:43:32.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:31 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi019.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T15:43:32.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:43:32.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi019.front.sepia.ceph.com:9093"}]: dispatch 2024-02-22T15:43:32.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:43:32.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:32.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi019.front.sepia.ceph.com:3000"}]: dispatch 2024-02-22T15:43:32.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:43:32.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:31 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi019.front.sepia.ceph.com:9095"}]: dispatch 2024-02-22T15:43:34.036 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:34.036 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:34.036 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:33 smithi019 ceph-mon[67418]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:33 smithi062 ceph-mon[48130]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:36.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:35 smithi019 ceph-mon[67418]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:36.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:36.180 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:35 smithi062 ceph-mon[48130]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:35 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:35 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:37.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:37.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:37.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:37.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:37.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:37.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:38.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:37 smithi019 ceph-mon[67418]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:38.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:38.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:43:38.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:38.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:43:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:37 smithi062 ceph-mon[48130]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:37 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:37 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:43:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:37 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:37 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:43:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.531+0000 7fa3067fc700 1 -- 172.21.15.19:0/1673368676 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa310104570 con 0x7fa2fc069210 2024-02-22T15:43:39.516 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-02-22T15:43:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa2fc069210 msgr2=0x7fa2fc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:43:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa2fc069210 0x7fa2fc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa3000041d0 tx=0x7fa300009040 comp rx=0 tx=0).stop 2024-02-22T15:43:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 msgr2=0x7fa31010faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:43:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 0x7fa31010faa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009fd0 tx=0x7fa30c009e30 comp rx=0 tx=0).stop 2024-02-22T15:43:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 shutdown_connections 2024-02-22T15:43:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa2fc069210 0x7fa2fc06b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa310070840 0x7fa31010faa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 --2- 172.21.15.19:0/1673368676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa310071e90 0x7fa31010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 >> 172.21.15.19:0/1673368676 conn(0x7fa3100f8620 msgr2=0x7fa310102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:43:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 shutdown_connections 2024-02-22T15:43:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:39.533+0000 7fa317f6e700 1 -- 172.21.15.19:0/1673368676 wait complete. 2024-02-22T15:43:40.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:39 smithi062 ceph-mon[48130]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:40.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:39 smithi062 ceph-mon[48130]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T15:43:40.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:39 smithi019 ceph-mon[67418]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:40.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:40.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:39 smithi019 ceph-mon[67418]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T15:43:40.450 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.0.service 2024-02-22T15:43:40.453 INFO:tasks.cephadm:Deploying osd.1 on smithi019 with /dev/vg_nvme/lv_3... 2024-02-22T15:43:40.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-22T15:43:40.483 INFO:journalctl@ceph.osd.0.smithi019.stdout:-- Logs begin at Thu 2024-02-22 14:59:28 UTC. -- 2024-02-22T15:43:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:43:41.786 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-mon[67418]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T15:43:41.786 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-mon[67418]: osdmap e7: 1 total, 0 up, 1 in 2024-02-22T15:43:41.786 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:41.786 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-mon[67418]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:43:41.786 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-mon[67418]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:41.786 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[91179]: 2024-02-22T15:43:41.605+0000 7effa46ae700 -1 osd.0 0 waiting for initial osdmap 2024-02-22T15:43:41.786 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:43:41 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[91179]: 2024-02-22T15:43:41.609+0000 7eff9d4a5700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:43:41.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:41 smithi062 ceph-mon[48130]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T15:43:41.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:41 smithi062 ceph-mon[48130]: osdmap e7: 1 total, 0 up, 1 in 2024-02-22T15:43:41.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:41 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:41.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:41 smithi062 ceph-mon[48130]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:43:41.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:41 smithi062 ceph-mon[48130]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:42.637 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:42 smithi019 ceph-mon[67418]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-02-22T15:43:42.637 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:42 smithi019 ceph-mon[67418]: osdmap e8: 1 total, 0 up, 1 in 2024-02-22T15:43:42.637 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:42.637 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:42.637 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:42 smithi062 ceph-mon[48130]: from='osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-02-22T15:43:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:42 smithi062 ceph-mon[48130]: osdmap e8: 1 total, 0 up, 1 in 2024-02-22T15:43:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-22T15:43:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-22T15:43:43.820 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-02-22T15:43:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-02-22T15:43:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111361 s, 942 MB/s 2024-02-22T15:43:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-02-22T15:43:43.861 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:43 smithi019 ceph-mon[67418]: purged_snaps scrub starts 2024-02-22T15:43:43.862 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:43 smithi019 ceph-mon[67418]: purged_snaps scrub ok 2024-02-22T15:43:43.862 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:43 smithi019 ceph-mon[67418]: osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] boot 2024-02-22T15:43:43.862 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:43 smithi019 ceph-mon[67418]: osdmap e9: 1 total, 1 up, 1 in 2024-02-22T15:43:43.862 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:43 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:43.862 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:43 smithi019 ceph-mon[67418]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:43 smithi062 ceph-mon[48130]: purged_snaps scrub starts 2024-02-22T15:43:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:43 smithi062 ceph-mon[48130]: purged_snaps scrub ok 2024-02-22T15:43:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:43 smithi062 ceph-mon[48130]: osd.0 [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] boot 2024-02-22T15:43:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:43 smithi062 ceph-mon[48130]: osdmap e9: 1 total, 1 up, 1 in 2024-02-22T15:43:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:43 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:43:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:43 smithi062 ceph-mon[48130]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-22T15:43:44.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:44 smithi062 ceph-mon[48130]: osdmap e10: 1 total, 1 up, 1 in 2024-02-22T15:43:44.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:44 smithi019 ceph-mon[67418]: osdmap e10: 1 total, 1 up, 1 in 2024-02-22T15:43:45.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-02-22T15:43:45.491 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:43:45.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:45 smithi062 ceph-mon[48130]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:45.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:45 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:45.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:45 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:45.978 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:45 smithi019 ceph-mon[67418]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:45.978 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:45.978 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:48.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:47 smithi062 ceph-mon[48130]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:48.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:47 smithi019 ceph-mon[67418]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- 172.21.15.19:0/2323602728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 msgr2=0x7f7e20102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 --2- 172.21.15.19:0/2323602728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e20102780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7e14004940 tx=0x7f7e14009e30 comp rx=0 tx=0).stop 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- 172.21.15.19:0/2323602728 shutdown_connections 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 --2- 172.21.15.19:0/2323602728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e20102780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 --2- 172.21.15.19:0/2323602728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200fda80 0x7f7e200ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- 172.21.15.19:0/2323602728 >> 172.21.15.19:0/2323602728 conn(0x7f7e200f7520 msgr2=0x7f7e200f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- 172.21.15.19:0/2323602728 shutdown_connections 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- 172.21.15.19:0/2323602728 wait complete. 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 Processor -- start 2024-02-22T15:43:48.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- start start 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200fda80 0x7f7e2010d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e2010ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010e3c0 con 0x7f7e201003a0 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.299+0000 7f7e253a9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2010ec60 con 0x7f7e200fda80 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e2010ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e2010ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51674/0 (socket says 172.21.15.19:51674) 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 -- 172.21.15.19:0/1309145392 learned_addr learned my addr 172.21.15.19:0/1309145392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1effd700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200fda80 0x7f7e2010d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 -- 172.21.15.19:0/1309145392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200fda80 msgr2=0x7f7e2010d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200fda80 0x7f7e2010d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 -- 172.21.15.19:0/1309145392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e14005040 con 0x7f7e201003a0 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e1e7fc700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e2010ddc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009fd0 tx=0x7f7e14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1400e040 con 0x7f7e201003a0 2024-02-22T15:43:48.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e14004290 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2010eee0 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1401c3f0 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.300+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2010f420 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.301+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e20107910 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.302+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f7e14013420 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.302+0000 7f7e077fe700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e08069210 0x7f7e0806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.302+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f7e14087740 con 0x7f7e201003a0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.302+0000 7f7e1effd700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e08069210 0x7f7e0806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:43:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.302+0000 7f7e1effd700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e08069210 0x7f7e0806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e100041d0 tx=0x7f7e10009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:43:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.306+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7e14058300 con 0x7f7e201003a0 2024-02-22T15:43:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:43:48.510+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e2002b310 con 0x7f7e08069210 2024-02-22T15:43:49.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:43:49.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:43:49.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:49.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:48 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:43:49.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:48 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:43:49.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:48 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:50.188 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:49 smithi019 ceph-mon[67418]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:43:50.188 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:49 smithi019 ceph-mon[67418]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:50.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:49 smithi062 ceph-mon[48130]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:43:50.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:49 smithi062 ceph-mon[48130]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi019 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi019 to 12902M 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:51.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:43:51.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:43:51.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:51.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:51.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: Detected new or changed devices on smithi019 2024-02-22T15:43:51.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi019 to 12902M 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:51.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/501515849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc2b0a0-987e-48ca-895f-f773652ceb17"}]: dispatch 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/501515849' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc2b0a0-987e-48ca-895f-f773652ceb17"}]': finished 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: osdmap e11: 2 total, 1 up, 2 in 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:52.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:52 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2110932356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/501515849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc2b0a0-987e-48ca-895f-f773652ceb17"}]: dispatch 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/501515849' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc2b0a0-987e-48ca-895f-f773652ceb17"}]': finished 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: osdmap e11: 2 total, 1 up, 2 in 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:53.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:52 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2110932356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:43:53.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:53 smithi019 ceph-mon[67418]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:54.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:53 smithi062 ceph-mon[48130]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:56.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:55 smithi062 ceph-mon[48130]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:56.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:55 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:56.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:55 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:56.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:55 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:56.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:55 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:43:56.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:55 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:56.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:55 smithi019 ceph-mon[67418]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:56.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:55 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:56.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:55 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:56.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:55 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:43:56.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:55 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:43:56.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:55 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:43:57.950 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:57 smithi019 ceph-mon[67418]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:58.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:57 smithi062 ceph-mon[48130]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:59.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:59 smithi019 ceph-mon[67418]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:43:59.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T15:43:59.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:43:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:59 smithi062 ceph-mon[48130]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T15:44:00.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:43:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:00.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:00 smithi019 ceph-mon[67418]: Deploying daemon osd.1 on smithi019 2024-02-22T15:44:01.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:00 smithi062 ceph-mon[48130]: Deploying daemon osd.1 on smithi019 2024-02-22T15:44:02.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:01 smithi062 ceph-mon[48130]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:02.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:01 smithi019 ceph-mon[67418]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:04.120 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:03 smithi019 ceph-mon[67418]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:04.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:03 smithi062 ceph-mon[48130]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:06.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:06 smithi062 ceph-mon[48130]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:06.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:06.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:06.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:06.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:06.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:06 smithi019 ceph-mon[67418]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:06.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:06.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:06.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:06.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:08.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:08 smithi019 ceph-mon[67418]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:08.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:08 smithi062 ceph-mon[48130]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.169+0000 7f7e077fe700 1 -- 172.21.15.19:0/1309145392 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e2002b310 con 0x7f7e08069210 2024-02-22T15:44:09.156 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 1 on host 'smithi019' 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.171+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e08069210 msgr2=0x7f7e0806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.171+0000 7f7e253a9700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e08069210 0x7f7e0806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e100041d0 tx=0x7f7e10009040 comp rx=0 tx=0).stop 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.171+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 msgr2=0x7f7e2010ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.171+0000 7f7e253a9700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e2010ddc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009fd0 tx=0x7f7e14009e30 comp rx=0 tx=0).stop 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.171+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 shutdown_connections 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.171+0000 7f7e253a9700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e08069210 0x7f7e0806b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.172+0000 7f7e253a9700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e200fda80 0x7f7e2010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.172+0000 7f7e253a9700 1 --2- 172.21.15.19:0/1309145392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e201003a0 0x7f7e2010ddc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.172+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 >> 172.21.15.19:0/1309145392 conn(0x7f7e200f7520 msgr2=0x7f7e200f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:44:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.172+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 shutdown_connections 2024-02-22T15:44:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:09.172+0000 7f7e253a9700 1 -- 172.21.15.19:0/1309145392 wait complete. 2024-02-22T15:44:10.049 DEBUG:teuthology.orchestra.run.smithi019:osd.1> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.1.service 2024-02-22T15:44:10.051 INFO:tasks.cephadm:Deploying osd.2 on smithi019 with /dev/vg_nvme/lv_2... 2024-02-22T15:44:10.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-22T15:44:10.083 INFO:journalctl@ceph.osd.1.smithi019.stdout:-- Logs begin at Thu 2024-02-22 14:59:28 UTC. -- 2024-02-22T15:44:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:09 smithi062 ceph-mon[48130]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:09 smithi062 ceph-mon[48130]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T15:44:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:09 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:10.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:09 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:10.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:09 smithi019 ceph-mon[67418]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:10.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:09 smithi019 ceph-mon[67418]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T15:44:10.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:09 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:10.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:09 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:10.998 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:10 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[100582]: 2024-02-22T15:44:10.954+0000 7fca01da1700 -1 osd.1 0 waiting for initial osdmap 2024-02-22T15:44:10.998 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:10 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[100582]: 2024-02-22T15:44:10.961+0000 7fc9fb399700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:44:10.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:10 smithi019 ceph-mon[67418]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T15:44:10.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:10 smithi019 ceph-mon[67418]: osdmap e12: 2 total, 1 up, 2 in 2024-02-22T15:44:10.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:10 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:10.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:10 smithi019 ceph-mon[67418]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:44:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:44:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:10 smithi062 ceph-mon[48130]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T15:44:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:10 smithi062 ceph-mon[48130]: osdmap e12: 2 total, 1 up, 2 in 2024-02-22T15:44:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:10 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:10 smithi062 ceph-mon[48130]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:44:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:11 smithi062 ceph-mon[48130]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:11 smithi062 ceph-mon[48130]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-02-22T15:44:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:11 smithi062 ceph-mon[48130]: osdmap e13: 2 total, 1 up, 2 in 2024-02-22T15:44:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:11 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:12.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:11 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:12.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:11 smithi019 ceph-mon[67418]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:12.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:11 smithi019 ceph-mon[67418]: from='osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-02-22T15:44:12.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:11 smithi019 ceph-mon[67418]: osdmap e13: 2 total, 1 up, 2 in 2024-02-22T15:44:12.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:11 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:12.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:11 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:13.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:12 smithi062 ceph-mon[48130]: purged_snaps scrub starts 2024-02-22T15:44:13.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:12 smithi062 ceph-mon[48130]: purged_snaps scrub ok 2024-02-22T15:44:13.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:12 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:13.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:12 smithi062 ceph-mon[48130]: osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] boot 2024-02-22T15:44:13.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:12 smithi062 ceph-mon[48130]: osdmap e14: 2 total, 2 up, 2 in 2024-02-22T15:44:13.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:12 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:13.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:12 smithi019 ceph-mon[67418]: purged_snaps scrub starts 2024-02-22T15:44:13.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:12 smithi019 ceph-mon[67418]: purged_snaps scrub ok 2024-02-22T15:44:13.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:12 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:13.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:12 smithi019 ceph-mon[67418]: osd.1 [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] boot 2024-02-22T15:44:13.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:12 smithi019 ceph-mon[67418]: osdmap e14: 2 total, 2 up, 2 in 2024-02-22T15:44:13.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:12 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:44:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-22T15:44:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-22T15:44:13.982 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-02-22T15:44:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-02-22T15:44:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011587 s, 905 MB/s 2024-02-22T15:44:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-02-22T15:44:14.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:13 smithi062 ceph-mon[48130]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:13 smithi062 ceph-mon[48130]: osdmap e15: 2 total, 2 up, 2 in 2024-02-22T15:44:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:13 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:13 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:13 smithi019 ceph-mon[67418]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-22T15:44:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:13 smithi019 ceph-mon[67418]: osdmap e15: 2 total, 2 up, 2 in 2024-02-22T15:44:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:13 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:14.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:13 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:15.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-02-22T15:44:15.617 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:44:16.192 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:15 smithi019 ceph-mon[67418]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:16.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:15 smithi062 ceph-mon[48130]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.982+0000 7f39febb3700 1 -- 172.21.15.19:0/1816052121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 msgr2=0x7f39f80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.982+0000 7f39febb3700 1 --2- 172.21.15.19:0/1816052121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f80fd420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f39e4004940 tx=0x7f39e4009e30 comp rx=0 tx=0).stop 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 -- 172.21.15.19:0/1816052121 shutdown_connections 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 --2- 172.21.15.19:0/1816052121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f80fd420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 --2- 172.21.15.19:0/1816052121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 0x7f39f80fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 -- 172.21.15.19:0/1816052121 >> 172.21.15.19:0/1816052121 conn(0x7f39f80f74e0 msgr2=0x7f39f80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 -- 172.21.15.19:0/1816052121 shutdown_connections 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 -- 172.21.15.19:0/1816052121 wait complete. 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 Processor -- start 2024-02-22T15:44:17.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.983+0000 7f39febb3700 1 -- start start 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39febb3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 0x7f39f8113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39febb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f81143c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39febb3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8114a50 con 0x7f39f80fcfb0 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39febb3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8115290 con 0x7f39f80fbe40 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f81143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f81143c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47206/0 (socket says 172.21.15.19:47206) 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 -- 172.21.15.19:0/1025523388 learned_addr learned my addr 172.21.15.19:0/1025523388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39fc94f700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 0x7f39f8113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 -- 172.21.15.19:0/1025523388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 msgr2=0x7f39f8113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 0x7f39f8113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 -- 172.21.15.19:0/1025523388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4005040 con 0x7f39f80fcfb0 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39fc94f700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 0x7f39f8113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:44:17.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f7fff700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f81143c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009360 tx=0x7f39e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:44:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e400e040 con 0x7f39f80fcfb0 2024-02-22T15:44:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f39e400bcd0 con 0x7f39f80fcfb0 2024-02-22T15:44:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8115510 con 0x7f39f80fcfb0 2024-02-22T15:44:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e401c630 con 0x7f39f80fcfb0 2024-02-22T15:44:17.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.984+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f8115a50 con 0x7f39f80fcfb0 2024-02-22T15:44:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.985+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f804bed0 con 0x7f39f80fcfb0 2024-02-22T15:44:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.986+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f39e4013960 con 0x7f39f80fcfb0 2024-02-22T15:44:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.986+0000 7f39f57fa700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f39e8069210 0x7f39e806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:44:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.986+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f39e4087810 con 0x7f39f80fcfb0 2024-02-22T15:44:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.986+0000 7f39fc94f700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f39e8069210 0x7f39e806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:44:17.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.987+0000 7f39fc94f700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f39e8069210 0x7f39e806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009b10 tx=0x7f39ec008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:44:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:17.991+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f39e40583f0 con 0x7f39f80fcfb0 2024-02-22T15:44:17.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:17 smithi019 ceph-mon[67418]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:18.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:18.193+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f39f8115fa0 con 0x7f39e8069210 2024-02-22T15:44:18.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:17 smithi062 ceph-mon[48130]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:19.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:44:19.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:44:19.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:44:19.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:19.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:18 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:19.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:19.461 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:18 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:20.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:19 smithi019 ceph-mon[67418]: Detected new or changed devices on smithi019 2024-02-22T15:44:20.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:19 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi019 to 6451M 2024-02-22T15:44:20.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:19 smithi019 ceph-mon[67418]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:20.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:19 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi019 2024-02-22T15:44:20.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:19 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi019 to 6451M 2024-02-22T15:44:20.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:19 smithi062 ceph-mon[48130]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:20.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:21.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:22.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:21 smithi019 ceph-mon[67418]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:22.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:21 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1551354686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae746a74-93bc-4d46-a925-f5824de285ce"}]: dispatch 2024-02-22T15:44:22.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:21 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1551354686' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae746a74-93bc-4d46-a925-f5824de285ce"}]': finished 2024-02-22T15:44:22.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:21 smithi019 ceph-mon[67418]: osdmap e16: 3 total, 2 up, 3 in 2024-02-22T15:44:22.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:22.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:21 smithi062 ceph-mon[48130]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:22.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:21 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1551354686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae746a74-93bc-4d46-a925-f5824de285ce"}]: dispatch 2024-02-22T15:44:22.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:21 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1551354686' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae746a74-93bc-4d46-a925-f5824de285ce"}]': finished 2024-02-22T15:44:22.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:21 smithi062 ceph-mon[48130]: osdmap e16: 3 total, 2 up, 3 in 2024-02-22T15:44:22.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:21 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:22 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1641229637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:44:23.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:22 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1641229637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:44:24.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:24 smithi019 ceph-mon[67418]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:24.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:24 smithi062 ceph-mon[48130]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:25.672 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:25 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:25.672 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:25 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:25.672 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:25 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:25.672 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:25 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:25.672 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:25 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:25.673 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:25 smithi019 ceph-mon[67418]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:25.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:25 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:25.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:25 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:25.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:25 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:25.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:25 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:25.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:25 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:25.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:25 smithi062 ceph-mon[48130]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:28.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:27 smithi062 ceph-mon[48130]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:28.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:27 smithi019 ceph-mon[67418]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:30.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:29 smithi062 ceph-mon[48130]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T15:44:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:30.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:29 smithi062 ceph-mon[48130]: Deploying daemon osd.2 on smithi019 2024-02-22T15:44:30.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:29 smithi019 ceph-mon[67418]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:30.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T15:44:30.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:30.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:29 smithi019 ceph-mon[67418]: Deploying daemon osd.2 on smithi019 2024-02-22T15:44:32.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:31 smithi062 ceph-mon[48130]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:32.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:31 smithi019 ceph-mon[67418]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:34.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:33 smithi019 ceph-mon[67418]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:34.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:33 smithi062 ceph-mon[48130]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:35 smithi062 ceph-mon[48130]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:35 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:35 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:36.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:35 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:36.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:35 smithi019 ceph-mon[67418]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:36.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:36.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:36.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:36.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:37.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:38.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:37 smithi062 ceph-mon[48130]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:38.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:37 smithi019 ceph-mon[67418]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:38.687 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 2 on host 'smithi019' 2024-02-22T15:44:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.702+0000 7f39f57fa700 1 -- 172.21.15.19:0/1025523388 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f39f8115fa0 con 0x7f39e8069210 2024-02-22T15:44:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f39e8069210 msgr2=0x7f39e806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f39e8069210 0x7f39e806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009b10 tx=0x7f39ec008040 comp rx=0 tx=0).stop 2024-02-22T15:44:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 msgr2=0x7f39f81143c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f81143c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009360 tx=0x7f39e4009e30 comp rx=0 tx=0).stop 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 shutdown_connections 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f39e8069210 0x7f39e806b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80fbe40 0x7f39f8113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 --2- 172.21.15.19:0/1025523388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39f80fcfb0 0x7f39f81143c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 >> 172.21.15.19:0/1025523388 conn(0x7f39f80f74e0 msgr2=0x7f39f8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 shutdown_connections 2024-02-22T15:44:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:44:38.705+0000 7f39febb3700 1 -- 172.21.15.19:0/1025523388 wait complete. 2024-02-22T15:44:39.778 DEBUG:teuthology.orchestra.run.smithi019:osd.2> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.2.service 2024-02-22T15:44:39.780 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_4... 2024-02-22T15:44:39.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-22T15:44:39.807 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:39.807 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:39.807 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:39 smithi019 ceph-mon[67418]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T15:44:39.807 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:39 smithi019 ceph-mon[67418]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:39.812 INFO:journalctl@ceph.osd.2.smithi019.stdout:-- Logs begin at Thu 2024-02-22 14:59:28 UTC. -- 2024-02-22T15:44:39.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:39.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:39.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:39 smithi062 ceph-mon[48130]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T15:44:39.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:39 smithi062 ceph-mon[48130]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:40.059 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:44:40.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:40 smithi062 ceph-mon[48130]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T15:44:40.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:40 smithi062 ceph-mon[48130]: osdmap e17: 3 total, 2 up, 3 in 2024-02-22T15:44:40.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:40 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:40.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:40 smithi062 ceph-mon[48130]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:44:40.997 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:40 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[110364]: 2024-02-22T15:44:40.666+0000 7f856150f700 -1 osd.2 0 waiting for initial osdmap 2024-02-22T15:44:40.998 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:40 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[110364]: 2024-02-22T15:44:40.672+0000 7f855ab07700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:44:40.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:40 smithi019 ceph-mon[67418]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T15:44:40.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:40 smithi019 ceph-mon[67418]: osdmap e17: 3 total, 2 up, 3 in 2024-02-22T15:44:40.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:40 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:40.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:40 smithi019 ceph-mon[67418]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:44:41.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:41 smithi062 ceph-mon[48130]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-02-22T15:44:41.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:41 smithi062 ceph-mon[48130]: osdmap e18: 3 total, 2 up, 3 in 2024-02-22T15:44:41.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:41 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:41.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:41 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:41.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:41 smithi062 ceph-mon[48130]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:41.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:41 smithi019 ceph-mon[67418]: from='osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-02-22T15:44:41.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:41 smithi019 ceph-mon[67418]: osdmap e18: 3 total, 2 up, 3 in 2024-02-22T15:44:41.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:41.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:41 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:41.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:41 smithi019 ceph-mon[67418]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-22T15:44:41.999 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-22T15:44:41.999 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-22T15:44:41.999 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-02-22T15:44:41.999 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-02-22T15:44:41.999 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110353 s, 950 MB/s 2024-02-22T15:44:41.999 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-02-22T15:44:42.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-02-22T15:44:42.887 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:44:42.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: purged_snaps scrub starts 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: purged_snaps scrub ok 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] boot 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: osdmap e19: 3 total, 3 up, 3 in 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:42.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:42 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:42.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: purged_snaps scrub starts 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: purged_snaps scrub ok 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: osd.2 [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] boot 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: osdmap e19: 3 total, 3 up, 3 in 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:42.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:42 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:43.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:43 smithi019 ceph-mon[67418]: osdmap e20: 3 total, 3 up, 3 in 2024-02-22T15:44:43.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:43 smithi019 ceph-mon[67418]: pgmap v79: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:43.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:43 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T15:44:43.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:43 smithi062 ceph-mon[48130]: osdmap e20: 3 total, 3 up, 3 in 2024-02-22T15:44:43.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:43 smithi062 ceph-mon[48130]: pgmap v79: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:43.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:43 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.633+0000 7fefd9157700 1 -- 172.21.15.62:0/1382593830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fcfb0 msgr2=0x7fefd40fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.633+0000 7fefd9157700 1 --2- 172.21.15.62:0/1382593830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fcfb0 0x7fefd40fd420 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fefc8004940 tx=0x7fefc8009e30 comp rx=0 tx=0).stop 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 -- 172.21.15.62:0/1382593830 shutdown_connections 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 --2- 172.21.15.62:0/1382593830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fcfb0 0x7fefd40fd420 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 --2- 172.21.15.62:0/1382593830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefd40fbe40 0x7fefd40fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 -- 172.21.15.62:0/1382593830 >> 172.21.15.62:0/1382593830 conn(0x7fefd40f7500 msgr2=0x7fefd40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 -- 172.21.15.62:0/1382593830 shutdown_connections 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 -- 172.21.15.62:0/1382593830 wait complete. 2024-02-22T15:44:44.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 Processor -- start 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.634+0000 7fefd9157700 1 -- start start 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd9157700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 0x7fefd410c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd9157700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefd40fcfb0 0x7fefd410cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd9157700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd410d230 con 0x7fefd40fcfb0 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd9157700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd410dad0 con 0x7fefd40fbe40 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 0x7fefd410c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 0x7fefd410c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44286/0 (socket says 172.21.15.62:44286) 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 -- 172.21.15.62:0/3492688182 learned_addr learned my addr 172.21.15.62:0/3492688182 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd259c700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefd40fcfb0 0x7fefd410cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 -- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefd40fcfb0 msgr2=0x7fefd410cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefd40fcfb0 0x7fefd410cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:44:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 -- 172.21.15.62:0/3492688182 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc8005040 con 0x7fefd40fbe40 2024-02-22T15:44:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd2d9d700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 0x7fefd410c6f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fefc400eb40 tx=0x7fefc400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:44:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefc4007cc0 con 0x7fefd40fbe40 2024-02-22T15:44:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefc400f040 con 0x7fefd40fbe40 2024-02-22T15:44:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd410dd50 con 0x7fefd40fbe40 2024-02-22T15:44:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefc400c990 con 0x7fefd40fbe40 2024-02-22T15:44:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.635+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd410e290 con 0x7fefd40fbe40 2024-02-22T15:44:44.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.636+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd404bed0 con 0x7fefd40fbe40 2024-02-22T15:44:44.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.637+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fefc400caf0 con 0x7fefd40fbe40 2024-02-22T15:44:44.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.637+0000 7fefbb7fe700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fefbc069210 0x7fefbc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:44:44.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.637+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7fefc40862d0 con 0x7fefd40fbe40 2024-02-22T15:44:44.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.637+0000 7fefd259c700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fefbc069210 0x7fefbc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:44:44.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.638+0000 7fefd259c700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fefbc069210 0x7fefbc06b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fefc8013680 tx=0x7fefc8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:44:44.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.642+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fefc404dbd0 con 0x7fefd40fbe40 2024-02-22T15:44:44.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:44.843+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fefd410e700 con 0x7fefbc069210 2024-02-22T15:44:44.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-22T15:44:44.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:44 smithi062 ceph-mon[48130]: osdmap e21: 3 total, 3 up, 3 in 2024-02-22T15:44:44.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T15:44:44.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-22T15:44:44.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:44 smithi019 ceph-mon[67418]: osdmap e21: 3 total, 3 up, 3 in 2024-02-22T15:44:44.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-22T15:44:45.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:44:45.689+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fefc4053420 con 0x7fefd40fbe40 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: osdmap e22: 3 total, 3 up, 3 in 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:45.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:45 smithi019 ceph-mon[67418]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:45.748 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:45 smithi019 sudo[115727]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-22T15:44:45.749 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:45 smithi019 sudo[115727]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:45.749 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:45 smithi019 sudo[115727]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:45.749 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:45 smithi019 sudo[115727]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:45.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-22T15:44:45.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: osdmap e22: 3 total, 3 up, 3 in 2024-02-22T15:44:45.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:44:45.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:44:45.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:44:45.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:45.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:45 smithi062 ceph-mon[48130]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:46.497 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-22T15:44:46.497 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:46.497 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115804]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:46.497 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115804]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: mgrmap e19: smithi019.yvjrkp(active, since 2m), standbys: smithi062.ecubmv 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: osdmap e23: 3 total, 3 up, 3 in 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi019 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:46.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:46.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:46.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi019 to 4300M 2024-02-22T15:44:46.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:46.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:46.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:46.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:46.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:46.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.997 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-22T15:44:46.997 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:46.998 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115807]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:46.998 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:46 smithi019 sudo[115807]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: mgrmap e19: smithi019.yvjrkp(active, since 2m), standbys: smithi062.ecubmv 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: osdmap e23: 3 total, 3 up, 3 in 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: Detected new or changed devices on smithi019 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:46.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi019 to 4300M 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:44:46.999 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:47.000 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:47.485 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-22T15:44:47.485 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:47.486 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115810]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:47.486 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115810]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:47.747 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115813]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-22T15:44:47.747 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:47.747 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115813]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:47.748 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115813]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:47.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:47 smithi019 ceph-mon[67418]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:47.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:47 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2463824122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8e553ca-1a18-42a9-b3d2-fef6669ef7d4"}]: dispatch 2024-02-22T15:44:47.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:47 smithi019 ceph-mon[67418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8e553ca-1a18-42a9-b3d2-fef6669ef7d4"}]: dispatch 2024-02-22T15:44:47.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:47 smithi019 ceph-mon[67418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8e553ca-1a18-42a9-b3d2-fef6669ef7d4"}]': finished 2024-02-22T15:44:47.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:47 smithi019 ceph-mon[67418]: osdmap e24: 4 total, 3 up, 4 in 2024-02-22T15:44:47.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:47 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:44:47.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:47 smithi062 ceph-mon[48130]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:47.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:47 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2463824122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8e553ca-1a18-42a9-b3d2-fef6669ef7d4"}]: dispatch 2024-02-22T15:44:47.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:47 smithi062 ceph-mon[48130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8e553ca-1a18-42a9-b3d2-fef6669ef7d4"}]: dispatch 2024-02-22T15:44:47.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:47 smithi062 ceph-mon[48130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8e553ca-1a18-42a9-b3d2-fef6669ef7d4"}]': finished 2024-02-22T15:44:47.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:47 smithi062 ceph-mon[48130]: osdmap e24: 4 total, 3 up, 4 in 2024-02-22T15:44:47.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:47 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:44:48.247 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-22T15:44:48.247 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:48.247 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115816]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:48.247 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:44:47 smithi019 sudo[115816]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:48.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-22T15:44:48.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:48.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115819]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:48.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115819]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:49.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-22T15:44:49.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:49.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115822]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:49.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:48 smithi019 sudo[115822]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:49.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[52832]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-22T15:44:49.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[52832]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:49.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[52832]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:49.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[52832]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:49.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[53024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[53024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[53024]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 sudo[53024]: pam_unix(sudo:session): session closed for user root 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/1170387364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:44:49.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:49 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:44:49.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-22T15:44:49.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/1170387364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:44:49.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-22T15:44:49.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:44:49.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:44:49.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:44:49.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:49 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:44:50.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:50 smithi062 ceph-mon[48130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-22T15:44:50.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:50 smithi062 ceph-mon[48130]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:50.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:50 smithi062 ceph-mon[48130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-22T15:44:50.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:50 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:50.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:50 smithi019 ceph-mon[67418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-22T15:44:50.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:50 smithi019 ceph-mon[67418]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:50.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:50 smithi019 ceph-mon[67418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-22T15:44:50.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:50 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:44:51.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:51 smithi019 ceph-mon[67418]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:51.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:52.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:51 smithi062 ceph-mon[48130]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:52.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:44:54.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:53 smithi062 ceph-mon[48130]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:54.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:53 smithi019 ceph-mon[67418]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:44:55.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T15:44:55.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:54 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:55.310 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T15:44:55.310 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:54 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:44:56.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:55 smithi019 ceph-mon[67418]: Deploying daemon osd.3 on smithi062 2024-02-22T15:44:56.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:55 smithi019 ceph-mon[67418]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-22T15:44:56.304 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:55 smithi062 ceph-mon[48130]: Deploying daemon osd.3 on smithi062 2024-02-22T15:44:56.305 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:55 smithi062 ceph-mon[48130]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-22T15:44:58.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:57 smithi062 ceph-mon[48130]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-22T15:44:58.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:57 smithi019 ceph-mon[67418]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-22T15:45:00.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:59 smithi019 ceph-mon[67418]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-22T15:45:00.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:00.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:00.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:44:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:00.295 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:59 smithi062 ceph-mon[48130]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-22T15:45:00.295 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:00.296 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:00.296 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:44:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:02.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:01 smithi062 ceph-mon[48130]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-22T15:45:02.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:01 smithi019 ceph-mon[67418]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-22T15:45:03.443 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-02-22T15:45:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.458+0000 7fefbb7fe700 1 -- 172.21.15.62:0/3492688182 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fefd410e700 con 0x7fefbc069210 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.460+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fefbc069210 msgr2=0x7fefbc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.460+0000 7fefd9157700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fefbc069210 0x7fefbc06b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fefc8013680 tx=0x7fefc8009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 msgr2=0x7fefd410c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 0x7fefd410c6f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fefc400eb40 tx=0x7fefc400b040 comp rx=0 tx=0).stop 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 shutdown_connections 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fefbc069210 0x7fefbc06b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefd40fbe40 0x7fefd410c6f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 --2- 172.21.15.62:0/3492688182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefd40fcfb0 0x7fefd410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 >> 172.21.15.62:0/3492688182 conn(0x7fefd40f7500 msgr2=0x7fefd4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 shutdown_connections 2024-02-22T15:45:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:03.461+0000 7fefd9157700 1 -- 172.21.15.62:0/3492688182 wait complete. 2024-02-22T15:45:04.093 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-22T15:45:04.106 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.106 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.106 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.106 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.106 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: from='osd.3 [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:45:04.107 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:03 smithi062 ceph-mon[48130]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:45:04.116 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.3.service 2024-02-22T15:45:04.118 INFO:tasks.cephadm:Deploying osd.4 on smithi062 with /dev/vg_nvme/lv_3... 2024-02-22T15:45:04.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-22T15:45:04.153 INFO:journalctl@ceph.osd.3.smithi062.stdout:-- Logs begin at Thu 2024-02-22 14:59:35 UTC. -- 2024-02-22T15:45:04.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-02-22T15:45:04.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:04.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: from='osd.3 [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:45:04.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:03 smithi019 ceph-mon[67418]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:45:05.228 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:45:05.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-mon[48130]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T15:45:05.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-mon[48130]: osdmap e25: 4 total, 3 up, 4 in 2024-02-22T15:45:05.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:05.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-mon[48130]: from='osd.3 [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:05.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-mon[48130]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:05.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-mon[48130]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:45:05.710 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[54690]: 2024-02-22T15:45:05.547+0000 7f4c29117700 -1 osd.3 0 waiting for initial osdmap 2024-02-22T15:45:05.710 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:45:05 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[54690]: 2024-02-22T15:45:05.553+0000 7f4c25f16700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:45:05.948 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:05 smithi019 ceph-mon[67418]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T15:45:05.948 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:05 smithi019 ceph-mon[67418]: osdmap e25: 4 total, 3 up, 4 in 2024-02-22T15:45:05.948 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:05 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:05.948 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:05 smithi019 ceph-mon[67418]: from='osd.3 [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:05.948 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:05 smithi019 ceph-mon[67418]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:05.948 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:05 smithi019 ceph-mon[67418]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:45:06.671 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:06 smithi062 ceph-mon[48130]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-02-22T15:45:06.671 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:06 smithi062 ceph-mon[48130]: osdmap e26: 4 total, 3 up, 4 in 2024-02-22T15:45:06.671 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:06.671 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:06.671 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:06.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:06 smithi019 ceph-mon[67418]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-02-22T15:45:06.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:06 smithi019 ceph-mon[67418]: osdmap e26: 4 total, 3 up, 4 in 2024-02-22T15:45:06.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:06.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:06.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-22T15:45:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-22T15:45:07.632 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-02-22T15:45:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-02-22T15:45:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110265 s, 951 MB/s 2024-02-22T15:45:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-02-22T15:45:07.927 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:07 smithi062 ceph-mon[48130]: purged_snaps scrub starts 2024-02-22T15:45:07.927 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:07 smithi062 ceph-mon[48130]: purged_snaps scrub ok 2024-02-22T15:45:07.927 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:07 smithi062 ceph-mon[48130]: osd.3 [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] boot 2024-02-22T15:45:07.928 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:07 smithi062 ceph-mon[48130]: osdmap e27: 4 total, 4 up, 4 in 2024-02-22T15:45:07.928 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:07 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:07.928 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:07 smithi062 ceph-mon[48130]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:45:07.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:07 smithi019 ceph-mon[67418]: purged_snaps scrub starts 2024-02-22T15:45:07.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:07 smithi019 ceph-mon[67418]: purged_snaps scrub ok 2024-02-22T15:45:07.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:07 smithi019 ceph-mon[67418]: osd.3 [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] boot 2024-02-22T15:45:07.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:07 smithi019 ceph-mon[67418]: osdmap e27: 4 total, 4 up, 4 in 2024-02-22T15:45:07.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:07 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:45:07.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:07 smithi019 ceph-mon[67418]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-02-22T15:45:08.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-02-22T15:45:08.822 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:45:08.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: osdmap e28: 4 total, 4 up, 4 in 2024-02-22T15:45:08.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi062 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi062 to 15970M 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:08.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:08.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:08 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: osdmap e28: 4 total, 4 up, 4 in 2024-02-22T15:45:08.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: Detected new or changed devices on smithi062 2024-02-22T15:45:08.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi062 to 15970M 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:08.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:08 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:09.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:09 smithi062 ceph-mon[48130]: osdmap e29: 4 total, 4 up, 4 in 2024-02-22T15:45:09.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:09 smithi062 ceph-mon[48130]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:09.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:09 smithi062 ceph-mon[48130]: osdmap e30: 4 total, 4 up, 4 in 2024-02-22T15:45:09.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:09 smithi019 ceph-mon[67418]: osdmap e29: 4 total, 4 up, 4 in 2024-02-22T15:45:09.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:09 smithi019 ceph-mon[67418]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:09.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:09 smithi019 ceph-mon[67418]: osdmap e30: 4 total, 4 up, 4 in 2024-02-22T15:45:10.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 -- 172.21.15.62:0/1655707257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fe0e0 msgr2=0x7f3b200fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:10.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1655707257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fe0e0 0x7f3b200fe550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b10004970 tx=0x7f3b10009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 -- 172.21.15.62:0/1655707257 shutdown_connections 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1655707257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fe0e0 0x7f3b200fe550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1655707257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b200fcef0 0x7f3b200fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 -- 172.21.15.62:0/1655707257 >> 172.21.15.62:0/1655707257 conn(0x7f3b200f8620 msgr2=0x7f3b200faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 -- 172.21.15.62:0/1655707257 shutdown_connections 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 -- 172.21.15.62:0/1655707257 wait complete. 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.527+0000 7f3b274c4700 1 Processor -- start 2024-02-22T15:45:10.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b274c4700 1 -- start start 2024-02-22T15:45:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b274c4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 0x7f3b2010d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b274c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b200fe0e0 0x7f3b2010dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b274c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2010e380 con 0x7f3b200fe0e0 2024-02-22T15:45:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b274c4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2010ec20 con 0x7f3b200fcef0 2024-02-22T15:45:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 0x7f3b2010d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:10.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 0x7f3b2010d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35956/0 (socket says 172.21.15.62:35956) 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 -- 172.21.15.62:0/1172096183 learned_addr learned my addr 172.21.15.62:0/1172096183 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b24a5f700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b200fe0e0 0x7f3b2010dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 -- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b200fe0e0 msgr2=0x7f3b2010dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b200fe0e0 0x7f3b2010dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 -- 172.21.15.62:0/1172096183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10005040 con 0x7f3b200fcef0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.528+0000 7f3b25260700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 0x7f3b2010d840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c002a50 tx=0x7f3b1c004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.529+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b1c007cc0 con 0x7f3b200fcef0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.529+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b1c00f040 con 0x7f3b200fcef0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.529+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2010ef00 con 0x7f3b200fcef0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.529+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b1c00d990 con 0x7f3b200fcef0 2024-02-22T15:45:10.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.529+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2010f440 con 0x7f3b200fcef0 2024-02-22T15:45:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.530+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2004bed0 con 0x7f3b200fcef0 2024-02-22T15:45:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.530+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3b1c009710 con 0x7f3b200fcef0 2024-02-22T15:45:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.531+0000 7f3b15ffb700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b0c069370 0x7f3b0c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.531+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f3b1c086810 con 0x7f3b200fcef0 2024-02-22T15:45:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.531+0000 7f3b24a5f700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b0c069370 0x7f3b0c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.531+0000 7f3b24a5f700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b0c069370 0x7f3b0c06b820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b370 tx=0x7f3b10009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:45:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.535+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3b1c056bd0 con 0x7f3b200fcef0 2024-02-22T15:45:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:10.733+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b20102ac0 con 0x7f3b0c069370 2024-02-22T15:45:10.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:10 smithi019 ceph-mon[67418]: osdmap e31: 4 total, 4 up, 4 in 2024-02-22T15:45:10.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:10 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:45:10.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:10 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:45:10.998 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:10 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:11.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:10 smithi062 ceph-mon[48130]: osdmap e31: 4 total, 4 up, 4 in 2024-02-22T15:45:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:10 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:45:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:10 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:45:11.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:10 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:12.130 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:11 smithi062 ceph-mon[48130]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:45:12.130 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:11 smithi062 ceph-mon[48130]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:12.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:11 smithi019 ceph-mon[67418]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:45:12.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:11 smithi019 ceph-mon[67418]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:14.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:13 smithi062 ceph-mon[48130]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:13 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/3757088123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08c28eb8-43da-423f-a330-dc773450fc4d"}]: dispatch 2024-02-22T15:45:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:13 smithi062 ceph-mon[48130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08c28eb8-43da-423f-a330-dc773450fc4d"}]: dispatch 2024-02-22T15:45:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:13 smithi062 ceph-mon[48130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08c28eb8-43da-423f-a330-dc773450fc4d"}]': finished 2024-02-22T15:45:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:13 smithi062 ceph-mon[48130]: osdmap e32: 5 total, 4 up, 5 in 2024-02-22T15:45:14.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:13 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:14.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:13 smithi019 ceph-mon[67418]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:14.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:13 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/3757088123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08c28eb8-43da-423f-a330-dc773450fc4d"}]: dispatch 2024-02-22T15:45:14.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:13 smithi019 ceph-mon[67418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08c28eb8-43da-423f-a330-dc773450fc4d"}]: dispatch 2024-02-22T15:45:14.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:13 smithi019 ceph-mon[67418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08c28eb8-43da-423f-a330-dc773450fc4d"}]': finished 2024-02-22T15:45:14.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:13 smithi019 ceph-mon[67418]: osdmap e32: 5 total, 4 up, 5 in 2024-02-22T15:45:14.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:13 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:15.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:14 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2832137119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:45:15.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:14 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2832137119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:45:16.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:15 smithi019 ceph-mon[67418]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-22T15:45:16.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:15 smithi062 ceph-mon[48130]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-22T15:45:18.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:17 smithi019 ceph-mon[67418]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-22T15:45:18.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:17 smithi062 ceph-mon[48130]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-22T15:45:20.212 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:19 smithi062 ceph-mon[48130]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-02-22T15:45:20.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:19 smithi019 ceph-mon[67418]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-02-22T15:45:20.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T15:45:20.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:20.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:21.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T15:45:21.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:21.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:22.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:21 smithi019 ceph-mon[67418]: Deploying daemon osd.4 on smithi062 2024-02-22T15:45:22.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:21 smithi019 ceph-mon[67418]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-22T15:45:22.276 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:21 smithi062 ceph-mon[48130]: Deploying daemon osd.4 on smithi062 2024-02-22T15:45:22.277 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:21 smithi062 ceph-mon[48130]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-22T15:45:24.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:23 smithi019 ceph-mon[67418]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-22T15:45:24.363 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:23 smithi062 ceph-mon[48130]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-22T15:45:26.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:25 smithi019 ceph-mon[67418]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:26.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:25 smithi062 ceph-mon[48130]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:27.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:28.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:28 smithi062 ceph-mon[48130]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:28.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:28 smithi019 ceph-mon[67418]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:29.844 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 4 on host 'smithi062' 2024-02-22T15:45:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.860+0000 7f3b15ffb700 1 -- 172.21.15.62:0/1172096183 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3b20102ac0 con 0x7f3b0c069370 2024-02-22T15:45:29.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b0c069370 msgr2=0x7f3b0c06b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:29.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b0c069370 0x7f3b0c06b820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b370 tx=0x7f3b10009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:29.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 msgr2=0x7f3b2010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 0x7f3b2010d840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c002a50 tx=0x7f3b1c004600 comp rx=0 tx=0).stop 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 shutdown_connections 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b0c069370 0x7f3b0c06b820 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b200fcef0 0x7f3b2010d840 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 --2- 172.21.15.62:0/1172096183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b200fe0e0 0x7f3b2010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 >> 172.21.15.62:0/1172096183 conn(0x7f3b200f8620 msgr2=0x7f3b201013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.862+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 shutdown_connections 2024-02-22T15:45:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:29.863+0000 7f3b274c4700 1 -- 172.21.15.62:0/1172096183 wait complete. 2024-02-22T15:45:30.092 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:29 smithi062 ceph-mon[48130]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:30.093 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.093 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:29 smithi019 ceph-mon[67418]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:30.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.668 DEBUG:teuthology.orchestra.run.smithi062:osd.4> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.4.service 2024-02-22T15:45:30.670 INFO:tasks.cephadm:Deploying osd.5 on smithi062 with /dev/vg_nvme/lv_2... 2024-02-22T15:45:30.670 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-22T15:45:30.705 INFO:journalctl@ceph.osd.4.smithi062.stdout:-- Logs begin at Thu 2024-02-22 14:59:35 UTC. -- 2024-02-22T15:45:30.949 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:30 smithi062 ceph-mon[48130]: from='osd.4 [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:45:30.949 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:30 smithi062 ceph-mon[48130]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:45:30.949 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.949 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:30 smithi019 ceph-mon[67418]: from='osd.4 [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:45:30.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:30 smithi019 ceph-mon[67418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:45:30.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:30.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:45:32.080 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-mon[48130]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:32.080 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-mon[48130]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T15:45:32.080 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-mon[48130]: osdmap e33: 5 total, 4 up, 5 in 2024-02-22T15:45:32.080 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:32.080 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-mon[48130]: from='osd.4 [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:32.080 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-mon[48130]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:32.080 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[63236]: 2024-02-22T15:45:31.987+0000 7f358e18f700 -1 osd.4 0 waiting for initial osdmap 2024-02-22T15:45:32.080 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:45:31 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[63236]: 2024-02-22T15:45:31.994+0000 7f358978b700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:45:32.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:31 smithi019 ceph-mon[67418]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:32.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:31 smithi019 ceph-mon[67418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T15:45:32.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:31 smithi019 ceph-mon[67418]: osdmap e33: 5 total, 4 up, 5 in 2024-02-22T15:45:32.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:31 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:32.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:31 smithi019 ceph-mon[67418]: from='osd.4 [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:32.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:31 smithi019 ceph-mon[67418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:33.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-02-22T15:45:33.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: osdmap e34: 5 total, 4 up, 5 in 2024-02-22T15:45:33.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:33.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:33.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: osd.4 [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] boot 2024-02-22T15:45:33.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: osdmap e35: 5 total, 5 up, 5 in 2024-02-22T15:45:33.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:32 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:33.251 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-02-22T15:45:33.252 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: osdmap e34: 5 total, 4 up, 5 in 2024-02-22T15:45:33.252 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:33.252 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:33.252 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: osd.4 [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] boot 2024-02-22T15:45:33.252 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: osdmap e35: 5 total, 5 up, 5 in 2024-02-22T15:45:33.252 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:32 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:45:34.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:33 smithi062 ceph-mon[48130]: purged_snaps scrub starts 2024-02-22T15:45:34.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:33 smithi062 ceph-mon[48130]: purged_snaps scrub ok 2024-02-22T15:45:34.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:33 smithi062 ceph-mon[48130]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:34.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:34.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:33 smithi019 ceph-mon[67418]: purged_snaps scrub starts 2024-02-22T15:45:34.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:33 smithi019 ceph-mon[67418]: purged_snaps scrub ok 2024-02-22T15:45:34.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:33 smithi019 ceph-mon[67418]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-22T15:45:34.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-22T15:45:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-22T15:45:34.762 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-02-22T15:45:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-02-22T15:45:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011118 s, 943 MB/s 2024-02-22T15:45:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi062 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: osdmap e36: 5 total, 5 up, 5 in 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi062 to 7985M 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.054 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:35.055 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:35.055 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.055 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:35.055 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:35.055 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:35.055 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: Detected new or changed devices on smithi062 2024-02-22T15:45:35.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:45:35.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: osdmap e36: 5 total, 5 up, 5 in 2024-02-22T15:45:35.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:45:35.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi062 to 7985M 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:45:35.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:35.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-02-22T15:45:35.611 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:45:36.444 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:36 smithi062 ceph-mon[48130]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:36.444 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:36.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:36 smithi019 ceph-mon[67418]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:36.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:37.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.210+0000 7f79497ee700 1 -- 172.21.15.62:0/564244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 msgr2=0x7f79440fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:37.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.210+0000 7f79497ee700 1 --2- 172.21.15.62:0/564244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f79440fe550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7938004970 tx=0x7f7938009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:37.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.210+0000 7f79497ee700 1 -- 172.21.15.62:0/564244 shutdown_connections 2024-02-22T15:45:37.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.210+0000 7f79497ee700 1 --2- 172.21.15.62:0/564244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f79440fe550 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:37.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.210+0000 7f79497ee700 1 --2- 172.21.15.62:0/564244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79440fcef0 0x7f79440fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 -- 172.21.15.62:0/564244 >> 172.21.15.62:0/564244 conn(0x7f79440f8620 msgr2=0x7f79440faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 -- 172.21.15.62:0/564244 shutdown_connections 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 -- 172.21.15.62:0/564244 wait complete. 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 Processor -- start 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 -- start start 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79440fcef0 0x7f794410d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f794410dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794410e340 con 0x7f79440fcef0 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.211+0000 7f79497ee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794410ebe0 con 0x7f79440fe0e0 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f794410dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f794410dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43628/0 (socket says 172.21.15.62:43628) 2024-02-22T15:45:37.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 -- 172.21.15.62:0/2557528698 learned_addr learned my addr 172.21.15.62:0/2557528698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f7942ffd700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79440fcef0 0x7f794410d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 -- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79440fcef0 msgr2=0x7f794410d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79440fcef0 0x7f794410d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 -- 172.21.15.62:0/2557528698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938005040 con 0x7f79440fe0e0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79427fc700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f794410dd40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7938000c00 tx=0x7f7938009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793800e040 con 0x7f79440fe0e0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794410ee60 con 0x7f79440fe0e0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f793800bc90 con 0x7f79440fe0e0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793801c650 con 0x7f79440fe0e0 2024-02-22T15:45:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.212+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794410f3a0 con 0x7f79440fe0e0 2024-02-22T15:45:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.213+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794404bed0 con 0x7f79440fe0e0 2024-02-22T15:45:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.214+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7938013420 con 0x7f79440fe0e0 2024-02-22T15:45:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.214+0000 7f792b7fe700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f792c0692a0 0x7f792c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.214+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f7938088c40 con 0x7f79440fe0e0 2024-02-22T15:45:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.215+0000 7f7942ffd700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f792c0692a0 0x7f792c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.215+0000 7f7942ffd700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f792c0692a0 0x7f792c06b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79440fcc30 tx=0x7f793400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:45:37.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.218+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7938058ed0 con 0x7f79440fe0e0 2024-02-22T15:45:37.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:37.420+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7944102ac0 con 0x7f792c0692a0 2024-02-22T15:45:38.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:38 smithi062 ceph-mon[48130]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:38.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:38 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:45:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:38 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:45:38.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:38 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:38.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:38 smithi019 ceph-mon[67418]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:38.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:38 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-22T15:45:38.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:38 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-22T15:45:38.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:38 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:39.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:39 smithi062 ceph-mon[48130]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:45:39.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:39 smithi019 ceph-mon[67418]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:45:40.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:40 smithi062 ceph-mon[48130]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:40.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:40 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/1042048932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7"}]: dispatch 2024-02-22T15:45:40.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:40 smithi062 ceph-mon[48130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7"}]: dispatch 2024-02-22T15:45:40.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:40 smithi062 ceph-mon[48130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7"}]': finished 2024-02-22T15:45:40.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:40 smithi062 ceph-mon[48130]: osdmap e37: 6 total, 5 up, 6 in 2024-02-22T15:45:40.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:40 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:40.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:40 smithi019 ceph-mon[67418]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:40.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:40 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/1042048932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7"}]: dispatch 2024-02-22T15:45:40.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:40 smithi019 ceph-mon[67418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7"}]: dispatch 2024-02-22T15:45:40.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:40 smithi019 ceph-mon[67418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7"}]': finished 2024-02-22T15:45:40.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:40 smithi019 ceph-mon[67418]: osdmap e37: 6 total, 5 up, 6 in 2024-02-22T15:45:40.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:40 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:41.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:41 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/2370299539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:45:41.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:41 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/2370299539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-22T15:45:42.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:42 smithi062 ceph-mon[48130]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:42.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:42 smithi019 ceph-mon[67418]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:44.236 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:44 smithi062 ceph-mon[48130]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:44.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:44 smithi019 ceph-mon[67418]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:46.245 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:46 smithi062 ceph-mon[48130]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:46.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:46 smithi019 ceph-mon[67418]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:48.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:48 smithi062 ceph-mon[48130]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:48.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T15:45:48.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:48 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:48.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:48 smithi019 ceph-mon[67418]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:48.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:48 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T15:45:48.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:48 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:45:49.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:49 smithi062 ceph-mon[48130]: Deploying daemon osd.5 on smithi062 2024-02-22T15:45:49.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:49 smithi019 ceph-mon[67418]: Deploying daemon osd.5 on smithi062 2024-02-22T15:45:50.379 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:50 smithi062 ceph-mon[48130]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:50.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:50 smithi019 ceph-mon[67418]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:51.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:51.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:45:52.423 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:52 smithi062 ceph-mon[48130]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:52.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:52 smithi019 ceph-mon[67418]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:53.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:53.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:53.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:53.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:53 smithi019 ceph-mon[67418]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:53.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:53.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:45:53.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:53.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:53 smithi062 ceph-mon[48130]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:56.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:55 smithi062 ceph-mon[48130]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:56.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:55 smithi019 ceph-mon[67418]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:56.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.342+0000 7f792b7fe700 1 -- 172.21.15.62:0/2557528698 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7944102ac0 con 0x7f792c0692a0 2024-02-22T15:45:56.327 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 5 on host 'smithi062' 2024-02-22T15:45:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f792c0692a0 msgr2=0x7f792c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f792c0692a0 0x7f792c06b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79440fcc30 tx=0x7f793400b040 comp rx=0 tx=0).stop 2024-02-22T15:45:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 msgr2=0x7f794410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:56.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f794410dd40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7938000c00 tx=0x7f7938009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 shutdown_connections 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f792c0692a0 0x7f792c06b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79440fcef0 0x7f794410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 --2- 172.21.15.62:0/2557528698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79440fe0e0 0x7f794410dd40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 >> 172.21.15.62:0/2557528698 conn(0x7f79440f8620 msgr2=0x7f79441013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 shutdown_connections 2024-02-22T15:45:56.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:45:56.345+0000 7f79497ee700 1 -- 172.21.15.62:0/2557528698 wait complete. 2024-02-22T15:45:57.060 DEBUG:teuthology.orchestra.run.smithi062:osd.5> sudo journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.5.service 2024-02-22T15:45:57.063 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-22T15:45:57.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd stat -f json 2024-02-22T15:45:57.092 INFO:journalctl@ceph.osd.5.smithi062.stdout:-- Logs begin at Thu 2024-02-22 14:59:35 UTC. -- 2024-02-22T15:45:57.355 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: from='osd.5 [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.422 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:57 smithi062 ceph-mon[48130]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:57.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: from='osd.5 [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:45:57.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:45:57.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:45:57.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:57 smithi019 ceph-mon[67418]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:58.315 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-mon[48130]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T15:45:58.315 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-mon[48130]: osdmap e38: 6 total, 5 up, 6 in 2024-02-22T15:45:58.315 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:58.315 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-mon[48130]: from='osd.5 [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:58.315 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-mon[48130]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:58.316 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[71291]: 2024-02-22T15:45:58.306+0000 7f05c7a1f700 -1 osd.5 0 waiting for initial osdmap 2024-02-22T15:45:58.316 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:45:58 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[71291]: 2024-02-22T15:45:58.317+0000 7f05c381c700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:45:58.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:58 smithi019 ceph-mon[67418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T15:45:58.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:58 smithi019 ceph-mon[67418]: osdmap e38: 6 total, 5 up, 6 in 2024-02-22T15:45:58.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:58 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:58.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:58 smithi019 ceph-mon[67418]: from='osd.5 [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:58.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:58 smithi019 ceph-mon[67418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.192+0000 7f9057727700 1 -- 172.21.15.19:0/1220031626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90501003a0 msgr2=0x7f9050102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.192+0000 7f9057727700 1 --2- 172.21.15.19:0/1220031626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90501003a0 0x7f9050102780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9044004940 tx=0x7f9044009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.192+0000 7f9057727700 1 -- 172.21.15.19:0/1220031626 shutdown_connections 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.192+0000 7f9057727700 1 --2- 172.21.15.19:0/1220031626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90501003a0 0x7f9050102780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.192+0000 7f9057727700 1 --2- 172.21.15.19:0/1220031626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90500fda80 0x7f90500ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.192+0000 7f9057727700 1 -- 172.21.15.19:0/1220031626 >> 172.21.15.19:0/1220031626 conn(0x7f90500f74e0 msgr2=0x7f90500f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 -- 172.21.15.19:0/1220031626 shutdown_connections 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 -- 172.21.15.19:0/1220031626 wait complete. 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 Processor -- start 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 -- start start 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 0x7f905010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90501003a0 0x7f905010ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050110610 con 0x7f90500fda80 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f9057727700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050110e50 con 0x7f90501003a0 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f90554c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 0x7f905010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.193+0000 7f90554c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 0x7f905010fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58940/0 (socket says 172.21.15.19:58940) 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f90554c3700 1 -- 172.21.15.19:0/2354189604 learned_addr learned my addr 172.21.15.19:0/2354189604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f9054cc2700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90501003a0 0x7f905010ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f90554c3700 1 -- 172.21.15.19:0/2354189604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90501003a0 msgr2=0x7f905010ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f90554c3700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90501003a0 0x7f905010ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f90554c3700 1 -- 172.21.15.19:0/2354189604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044005040 con 0x7f90500fda80 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f90554c3700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 0x7f905010fa40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b790 tx=0x7f904c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f904c00f770 con 0x7f90500fda80 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f904c00fd80 con 0x7f90500fda80 2024-02-22T15:45:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9050111130 con 0x7f90500fda80 2024-02-22T15:45:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f904c00d310 con 0x7f90500fda80 2024-02-22T15:45:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.194+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9050111670 con 0x7f90500fda80 2024-02-22T15:45:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.195+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905004bed0 con 0x7f90500fda80 2024-02-22T15:45:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.196+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f904c010420 con 0x7f90500fda80 2024-02-22T15:45:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.196+0000 7f9041ffb700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f903c0692a0 0x7f903c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:45:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.196+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f904c086be0 con 0x7f90500fda80 2024-02-22T15:45:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.196+0000 7f9054cc2700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f903c0692a0 0x7f903c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:45:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.197+0000 7f9054cc2700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f903c0692a0 0x7f903c06b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9044004560 tx=0x7f9044009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:45:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.202+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f904c056e90 con 0x7f90500fda80 2024-02-22T15:45:59.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.396+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9050105950 con 0x7f90500fda80 2024-02-22T15:45:59.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.396+0000 7f9041ffb700 1 -- 172.21.15.19:0/2354189604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f904c056a20 con 0x7f90500fda80 2024-02-22T15:45:59.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f903c0692a0 msgr2=0x7f903c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f903c0692a0 0x7f903c06b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9044004560 tx=0x7f9044009e30 comp rx=0 tx=0).stop 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 msgr2=0x7f905010fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 0x7f905010fa40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b790 tx=0x7f904c009250 comp rx=0 tx=0).stop 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 shutdown_connections 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f903c0692a0 0x7f903c06b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90500fda80 0x7f905010fa40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 --2- 172.21.15.19:0/2354189604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90501003a0 0x7f905010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 >> 172.21.15.19:0/2354189604 conn(0x7f90500f74e0 msgr2=0x7f90500fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:45:59.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 shutdown_connections 2024-02-22T15:45:59.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:45:59.398+0000 7f9057727700 1 -- 172.21.15.19:0/2354189604 wait complete. 2024-02-22T15:45:59.555 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:59 smithi062 ceph-mon[48130]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-02-22T15:45:59.555 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:59 smithi062 ceph-mon[48130]: osdmap e39: 6 total, 5 up, 6 in 2024-02-22T15:45:59.555 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:59.555 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:59 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:59.555 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:45:59 smithi062 ceph-mon[48130]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:45:59.661 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:59 smithi019 ceph-mon[67418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-02-22T15:45:59.661 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:59 smithi019 ceph-mon[67418]: osdmap e39: 6 total, 5 up, 6 in 2024-02-22T15:45:59.661 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:59.661 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:59 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:45:59.661 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:45:59 smithi019 ceph-mon[67418]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-22T15:46:00.037 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1708616759,"num_in_osds":6,"osd_in_since":1708616740,"num_remapped_pgs":0} 2024-02-22T15:46:00.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T15:46:00.295 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: purged_snaps scrub starts 2024-02-22T15:46:00.295 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: purged_snaps scrub ok 2024-02-22T15:46:00.295 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: osd.5 [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] boot 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: osdmap e40: 6 total, 6 up, 6 in 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2354189604' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: Detected new or changed devices on smithi062 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi062 to 5323M 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:46:00.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:00 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.323 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: purged_snaps scrub starts 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: purged_snaps scrub ok 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: osd.5 [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] boot 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: osdmap e40: 6 total, 6 up, 6 in 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2354189604' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi062 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi062 to 5323M 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:46:00.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:00 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:01.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:01 smithi019 ceph-mon[67418]: osdmap e41: 6 total, 6 up, 6 in 2024-02-22T15:46:01.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:46:01.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:46:01.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:46:01.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:01.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:01 smithi019 ceph-mon[67418]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:01.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:01 smithi062 ceph-mon[48130]: osdmap e41: 6 total, 6 up, 6 in 2024-02-22T15:46:01.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:46:01.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:46:01.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:46:01.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:46:01.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:01 smithi062 ceph-mon[48130]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:02.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 -- 172.21.15.19:0/992210329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fdfe0 msgr2=0x7fea640fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:02.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 --2- 172.21.15.19:0/992210329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fdfe0 0x7fea640fe3b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea54004970 tx=0x7fea54009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:02.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 -- 172.21.15.19:0/992210329 shutdown_connections 2024-02-22T15:46:02.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 --2- 172.21.15.19:0/992210329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea640fe8f0 0x7fea64107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 --2- 172.21.15.19:0/992210329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fdfe0 0x7fea640fe3b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 -- 172.21.15.19:0/992210329 >> 172.21.15.19:0/992210329 conn(0x7fea64071d90 msgr2=0x7fea64072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 -- 172.21.15.19:0/992210329 shutdown_connections 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 -- 172.21.15.19:0/992210329 wait complete. 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 Processor -- start 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.050+0000 7fea691fa700 1 -- start start 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea691fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea640fdfe0 0x7fea64113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea691fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 0x7fea64113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea691fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea641144c0 con 0x7fea640fe8f0 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea691fa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6410d7f0 con 0x7fea640fdfe0 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 0x7fea64113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 0x7fea64113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58956/0 (socket says 172.21.15.19:58956) 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 -- 172.21.15.19:0/1231967562 learned_addr learned my addr 172.21.15.19:0/1231967562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea62d9d700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea640fdfe0 0x7fea64113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 -- 172.21.15.19:0/1231967562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea640fdfe0 msgr2=0x7fea64113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:02.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea640fdfe0 0x7fea64113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:02.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 -- 172.21.15.19:0/1231967562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea54005040 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea6259c700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 0x7fea64113e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c830 tx=0x7fea58004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea5800a760 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6410dad0 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea5800ad70 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea580113a0 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.051+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6410e010 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.052+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea641046c0 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.053+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fea58012400 con 0x7fea640fe8f0 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.053+0000 7fea4b7fe700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fea4c0692a0 0x7fea4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:02.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.053+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fea58086ee0 con 0x7fea640fe8f0 2024-02-22T15:46:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.053+0000 7fea62d9d700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fea4c0692a0 0x7fea4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.054+0000 7fea62d9d700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fea4c0692a0 0x7fea4c06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea54004f50 tx=0x7fea54009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.058+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fea58056fe0 con 0x7fea640fe8f0 2024-02-22T15:46:02.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.250+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea64109ef0 con 0x7fea640fe8f0 2024-02-22T15:46:02.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.250+0000 7fea4b7fe700 1 -- 172.21.15.19:0/1231967562 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11032 (secure 0 0 0) 0x7fea58056b70 con 0x7fea640fe8f0 2024-02-22T15:46:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:02.234 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:46:00.310497+0000","last_up_change":"2024-02-22T15:45:59.303021+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6803","nonce":3426164274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6805","nonce":3426164274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6809","nonce":3426164274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6807","nonce":3426164274}]},"public_addr":"172.21.15.19:6803/3426164274","cluster_addr":"172.21.15.19:6805/3426164274","heartbeat_back_addr":"172.21.15.19:6809/3426164274","heartbeat_front_addr":"172.21.15.19:6807/3426164274","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6811","nonce":1226754076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6813","nonce":1226754076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6817","nonce":1226754076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6815","nonce":1226754076}]},"public_addr":"172.21.15.19:6811/1226754076","cluster_addr":"172.21.15.19:6813/1226754076","heartbeat_back_addr":"172.21.15.19:6817/1226754076","heartbeat_front_addr":"172.21.15.19:6815/1226754076","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6819","nonce":2801184587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6821","nonce":2801184587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6825","nonce":2801184587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6823","nonce":2801184587}]},"public_addr":"172.21.15.19:6819/2801184587","cluster_addr":"172.21.15.19:6821/2801184587","heartbeat_back_addr":"172.21.15.19:6825/2801184587","heartbeat_front_addr":"172.21.15.19:6823/2801184587","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6801","nonce":179328076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6803","nonce":179328076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6807","nonce":179328076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6805","nonce":179328076}]},"public_addr":"172.21.15.62:6801/179328076","cluster_addr":"172.21.15.62:6803/179328076","heartbeat_back_addr":"172.21.15.62:6807/179328076","heartbeat_front_addr":"172.21.15.62:6805/179328076","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6809","nonce":925300074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6811","nonce":925300074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6815","nonce":925300074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6813","nonce":925300074}]},"public_addr":"172.21.15.62:6809/925300074","cluster_addr":"172.21.15.62:6811/925300074","heartbeat_back_addr":"172.21.15.62:6815/925300074","heartbeat_front_addr":"172.21.15.62:6813/925300074","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6817","nonce":3437650181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6819","nonce":3437650181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6823","nonce":3437650181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6821","nonce":3437650181}]},"public_addr":"172.21.15.62:6817/3437650181","cluster_addr":"172.21.15.62:6819/3437650181","heartbeat_back_addr":"172.21.15.62:6823/3437650181","heartbeat_front_addr":"172.21.15.62:6821/3437650181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fea4c0692a0 msgr2=0x7fea4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fea4c0692a0 0x7fea4c06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea54004f50 tx=0x7fea54009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 msgr2=0x7fea64113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 0x7fea64113e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c830 tx=0x7fea58004470 comp rx=0 tx=0).stop 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 shutdown_connections 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fea4c0692a0 0x7fea4c06b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea640fdfe0 0x7fea64113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 --2- 172.21.15.19:0/1231967562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea640fe8f0 0x7fea64113e70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:02.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 >> 172.21.15.19:0/1231967562 conn(0x7fea64071d90 msgr2=0x7fea64101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:02.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 shutdown_connections 2024-02-22T15:46:02.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:02.252+0000 7fea691fa700 1 -- 172.21.15.19:0/1231967562 wait complete. 2024-02-22T15:46:02.592 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:02 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1231967562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:46:02.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:02 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1231967562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:46:02.987 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-22T15:44:43.417715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-02-22T15:46:02.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-22T15:46:03.269 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:03.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:03 smithi062 ceph-mon[48130]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:03.747 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:03 smithi019 ceph-mon[67418]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 -- 172.21.15.19:0/1145414222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481027e0 msgr2=0x7f46480fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1145414222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481027e0 0x7f46480fbea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4630004940 tx=0x7f4630009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 -- 172.21.15.19:0/1145414222 shutdown_connections 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1145414222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46481027e0 0x7f46480fbea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1145414222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4648101e40 0x7f4648102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 -- 172.21.15.19:0/1145414222 >> 172.21.15.19:0/1145414222 conn(0x7f4648071d90 msgr2=0x7f4648072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 -- 172.21.15.19:0/1145414222 shutdown_connections 2024-02-22T15:46:05.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.157+0000 7f464e2f3700 1 -- 172.21.15.19:0/1145414222 wait complete. 2024-02-22T15:46:05.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f464e2f3700 1 Processor -- start 2024-02-22T15:46:05.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f464e2f3700 1 -- start start 2024-02-22T15:46:05.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f464e2f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 0x7f4648112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:05.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f464e2f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46481027e0 0x7f4648112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f464e2f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46481134b0 con 0x7f4648101e40 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f464e2f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464810c750 con 0x7f46481027e0 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.158+0000 7f4647fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 0x7f4648112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4647fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 0x7f4648112890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58972/0 (socket says 172.21.15.19:58972) 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4647fff700 1 -- 172.21.15.19:0/1465891250 learned_addr learned my addr 172.21.15.19:0/1465891250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4647fff700 1 -- 172.21.15.19:0/1465891250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46481027e0 msgr2=0x7f4648112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f46477fe700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46481027e0 0x7f4648112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4647fff700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46481027e0 0x7f4648112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4647fff700 1 -- 172.21.15.19:0/1465891250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4630005040 con 0x7f4648101e40 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4647fff700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 0x7f4648112890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f463800b790 tx=0x7f4638009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463800f770 con 0x7f4648101e40 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f463800fd80 con 0x7f4648101e40 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464810ca30 con 0x7f4648101e40 2024-02-22T15:46:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463800d310 con 0x7f4648101e40 2024-02-22T15:46:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.159+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464810cf70 con 0x7f4648101e40 2024-02-22T15:46:05.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.160+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46480738b0 con 0x7f4648101e40 2024-02-22T15:46:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.161+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4638010420 con 0x7f4648101e40 2024-02-22T15:46:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.161+0000 7f4644ff9700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f46340692a0 0x7f463406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.161+0000 7f46477fe700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f46340692a0 0x7f463406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.161+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4638086e90 con 0x7f4648101e40 2024-02-22T15:46:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.162+0000 7f46477fe700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f46340692a0 0x7f463406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f463000b3b0 tx=0x7f4630009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:05.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.166+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4638056ef0 con 0x7f4648101e40 2024-02-22T15:46:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.356+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f464804bf00 con 0x7f4648101e40 2024-02-22T15:46:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.356+0000 7f4644ff9700 1 -- 172.21.15.19:0/1465891250 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4638056a80 con 0x7f4648101e40 2024-02-22T15:46:05.340 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f46340692a0 msgr2=0x7f463406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f46340692a0 0x7f463406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f463000b3b0 tx=0x7f4630009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 msgr2=0x7f4648112890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 0x7f4648112890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f463800b790 tx=0x7f4638009250 comp rx=0 tx=0).stop 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 shutdown_connections 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f46340692a0 0x7f463406b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4648101e40 0x7f4648112890 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 --2- 172.21.15.19:0/1465891250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46481027e0 0x7f4648112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 >> 172.21.15.19:0/1465891250 conn(0x7f4648071d90 msgr2=0x7f46480facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:05.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 shutdown_connections 2024-02-22T15:46:05.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:05.358+0000 7f464e2f3700 1 -- 172.21.15.19:0/1465891250 wait complete. 2024-02-22T15:46:05.961 INFO:tasks.cephadm:Setting up client nodes... 2024-02-22T15:46:05.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-22T15:46:06.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:05 smithi062 ceph-mon[48130]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:06.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:05 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1465891250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T15:46:06.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:05 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:06.237 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:06.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:05 smithi019 ceph-mon[67418]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:06.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:05 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1465891250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T15:46:06.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:05 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 -- 172.21.15.19:0/3375337275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 msgr2=0x7fd7a80fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/3375337275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80fe550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd798004940 tx=0x7fd798009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 -- 172.21.15.19:0/3375337275 shutdown_connections 2024-02-22T15:46:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/3375337275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80fe550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/3375337275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a80fcef0 0x7fd7a80fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 -- 172.21.15.19:0/3375337275 >> 172.21.15.19:0/3375337275 conn(0x7fd7a80f8620 msgr2=0x7fd7a80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 -- 172.21.15.19:0/3375337275 shutdown_connections 2024-02-22T15:46:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.122+0000 7fd7adf6e700 1 -- 172.21.15.19:0/3375337275 wait complete. 2024-02-22T15:46:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7adf6e700 1 Processor -- start 2024-02-22T15:46:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7adf6e700 1 -- start start 2024-02-22T15:46:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7adf6e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a80fcef0 0x7fd7a8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7adf6e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7adf6e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80708e0 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7adf6e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a8070a50 con 0x7fd7a80fcef0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7a6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7a6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42530/0 (socket says 172.21.15.19:42530) 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.123+0000 7fd7a6ffd700 1 -- 172.21.15.19:0/2514400954 learned_addr learned my addr 172.21.15.19:0/2514400954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7a77fe700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a80fcef0 0x7fd7a8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7a6ffd700 1 -- 172.21.15.19:0/2514400954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a80fcef0 msgr2=0x7fd7a8071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7a6ffd700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a80fcef0 0x7fd7a8071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7a6ffd700 1 -- 172.21.15.19:0/2514400954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798005040 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7a6ffd700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80702e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd798009380 tx=0x7fd798009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd79800e040 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd79800bc50 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a8070cd0 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd798013420 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.124+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a8115920 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.125+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a804bed0 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.125+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd79801c430 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.126+0000 7fd78ffff700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7900692a0 0x7fd79006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.126+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd798087bc0 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.126+0000 7fd7a77fe700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7900692a0 0x7fd79006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.126+0000 7fd7a77fe700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7900692a0 0x7fd79006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8071bf0 tx=0x7fd79c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.131+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd798057d70 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:07 smithi062 ceph-mon[48130]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:08.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:07 smithi019 ceph-mon[67418]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.401+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd7a81058f0 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.405+0000 7fd78ffff700 1 -- 172.21.15.19:0/2514400954 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd798057900 con 0x7fd7a80fe0e0 2024-02-22T15:46:08.389 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-02-22T15:46:08.389 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQBAbNdlBzcIGBAATV6uCJbC/raMh2zb9L3xPg== 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7900692a0 msgr2=0x7fd79006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7900692a0 0x7fd79006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8071bf0 tx=0x7fd79c00b040 comp rx=0 tx=0).stop 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 msgr2=0x7fd7a80702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80702e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd798009380 tx=0x7fd798009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 shutdown_connections 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7900692a0 0x7fd79006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7a80fcef0 0x7fd7a8071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 --2- 172.21.15.19:0/2514400954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a80fe0e0 0x7fd7a80702e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 >> 172.21.15.19:0/2514400954 conn(0x7fd7a80f8620 msgr2=0x7fd7a81013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 shutdown_connections 2024-02-22T15:46:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:08.407+0000 7fd7adf6e700 1 -- 172.21.15.19:0/2514400954 wait complete. 2024-02-22T15:46:09.128 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:08 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2514400954' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T15:46:09.128 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:08 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2514400954' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T15:46:09.150 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:46:09.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-22T15:46:09.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-22T15:46:09.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-22T15:46:09.229 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:08 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2514400954' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T15:46:09.229 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:08 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2514400954' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T15:46:09.486 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T15:46:10.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:09 smithi062 ceph-mon[48130]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:10.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:09 smithi019 ceph-mon[67418]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 -- 172.21.15.62:0/2280217009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b14066c70 msgr2=0x7f5b140ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/2280217009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b14066c70 0x7f5b140ffaf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b04004940 tx=0x7f5b04009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 -- 172.21.15.62:0/2280217009 shutdown_connections 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/2280217009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b141000c0 0x7f5b14102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/2280217009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b14066c70 0x7f5b140ffaf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 -- 172.21.15.62:0/2280217009 >> 172.21.15.62:0/2280217009 conn(0x7f5b140f7500 msgr2=0x7f5b140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 -- 172.21.15.62:0/2280217009 shutdown_connections 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.123+0000 7f5b1bd21700 1 -- 172.21.15.62:0/2280217009 wait complete. 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b1bd21700 1 Processor -- start 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b1bd21700 1 -- start start 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b1bd21700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b14066c70 0x7f5b1410d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:11.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b1bd21700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 0x7f5b1410dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b1bd21700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1410e430 con 0x7f5b14066c70 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b1bd21700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1410ec70 con 0x7f5b141000c0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b192bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 0x7f5b1410dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b192bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 0x7f5b1410dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50952/0 (socket says 172.21.15.62:50952) 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b192bc700 1 -- 172.21.15.62:0/486207587 learned_addr learned my addr 172.21.15.62:0/486207587 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b19abd700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b14066c70 0x7f5b1410d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b192bc700 1 -- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b14066c70 msgr2=0x7f5b1410d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b192bc700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b14066c70 0x7f5b1410d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.124+0000 7f5b192bc700 1 -- 172.21.15.62:0/486207587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04005040 con 0x7f5b141000c0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.125+0000 7f5b192bc700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 0x7f5b1410dda0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000b790 tx=0x7f5b10009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.125+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b1000f6f0 con 0x7f5b141000c0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.125+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b1000fd00 con 0x7f5b141000c0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.125+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b100103a0 con 0x7f5b141000c0 2024-02-22T15:46:11.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.125+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1410ef50 con 0x7f5b141000c0 2024-02-22T15:46:11.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.125+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1410f490 con 0x7f5b141000c0 2024-02-22T15:46:11.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.126+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1404bed0 con 0x7f5b141000c0 2024-02-22T15:46:11.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.126+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5b1000f850 con 0x7f5b141000c0 2024-02-22T15:46:11.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.126+0000 7f5b0a7fc700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5b00069370 0x7f5b0006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:11.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.127+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5b10086cb0 con 0x7f5b141000c0 2024-02-22T15:46:11.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.127+0000 7f5b19abd700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5b00069370 0x7f5b0006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:11.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.127+0000 7f5b19abd700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5b00069370 0x7f5b0006b820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b04004560 tx=0x7f5b04009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:11.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.131+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5b10056d10 con 0x7f5b141000c0 2024-02-22T15:46:11.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.400+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5b1410f920 con 0x7f5b141000c0 2024-02-22T15:46:11.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.407+0000 7f5b0a7fc700 1 -- 172.21.15.62:0/486207587 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5b100568a0 con 0x7f5b141000c0 2024-02-22T15:46:11.391 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2024-02-22T15:46:11.391 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQBDbNdlgD0EGBAA66dt64Sv2E0UkvQkak+www== 2024-02-22T15:46:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5b00069370 msgr2=0x7f5b0006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5b00069370 0x7f5b0006b820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b04004560 tx=0x7f5b04009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 msgr2=0x7f5b1410dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 0x7f5b1410dda0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000b790 tx=0x7f5b10009220 comp rx=0 tx=0).stop 2024-02-22T15:46:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 shutdown_connections 2024-02-22T15:46:11.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5b00069370 0x7f5b0006b820 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b14066c70 0x7f5b1410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 --2- 172.21.15.62:0/486207587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b141000c0 0x7f5b1410dda0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 >> 172.21.15.62:0/486207587 conn(0x7f5b140f7500 msgr2=0x7f5b140f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 shutdown_connections 2024-02-22T15:46:11.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-02-22T15:46:11.409+0000 7f5b1bd21700 1 -- 172.21.15.62:0/486207587 wait complete. 2024-02-22T15:46:12.025 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:11 smithi062 ceph-mon[48130]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:12.025 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:11 smithi062 ceph-mon[48130]: from='client.? 172.21.15.62:0/486207587' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T15:46:12.025 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:11 smithi062 ceph-mon[48130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T15:46:12.025 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:11 smithi062 ceph-mon[48130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T15:46:12.045 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:46:12.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-22T15:46:12.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-22T15:46:12.129 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-22T15:46:12.129 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-22T15:46:12.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph mgr dump --format=json 2024-02-22T15:46:12.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:11 smithi019 ceph-mon[67418]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:12.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:11 smithi019 ceph-mon[67418]: from='client.? 172.21.15.62:0/486207587' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T15:46:12.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:11 smithi019 ceph-mon[67418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-22T15:46:12.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:11 smithi019 ceph-mon[67418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-22T15:46:12.419 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 -- 172.21.15.19:0/480594910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 msgr2=0x7f69ec0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 --2- 172.21.15.19:0/480594910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec0fbea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f69e0004940 tx=0x7f69e0009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 -- 172.21.15.19:0/480594910 shutdown_connections 2024-02-22T15:46:14.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 --2- 172.21.15.19:0/480594910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec0fbea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 --2- 172.21.15.19:0/480594910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 0x7f69ec102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 -- 172.21.15.19:0/480594910 >> 172.21.15.19:0/480594910 conn(0x7f69ec071d90 msgr2=0x7f69ec072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 -- 172.21.15.19:0/480594910 shutdown_connections 2024-02-22T15:46:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.183+0000 7f69f2400700 1 -- 172.21.15.19:0/480594910 wait complete. 2024-02-22T15:46:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.184+0000 7f69f2400700 1 Processor -- start 2024-02-22T15:46:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.184+0000 7f69f2400700 1 -- start start 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.184+0000 7f69f2400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 0x7f69ec112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.184+0000 7f69f2400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.184+0000 7f69f2400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1134b0 con 0x7f69ec1027e0 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.184+0000 7f69f2400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec10c750 con 0x7f69ec101e40 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42554/0 (socket says 172.21.15.19:42554) 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 -- 172.21.15.19:0/509744951 learned_addr learned my addr 172.21.15.19:0/509744951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69ebfff700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 0x7f69ec112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 -- 172.21.15.19:0/509744951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 msgr2=0x7f69ec112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:14.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 0x7f69ec112890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 -- 172.21.15.19:0/509744951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0005040 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69ebfff700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 0x7f69ec112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69eb7fe700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec112dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009fd0 tx=0x7f69e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e000e040 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec10c9d0 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69e000bc50 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e001c3f0 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.185+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec10cf10 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.186+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec0738b0 con 0x7f69ec1027e0 2024-02-22T15:46:14.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.187+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f69e0013420 con 0x7f69ec1027e0 2024-02-22T15:46:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.187+0000 7f69e8ff9700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f69d40692a0 0x7f69d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.187+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f69e0088210 con 0x7f69ec1027e0 2024-02-22T15:46:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.187+0000 7f69ebfff700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f69d40692a0 0x7f69d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:14.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.187+0000 7f69ebfff700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f69d40692a0 0x7f69d406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006fd0 tx=0x7f69dc008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:14.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.192+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f69e0058310 con 0x7f69ec1027e0 2024-02-22T15:46:14.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:13 smithi019 ceph-mon[67418]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.437+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f69ec10de70 con 0x7f69ec1027e0 2024-02-22T15:46:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.440+0000 7f69e8ff9700 1 -- 172.21.15.19:0/509744951 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167167 (secure 0 0 0) 0x7f69e0057ea0 con 0x7f69ec1027e0 2024-02-22T15:46:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f69d40692a0 msgr2=0x7f69d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f69d40692a0 0x7f69d406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006fd0 tx=0x7f69dc008040 comp rx=0 tx=0).stop 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 msgr2=0x7f69ec112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec112dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009fd0 tx=0x7f69e0009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 shutdown_connections 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f69d40692a0 0x7f69d406b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69ec101e40 0x7f69ec112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 --2- 172.21.15.19:0/509744951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69ec1027e0 0x7f69ec112dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 >> 172.21.15.19:0/509744951 conn(0x7f69ec071d90 msgr2=0x7f69ec0facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 shutdown_connections 2024-02-22T15:46:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:14.443+0000 7f69f2400700 1 -- 172.21.15.19:0/509744951 wait complete. 2024-02-22T15:46:14.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:13 smithi062 ceph-mon[48130]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:15.037 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":19,"active_gid":14206,"active_name":"smithi019.yvjrkp","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":236956964},{"type":"v1","addr":"172.21.15.19:6801","nonce":236956964}]},"active_addr":"172.21.15.19:6801/236956964","active_change":"2024-02-22T15:42:20.890971+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi062.ecubmv","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/","prometheus":"http://172.21.15.19:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1375214472}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1542879988}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2452913904}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":576195047}]}]} 2024-02-22T15:46:15.043 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-22T15:46:15.043 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-22T15:46:15.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T15:46:15.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:14 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/509744951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-22T15:46:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:15.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:14 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/509744951' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-22T15:46:16.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:15 smithi019 ceph-mon[67418]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:16.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:15 smithi062 ceph-mon[48130]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 -- 172.21.15.19:0/519644281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 msgr2=0x7fcc20102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 --2- 172.21.15.19:0/519644281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc20102210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c004940 tx=0x7fcc1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 -- 172.21.15.19:0/519644281 shutdown_connections 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 --2- 172.21.15.19:0/519644281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc201027e0 0x7fcc200fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 --2- 172.21.15.19:0/519644281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc20102210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 -- 172.21.15.19:0/519644281 >> 172.21.15.19:0/519644281 conn(0x7fcc20071d90 msgr2=0x7fcc20072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 -- 172.21.15.19:0/519644281 shutdown_connections 2024-02-22T15:46:17.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.270+0000 7fcc27b58700 1 -- 172.21.15.19:0/519644281 wait complete. 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc27b58700 1 Processor -- start 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc27b58700 1 -- start start 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc27b58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc2010ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc27b58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc201027e0 0x7fcc2010ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc27b58700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2010f590 con 0x7fcc20101e40 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc27b58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2010f700 con 0x7fcc201027e0 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc258f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc2010ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc258f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc2010ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42576/0 (socket says 172.21.15.19:42576) 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc258f4700 1 -- 172.21.15.19:0/3947460315 learned_addr learned my addr 172.21.15.19:0/3947460315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.271+0000 7fcc250f3700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc201027e0 0x7fcc2010ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc258f4700 1 -- 172.21.15.19:0/3947460315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc201027e0 msgr2=0x7fcc2010ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc258f4700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc201027e0 0x7fcc2010ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc258f4700 1 -- 172.21.15.19:0/3947460315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c005040 con 0x7fcc20101e40 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc258f4700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc2010ea00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b3b0 tx=0x7fcc1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc1c00e040 con 0x7fcc20101e40 2024-02-22T15:46:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc1c00bc50 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc201179f0 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc1c013420 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.272+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc20117f30 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.273+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc200ff150 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.273+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcc1c01c430 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.274+0000 7fcc127fc700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcc0c0692a0 0x7fcc0c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.274+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcc1c087b20 con 0x7fcc20101e40 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.274+0000 7fcc250f3700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcc0c0692a0 0x7fcc0c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:17.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.274+0000 7fcc250f3700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcc0c0692a0 0x7fcc0c06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.278+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcc1c0542e0 con 0x7fcc20101e40 2024-02-22T15:46:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.467+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc200738b0 con 0x7fcc20101e40 2024-02-22T15:46:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.468+0000 7fcc127fc700 1 -- 172.21.15.19:0/3947460315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11032 (secure 0 0 0) 0x7fcc1c057900 con 0x7fcc20101e40 2024-02-22T15:46:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:17.451 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:46:00.310497+0000","last_up_change":"2024-02-22T15:45:59.303021+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6803","nonce":3426164274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6805","nonce":3426164274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6809","nonce":3426164274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6807","nonce":3426164274}]},"public_addr":"172.21.15.19:6803/3426164274","cluster_addr":"172.21.15.19:6805/3426164274","heartbeat_back_addr":"172.21.15.19:6809/3426164274","heartbeat_front_addr":"172.21.15.19:6807/3426164274","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6811","nonce":1226754076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6813","nonce":1226754076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6817","nonce":1226754076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6815","nonce":1226754076}]},"public_addr":"172.21.15.19:6811/1226754076","cluster_addr":"172.21.15.19:6813/1226754076","heartbeat_back_addr":"172.21.15.19:6817/1226754076","heartbeat_front_addr":"172.21.15.19:6815/1226754076","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6819","nonce":2801184587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6821","nonce":2801184587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6825","nonce":2801184587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6823","nonce":2801184587}]},"public_addr":"172.21.15.19:6819/2801184587","cluster_addr":"172.21.15.19:6821/2801184587","heartbeat_back_addr":"172.21.15.19:6825/2801184587","heartbeat_front_addr":"172.21.15.19:6823/2801184587","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6801","nonce":179328076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6803","nonce":179328076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6807","nonce":179328076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6805","nonce":179328076}]},"public_addr":"172.21.15.62:6801/179328076","cluster_addr":"172.21.15.62:6803/179328076","heartbeat_back_addr":"172.21.15.62:6807/179328076","heartbeat_front_addr":"172.21.15.62:6805/179328076","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6809","nonce":925300074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6811","nonce":925300074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6815","nonce":925300074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6813","nonce":925300074}]},"public_addr":"172.21.15.62:6809/925300074","cluster_addr":"172.21.15.62:6811/925300074","heartbeat_back_addr":"172.21.15.62:6815/925300074","heartbeat_front_addr":"172.21.15.62:6813/925300074","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6817","nonce":3437650181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6819","nonce":3437650181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6823","nonce":3437650181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6821","nonce":3437650181}]},"public_addr":"172.21.15.62:6817/3437650181","cluster_addr":"172.21.15.62:6819/3437650181","heartbeat_back_addr":"172.21.15.62:6823/3437650181","heartbeat_front_addr":"172.21.15.62:6821/3437650181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T15:46:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcc0c0692a0 msgr2=0x7fcc0c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcc0c0692a0 0x7fcc0c06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006fd0 tx=0x7fcc14008040 comp rx=0 tx=0).stop 2024-02-22T15:46:17.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 msgr2=0x7fcc2010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc2010ea00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b3b0 tx=0x7fcc1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 shutdown_connections 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fcc0c0692a0 0x7fcc0c06b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc20101e40 0x7fcc2010ea00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 --2- 172.21.15.19:0/3947460315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc201027e0 0x7fcc2010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 >> 172.21.15.19:0/3947460315 conn(0x7fcc20071d90 msgr2=0x7fcc200fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 shutdown_connections 2024-02-22T15:46:17.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:17.470+0000 7fcc27b58700 1 -- 172.21.15.19:0/3947460315 wait complete. 2024-02-22T15:46:18.098 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:17 smithi019 ceph-mon[67418]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:18.098 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:17 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3947460315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:46:18.118 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-22T15:46:18.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd dump --format=json 2024-02-22T15:46:18.395 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:18.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:17 smithi062 ceph-mon[48130]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:18.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:17 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3947460315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:46:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 -- 172.21.15.19:0/1238427702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750102800 msgr2=0x7f17500fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 --2- 172.21.15.19:0/1238427702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750102800 0x7f17500fbe70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1740004940 tx=0x7f1740009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 -- 172.21.15.19:0/1238427702 shutdown_connections 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 --2- 172.21.15.19:0/1238427702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750102800 0x7f17500fbe70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 --2- 172.21.15.19:0/1238427702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1750101e60 0x7f1750102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 -- 172.21.15.19:0/1238427702 >> 172.21.15.19:0/1238427702 conn(0x7f17500f7a10 msgr2=0x7f17500f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 -- 172.21.15.19:0/1238427702 shutdown_connections 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.243+0000 7f17558d5700 1 -- 172.21.15.19:0/1238427702 wait complete. 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17558d5700 1 Processor -- start 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17558d5700 1 -- start start 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17558d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 0x7f17501139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17558d5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1750102800 0x7f1750113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17558d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1750114580 con 0x7f1750101e60 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17558d5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175010d900 con 0x7f1750102800 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 0x7f17501139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 0x7f17501139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58546/0 (socket says 172.21.15.19:58546) 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 -- 172.21.15.19:0/3163900667 learned_addr learned my addr 172.21.15.19:0/3163900667 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174e7fc700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1750102800 0x7f1750113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 -- 172.21.15.19:0/3163900667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1750102800 msgr2=0x7f1750113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1750102800 0x7f1750113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 -- 172.21.15.19:0/3163900667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740005040 con 0x7f1750101e60 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f174effd700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 0x7f17501139f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1744002a50 tx=0x7f17440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.244+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1744007cc0 con 0x7f1750101e60 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.245+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175010dbe0 con 0x7f1750101e60 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.245+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f174400f040 con 0x7f1750101e60 2024-02-22T15:46:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.245+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174400d9c0 con 0x7f1750101e60 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.245+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175010e0f0 con 0x7f1750101e60 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.246+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17500fb610 con 0x7f1750101e60 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.246+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f174400db20 con 0x7f1750101e60 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.246+0000 7f17377fe700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f17380692a0 0x7f173806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.247+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1744086b70 con 0x7f1750101e60 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.247+0000 7f174e7fc700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f17380692a0 0x7f173806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.247+0000 7f174e7fc700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f17380692a0 0x7f173806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1740004890 tx=0x7f1740009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.251+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1744056bf0 con 0x7f1750101e60 2024-02-22T15:46:20.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:19 smithi019 ceph-mon[67418]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.445+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f175010efd0 con 0x7f1750101e60 2024-02-22T15:46:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.446+0000 7f17377fe700 1 -- 172.21.15.19:0/3163900667 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11032 (secure 0 0 0) 0x7f1744056780 con 0x7f1750101e60 2024-02-22T15:46:20.429 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:20.429 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:46:00.310497+0000","last_up_change":"2024-02-22T15:45:59.303021+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6803","nonce":3426164274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6805","nonce":3426164274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6809","nonce":3426164274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6807","nonce":3426164274}]},"public_addr":"172.21.15.19:6803/3426164274","cluster_addr":"172.21.15.19:6805/3426164274","heartbeat_back_addr":"172.21.15.19:6809/3426164274","heartbeat_front_addr":"172.21.15.19:6807/3426164274","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6811","nonce":1226754076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6813","nonce":1226754076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6817","nonce":1226754076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6815","nonce":1226754076}]},"public_addr":"172.21.15.19:6811/1226754076","cluster_addr":"172.21.15.19:6813/1226754076","heartbeat_back_addr":"172.21.15.19:6817/1226754076","heartbeat_front_addr":"172.21.15.19:6815/1226754076","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6819","nonce":2801184587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6821","nonce":2801184587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6825","nonce":2801184587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6823","nonce":2801184587}]},"public_addr":"172.21.15.19:6819/2801184587","cluster_addr":"172.21.15.19:6821/2801184587","heartbeat_back_addr":"172.21.15.19:6825/2801184587","heartbeat_front_addr":"172.21.15.19:6823/2801184587","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6801","nonce":179328076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6803","nonce":179328076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6807","nonce":179328076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6805","nonce":179328076}]},"public_addr":"172.21.15.62:6801/179328076","cluster_addr":"172.21.15.62:6803/179328076","heartbeat_back_addr":"172.21.15.62:6807/179328076","heartbeat_front_addr":"172.21.15.62:6805/179328076","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6809","nonce":925300074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6811","nonce":925300074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6815","nonce":925300074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6813","nonce":925300074}]},"public_addr":"172.21.15.62:6809/925300074","cluster_addr":"172.21.15.62:6811/925300074","heartbeat_back_addr":"172.21.15.62:6815/925300074","heartbeat_front_addr":"172.21.15.62:6813/925300074","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6817","nonce":3437650181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6819","nonce":3437650181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6823","nonce":3437650181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6821","nonce":3437650181}]},"public_addr":"172.21.15.62:6817/3437650181","cluster_addr":"172.21.15.62:6819/3437650181","heartbeat_back_addr":"172.21.15.62:6823/3437650181","heartbeat_front_addr":"172.21.15.62:6821/3437650181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T15:46:20.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f17380692a0 msgr2=0x7f173806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:20.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f17380692a0 0x7f173806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1740004890 tx=0x7f1740009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:20.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 msgr2=0x7f17501139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:20.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 0x7f17501139f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1744002a50 tx=0x7f17440046b0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 shutdown_connections 2024-02-22T15:46:20.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f17380692a0 0x7f173806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1750101e60 0x7f17501139f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 --2- 172.21.15.19:0/3163900667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1750102800 0x7f1750113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 >> 172.21.15.19:0/3163900667 conn(0x7f17500f7a10 msgr2=0x7f1750100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 shutdown_connections 2024-02-22T15:46:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:20.448+0000 7f17558d5700 1 -- 172.21.15.19:0/3163900667 wait complete. 2024-02-22T15:46:20.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:19 smithi062 ceph-mon[48130]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:20.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:20 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3163900667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:46:20.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:21.141 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-22T15:46:21.240 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-22T15:46:21.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-22T15:46:21.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-22T15:46:21.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-22T15:46:21.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-22T15:46:21.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:20 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3163900667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:46:21.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:22.200 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:22 smithi019 ceph-mon[67418]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:22.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:22 smithi062 ceph-mon[48130]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:23.990 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:24.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 -- 172.21.15.19:0/1344630343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b34101e40 msgr2=0x7f3b34102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 --2- 172.21.15.19:0/1344630343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b34101e40 0x7f3b34102210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3b24004940 tx=0x7f3b24009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 -- 172.21.15.19:0/1344630343 shutdown_connections 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 --2- 172.21.15.19:0/1344630343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b341027e0 0x7f3b340fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 --2- 172.21.15.19:0/1344630343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b34101e40 0x7f3b34102210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 -- 172.21.15.19:0/1344630343 >> 172.21.15.19:0/1344630343 conn(0x7f3b34071d90 msgr2=0x7f3b34072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 -- 172.21.15.19:0/1344630343 shutdown_connections 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.115+0000 7f3b39929700 1 -- 172.21.15.19:0/1344630343 wait complete. 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b39929700 1 Processor -- start 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b39929700 1 -- start start 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b39929700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b34101e40 0x7f3b34115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b39929700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 0x7f3b34116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b39929700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341167a0 con 0x7f3b341027e0 2024-02-22T15:46:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b39929700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3410fad0 con 0x7f3b34101e40 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 0x7f3b34116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b327fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 0x7f3b34116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58558/0 (socket says 172.21.15.19:58558) 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b327fc700 1 -- 172.21.15.19:0/1831836107 learned_addr learned my addr 172.21.15.19:0/1831836107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.116+0000 7f3b32ffd700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b34101e40 0x7f3b34115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b327fc700 1 -- 172.21.15.19:0/1831836107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b34101e40 msgr2=0x7f3b34115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b327fc700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b34101e40 0x7f3b34115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b327fc700 1 -- 172.21.15.19:0/1831836107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24005040 con 0x7f3b341027e0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b32ffd700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b34101e40 0x7f3b34115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b327fc700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 0x7f3b34116150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c800 tx=0x7f3b1c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b1c00a760 con 0x7f3b341027e0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3410fdb0 con 0x7f3b341027e0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b1c00ad70 con 0x7f3b341027e0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b1c0113a0 con 0x7f3b341027e0 2024-02-22T15:46:24.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.117+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b341102f0 con 0x7f3b341027e0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.118+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b341107b0 con 0x7f3b341027e0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.118+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3b1c012400 con 0x7f3b341027e0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b2b7fe700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b200692a0 0x7f3b2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3b1c086f20 con 0x7f3b341027e0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b32ffd700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b200692a0 0x7f3b2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b2b7fe700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] conn(0x7f3b2006ef70 0x7f3b20071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 --> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b20071a30 con 0x7f3b2006ef70 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b1c08c4f0 con 0x7f3b341027e0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b337fe700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] conn(0x7f3b2006ef70 0x7f3b20071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b32ffd700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b200692a0 0x7f3b2006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b24004f90 tx=0x7f3b24009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.119+0000 7f3b337fe700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] conn(0x7f3b2006ef70 0x7f3b20071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:24.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.120+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== osd.1 v2:172.21.15.19:6810/1226754076 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3b20071a30 con 0x7f3b2006ef70 2024-02-22T15:46:24.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 --> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3b3405e610 con 0x7f3b2006ef70 2024-02-22T15:46:24.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b2b7fe700 1 -- 172.21.15.19:0/1831836107 <== osd.1 v2:172.21.15.19:6810/1226754076 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3b3405e610 con 0x7f3b2006ef70 2024-02-22T15:46:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] conn(0x7f3b2006ef70 msgr2=0x7f3b20071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] conn(0x7f3b2006ef70 0x7f3b20071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b200692a0 msgr2=0x7f3b2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b200692a0 0x7f3b2006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b24004f90 tx=0x7f3b24009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 msgr2=0x7f3b34116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 0x7f3b34116150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c800 tx=0x7f3b1c004470 comp rx=0 tx=0).stop 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 shutdown_connections 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3b200692a0 0x7f3b2006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b34101e40 0x7f3b34115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:6810/1226754076,v1:172.21.15.19:6811/1226754076] conn(0x7f3b2006ef70 0x7f3b20071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.136+0000 7f3b39929700 1 --2- 172.21.15.19:0/1831836107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b341027e0 0x7f3b34116150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.137+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 >> 172.21.15.19:0/1831836107 conn(0x7f3b34071d90 msgr2=0x7f3b34109ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.137+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 shutdown_connections 2024-02-22T15:46:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:24.137+0000 7f3b39929700 1 -- 172.21.15.19:0/1831836107 wait complete. 2024-02-22T15:46:24.148 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:24 smithi019 ceph-mon[67418]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:24.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:24 smithi062 ceph-mon[48130]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:25.821 INFO:teuthology.orchestra.run.smithi019.stdout:60129542174 2024-02-22T15:46:25.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-22T15:46:26.054 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:26 smithi019 ceph-mon[67418]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:26.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:26 smithi062 ceph-mon[48130]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:28.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:28 smithi019 ceph-mon[67418]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:28.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:28 smithi062 ceph-mon[48130]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:28.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 -- 172.21.15.19:0/3054204268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc101e40 msgr2=0x7f78fc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:28.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 --2- 172.21.15.19:0/3054204268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc101e40 0x7f78fc102210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78ec004940 tx=0x7f78ec009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:28.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 -- 172.21.15.19:0/3054204268 shutdown_connections 2024-02-22T15:46:28.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 --2- 172.21.15.19:0/3054204268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78fc1027e0 0x7f78fc0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 --2- 172.21.15.19:0/3054204268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc101e40 0x7f78fc102210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 -- 172.21.15.19:0/3054204268 >> 172.21.15.19:0/3054204268 conn(0x7f78fc071d90 msgr2=0x7f78fc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 -- 172.21.15.19:0/3054204268 shutdown_connections 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 -- 172.21.15.19:0/3054204268 wait complete. 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.642+0000 7f7902509700 1 Processor -- start 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f7902509700 1 -- start start 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f7902509700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78fc101e40 0x7f78fc10c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f7902509700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 0x7f78fc10cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f7902509700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc10d0d0 con 0x7f78fc1027e0 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f7902509700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc10d240 con 0x7f78fc101e40 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 0x7f78fc10cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 0x7f78fc10cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56722/0 (socket says 172.21.15.19:56722) 2024-02-22T15:46:28.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 -- 172.21.15.19:0/1624603161 learned_addr learned my addr 172.21.15.19:0/1624603161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fbfff700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78fc101e40 0x7f78fc10c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 -- 172.21.15.19:0/1624603161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78fc101e40 msgr2=0x7f78fc10c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78fc101e40 0x7f78fc10c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 -- 172.21.15.19:0/1624603161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec005040 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.643+0000 7f78fb7fe700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 0x7f78fc10cb90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c800 tx=0x7f78e4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.644+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e40106f0 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.644+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f78e4010d00 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.644+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc119cc0 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.644+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e40113a0 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.644+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc11a0c0 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.645+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78fc05e690 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.645+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f78e4012400 con 0x7f78fc1027e0 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.645+0000 7f78f8ff9700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f78e80692a0 0x7f78e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:28.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f78e4086dd0 con 0x7f78fc1027e0 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f78fbfff700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f78e80692a0 0x7f78e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f78f8ff9700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] conn(0x7f78e806ef70 0x7f78e8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 --> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78e8071a30 con 0x7f78e806ef70 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78e408c430 con 0x7f78fc1027e0 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f7900aa6700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] conn(0x7f78e806ef70 0x7f78e8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f78fbfff700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f78e80692a0 0x7f78e806b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78ec004f90 tx=0x7f78ec009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.646+0000 7f7900aa6700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] conn(0x7f78e806ef70 0x7f78e8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.647+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== osd.2 v2:172.21.15.19:6818/2801184587 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f78e8071a30 con 0x7f78e806ef70 2024-02-22T15:46:28.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 --> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78fc05eac0 con 0x7f78e806ef70 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f78f8ff9700 1 -- 172.21.15.19:0/1624603161 <== osd.2 v2:172.21.15.19:6818/2801184587 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f78fc05eac0 con 0x7f78e806ef70 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] conn(0x7f78e806ef70 msgr2=0x7f78e8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] conn(0x7f78e806ef70 0x7f78e8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f78e80692a0 msgr2=0x7f78e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f78e80692a0 0x7f78e806b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78ec004f90 tx=0x7f78ec009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 msgr2=0x7f78fc10cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 0x7f78fc10cb90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c800 tx=0x7f78e4004470 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 shutdown_connections 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f78e80692a0 0x7f78e806b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78fc101e40 0x7f78fc10c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:6818/2801184587,v1:172.21.15.19:6819/2801184587] conn(0x7f78e806ef70 0x7f78e8071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 --2- 172.21.15.19:0/1624603161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78fc1027e0 0x7f78fc10cb90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:28.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.662+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 >> 172.21.15.19:0/1624603161 conn(0x7f78fc071d90 msgr2=0x7f78fc0fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.663+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 shutdown_connections 2024-02-22T15:46:28.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.663+0000 7f7902509700 1 -- 172.21.15.19:0/1624603161 wait complete. 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 -- 172.21.15.19:0/488426645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10901027e0 msgr2=0x7f10900fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 --2- 172.21.15.19:0/488426645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10901027e0 0x7f10900fbea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1080004940 tx=0x7f1080009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 -- 172.21.15.19:0/488426645 shutdown_connections 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 --2- 172.21.15.19:0/488426645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10901027e0 0x7f10900fbea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 --2- 172.21.15.19:0/488426645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1090101e40 0x7f1090102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 -- 172.21.15.19:0/488426645 >> 172.21.15.19:0/488426645 conn(0x7f1090071d90 msgr2=0x7f1090072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 -- 172.21.15.19:0/488426645 shutdown_connections 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 -- 172.21.15.19:0/488426645 wait complete. 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 Processor -- start 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.160+0000 7f1097dfe700 1 -- start start 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1097dfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 0x7f109010ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1097dfe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10901027e0 0x7f109010ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1097dfe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109010f520 con 0x7f1090101e40 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1097dfe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109010f690 con 0x7f10901027e0 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 0x7f109010ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 0x7f109010ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58572/0 (socket says 172.21.15.19:58572) 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 -- 172.21.15.19:0/3668457833 learned_addr learned my addr 172.21.15.19:0/3668457833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095399700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10901027e0 0x7f109010ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 -- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10901027e0 msgr2=0x7f109010ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10901027e0 0x7f109010ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 -- 172.21.15.19:0/3668457833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080005040 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f1095b9a700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 0x7f109010ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f108c002a50 tx=0x7f108c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.161+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c007cc0 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.162+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f108c00f040 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.162+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1090117a20 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.162+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c00d9c0 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.162+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1090117ee0 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.162+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f109005e690 con 0x7f1090101e40 2024-02-22T15:46:29.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.163+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f108c00db20 con 0x7f1090101e40 2024-02-22T15:46:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.163+0000 7f10867fc700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f107c0692a0 0x7f107c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.163+0000 7f1095399700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f107c0692a0 0x7f107c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.163+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f108c0875e0 con 0x7f1090101e40 2024-02-22T15:46:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.163+0000 7f10867fc700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] conn(0x7f107c06ef70 0x7f107c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.164+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 --> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f107c071a30 con 0x7f107c06ef70 2024-02-22T15:46:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.164+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c087920 con 0x7f1090101e40 2024-02-22T15:46:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.164+0000 7f1095399700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f107c0692a0 0x7f107c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1080004890 tx=0x7f1080009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.164+0000 7f109639b700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] conn(0x7f107c06ef70 0x7f107c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.164+0000 7f109639b700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] conn(0x7f107c06ef70 0x7f107c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.165+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== osd.3 v2:172.21.15.62:6800/179328076 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f107c071a30 con 0x7f107c06ef70 2024-02-22T15:46:29.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.182+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 --> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f109005eb40 con 0x7f107c06ef70 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f10867fc700 1 -- 172.21.15.19:0/3668457833 <== osd.3 v2:172.21.15.62:6800/179328076 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f109005eb40 con 0x7f107c06ef70 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] conn(0x7f107c06ef70 msgr2=0x7f107c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] conn(0x7f107c06ef70 0x7f107c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f107c0692a0 msgr2=0x7f107c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f107c0692a0 0x7f107c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1080004890 tx=0x7f1080009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 msgr2=0x7f109010ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 0x7f109010ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f108c002a50 tx=0x7f108c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 shutdown_connections 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:6800/179328076,v1:172.21.15.62:6801/179328076] conn(0x7f107c06ef70 0x7f107c071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f107c0692a0 0x7f107c06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1090101e40 0x7f109010ea50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 --2- 172.21.15.19:0/3668457833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10901027e0 0x7f109010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.183+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 >> 172.21.15.19:0/3668457833 conn(0x7f1090071d90 msgr2=0x7f10900faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.184+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 shutdown_connections 2024-02-22T15:46:29.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:27.184+0000 7f1097dfe700 1 -- 172.21.15.19:0/3668457833 wait complete. 2024-02-22T15:46:29.762 INFO:teuthology.orchestra.run.smithi019.stdout:115964117010 2024-02-22T15:46:29.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-22T15:46:30.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:30 smithi019 ceph-mon[67418]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:30.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:30 smithi062 ceph-mon[48130]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:32.356 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:32 smithi019 ceph-mon[67418]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:32.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:32 smithi062 ceph-mon[48130]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:32.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.919+0000 7f1044983700 1 -- 172.21.15.19:0/2394324204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10401038f0 msgr2=0x7f10400fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:32.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.919+0000 7f1044983700 1 --2- 172.21.15.19:0/2394324204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10401038f0 0x7f10400fcf60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1030004940 tx=0x7f1030009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:32.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.919+0000 7f1044983700 1 -- 172.21.15.19:0/2394324204 shutdown_connections 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.919+0000 7f1044983700 1 --2- 172.21.15.19:0/2394324204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10401038f0 0x7f10400fcf60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.919+0000 7f1044983700 1 --2- 172.21.15.19:0/2394324204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1040102f50 0x7f1040103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.919+0000 7f1044983700 1 -- 172.21.15.19:0/2394324204 >> 172.21.15.19:0/2394324204 conn(0x7f10400f8ae0 msgr2=0x7f10400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 -- 172.21.15.19:0/2394324204 shutdown_connections 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 -- 172.21.15.19:0/2394324204 wait complete. 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 Processor -- start 2024-02-22T15:46:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 -- start start 2024-02-22T15:46:32.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 0x7f1040070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10401038f0 0x7f10400707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040070e40 con 0x7f1040102f50 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.920+0000 7f1044983700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040070fb0 con 0x7f10401038f0 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 0x7f1040070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 0x7f1040070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56750/0 (socket says 172.21.15.19:56750) 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 -- 172.21.15.19:0/577043098 learned_addr learned my addr 172.21.15.19:0/577043098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103dd9b700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10401038f0 0x7f10400707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 -- 172.21.15.19:0/577043098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10401038f0 msgr2=0x7f10400707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10401038f0 0x7f10400707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 -- 172.21.15.19:0/577043098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030005040 con 0x7f1040102f50 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f103e59c700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 0x7f1040070290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f102800c830 tx=0x7f1028004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102800a760 con 0x7f1040102f50 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1040119be0 con 0x7f1040102f50 2024-02-22T15:46:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f102800ad70 con 0x7f1040102f50 2024-02-22T15:46:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10280113a0 con 0x7f1040102f50 2024-02-22T15:46:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.921+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104011a120 con 0x7f1040102f50 2024-02-22T15:46:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.922+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f104005e610 con 0x7f1040102f50 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1028012400 con 0x7f1040102f50 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f1036ffd700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f102c0692a0 0x7f102c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1028087950 con 0x7f1040102f50 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f103dd9b700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f102c0692a0 0x7f102c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f1036ffd700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] conn(0x7f102c06ef70 0x7f102c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 --> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f102c071a30 con 0x7f102c06ef70 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.923+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1028087cd0 con 0x7f1040102f50 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.924+0000 7f103ed9d700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] conn(0x7f102c06ef70 0x7f102c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:32.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.924+0000 7f103dd9b700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f102c0692a0 0x7f102c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1030004f90 tx=0x7f1030009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.924+0000 7f103ed9d700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] conn(0x7f102c06ef70 0x7f102c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.924+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== osd.0 v2:172.21.15.19:6802/3426164274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f102c071a30 con 0x7f102c06ef70 2024-02-22T15:46:32.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 --> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f104005ea90 con 0x7f102c06ef70 2024-02-22T15:46:32.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1036ffd700 1 -- 172.21.15.19:0/577043098 <== osd.0 v2:172.21.15.19:6802/3426164274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f104005ea90 con 0x7f102c06ef70 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] conn(0x7f102c06ef70 msgr2=0x7f102c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] conn(0x7f102c06ef70 0x7f102c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f102c0692a0 msgr2=0x7f102c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f102c0692a0 0x7f102c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1030004f90 tx=0x7f1030009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 msgr2=0x7f1040070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 0x7f1040070290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f102800c830 tx=0x7f1028004470 comp rx=0 tx=0).stop 2024-02-22T15:46:32.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 shutdown_connections 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f102c0692a0 0x7f102c06b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1040102f50 0x7f1040070290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.19:6802/3426164274,v1:172.21.15.19:6803/3426164274] conn(0x7f102c06ef70 0x7f102c071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 --2- 172.21.15.19:0/577043098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10401038f0 0x7f10400707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 >> 172.21.15.19:0/577043098 conn(0x7f10400f8ae0 msgr2=0x7f10400fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 shutdown_connections 2024-02-22T15:46:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:32.940+0000 7f1044983700 1 -- 172.21.15.19:0/577043098 wait complete. 2024-02-22T15:46:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:33.218 INFO:teuthology.orchestra.run.smithi019.stdout:81604378648 2024-02-22T15:46:33.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-22T15:46:34.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 -- 172.21.15.19:0/49818141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fc6e0 msgr2=0x7fe174109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 --2- 172.21.15.19:0/49818141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fc6e0 0x7fe174109440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe15c004940 tx=0x7fe15c009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 -- 172.21.15.19:0/49818141 shutdown_connections 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 --2- 172.21.15.19:0/49818141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fc6e0 0x7fe174109440 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 --2- 172.21.15.19:0/49818141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1740fbdd0 0x7fe1740fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 -- 172.21.15.19:0/49818141 >> 172.21.15.19:0/49818141 conn(0x7fe174071d90 msgr2=0x7fe174072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 -- 172.21.15.19:0/49818141 shutdown_connections 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 -- 172.21.15.19:0/49818141 wait complete. 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.943+0000 7fe17a07e700 1 Processor -- start 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe17a07e700 1 -- start start 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe17a07e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 0x7fe174115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe17a07e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1740fc6e0 0x7fe1741160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:34.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe17a07e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1741167d0 con 0x7fe1740fbdd0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe17a07e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17410fa70 con 0x7fe1740fc6e0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 0x7fe174115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 0x7fe174115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56768/0 (socket says 172.21.15.19:56768) 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 -- 172.21.15.19:0/3706250272 learned_addr learned my addr 172.21.15.19:0/3706250272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe172ffd700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1740fc6e0 0x7fe1741160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 -- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1740fc6e0 msgr2=0x7fe1741160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1740fc6e0 0x7fe1741160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 -- 172.21.15.19:0/3706250272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c005040 con 0x7fe1740fbdd0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe1737fe700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 0x7fe174115bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe16400b790 tx=0x7fe164009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe16400f6f0 con 0x7fe1740fbdd0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.944+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe16400fd00 con 0x7fe1740fbdd0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.945+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17410fd50 con 0x7fe1740fbdd0 2024-02-22T15:46:34.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.945+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1640103a0 con 0x7fe1740fbdd0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.945+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe174110290 con 0x7fe1740fbdd0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.945+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe174110810 con 0x7fe1740fbdd0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe16400f850 con 0x7fe1740fbdd0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe16bfff700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe1600692a0 0x7fe16006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe164086c70 con 0x7fe1740fbdd0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe172ffd700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe1600692a0 0x7fe16006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe16bfff700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] conn(0x7fe16006ef70 0x7fe160071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 --> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe160071a30 con 0x7fe16006ef70 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.946+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe16408c070 con 0x7fe1740fbdd0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.947+0000 7fe172ffd700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe1600692a0 0x7fe16006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1741113e0 tx=0x7fe15c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.947+0000 7fe173fff700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] conn(0x7fe16006ef70 0x7fe160071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.947+0000 7fe173fff700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] conn(0x7fe16006ef70 0x7fe160071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:34.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.948+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== osd.4 v2:172.21.15.62:6808/925300074 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe160071a30 con 0x7fe16006ef70 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.964+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 --> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1740738b0 con 0x7fe16006ef70 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.964+0000 7fe16bfff700 1 -- 172.21.15.19:0/3706250272 <== osd.4 v2:172.21.15.62:6808/925300074 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe1740738b0 con 0x7fe16006ef70 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.964+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] conn(0x7fe16006ef70 msgr2=0x7fe160071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.964+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] conn(0x7fe16006ef70 0x7fe160071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe1600692a0 msgr2=0x7fe16006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe1600692a0 0x7fe16006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1741113e0 tx=0x7fe15c009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 msgr2=0x7fe174115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 0x7fe174115bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe16400b790 tx=0x7fe164009250 comp rx=0 tx=0).stop 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 shutdown_connections 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:6808/925300074,v1:172.21.15.62:6809/925300074] conn(0x7fe16006ef70 0x7fe160071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe1600692a0 0x7fe16006b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1740fbdd0 0x7fe174115bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 --2- 172.21.15.19:0/3706250272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1740fc6e0 0x7fe1741160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 >> 172.21.15.19:0/3706250272 conn(0x7fe174071d90 msgr2=0x7fe1740fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 shutdown_connections 2024-02-22T15:46:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:33.965+0000 7fe17a07e700 1 -- 172.21.15.19:0/3706250272 wait complete. 2024-02-22T15:46:34.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:34 smithi019 ceph-mon[67418]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:34.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:34 smithi062 ceph-mon[48130]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:34.774 INFO:teuthology.orchestra.run.smithi019.stdout:38654705701 2024-02-22T15:46:34.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-22T15:46:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 -- 172.21.15.19:0/778071027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc1027e0 msgr2=0x7ff3fc0fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 --2- 172.21.15.19:0/778071027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc1027e0 0x7ff3fc0fbe50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0004940 tx=0x7ff3f0009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 -- 172.21.15.19:0/778071027 shutdown_connections 2024-02-22T15:46:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 --2- 172.21.15.19:0/778071027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc1027e0 0x7ff3fc0fbe50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 --2- 172.21.15.19:0/778071027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3fc101e40 0x7ff3fc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 -- 172.21.15.19:0/778071027 >> 172.21.15.19:0/778071027 conn(0x7ff3fc0f79d0 msgr2=0x7ff3fc0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 -- 172.21.15.19:0/778071027 shutdown_connections 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.524+0000 7ff401d50700 1 -- 172.21.15.19:0/778071027 wait complete. 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff401d50700 1 Processor -- start 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff401d50700 1 -- start start 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff401d50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 0x7ff3fc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff401d50700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3fc1027e0 0x7ff3fc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff401d50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc114530 con 0x7ff3fc101e40 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff401d50700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc10d860 con 0x7ff3fc1027e0 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff3fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 0x7ff3fc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff3fb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 0x7ff3fc1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56786/0 (socket says 172.21.15.19:56786) 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.525+0000 7ff3fb7fe700 1 -- 172.21.15.19:0/1006055054 learned_addr learned my addr 172.21.15.19:0/1006055054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3faffd700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3fc1027e0 0x7ff3fc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3fb7fe700 1 -- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3fc1027e0 msgr2=0x7ff3fc113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3fb7fe700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3fc1027e0 0x7ff3fc113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3fb7fe700 1 -- 172.21.15.19:0/1006055054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0005040 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3fb7fe700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 0x7ff3fc1139a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec002a50 tx=0x7ff3ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3ec007cc0 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc10db40 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3ec00f040 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3ec00d9c0 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.526+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc10e050 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.527+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3fc10e530 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.527+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff3ec00db20 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3e3fff700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff3e40692b0 0x7ff3e406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff3ec086b70 con 0x7ff3fc101e40 2024-02-22T15:46:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3faffd700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff3e40692b0 0x7ff3e406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3e3fff700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] conn(0x7ff3e406ef80 0x7ff3e4071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 --> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3e4071a40 con 0x7ff3e406ef80 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3ec08b220 con 0x7ff3fc101e40 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3faffd700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff3e40692b0 0x7ff3e406b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc10eb90 tx=0x7ff3f0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.528+0000 7ff3fbfff700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] conn(0x7ff3e406ef80 0x7ff3e4071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.529+0000 7ff3fbfff700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] conn(0x7ff3e406ef80 0x7ff3e4071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.529+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== osd.5 v2:172.21.15.62:6816/3437650181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff3e4071a40 con 0x7ff3e406ef80 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.544+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 --> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3fc10e6a0 con 0x7ff3e406ef80 2024-02-22T15:46:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.545+0000 7ff3e3fff700 1 -- 172.21.15.19:0/1006055054 <== osd.5 v2:172.21.15.62:6816/3437650181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff3fc10e6a0 con 0x7ff3e406ef80 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] conn(0x7ff3e406ef80 msgr2=0x7ff3e4071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] conn(0x7ff3e406ef80 0x7ff3e4071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff3e40692b0 msgr2=0x7ff3e406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff3e40692b0 0x7ff3e406b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc10eb90 tx=0x7ff3f0009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 msgr2=0x7ff3fc1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 0x7ff3fc1139a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec002a50 tx=0x7ff3ec0046b0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 shutdown_connections 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:6816/3437650181,v1:172.21.15.62:6817/3437650181] conn(0x7ff3e406ef80 0x7ff3e4071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff3e40692b0 0x7ff3e406b760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3fc101e40 0x7ff3fc1139a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 --2- 172.21.15.19:0/1006055054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3fc1027e0 0x7ff3fc113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 >> 172.21.15.19:0/1006055054 conn(0x7ff3fc0f79d0 msgr2=0x7ff3fc066050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 shutdown_connections 2024-02-22T15:46:35.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:35.546+0000 7ff401d50700 1 -- 172.21.15.19:0/1006055054 wait complete. 2024-02-22T15:46:36.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:36 smithi019 ceph-mon[67418]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:36.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:36.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:36 smithi062 ceph-mon[48130]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:36.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:37.027 INFO:teuthology.orchestra.run.smithi019.stdout:150323855375 2024-02-22T15:46:37.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-22T15:46:37.455 INFO:teuthology.orchestra.run.smithi019.stdout:171798691851 2024-02-22T15:46:37.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 -- 172.21.15.19:0/675630794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 msgr2=0x7f33341022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 --2- 172.21.15.19:0/675630794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f33341022b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3330004940 tx=0x7f3330009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 -- 172.21.15.19:0/675630794 shutdown_connections 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 --2- 172.21.15.19:0/675630794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3334102880 0x7f33340fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 --2- 172.21.15.19:0/675630794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f33341022b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 -- 172.21.15.19:0/675630794 >> 172.21.15.19:0/675630794 conn(0x7f3334071560 msgr2=0x7f3334071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 -- 172.21.15.19:0/675630794 shutdown_connections 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.172+0000 7f333bff9700 1 -- 172.21.15.19:0/675630794 wait complete. 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f333bff9700 1 Processor -- start 2024-02-22T15:46:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f333bff9700 1 -- start start 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f333bff9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f333410ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f333bff9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3334102880 0x7f333410ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f333bff9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333410f500 con 0x7f3334101ee0 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f333bff9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333410f640 con 0x7f3334102880 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f3339d95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f333410ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f3339d95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f333410ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58594/0 (socket says 172.21.15.19:58594) 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.173+0000 7f3339d95700 1 -- 172.21.15.19:0/588508191 learned_addr learned my addr 172.21.15.19:0/588508191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f3339d95700 1 -- 172.21.15.19:0/588508191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3334102880 msgr2=0x7f333410ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f3339594700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3334102880 0x7f333410ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f3339d95700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3334102880 0x7f333410ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f3339d95700 1 -- 172.21.15.19:0/588508191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330005040 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f3339d95700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f333410ea30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f333000b3b0 tx=0x7f3330009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333000e040 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3334117940 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f333000bc50 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3330013420 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.174+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3334117e50 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.175+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33340ff1f0 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.175+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f333001c430 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.176+0000 7f33267fc700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f33200692a0 0x7f332006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.176+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3330087c50 con 0x7f3334101ee0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.176+0000 7f3339594700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f33200692a0 0x7f332006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.176+0000 7f3339594700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f33200692a0 0x7f332006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33340fc030 tx=0x7f3328009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:38.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.180+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3330054370 con 0x7f3334101ee0 2024-02-22T15:46:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:38 smithi019 ceph-mon[67418]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:38.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.374+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f33340fbd70 con 0x7f3334101ee0 2024-02-22T15:46:38.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.374+0000 7f33267fc700 1 -- 172.21.15.19:0/588508191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3330057990 con 0x7f3334101ee0 2024-02-22T15:46:38.357 INFO:teuthology.orchestra.run.smithi019.stdout:60129542176 2024-02-22T15:46:38.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f33200692a0 msgr2=0x7f332006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:38.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f33200692a0 0x7f332006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33340fc030 tx=0x7f3328009040 comp rx=0 tx=0).stop 2024-02-22T15:46:38.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 msgr2=0x7f333410ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f333410ea30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f333000b3b0 tx=0x7f3330009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 shutdown_connections 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f33200692a0 0x7f332006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3334101ee0 0x7f333410ea30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 --2- 172.21.15.19:0/588508191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3334102880 0x7f333410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 >> 172.21.15.19:0/588508191 conn(0x7f3334071560 msgr2=0x7f33340fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 shutdown_connections 2024-02-22T15:46:38.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:38.376+0000 7f333bff9700 1 -- 172.21.15.19:0/588508191 wait complete. 2024-02-22T15:46:38.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:38 smithi062 ceph-mon[48130]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:38.605 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:39.443 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:39 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/588508191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T15:46:39.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:39 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/588508191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-22T15:46:39.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-02-22T15:46:39.774 DEBUG:teuthology.parallel:result is None 2024-02-22T15:46:40.293 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:40 smithi019 ceph-mon[67418]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:40.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:40 smithi062 ceph-mon[48130]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:40.782 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/2533764570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fe8f0 msgr2=0x7fa7e4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/2533764570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fe8f0 0x7fa7e4107050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8004970 tx=0x7fa7d8009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/2533764570 shutdown_connections 2024-02-22T15:46:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/2533764570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fe8f0 0x7fa7e4107050 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/2533764570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e40fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/2533764570 >> 172.21.15.19:0/2533764570 conn(0x7fa7e4071d90 msgr2=0x7fa7e4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/2533764570 shutdown_connections 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.595+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/2533764570 wait complete. 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7ea6f0700 1 Processor -- start 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7ea6f0700 1 -- start start 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7ea6f0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7ea6f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e40fe8f0 0x7fa7e4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7ea6f0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4114500 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7ea6f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e410d830 con 0x7fa7e40fe8f0 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e4113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58618/0 (socket says 172.21.15.19:58618) 2024-02-22T15:46:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.596+0000 7fa7e3fff700 1 -- 172.21.15.19:0/30910580 learned_addr learned my addr 172.21.15.19:0/30910580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e37fe700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e40fe8f0 0x7fa7e4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e3fff700 1 -- 172.21.15.19:0/30910580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e40fe8f0 msgr2=0x7fa7e4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e3fff700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e40fe8f0 0x7fa7e4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e3fff700 1 -- 172.21.15.19:0/30910580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8005040 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e3fff700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e4113970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4002a50 tx=0x7fa7d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d4007cc0 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7d400f040 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d400d9c0 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e410db10 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.597+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e410e020 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.598+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e41046c0 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.598+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa7d400db20 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.599+0000 7fa7e0ff9700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa7cc0692a0 0x7fa7cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.599+0000 7fa7e37fe700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa7cc0692a0 0x7fa7cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.599+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa7d4086aa0 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.599+0000 7fa7e37fe700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa7cc0692a0 0x7fa7cc06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8004f50 tx=0x7fa7d8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.604+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa7d4056ba0 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.801+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa7e404bed0 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.801+0000 7fa7e0ff9700 1 -- 172.21.15.19:0/30910580 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7d4056730 con 0x7fa7e40fdfe0 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stdout:115964117013 2024-02-22T15:46:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.804+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa7cc0692a0 msgr2=0x7fa7cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.804+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa7cc0692a0 0x7fa7cc06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8004f50 tx=0x7fa7d8009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.804+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 msgr2=0x7fa7e4113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.804+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e4113970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4002a50 tx=0x7fa7d40046b0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 shutdown_connections 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa7cc0692a0 0x7fa7cc06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7e40fdfe0 0x7fa7e4113970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 --2- 172.21.15.19:0/30910580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7e40fe8f0 0x7fa7e4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 >> 172.21.15.19:0/30910580 conn(0x7fa7e4071d90 msgr2=0x7fa7e4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 shutdown_connections 2024-02-22T15:46:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:40.805+0000 7fa7ea6f0700 1 -- 172.21.15.19:0/30910580 wait complete. 2024-02-22T15:46:41.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:41 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/30910580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T15:46:41.424 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:41.424 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:41.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:41 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/30910580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-22T15:46:42.016 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:42 smithi019 ceph-mon[67418]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:42.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:42 smithi062 ceph-mon[48130]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:44.123 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:44 smithi019 ceph-mon[67418]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:44.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:44 smithi062 ceph-mon[48130]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 -- 172.21.15.19:0/2558547328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f81027e0 msgr2=0x7f86f80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 --2- 172.21.15.19:0/2558547328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f81027e0 0x7f86f80fbea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f86e8004940 tx=0x7f86e8009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 -- 172.21.15.19:0/2558547328 shutdown_connections 2024-02-22T15:46:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 --2- 172.21.15.19:0/2558547328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f81027e0 0x7f86f80fbea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 --2- 172.21.15.19:0/2558547328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f8101e40 0x7f86f8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 -- 172.21.15.19:0/2558547328 >> 172.21.15.19:0/2558547328 conn(0x7f86f8071d90 msgr2=0x7f86f8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 -- 172.21.15.19:0/2558547328 shutdown_connections 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.747+0000 7f87005e4700 1 -- 172.21.15.19:0/2558547328 wait complete. 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f87005e4700 1 Processor -- start 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f87005e4700 1 -- start start 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f87005e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 0x7f86f8113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f87005e4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f81027e0 0x7f86f8113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f87005e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81144f0 con 0x7f86f8101e40 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f87005e4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f810d820 con 0x7f86f81027e0 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fe380700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 0x7f86f8113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fe380700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 0x7f86f8113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58630/0 (socket says 172.21.15.19:58630) 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fe380700 1 -- 172.21.15.19:0/1579127694 learned_addr learned my addr 172.21.15.19:0/1579127694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fdb7f700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f81027e0 0x7f86f8113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fe380700 1 -- 172.21.15.19:0/1579127694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f81027e0 msgr2=0x7f86f8113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fe380700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f81027e0 0x7f86f8113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.748+0000 7f86fe380700 1 -- 172.21.15.19:0/1579127694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8005040 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.749+0000 7f86fe380700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 0x7f86f8113960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86f400eb70 tx=0x7f86f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.749+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f4007cc0 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.749+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86f400f040 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.749+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f810daa0 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.749+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f400c9f0 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.749+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f810dfb0 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.750+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f80738b0 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.750+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f86f400cb50 con 0x7f86f8101e40 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.751+0000 7f86eeffd700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86e40692a0 0x7f86e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.751+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f86f40868d0 con 0x7f86f8101e40 2024-02-22T15:46:45.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.751+0000 7f86fdb7f700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86e40692a0 0x7f86e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:45.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.751+0000 7f86fdb7f700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86e40692a0 0x7f86e406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000c00 tx=0x7f86e8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:45.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.755+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f86f40569d0 con 0x7f86f8101e40 2024-02-22T15:46:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.825+0000 7fc6956ca700 1 -- 172.21.15.19:0/451136716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 msgr2=0x7fc690103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.825+0000 7fc6956ca700 1 --2- 172.21.15.19:0/451136716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc690103320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc680004940 tx=0x7fc680009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 -- 172.21.15.19:0/451136716 shutdown_connections 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 --2- 172.21.15.19:0/451136716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6901038f0 0x7fc6900fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 --2- 172.21.15.19:0/451136716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc690103320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 -- 172.21.15.19:0/451136716 >> 172.21.15.19:0/451136716 conn(0x7fc6900f8ae0 msgr2=0x7fc6900faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 -- 172.21.15.19:0/451136716 shutdown_connections 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 -- 172.21.15.19:0/451136716 wait complete. 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 Processor -- start 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 -- start start 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc6901139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.826+0000 7fc6956ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6901038f0 0x7fc690113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6956ca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc690114530 con 0x7fc690102f50 2024-02-22T15:46:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6956ca700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69010d860 con 0x7fc6901038f0 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc6901139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc6901139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58658/0 (socket says 172.21.15.19:58658) 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 -- 172.21.15.19:0/2391346034 learned_addr learned my addr 172.21.15.19:0/2391346034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68e7fc700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6901038f0 0x7fc690113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 -- 172.21.15.19:0/2391346034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6901038f0 msgr2=0x7fc690113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6901038f0 0x7fc690113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 -- 172.21.15.19:0/2391346034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684005040 con 0x7fc690102f50 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc68effd700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc6901139a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc680009fd0 tx=0x7fc680009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc680021070 con 0x7fc690102f50 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc68001c400 con 0x7fc690102f50 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc680005040 con 0x7fc690102f50 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc68000e070 con 0x7fc690102f50 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.827+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69010de30 con 0x7fc690102f50 2024-02-22T15:46:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.828+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6900fc6e0 con 0x7fc690102f50 2024-02-22T15:46:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.829+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc68001c570 con 0x7fc690102f50 2024-02-22T15:46:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.829+0000 7fc6777fe700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc6780692a0 0x7fc67806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.829+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc680088050 con 0x7fc690102f50 2024-02-22T15:46:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.829+0000 7fc68e7fc700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc6780692a0 0x7fc67806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.830+0000 7fc68e7fc700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc6780692a0 0x7fc67806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc69010f1d0 tx=0x7fc684009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:45.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.838+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc680058150 con 0x7fc690102f50 2024-02-22T15:46:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.958+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f86f810ef40 con 0x7f86f8101e40 2024-02-22T15:46:45.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.958+0000 7f86eeffd700 1 -- 172.21.15.19:0/1579127694 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86f4056560 con 0x7f86f8101e40 2024-02-22T15:46:45.942 INFO:teuthology.orchestra.run.smithi019.stdout:150323855377 2024-02-22T15:46:45.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.961+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86e40692a0 msgr2=0x7f86e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.961+0000 7f87005e4700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86e40692a0 0x7f86e406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000c00 tx=0x7f86e8009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.961+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 msgr2=0x7f86f8113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.961+0000 7f87005e4700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 0x7f86f8113960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86f400eb70 tx=0x7f86f400b040 comp rx=0 tx=0).stop 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 shutdown_connections 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86e40692a0 0x7f86e406b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f8101e40 0x7f86f8113960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 --2- 172.21.15.19:0/1579127694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f81027e0 0x7f86f8113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 >> 172.21.15.19:0/1579127694 conn(0x7f86f8071d90 msgr2=0x7f86f81000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 shutdown_connections 2024-02-22T15:46:45.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:45.962+0000 7f87005e4700 1 -- 172.21.15.19:0/1579127694 wait complete. 2024-02-22T15:46:46.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.041+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc69010ef80 con 0x7fc690102f50 2024-02-22T15:46:46.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.041+0000 7fc6777fe700 1 -- 172.21.15.19:0/2391346034 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc680057ce0 con 0x7fc690102f50 2024-02-22T15:46:46.025 INFO:teuthology.orchestra.run.smithi019.stdout:171798691852 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.043+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc6780692a0 msgr2=0x7fc67806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.043+0000 7fc6956ca700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc6780692a0 0x7fc67806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc69010f1d0 tx=0x7fc684009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.043+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 msgr2=0x7fc6901139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.043+0000 7fc6956ca700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc6901139a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc680009fd0 tx=0x7fc680009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 shutdown_connections 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fc6780692a0 0x7fc67806b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc690102f50 0x7fc6901139a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 --2- 172.21.15.19:0/2391346034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6901038f0 0x7fc690113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 >> 172.21.15.19:0/2391346034 conn(0x7fc6900f8ae0 msgr2=0x7fc6900fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 shutdown_connections 2024-02-22T15:46:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.044+0000 7fc6956ca700 1 -- 172.21.15.19:0/2391346034 wait complete. 2024-02-22T15:46:46.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:46 smithi019 ceph-mon[67418]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:46.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:46 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1579127694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T15:46:46.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:46 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2391346034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T15:46:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:46 smithi062 ceph-mon[48130]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:46.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:46 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1579127694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-22T15:46:46.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:46 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2391346034' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.660+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/1704259625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 msgr2=0x7fbb74106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.660+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/1704259625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74106f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbb70004940 tx=0x7fbb70009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.660+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/1704259625 shutdown_connections 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.660+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/1704259625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74106f30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.660+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/1704259625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb740fdf50 0x7fbb740fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.660+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/1704259625 >> 172.21.15.19:0/1704259625 conn(0x7fbb740f79e0 msgr2=0x7fbb740f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/1704259625 shutdown_connections 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/1704259625 wait complete. 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 Processor -- start 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 -- start start 2024-02-22T15:46:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb740fdf50 0x7fbb741139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74114540 con 0x7fbb740fe8f0 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.661+0000 7fbb7c2e4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7410d870 con 0x7fbb740fdf50 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58664/0 (socket says 172.21.15.19:58664) 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 -- 172.21.15.19:0/3164611353 learned_addr learned my addr 172.21.15.19:0/3164611353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7a080700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb740fdf50 0x7fbb741139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 -- 172.21.15.19:0/3164611353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb740fdf50 msgr2=0x7fbb741139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb740fdf50 0x7fbb741139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 -- 172.21.15.19:0/3164611353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70005040 con 0x7fbb740fe8f0 2024-02-22T15:46:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7987f700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74113ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b3b0 tx=0x7fbb70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:46.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7000e040 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb70009af0 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7410daf0 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7001c3f0 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.662+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7410e030 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.663+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb74104540 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.664+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbb70013800 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.664+0000 7fbb66ffd700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbb600692a0 0x7fbb6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.664+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbb70088c70 con 0x7fbb740fe8f0 2024-02-22T15:46:46.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.664+0000 7fbb7a080700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbb600692a0 0x7fbb6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:46.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.665+0000 7fbb7a080700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbb600692a0 0x7fbb6006b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb680041d0 tx=0x7fbb680043c0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:46.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.668+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbb70058d50 con 0x7fbb740fe8f0 2024-02-22T15:46:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.866+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbb74064b00 con 0x7fbb740fe8f0 2024-02-22T15:46:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.866+0000 7fbb66ffd700 1 -- 172.21.15.19:0/3164611353 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb700588e0 con 0x7fbb740fe8f0 2024-02-22T15:46:46.850 INFO:teuthology.orchestra.run.smithi019.stdout:81604378652 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbb600692a0 msgr2=0x7fbb6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbb600692a0 0x7fbb6006b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbb680041d0 tx=0x7fbb680043c0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 msgr2=0x7fbb74113ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74113ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b3b0 tx=0x7fbb70009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 shutdown_connections 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbb600692a0 0x7fbb6006b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb740fdf50 0x7fbb741139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 --2- 172.21.15.19:0/3164611353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb740fe8f0 0x7fbb74113ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 >> 172.21.15.19:0/3164611353 conn(0x7fbb740f79e0 msgr2=0x7fbb741016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 shutdown_connections 2024-02-22T15:46:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:46.868+0000 7fbb7c2e4700 1 -- 172.21.15.19:0/3164611353 wait complete. 2024-02-22T15:46:47.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:47 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3164611353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T15:46:47.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:47 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3164611353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-22T15:46:47.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691851 got 171798691852 for osd.5 2024-02-22T15:46:47.539 DEBUG:teuthology.parallel:result is None 2024-02-22T15:46:47.969 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378652 for osd.2 2024-02-22T15:46:47.969 DEBUG:teuthology.parallel:result is None 2024-02-22T15:46:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:48 smithi019 ceph-mon[67418]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:48.345 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855377 for osd.4 2024-02-22T15:46:48.345 DEBUG:teuthology.parallel:result is None 2024-02-22T15:46:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:48 smithi062 ceph-mon[48130]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:49.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117010 got 115964117013 for osd.3 2024-02-22T15:46:49.090 DEBUG:teuthology.parallel:result is None 2024-02-22T15:46:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 -- 172.21.15.19:0/2292523746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 msgr2=0x7feadc0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 --2- 172.21.15.19:0/2292523746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc0fcf60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fead0004940 tx=0x7fead0009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 -- 172.21.15.19:0/2292523746 shutdown_connections 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 --2- 172.21.15.19:0/2292523746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc0fcf60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 --2- 172.21.15.19:0/2292523746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc102f50 0x7feadc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 -- 172.21.15.19:0/2292523746 >> 172.21.15.19:0/2292523746 conn(0x7feadc0f8ae0 msgr2=0x7feadc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 -- 172.21.15.19:0/2292523746 shutdown_connections 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.323+0000 7feae3fc8700 1 -- 172.21.15.19:0/2292523746 wait complete. 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae3fc8700 1 Processor -- start 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae3fc8700 1 -- start start 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae3fc8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc102f50 0x7feadc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae3fc8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae3fc8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc116720 con 0x7feadc1038f0 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae3fc8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc10fa50 con 0x7feadc102f50 2024-02-22T15:46:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1563700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1563700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42852/0 (socket says 172.21.15.19:42852) 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1563700 1 -- 172.21.15.19:0/863319965 learned_addr learned my addr 172.21.15.19:0/863319965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1d64700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc102f50 0x7feadc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1563700 1 -- 172.21.15.19:0/863319965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc102f50 msgr2=0x7feadc115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1563700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc102f50 0x7feadc115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.324+0000 7feae1563700 1 -- 172.21.15.19:0/863319965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0005040 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.325+0000 7feae1563700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc1160d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fead0009380 tx=0x7fead0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.325+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead000e040 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.325+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fead000bc50 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.325+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc10fcd0 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.325+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead001c3f0 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.325+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc110210 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.326+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc0fc6e0 con 0x7feadc1038f0 2024-02-22T15:46:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.326+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fead0013420 con 0x7feadc1038f0 2024-02-22T15:46:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.327+0000 7feada7fc700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feac40692a0 0x7feac406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.327+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fead0088210 con 0x7feadc1038f0 2024-02-22T15:46:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.327+0000 7feae1d64700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feac40692a0 0x7feac406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.327+0000 7feae1d64700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feac40692a0 0x7feac406b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feacc004ec0 tx=0x7feacc009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:49.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.332+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fead0058310 con 0x7feadc1038f0 2024-02-22T15:46:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.525+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feadc04bed0 con 0x7feadc1038f0 2024-02-22T15:46:49.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.526+0000 7feada7fc700 1 -- 172.21.15.19:0/863319965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fead0057ea0 con 0x7feadc1038f0 2024-02-22T15:46:49.509 INFO:teuthology.orchestra.run.smithi019.stdout:38654705705 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feac40692a0 msgr2=0x7feac406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feac40692a0 0x7feac406b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feacc004ec0 tx=0x7feacc009040 comp rx=0 tx=0).stop 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 msgr2=0x7feadc1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc1160d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fead0009380 tx=0x7fead0009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 shutdown_connections 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feac40692a0 0x7feac406b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc102f50 0x7feadc115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 --2- 172.21.15.19:0/863319965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc1038f0 0x7feadc1160d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 >> 172.21.15.19:0/863319965 conn(0x7feadc0f8ae0 msgr2=0x7feadc0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 shutdown_connections 2024-02-22T15:46:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:49.528+0000 7feae3fc8700 1 -- 172.21.15.19:0/863319965 wait complete. 2024-02-22T15:46:50.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:50 smithi019 ceph-mon[67418]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:50.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:50 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/863319965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T15:46:50.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705701 got 38654705705 for osd.0 2024-02-22T15:46:50.147 DEBUG:teuthology.parallel:result is None 2024-02-22T15:46:50.147 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-22T15:46:50.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T15:46:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:50.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:50 smithi062 ceph-mon[48130]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:50.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:50 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/863319965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-22T15:46:51.408 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:51.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:46:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 -- 172.21.15.19:0/2719048068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 msgr2=0x7fd1081092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 --2- 172.21.15.19:0/2719048068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081092c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8004970 tx=0x7fd0f8009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 -- 172.21.15.19:0/2719048068 shutdown_connections 2024-02-22T15:46:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 --2- 172.21.15.19:0/2719048068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081092c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 --2- 172.21.15.19:0/2719048068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1080702d0 0x7fd1080706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 -- 172.21.15.19:0/2719048068 >> 172.21.15.19:0/2719048068 conn(0x7fd1080f8ae0 msgr2=0x7fd1080faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 -- 172.21.15.19:0/2719048068 shutdown_connections 2024-02-22T15:46:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.228+0000 7fd10d812700 1 -- 172.21.15.19:0/2719048068 wait complete. 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd10d812700 1 Processor -- start 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd10d812700 1 -- start start 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd10d812700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1080702d0 0x7fd108115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd10d812700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd10d812700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108116700 con 0x7fd108070be0 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd10d812700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10810fa30 con 0x7fd1080702d0 2024-02-22T15:46:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd1067fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd1067fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42870/0 (socket says 172.21.15.19:42870) 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd1067fc700 1 -- 172.21.15.19:0/4238514186 learned_addr learned my addr 172.21.15.19:0/4238514186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.229+0000 7fd106ffd700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1080702d0 0x7fd108115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd1067fc700 1 -- 172.21.15.19:0/4238514186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1080702d0 msgr2=0x7fd108115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd1067fc700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1080702d0 0x7fd108115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd1067fc700 1 -- 172.21.15.19:0/4238514186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8005040 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd1067fc700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081160b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f800e040 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0f800bc10 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10810fcb0 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f801c3f0 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.230+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1081101f0 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.231+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108106980 con 0x7fd108070be0 2024-02-22T15:46:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.231+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd0f8013420 con 0x7fd108070be0 2024-02-22T15:46:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.232+0000 7fd0ef7fe700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0f00692a0 0x7fd0f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.232+0000 7fd106ffd700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0f00692a0 0x7fd0f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.232+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd0f80881e0 con 0x7fd108070be0 2024-02-22T15:46:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.232+0000 7fd106ffd700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0f00692a0 0x7fd0f006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0041d0 tx=0x7fd0fc009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.236+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd0f80582e0 con 0x7fd108070be0 2024-02-22T15:46:52.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:52 smithi019 ceph-mon[67418]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:52.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.424+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd10810c110 con 0x7fd0f00692a0 2024-02-22T15:46:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.425+0000 7fd0ef7fe700 1 -- 172.21.15.19:0/4238514186 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28221 (secure 0 0 0) 0x7fd10810c110 con 0x7fd0f00692a0 2024-02-22T15:46:52.409 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0f00692a0 msgr2=0x7fd0f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0f00692a0 0x7fd0f006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0041d0 tx=0x7fd0fc009040 comp rx=0 tx=0).stop 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 msgr2=0x7fd1081160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081160b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f8009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 shutdown_connections 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0f00692a0 0x7fd0f006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1080702d0 0x7fd108115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 --2- 172.21.15.19:0/4238514186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd108070be0 0x7fd1081160b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 >> 172.21.15.19:0/4238514186 conn(0x7fd1080f8ae0 msgr2=0x7fd108103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 shutdown_connections 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:52.427+0000 7fd10d812700 1 -- 172.21.15.19:0/4238514186 wait complete. 2024-02-22T15:46:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-02-22T15:46:52.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:52 smithi062 ceph-mon[48130]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:53.193 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-02-22T15:46:50.942488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002142"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-22T15:45:59.308922+0000","last_change":"2024-02-22T15:45:32.987134+0000","last_active":"2024-02-22T15:45:59.308922+0000","last_peered":"2024-02-22T15:45:59.308922+0000","last_clean":"2024-02-22T15:45:59.308922+0000","last_became_active":"2024-02-22T15:45:32.986701+0000","last_became_peered":"2024-02-22T15:45:32.986701+0000","last_unstale":"2024-02-22T15:45:59.308922+0000","last_undegraded":"2024-02-22T15:45:59.308922+0000","last_fullsized":"2024-02-22T15:45:59.308922+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T15:44:43.673136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T15:44:43.673136+0000","last_clean_scrub_stamp":"2024-02-22T15:44:43.673136+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-24T03:32:28.539364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.42199999999999999}]},{"osd":2,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63200000000000001}]},{"osd":1,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.501},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.621}]},{"osd":4,"last update":"Thu Feb 22 15:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.38500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.375,"15min":0.378},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.377,"15min":0.377},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.28599999999999998}]},{"osd":1,"last update":"Thu Feb 22 15:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.30199999999999999}]},{"osd":3,"last update":"Thu Feb 22 15:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.495}]},{"osd":4,"last update":"Thu Feb 22 15:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 15:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.373,"15min":0.375},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Feb 22 15:46:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.373,"15min":0.377},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.40999999999999998,"15min":0.41799999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34399999999999997},"min":{"1min":0.247,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Feb 22 15:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.55800000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.41499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Feb 22 15:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Feb 22 15:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T15:46:53.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph pg dump --format=json 2024-02-22T15:46:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:54.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:54 smithi062 ceph-mon[48130]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T15:46:54.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:54 smithi062 ceph-mon[48130]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:54.473 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:54 smithi019 ceph-mon[67418]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T15:46:54.473 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:54 smithi019 ceph-mon[67418]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.389+0000 7fa83c2c3700 1 -- 172.21.15.19:0/802132143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 msgr2=0x7fa8340fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.389+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/802132143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fbea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa830004940 tx=0x7fa830009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 -- 172.21.15.19:0/802132143 shutdown_connections 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/802132143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fbea0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/802132143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa834101e40 0x7fa834102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 -- 172.21.15.19:0/802132143 >> 172.21.15.19:0/802132143 conn(0x7fa834071d90 msgr2=0x7fa834072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 -- 172.21.15.19:0/802132143 shutdown_connections 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 -- 172.21.15.19:0/802132143 wait complete. 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 Processor -- start 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.390+0000 7fa83c2c3700 1 -- start start 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83c2c3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa834101e40 0x7fa8340fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83c2c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83c2c3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340fe600 con 0x7fa8341027e0 2024-02-22T15:46:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83c2c3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340fe770 con 0x7fa834101e40 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42890/0 (socket says 172.21.15.19:42890) 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 -- 172.21.15.19:0/3814001732 learned_addr learned my addr 172.21.15.19:0/3814001732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83a05f700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa834101e40 0x7fa8340fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 -- 172.21.15.19:0/3814001732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa834101e40 msgr2=0x7fa8340fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa834101e40 0x7fa8340fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 -- 172.21.15.19:0/3814001732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830005040 con 0x7fa8341027e0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83985e700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fe000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa830009fd0 tx=0x7fa830009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83000e040 con 0x7fa8341027e0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa834119bf0 con 0x7fa8341027e0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa83000bc50 con 0x7fa8341027e0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83001c3f0 con 0x7fa8341027e0 2024-02-22T15:46:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.391+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83411a050 con 0x7fa8341027e0 2024-02-22T15:46:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.392+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8340738b0 con 0x7fa8341027e0 2024-02-22T15:46:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.393+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa830013420 con 0x7fa8341027e0 2024-02-22T15:46:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.393+0000 7fa826ffd700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8200692a0 0x7fa82006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.393+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa830088200 con 0x7fa8341027e0 2024-02-22T15:46:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.393+0000 7fa83a05f700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8200692a0 0x7fa82006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.394+0000 7fa83a05f700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8200692a0 0x7fa82006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa828004990 tx=0x7fa828009ee0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:55.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.398+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa830054870 con 0x7fa8341027e0 2024-02-22T15:46:55.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.584+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa834065ed0 con 0x7fa8200692a0 2024-02-22T15:46:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.585+0000 7fa826ffd700 1 -- 172.21.15.19:0/3814001732 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28191 (secure 0 0 0) 0x7fa834065ed0 con 0x7fa8200692a0 2024-02-22T15:46:55.568 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8200692a0 msgr2=0x7fa82006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8200692a0 0x7fa82006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa828004990 tx=0x7fa828009ee0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 msgr2=0x7fa8340fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fe000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa830009fd0 tx=0x7fa830009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 shutdown_connections 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8200692a0 0x7fa82006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa834101e40 0x7fa8340fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 --2- 172.21.15.19:0/3814001732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8341027e0 0x7fa8340fe000 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 >> 172.21.15.19:0/3814001732 conn(0x7fa834071d90 msgr2=0x7fa8340657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 shutdown_connections 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:55.587+0000 7fa83c2c3700 1 -- 172.21.15.19:0/3814001732 wait complete. 2024-02-22T15:46:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-02-22T15:46:56.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:56 smithi019 ceph-mon[67418]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:56.351 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-02-22T15:46:54.943218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002053"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-02-22T15:45:59.308922+0000","last_change":"2024-02-22T15:45:32.987134+0000","last_active":"2024-02-22T15:45:59.308922+0000","last_peered":"2024-02-22T15:45:59.308922+0000","last_clean":"2024-02-22T15:45:59.308922+0000","last_became_active":"2024-02-22T15:45:32.986701+0000","last_became_peered":"2024-02-22T15:45:32.986701+0000","last_unstale":"2024-02-22T15:45:59.308922+0000","last_undegraded":"2024-02-22T15:45:59.308922+0000","last_fullsized":"2024-02-22T15:45:59.308922+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-22T15:44:43.673136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-22T15:44:43.673136+0000","last_clean_scrub_stamp":"2024-02-22T15:44:43.673136+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-24T03:32:28.539364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.436},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.5},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.441},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Feb 22 15:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.375,"15min":0.378},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.377,"15min":0.377},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Feb 22 15:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41699999999999998,"15min":0.41799999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Feb 22 15:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Feb 22 15:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 22 15:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.373,"15min":0.375},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Feb 22 15:46:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.373,"15min":0.377},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.48199999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.371},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.40999999999999998,"15min":0.41799999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Feb 22 15:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Feb 22 15:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 22 15:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.22800000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34399999999999997},"min":{"1min":0.247,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Feb 22 15:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.55800000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.41499999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.26600000000000001}]},{"osd":3,"last update":"Thu Feb 22 15:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.39800000000000002}]},{"osd":4,"last update":"Thu Feb 22 15:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-22T15:46:56.352 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-22T15:46:56.352 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-22T15:46:56.353 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-22T15:46:56.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph health --format=json 2024-02-22T15:46:56.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:56 smithi062 ceph-mon[48130]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:56.635 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:46:57.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:57 smithi019 ceph-mon[67418]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T15:46:57.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:57 smithi062 ceph-mon[48130]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-22T15:46:58.336 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:58 smithi019 ceph-mon[67418]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:58.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:58 smithi062 ceph-mon[48130]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:46:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 -- 172.21.15.19:0/78758078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c1027e0 msgr2=0x7f1c7c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 --2- 172.21.15.19:0/78758078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c1027e0 0x7f1c7c0fbea0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c004940 tx=0x7f1c6c009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 -- 172.21.15.19:0/78758078 shutdown_connections 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 --2- 172.21.15.19:0/78758078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c1027e0 0x7f1c7c0fbea0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 --2- 172.21.15.19:0/78758078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c7c101e40 0x7f1c7c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 -- 172.21.15.19:0/78758078 >> 172.21.15.19:0/78758078 conn(0x7f1c7c071d90 msgr2=0x7f1c7c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 -- 172.21.15.19:0/78758078 shutdown_connections 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.619+0000 7f1c832b3700 1 -- 172.21.15.19:0/78758078 wait complete. 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c832b3700 1 Processor -- start 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c832b3700 1 -- start start 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c832b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 0x7f1c7c10ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c832b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c7c1027e0 0x7f1c7c10efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c832b3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c10f540 con 0x7f1c7c101e40 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c832b3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c10f680 con 0x7f1c7c1027e0 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 0x7f1c7c10ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 0x7f1c7c10ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35022/0 (socket says 172.21.15.19:35022) 2024-02-22T15:46:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 -- 172.21.15.19:0/2318814995 learned_addr learned my addr 172.21.15.19:0/2318814995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:46:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8084e700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c7c1027e0 0x7f1c7c10efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 -- 172.21.15.19:0/2318814995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c7c1027e0 msgr2=0x7f1c7c10efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c7c1027e0 0x7f1c7c10efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 -- 172.21.15.19:0/2318814995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c005040 con 0x7f1c7c101e40 2024-02-22T15:46:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.620+0000 7f1c8104f700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 0x7f1c7c10ea70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1c78002a50 tx=0x7f1c780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.621+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c78007cc0 con 0x7f1c7c101e40 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.621+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c7800f040 con 0x7f1c7c101e40 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.621+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7800d9c0 con 0x7f1c7c101e40 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.621+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c1179e0 con 0x7f1c7c101e40 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.621+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c117ef0 con 0x7f1c7c101e40 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.622+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c104310 con 0x7f1c7c101e40 2024-02-22T15:46:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.622+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1c7800db20 con 0x7f1c7c101e40 2024-02-22T15:46:58.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.622+0000 7f1c71ffb700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f1c680692a0 0x7f1c6806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:46:58.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.623+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1c78086ab0 con 0x7f1c7c101e40 2024-02-22T15:46:58.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.623+0000 7f1c8084e700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f1c680692a0 0x7f1c6806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:46:58.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.623+0000 7f1c8084e700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f1c680692a0 0x7f1c6806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00b3b0 tx=0x7f1c6c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:46:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.627+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1c78053120 con 0x7f1c7c101e40 2024-02-22T15:46:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.862+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1c7c0738b0 con 0x7f1c7c101e40 2024-02-22T15:46:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.862+0000 7f1c71ffb700 1 -- 172.21.15.19:0/2318814995 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1c78056740 con 0x7f1c7c101e40 2024-02-22T15:46:58.845 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:46:58.846 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f1c680692a0 msgr2=0x7f1c6806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f1c680692a0 0x7f1c6806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00b3b0 tx=0x7f1c6c009e30 comp rx=0 tx=0).stop 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 msgr2=0x7f1c7c10ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 0x7f1c7c10ea70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1c78002a50 tx=0x7f1c780046b0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 shutdown_connections 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f1c680692a0 0x7f1c6806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c7c101e40 0x7f1c7c10ea70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 --2- 172.21.15.19:0/2318814995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c7c1027e0 0x7f1c7c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:46:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 >> 172.21.15.19:0/2318814995 conn(0x7f1c7c071d90 msgr2=0x7f1c7c0faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:46:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 shutdown_connections 2024-02-22T15:46:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:46:58.864+0000 7f1c832b3700 1 -- 172.21.15.19:0/2318814995 wait complete. 2024-02-22T15:46:59.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:46:59 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2318814995' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-22T15:46:59.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:46:59 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2318814995' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-22T15:46:59.750 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-22T15:46:59.750 INFO:tasks.cephadm:Setup complete, yielding 2024-02-22T15:46:59.751 INFO:teuthology.run_tasks:Running task print... 2024-02-22T15:46:59.760 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-02-22T15:46:59.760 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:46:59.780 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:46:59.780 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-22T15:47:00.064 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:00.389 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:00 smithi019 ceph-mon[67418]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:00.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:00 smithi062 ceph-mon[48130]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:01.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:01 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:01.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:01 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 -- 172.21.15.19:0/373587836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 msgr2=0x7f0be0101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 --2- 172.21.15.19:0/373587836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be0101f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc004940 tx=0x7f0bdc009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 -- 172.21.15.19:0/373587836 shutdown_connections 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 --2- 172.21.15.19:0/373587836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0be0102460 0x7f0be0104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 --2- 172.21.15.19:0/373587836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be0101f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 -- 172.21.15.19:0/373587836 >> 172.21.15.19:0/373587836 conn(0x7f0be00f7510 msgr2=0x7f0be00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 -- 172.21.15.19:0/373587836 shutdown_connections 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 -- 172.21.15.19:0/373587836 wait complete. 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 Processor -- start 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.896+0000 7f0be7caa700 1 -- start start 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be7caa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be010fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be7caa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0be0102460 0x7f0be010fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:01.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be7caa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0110680 con 0x7f0be0066c50 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be7caa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0110ec0 con 0x7f0be0102460 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be010fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be010fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35028/0 (socket says 172.21.15.19:35028) 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 -- 172.21.15.19:0/834248731 learned_addr learned my addr 172.21.15.19:0/834248731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 -- 172.21.15.19:0/834248731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0be0102460 msgr2=0x7f0be010fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5245700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0be0102460 0x7f0be010fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0be0102460 0x7f0be010fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 -- 172.21.15.19:0/834248731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc005040 con 0x7f0be0066c50 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be5a46700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be010fab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009360 tx=0x7f0bdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bdc00e040 con 0x7f0be0066c50 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0bdc00bda0 con 0x7f0be0066c50 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be0111140 con 0x7f0be0066c50 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bdc01c630 con 0x7f0be0066c50 2024-02-22T15:47:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.897+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be0111680 con 0x7f0be0066c50 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.898+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be0109bd0 con 0x7f0be0066c50 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.899+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0bdc013a40 con 0x7f0be0066c50 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.899+0000 7f0bd67fc700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0bcc0692a0 0x7f0bcc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.899+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0bdc088350 con 0x7f0be0066c50 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.899+0000 7f0be5245700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0bcc0692a0 0x7f0bcc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.900+0000 7f0be5245700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0bcc0692a0 0x7f0bcc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0006fd0 tx=0x7f0bd0008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:01.904+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0bdc058450 con 0x7f0be0066c50 2024-02-22T15:47:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.097+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f0be004bed0 con 0x7f0be0066c50 2024-02-22T15:47:02.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.100+0000 7f0bd67fc700 1 -- 172.21.15.19:0/834248731 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f0bdc057fe0 con 0x7f0be0066c50 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0bcc0692a0 msgr2=0x7f0bcc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0bcc0692a0 0x7f0bcc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0006fd0 tx=0x7f0bd0008040 comp rx=0 tx=0).stop 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 msgr2=0x7f0be010fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be010fab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009360 tx=0x7f0bdc009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 shutdown_connections 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f0bcc0692a0 0x7f0bcc06b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0be0066c50 0x7f0be010fab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 --2- 172.21.15.19:0/834248731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0be0102460 0x7f0be010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 >> 172.21.15.19:0/834248731 conn(0x7f0be00f7510 msgr2=0x7f0be00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 shutdown_connections 2024-02-22T15:47:02.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:02.103+0000 7f0be7caa700 1 -- 172.21.15.19:0/834248731 wait complete. 2024-02-22T15:47:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:02 smithi062 ceph-mon[48130]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:02.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:02 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:02.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:02 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:02.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:02 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:02.477 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:02 smithi019 ceph-mon[67418]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:02.477 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:02 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:02.477 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:02 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:02.477 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:02 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:02.800 INFO:teuthology.run_tasks:Running task print... 2024-02-22T15:47:02.809 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-22T15:47:02.809 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:47:02.817 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:47:02.817 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph orch status' 2024-02-22T15:47:03.101 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:03.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:03 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/834248731' entity='client.admin' 2024-02-22T15:47:03.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:03.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:03.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:03.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:03 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:03.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:03 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/834248731' entity='client.admin' 2024-02-22T15:47:03.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:03.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:03.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:03.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:03 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:04.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:04 smithi019 ceph-mon[67418]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:04.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:04 smithi062 ceph-mon[48130]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:04.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2134968020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 msgr2=0x7fa5a40fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:04.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2134968020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a40fd420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa594004940 tx=0x7fa594009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2134968020 shutdown_connections 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2134968020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a40fd420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2134968020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a40fbe40 0x7fa5a40fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2134968020 >> 172.21.15.19:0/2134968020 conn(0x7fa5a40f74e0 msgr2=0x7fa5a40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2134968020 shutdown_connections 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2134968020 wait complete. 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.972+0000 7fa5a96a4700 1 Processor -- start 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a96a4700 1 -- start start 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a96a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a40fbe40 0x7fa5a410d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a96a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a96a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a410e3b0 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a96a4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a410ec50 con 0x7fa5a40fbe40 2024-02-22T15:47:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a410ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35036/0 (socket says 172.21.15.19:35036) 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 -- 172.21.15.19:0/2827234388 learned_addr learned my addr 172.21.15.19:0/2827234388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a2ffd700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a40fbe40 0x7fa5a410d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 -- 172.21.15.19:0/2827234388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a40fbe40 msgr2=0x7fa5a410d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a40fbe40 0x7fa5a410d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 -- 172.21.15.19:0/2827234388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594005040 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a27fc700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a410ddb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa594009fd0 tx=0x7fa594009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa59400e040 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa594004290 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa59401c3f0 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.973+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a410eed0 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.974+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a410f410 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.974+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a404bed0 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.975+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa59400bc50 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.975+0000 7fa59b7fe700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa5900692b0 0x7fa59006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:04.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.975+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa594088220 con 0x7fa5a40fcfb0 2024-02-22T15:47:04.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.975+0000 7fa5a2ffd700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa5900692b0 0x7fa59006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:04.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.976+0000 7fa5a2ffd700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa5900692b0 0x7fa59006b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa58c004f60 tx=0x7fa58c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:04.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:04.980+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa594058320 con 0x7fa5a40fcfb0 2024-02-22T15:47:05.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.186+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5a410f820 con 0x7fa5900692b0 2024-02-22T15:47:05.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.187+0000 7fa59b7fe700 1 -- 172.21.15.19:0/2827234388 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa5a410f820 con 0x7fa5900692b0 2024-02-22T15:47:05.170 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-02-22T15:47:05.170 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-02-22T15:47:05.170 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-02-22T15:47:05.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa5900692b0 msgr2=0x7fa59006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:05.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa5900692b0 0x7fa59006b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa58c004f60 tx=0x7fa58c00b040 comp rx=0 tx=0).stop 2024-02-22T15:47:05.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 msgr2=0x7fa5a410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:05.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a410ddb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa594009fd0 tx=0x7fa594009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:05.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 shutdown_connections 2024-02-22T15:47:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa5900692b0 0x7fa59006b760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a40fbe40 0x7fa5a410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 --2- 172.21.15.19:0/2827234388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a40fcfb0 0x7fa5a410ddb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 >> 172.21.15.19:0/2827234388 conn(0x7fa5a40f74e0 msgr2=0x7fa5a40656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 shutdown_connections 2024-02-22T15:47:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:05.189+0000 7fa5a96a4700 1 -- 172.21.15.19:0/2827234388 wait complete. 2024-02-22T15:47:05.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-22T15:47:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:06.390 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:06 smithi019 ceph-mon[67418]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:06.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:06 smithi019 ceph-mon[67418]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:06.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:06.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:06 smithi062 ceph-mon[48130]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:06.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:06 smithi062 ceph-mon[48130]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:06.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 -- 172.21.15.19:0/4015997202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 msgr2=0x7f64680fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 --2- 172.21.15.19:0/4015997202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f64680fc260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f645c004940 tx=0x7f645c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 -- 172.21.15.19:0/4015997202 shutdown_connections 2024-02-22T15:47:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 --2- 172.21.15.19:0/4015997202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680fcfc0 0x7f64680fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 --2- 172.21.15.19:0/4015997202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f64680fc260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 -- 172.21.15.19:0/4015997202 >> 172.21.15.19:0/4015997202 conn(0x7f64680f7510 msgr2=0x7f64680f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 -- 172.21.15.19:0/4015997202 shutdown_connections 2024-02-22T15:47:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 -- 172.21.15.19:0/4015997202 wait complete. 2024-02-22T15:47:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 Processor -- start 2024-02-22T15:47:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.027+0000 7f646eef6700 1 -- start start 2024-02-22T15:47:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646eef6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f6468110560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646eef6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680fcfc0 0x7f6468112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646eef6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468066d10 con 0x7f64680fbe50 2024-02-22T15:47:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646eef6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468066e50 con 0x7f64680fcfc0 2024-02-22T15:47:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f6468110560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f6468110560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36106/0 (socket says 172.21.15.19:36106) 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 -- 172.21.15.19:0/1469518923 learned_addr learned my addr 172.21.15.19:0/1469518923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f6467fff700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680fcfc0 0x7f6468112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 -- 172.21.15.19:0/1469518923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680fcfc0 msgr2=0x7f6468112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680fcfc0 0x7f6468112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 -- 172.21.15.19:0/1469518923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c005040 con 0x7f64680fbe50 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646cc92700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f6468110560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f645c009360 tx=0x7f645c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f645c00e040 con 0x7f64680fbe50 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f645c00bc50 con 0x7f64680fbe50 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.028+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64681130b0 con 0x7f64680fbe50 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.029+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f645c01c3f0 con 0x7f64680fbe50 2024-02-22T15:47:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.029+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64681135a0 con 0x7f64680fbe50 2024-02-22T15:47:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.030+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646804bed0 con 0x7f64680fbe50 2024-02-22T15:47:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.030+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f645c013420 con 0x7f64680fbe50 2024-02-22T15:47:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.030+0000 7f64657fa700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f64580692a0 0x7f645806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.030+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f645c0881d0 con 0x7f64680fbe50 2024-02-22T15:47:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.030+0000 7f6467fff700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f64580692a0 0x7f645806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.031+0000 7f6467fff700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f64580692a0 0x7f645806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6454004f40 tx=0x7f645400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:08.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.035+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f645c0547c0 con 0x7f64680fbe50 2024-02-22T15:47:08.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.237+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64681019a0 con 0x7f64580692a0 2024-02-22T15:47:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.245+0000 7f64657fa700 1 -- 172.21.15.19:0/1469518923 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f64681019a0 con 0x7f64580692a0 2024-02-22T15:47:08.228 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:47:08.228 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (3m) 2m ago 5m 26.5M - 0.25.0 c8568f914cd2 f9f67af4f00a 2024-02-22T15:47:08.228 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (5m) 2m ago 5m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:47:08.228 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (4m) 71s ago 4m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:47:08.228 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (3m) 2m ago 5m 76.3M - 9.4.7 954c08fa6188 1b6b49b44a2a 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:9283 running (6m) 2m ago 6m 469M - 17.2.7-527-g6ea2a55e e2e4ad29904c 1e57e04e2dad 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 running (4m) 71s ago 4m 431M - 17.2.7-527-g6ea2a55e e2e4ad29904c 8bcf1362db0f 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (6m) 2m ago 7m 41.6M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (4m) 71s ago 4m 41.5M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (5m) 2m ago 5m 17.3M - 1.5.0 0da6a335fe13 d890c8b7cdd8 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (4m) 71s ago 4m 16.0M - 1.5.0 0da6a335fe13 379a69e65053 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (3m) 2m ago 3m 37.7M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (3m) 2m ago 3m 36.8M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (2m) 2m ago 2m 11.0M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (2m) 71s ago 2m 38.1M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (102s) 71s ago 102s 38.9M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (76s) 71s ago 75s 12.7M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:47:08.229 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (3m) 2m ago 4m 38.1M - 2.43.0 a07b618ecd1d 05fe6b300615 2024-02-22T15:47:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f64580692a0 msgr2=0x7f645806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f64580692a0 0x7f645806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6454004f40 tx=0x7f645400b040 comp rx=0 tx=0).stop 2024-02-22T15:47:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 msgr2=0x7f6468110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f6468110560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f645c009360 tx=0x7f645c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 shutdown_connections 2024-02-22T15:47:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f64580692a0 0x7f645806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64680fbe50 0x7f6468110560 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 --2- 172.21.15.19:0/1469518923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64680fcfc0 0x7f6468112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 >> 172.21.15.19:0/1469518923 conn(0x7f64680f7510 msgr2=0x7f6468100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 shutdown_connections 2024-02-22T15:47:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:08.247+0000 7f646eef6700 1 -- 172.21.15.19:0/1469518923 wait complete. 2024-02-22T15:47:08.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:08 smithi062 ceph-mon[48130]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:08.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:08 smithi019 ceph-mon[67418]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:08.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-22T15:47:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:10.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:10 smithi062 ceph-mon[48130]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:10.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:10 smithi062 ceph-mon[48130]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:10.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:10 smithi019 ceph-mon[67418]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:10.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:10 smithi019 ceph-mon[67418]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:11.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.109+0000 7fe84f331700 1 -- 172.21.15.19:0/3927665776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 msgr2=0x7fe8480fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.109+0000 7fe84f331700 1 --2- 172.21.15.19:0/3927665776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe8480fc260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe844004940 tx=0x7fe844009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.109+0000 7fe84f331700 1 -- 172.21.15.19:0/3927665776 shutdown_connections 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.109+0000 7fe84f331700 1 --2- 172.21.15.19:0/3927665776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8480fcfc0 0x7fe8480fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.109+0000 7fe84f331700 1 --2- 172.21.15.19:0/3927665776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe8480fc260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.109+0000 7fe84f331700 1 -- 172.21.15.19:0/3927665776 >> 172.21.15.19:0/3927665776 conn(0x7fe8480f7510 msgr2=0x7fe8480f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 -- 172.21.15.19:0/3927665776 shutdown_connections 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 -- 172.21.15.19:0/3927665776 wait complete. 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 Processor -- start 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 -- start start 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe84810d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8480fcfc0 0x7fe84810ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84810e3b0 con 0x7fe8480fbe50 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84f331700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84810ec50 con 0x7fe8480fcfc0 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84d0cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe84810d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84d0cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe84810d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36128/0 (socket says 172.21.15.19:36128) 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84d0cd700 1 -- 172.21.15.19:0/639902825 learned_addr learned my addr 172.21.15.19:0/639902825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.110+0000 7fe84c8cc700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8480fcfc0 0x7fe84810ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe84d0cd700 1 -- 172.21.15.19:0/639902825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8480fcfc0 msgr2=0x7fe84810ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe84d0cd700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8480fcfc0 0x7fe84810ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe84d0cd700 1 -- 172.21.15.19:0/639902825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844005040 con 0x7fe8480fbe50 2024-02-22T15:47:11.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe84d0cd700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe84810d870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe844009fd0 tx=0x7fe844009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe84400e040 con 0x7fe8480fbe50 2024-02-22T15:47:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe844004290 con 0x7fe8480fbe50 2024-02-22T15:47:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84810eed0 con 0x7fe8480fbe50 2024-02-22T15:47:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe84401c3f0 con 0x7fe8480fbe50 2024-02-22T15:47:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.111+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84810f410 con 0x7fe8480fbe50 2024-02-22T15:47:11.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.112+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84804bed0 con 0x7fe8480fbe50 2024-02-22T15:47:11.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.113+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe84400bc50 con 0x7fe8480fbe50 2024-02-22T15:47:11.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.113+0000 7fe839ffb700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8340692b0 0x7fe83406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:11.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.113+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe844088150 con 0x7fe8480fbe50 2024-02-22T15:47:11.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.113+0000 7fe84c8cc700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8340692b0 0x7fe83406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:11.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.114+0000 7fe84c8cc700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8340692b0 0x7fe83406b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe83c004f40 tx=0x7fe83c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:11.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.118+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe8440582f0 con 0x7fe8480fbe50 2024-02-22T15:47:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.320+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe84810fad0 con 0x7fe8340692b0 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.324+0000 7fe839ffb700 1 -- 172.21.15.19:0/639902825 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7fe84810fad0 con 0x7fe8340692b0 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:crash 2/2 2m ago 6m * 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 2/2 2m ago 6m count:2 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:mon 2/2 2m ago 5m smithi019:172.21.15.19=smithi019;smithi062:172.21.15.62=smithi062;count:2 2024-02-22T15:47:11.307 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-02-22T15:47:11.308 INFO:teuthology.orchestra.run.smithi019.stdout:osd 6 2m ago - 2024-02-22T15:47:11.308 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-02-22T15:47:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8340692b0 msgr2=0x7fe83406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8340692b0 0x7fe83406b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe83c004f40 tx=0x7fe83c00b040 comp rx=0 tx=0).stop 2024-02-22T15:47:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 msgr2=0x7fe84810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe84810d870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe844009fd0 tx=0x7fe844009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 shutdown_connections 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8340692b0 0x7fe83406b760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8480fbe50 0x7fe84810d870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 --2- 172.21.15.19:0/639902825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8480fcfc0 0x7fe84810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 >> 172.21.15.19:0/639902825 conn(0x7fe8480f7510 msgr2=0x7fe848065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 shutdown_connections 2024-02-22T15:47:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:11.326+0000 7fe84f331700 1 -- 172.21.15.19:0/639902825 wait complete. 2024-02-22T15:47:12.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-22T15:47:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:12.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:12 smithi062 ceph-mon[48130]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:12.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:12 smithi019 ceph-mon[67418]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:13.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:13 smithi062 ceph-mon[48130]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:13.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:13 smithi019 ceph-mon[67418]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 -- 172.21.15.19:0/3947657126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 msgr2=0x7f40c40fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 --2- 172.21.15.19:0/3947657126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40fe550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f40ac004940 tx=0x7f40ac009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 -- 172.21.15.19:0/3947657126 shutdown_connections 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 --2- 172.21.15.19:0/3947657126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40fe550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 --2- 172.21.15.19:0/3947657126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 0x7f40c40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 -- 172.21.15.19:0/3947657126 >> 172.21.15.19:0/3947657126 conn(0x7f40c40f8620 msgr2=0x7f40c40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 -- 172.21.15.19:0/3947657126 shutdown_connections 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.252+0000 7f40c89df700 1 -- 172.21.15.19:0/3947657126 wait complete. 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c89df700 1 Processor -- start 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c89df700 1 -- start start 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c89df700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 0x7f40c4071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c89df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c89df700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4070970 con 0x7f40c40fe0e0 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c89df700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4070ae0 con 0x7f40c40fcef0 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36142/0 (socket says 172.21.15.19:36142) 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c1d9b700 1 -- 172.21.15.19:0/681851025 learned_addr learned my addr 172.21.15.19:0/681851025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c259c700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 0x7f40c4071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c1d9b700 1 -- 172.21.15.19:0/681851025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 msgr2=0x7f40c4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c1d9b700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 0x7f40c4071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c1d9b700 1 -- 172.21.15.19:0/681851025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac005040 con 0x7f40c40fe0e0 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.253+0000 7f40c259c700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 0x7f40c4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.254+0000 7f40c1d9b700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40702e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009360 tx=0x7f40ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.254+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ac00e040 con 0x7f40c40fe0e0 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.254+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f40ac00bc50 con 0x7f40c40fe0e0 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.254+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c4070d60 con 0x7f40c40fe0e0 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.254+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ac01c630 con 0x7f40c40fe0e0 2024-02-22T15:47:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.254+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c41158d0 con 0x7f40c40fe0e0 2024-02-22T15:47:14.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.255+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c404bed0 con 0x7f40c40fe0e0 2024-02-22T15:47:14.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.255+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f40ac013420 con 0x7f40c40fe0e0 2024-02-22T15:47:14.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.256+0000 7f40baffd700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f40b00692a0 0x7f40b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:14.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.256+0000 7f40c259c700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f40b00692a0 0x7f40b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:14.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.256+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f40ac088270 con 0x7f40c40fe0e0 2024-02-22T15:47:14.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.256+0000 7f40c259c700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f40b00692a0 0x7f40b006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006fd0 tx=0x7f40b4008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:14.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.260+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f40ac058350 con 0x7f40c40fe0e0 2024-02-22T15:47:14.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:14 smithi019 ceph-mon[67418]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:14.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.458+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f40c4102ac0 con 0x7f40b00692a0 2024-02-22T15:47:14.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.460+0000 7f40baffd700 1 -- 172.21.15.19:0/681851025 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f40c4102ac0 con 0x7f40b00692a0 2024-02-22T15:47:14.443 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-02-22T15:47:14.443 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-02-22T15:47:14.443 INFO:teuthology.orchestra.run.smithi019.stdout:smithi062 172.21.15.62 2024-02-22T15:47:14.443 INFO:teuthology.orchestra.run.smithi019.stdout:2 hosts in cluster 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f40b00692a0 msgr2=0x7f40b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f40b00692a0 0x7f40b006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006fd0 tx=0x7f40b4008040 comp rx=0 tx=0).stop 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 msgr2=0x7f40c40702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40702e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009360 tx=0x7f40ac009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 shutdown_connections 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f40b00692a0 0x7f40b006b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40c40fcef0 0x7f40c4071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 --2- 172.21.15.19:0/681851025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40fe0e0 0x7f40c40702e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 >> 172.21.15.19:0/681851025 conn(0x7f40c40f8620 msgr2=0x7f40c41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:14.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 shutdown_connections 2024-02-22T15:47:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:14.462+0000 7f40c89df700 1 -- 172.21.15.19:0/681851025 wait complete. 2024-02-22T15:47:14.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:14 smithi062 ceph-mon[48130]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:15.152 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-22T15:47:15.433 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:16.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:16 smithi062 ceph-mon[48130]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:16.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:16 smithi062 ceph-mon[48130]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:16.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:16 smithi019 ceph-mon[67418]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:16.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:16 smithi019 ceph-mon[67418]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 -- 172.21.15.19:0/2705712571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 msgr2=0x7fd7dc0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 --2- 172.21.15.19:0/2705712571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc0fc250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0004940 tx=0x7fd7d0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 -- 172.21.15.19:0/2705712571 shutdown_connections 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 --2- 172.21.15.19:0/2705712571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7dc0fcfb0 0x7fd7dc0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 --2- 172.21.15.19:0/2705712571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc0fc250 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 -- 172.21.15.19:0/2705712571 >> 172.21.15.19:0/2705712571 conn(0x7fd7dc0f74e0 msgr2=0x7fd7dc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 -- 172.21.15.19:0/2705712571 shutdown_connections 2024-02-22T15:47:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 -- 172.21.15.19:0/2705712571 wait complete. 2024-02-22T15:47:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 Processor -- start 2024-02-22T15:47:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.285+0000 7fd7e1039700 1 -- start start 2024-02-22T15:47:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7e1039700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc10fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7e1039700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7dc0fcfb0 0x7fd7dc10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7e1039700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc110640 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7e1039700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc110e80 con 0x7fd7dc0fcfb0 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc10fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc10fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36160/0 (socket says 172.21.15.19:36160) 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 -- 172.21.15.19:0/1001577951 learned_addr learned my addr 172.21.15.19:0/1001577951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7da59c700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7dc0fcfb0 0x7fd7dc10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 -- 172.21.15.19:0/1001577951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7dc0fcfb0 msgr2=0x7fd7dc10ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7dc0fcfb0 0x7fd7dc10ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 -- 172.21.15.19:0/1001577951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0005040 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7dad9d700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc10fa70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009360 tx=0x7fd7d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d000e040 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.286+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7d000bc50 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.287+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc111100 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.287+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d001c3f0 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.287+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc111640 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.288+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc04bed0 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.288+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd7d0013420 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.288+0000 7fd7c37fe700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7c40692a0 0x7fd7c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:17.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.288+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd7d00880f0 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.288+0000 7fd7da59c700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7c40692a0 0x7fd7c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:17.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.289+0000 7fd7da59c700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7c40692a0 0x7fd7c406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009bf0 tx=0x7fd7cc0091a0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.293+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd7d00581f0 con 0x7fd7dc0fbe40 2024-02-22T15:47:17.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.493+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7dc065dd0 con 0x7fd7c40692a0 2024-02-22T15:47:17.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.494+0000 7fd7c37fe700 1 -- 172.21.15.19:0/1001577951 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fd7dc065dd0 con 0x7fd7c40692a0 2024-02-22T15:47:17.477 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-22T15:47:17.477 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-22T15:47:17.477 INFO:teuthology.orchestra.run.smithi019.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 77s ago Has a FileSystem, LVM detected 2024-02-22T15:47:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7c40692a0 msgr2=0x7fd7c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7c40692a0 0x7fd7c406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009bf0 tx=0x7fd7cc0091a0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 msgr2=0x7fd7dc10fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc10fa70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009360 tx=0x7fd7d0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 shutdown_connections 2024-02-22T15:47:17.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd7c40692a0 0x7fd7c406b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7dc0fbe40 0x7fd7dc10fa70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 --2- 172.21.15.19:0/1001577951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7dc0fcfb0 0x7fd7dc10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 >> 172.21.15.19:0/1001577951 conn(0x7fd7dc0f74e0 msgr2=0x7fd7dc0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 shutdown_connections 2024-02-22T15:47:17.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:17.496+0000 7fd7e1039700 1 -- 172.21.15.19:0/1001577951 wait complete. 2024-02-22T15:47:18.212 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:47:18.222 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:47:18.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-22T15:47:18.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:18 smithi062 ceph-mon[48130]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:18.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:18 smithi019 ceph-mon[67418]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:18.505 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:19.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:19 smithi062 ceph-mon[48130]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:19.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:19 smithi019 ceph-mon[67418]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 -- 172.21.15.19:0/1556053728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 msgr2=0x7f8ab00ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/1556053728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab00ffe70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0004940 tx=0x7f8aa0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 -- 172.21.15.19:0/1556053728 shutdown_connections 2024-02-22T15:47:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/1556053728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01003b0 0x7f8ab0102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:20.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/1556053728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab00ffe70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 -- 172.21.15.19:0/1556053728 >> 172.21.15.19:0/1556053728 conn(0x7f8ab00f7510 msgr2=0x7f8ab00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 -- 172.21.15.19:0/1556053728 shutdown_connections 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.209+0000 7f8ab74bd700 1 -- 172.21.15.19:0/1556053728 wait complete. 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab74bd700 1 Processor -- start 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab74bd700 1 -- start start 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab74bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab010d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab74bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01003b0 0x7f8ab010dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab74bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab010e390 con 0x7f8ab00fda90 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab74bd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab010ec30 con 0x7f8ab01003b0 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab5259700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab010d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab5259700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab010d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41554/0 (socket says 172.21.15.19:41554) 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab5259700 1 -- 172.21.15.19:0/2071442689 learned_addr learned my addr 172.21.15.19:0/2071442689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab5259700 1 -- 172.21.15.19:0/2071442689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01003b0 msgr2=0x7f8ab010dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab5259700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01003b0 0x7f8ab010dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.210+0000 7f8ab5259700 1 -- 172.21.15.19:0/2071442689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa0005040 con 0x7f8ab00fda90 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.211+0000 7f8ab5259700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab010d850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009360 tx=0x7f8aa0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.211+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa000e040 con 0x7f8ab00fda90 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.211+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab010eeb0 con 0x7f8ab00fda90 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.211+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8aa000bcd0 con 0x7f8ab00fda90 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.211+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa001c630 con 0x7f8ab00fda90 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.211+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab010f3f0 con 0x7f8ab00fda90 2024-02-22T15:47:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.212+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab0107920 con 0x7f8ab00fda90 2024-02-22T15:47:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.213+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8aa0013960 con 0x7f8ab00fda90 2024-02-22T15:47:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.214+0000 7f8aa5ffb700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8a9c0692a0 0x7f8a9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.214+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8aa0088de0 con 0x7f8ab00fda90 2024-02-22T15:47:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.214+0000 7f8ab4a58700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8a9c0692a0 0x7f8a9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.214+0000 7f8ab4a58700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8a9c0692a0 0x7f8a9c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8aac004d00 tx=0x7f8aac009ce0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.218+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8aa0058ec0 con 0x7f8ab00fda90 2024-02-22T15:47:20.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:20 smithi019 ceph-mon[67418]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:20.438+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ab010f7e0 con 0x7f8a9c0692a0 2024-02-22T15:47:20.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:20 smithi062 ceph-mon[48130]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:21.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:21 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-22T15:47:21.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:21 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:21.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-22T15:47:21.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.213+0000 7f8aa5ffb700 1 -- 172.21.15.19:0/2071442689 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8ab010f7e0 con 0x7f8a9c0692a0 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8a9c0692a0 msgr2=0x7f8a9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8a9c0692a0 0x7f8a9c06b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8aac004d00 tx=0x7f8aac009ce0 comp rx=0 tx=0).stop 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 msgr2=0x7f8ab010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab010d850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009360 tx=0x7f8aa0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 shutdown_connections 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8a9c0692a0 0x7f8a9c06b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab00fda90 0x7f8ab010d850 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 --2- 172.21.15.19:0/2071442689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01003b0 0x7f8ab010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 >> 172.21.15.19:0/2071442689 conn(0x7f8ab00f7510 msgr2=0x7f8ab00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 shutdown_connections 2024-02-22T15:47:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:22.216+0000 7f8ab74bd700 1 -- 172.21.15.19:0/2071442689 wait complete. 2024-02-22T15:47:22.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:22 smithi062 ceph-mon[48130]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:22.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:22 smithi062 ceph-mon[48130]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:22.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:22 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-22T15:47:22.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:22 smithi062 ceph-mon[48130]: osdmap e42: 6 total, 6 up, 6 in 2024-02-22T15:47:22.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:22 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-22T15:47:22.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:22 smithi019 ceph-mon[67418]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:47:22.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:22 smithi019 ceph-mon[67418]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:22.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:22 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-22T15:47:22.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:22 smithi019 ceph-mon[67418]: osdmap e42: 6 total, 6 up, 6 in 2024-02-22T15:47:22.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:22 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-22T15:47:22.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[67395]: 2024-02-22T15:47:22.183+0000 7fb65ddfb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-22T15:47:22.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-22T15:47:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: osdmap e43: 6 total, 6 up, 6 in 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: osdmap e44: 6 total, 6 up, 6 in 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: fsmap cephfs:0 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: Saving service mds.cephfs spec with placement count:4 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:23.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:23.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:23.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:23.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:23 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:23.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: osdmap e43: 6 total, 6 up, 6 in 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: osdmap e44: 6 total, 6 up, 6 in 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: fsmap cephfs:0 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: Saving service mds.cephfs spec with placement count:4 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:23.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:23.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:23.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:23.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:23 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:24.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:24 smithi062 ceph-mon[48130]: Deploying daemon mds.cephfs.smithi019.lkzblb on smithi019 2024-02-22T15:47:24.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:24 smithi062 ceph-mon[48130]: pgmap v183: 65 pgs: 10 creating+peering, 40 unknown, 15 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:24.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:24 smithi062 ceph-mon[48130]: osdmap e45: 6 total, 6 up, 6 in 2024-02-22T15:47:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:24 smithi019 ceph-mon[67418]: Deploying daemon mds.cephfs.smithi019.lkzblb on smithi019 2024-02-22T15:47:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:24 smithi019 ceph-mon[67418]: pgmap v183: 65 pgs: 10 creating+peering, 40 unknown, 15 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:24 smithi019 ceph-mon[67418]: osdmap e45: 6 total, 6 up, 6 in 2024-02-22T15:47:25.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:25 smithi019 ceph-mon[67418]: osdmap e46: 6 total, 6 up, 6 in 2024-02-22T15:47:25.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:25 smithi062 ceph-mon[48130]: osdmap e46: 6 total, 6 up, 6 in 2024-02-22T15:47:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 -- 172.21.15.19:0/3101733753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fcef0 msgr2=0x7fccd00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 --2- 172.21.15.19:0/3101733753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fcef0 0x7fccd00fd320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fccc0004940 tx=0x7fccc0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 -- 172.21.15.19:0/3101733753 shutdown_connections 2024-02-22T15:47:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 --2- 172.21.15.19:0/3101733753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccd00fe0e0 0x7fccd00fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 --2- 172.21.15.19:0/3101733753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fcef0 0x7fccd00fd320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 -- 172.21.15.19:0/3101733753 >> 172.21.15.19:0/3101733753 conn(0x7fccd00f8620 msgr2=0x7fccd00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 -- 172.21.15.19:0/3101733753 shutdown_connections 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 -- 172.21.15.19:0/3101733753 wait complete. 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 Processor -- start 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.982+0000 7fccd829a700 1 -- start start 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd829a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccd00fcef0 0x7fccd010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd829a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 0x7fccd010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd829a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd010e3d0 con 0x7fccd00fe0e0 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd829a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd010ec70 con 0x7fccd00fcef0 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 0x7fccd010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 0x7fccd010ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41572/0 (socket says 172.21.15.19:41572) 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 -- 172.21.15.19:0/1142372501 learned_addr learned my addr 172.21.15.19:0/1142372501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd6036700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccd00fcef0 0x7fccd010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 -- 172.21.15.19:0/1142372501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccd00fcef0 msgr2=0x7fccd010d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccd00fcef0 0x7fccd010d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 -- 172.21.15.19:0/1142372501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0005040 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd5835700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 0x7fccd010ddd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcccc002a50 tx=0x7fcccc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc007cc0 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcccc00f040 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd010ef50 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc009540 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.983+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd010f490 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.984+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd004bed0 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.985+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcccc0096a0 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.986+0000 7fccc6ffd700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fccbc0692a0 0x7fccbc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.986+0000 7fccd6036700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fccbc0692a0 0x7fccbc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.986+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcccc0875a0 con 0x7fccd00fe0e0 2024-02-22T15:47:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.986+0000 7fccd6036700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fccbc0692a0 0x7fccbc06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b3b0 tx=0x7fccc0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:25.990+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcccc057360 con 0x7fccd00fe0e0 2024-02-22T15:47:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.242+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fccd01058f0 con 0x7fccd00fe0e0 2024-02-22T15:47:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.242+0000 7fccc6ffd700 1 -- 172.21.15.19:0/1142372501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fcccc056ef0 con 0x7fccd00fe0e0 2024-02-22T15:47:26.226 INFO:teuthology.orchestra.run.smithi019.stdout:e2 2024-02-22T15:47:26.226 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:47:26.226 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:47:26.226 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 2 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:47:22.185591+0000 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:47:26.227 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 1 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:in 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:26.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fccbc0692a0 msgr2=0x7fccbc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fccbc0692a0 0x7fccbc06b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b3b0 tx=0x7fccc0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 msgr2=0x7fccd010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 0x7fccd010ddd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcccc002a50 tx=0x7fcccc0046b0 comp rx=0 tx=0).stop 2024-02-22T15:47:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 shutdown_connections 2024-02-22T15:47:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fccbc0692a0 0x7fccbc06b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccd00fcef0 0x7fccd010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 --2- 172.21.15.19:0/1142372501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00fe0e0 0x7fccd010ddd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 >> 172.21.15.19:0/1142372501 conn(0x7fccd00f8620 msgr2=0x7fccd01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 shutdown_connections 2024-02-22T15:47:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:26.244+0000 7fccd829a700 1 -- 172.21.15.19:0/1142372501 wait complete. 2024-02-22T15:47:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-02-22T15:47:26.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:26 smithi062 ceph-mon[48130]: pgmap v186: 65 pgs: 15 creating+peering, 13 unknown, 37 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:26.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:26 smithi019 ceph-mon[67418]: pgmap v186: 65 pgs: 15 creating+peering, 13 unknown, 37 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:27.122 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:47:27.133 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:47:27.133 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-02-22T15:47:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1142372501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:27.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:27 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:27.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1142372501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:47:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:27.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:27 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:28.334 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: Deploying daemon mds.cephfs.smithi062.oyfagg on smithi062 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: pgmap v187: 65 pgs: 15 creating+peering, 50 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6826/2235854865,v1:172.21.15.19:6827/2235854865] up:boot 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: daemon mds.cephfs.smithi019.lkzblb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: Cluster is now healthy 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: fsmap cephfs:0 1 up:standby 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: fsmap cephfs:1 {0=cephfs.smithi019.lkzblb=up:creating} 2024-02-22T15:47:28.335 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:28 smithi062 ceph-mon[48130]: daemon mds.cephfs.smithi019.lkzblb is now active in filesystem cephfs as rank 0 2024-02-22T15:47:28.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: Deploying daemon mds.cephfs.smithi062.oyfagg on smithi062 2024-02-22T15:47:28.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: pgmap v187: 65 pgs: 15 creating+peering, 50 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:47:28.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6826/2235854865,v1:172.21.15.19:6827/2235854865] up:boot 2024-02-22T15:47:28.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: daemon mds.cephfs.smithi019.lkzblb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-22T15:47:28.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-22T15:47:28.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-22T15:47:28.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: Cluster is now healthy 2024-02-22T15:47:28.501 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: fsmap cephfs:0 1 up:standby 2024-02-22T15:47:28.501 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:47:28.501 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: fsmap cephfs:1 {0=cephfs.smithi019.lkzblb=up:creating} 2024-02-22T15:47:28.501 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:28 smithi019 ceph-mon[67418]: daemon mds.cephfs.smithi019.lkzblb is now active in filesystem cephfs as rank 0 2024-02-22T15:47:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 -- 172.21.15.19:0/2516504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 msgr2=0x7f428c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 --2- 172.21.15.19:0/2516504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c0fc250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f427c004940 tx=0x7f427c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 -- 172.21.15.19:0/2516504 shutdown_connections 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 --2- 172.21.15.19:0/2516504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f428c0fcfb0 0x7f428c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 --2- 172.21.15.19:0/2516504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c0fc250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 -- 172.21.15.19:0/2516504 >> 172.21.15.19:0/2516504 conn(0x7f428c0f74e0 msgr2=0x7f428c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 -- 172.21.15.19:0/2516504 shutdown_connections 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.197+0000 7f4291a8d700 1 -- 172.21.15.19:0/2516504 wait complete. 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f4291a8d700 1 Processor -- start 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f4291a8d700 1 -- start start 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f4291a8d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c10d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f4291a8d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f428c0fcfb0 0x7f428c10dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f4291a8d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c10e340 con 0x7f428c0fbe40 2024-02-22T15:47:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f4291a8d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c10ebe0 con 0x7f428c0fcfb0 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f428b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c10d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f428b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c10d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42972/0 (socket says 172.21.15.19:42972) 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f428b7fe700 1 -- 172.21.15.19:0/1194740300 learned_addr learned my addr 172.21.15.19:0/1194740300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.198+0000 7f428affd700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f428c0fcfb0 0x7f428c10dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f428b7fe700 1 -- 172.21.15.19:0/1194740300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f428c0fcfb0 msgr2=0x7f428c10dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f428b7fe700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f428c0fcfb0 0x7f428c10dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f428b7fe700 1 -- 172.21.15.19:0/1194740300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c005040 con 0x7f428c0fbe40 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f428b7fe700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c10d800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f427c021070 con 0x7f428c0fbe40 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f427c01c400 con 0x7f428c0fbe40 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c10ee60 con 0x7f428c0fbe40 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f427c00e070 con 0x7f428c0fbe40 2024-02-22T15:47:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.199+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c10f340 con 0x7f428c0fbe40 2024-02-22T15:47:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.200+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c04bed0 con 0x7f428c0fbe40 2024-02-22T15:47:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.201+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f427c01c9c0 con 0x7f428c0fbe40 2024-02-22T15:47:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.201+0000 7f4283fff700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f42780692a0 0x7f427806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.201+0000 7f428affd700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f42780692a0 0x7f427806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:29.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.201+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f427c088290 con 0x7f428c0fbe40 2024-02-22T15:47:29.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.201+0000 7f428affd700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f42780692a0 0x7f427806b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4274009b80 tx=0x7f427400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:29.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.206+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f427c0580f0 con 0x7f428c0fbe40 2024-02-22T15:47:29.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:29.449+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f428c10f7f0 con 0x7f428c0fbe40 2024-02-22T15:47:29.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:29 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6826/2235854865,v1:172.21.15.19:6827/2235854865] up:active 2024-02-22T15:47:29.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:29 smithi062 ceph-mon[48130]: fsmap cephfs:1 {0=cephfs.smithi019.lkzblb=up:active} 2024-02-22T15:47:29.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:29 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6826/2235854865,v1:172.21.15.19:6827/2235854865] up:active 2024-02-22T15:47:29.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:29 smithi019 ceph-mon[67418]: fsmap cephfs:1 {0=cephfs.smithi019.lkzblb=up:active} 2024-02-22T15:47:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.236+0000 7f4283fff700 1 -- 172.21.15.19:0/1194740300 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f427c057c80 con 0x7f428c0fbe40 2024-02-22T15:47:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.238+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f42780692a0 msgr2=0x7f427806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.238+0000 7f4291a8d700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f42780692a0 0x7f427806b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4274009b80 tx=0x7f427400b040 comp rx=0 tx=0).stop 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.238+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 msgr2=0x7f428c10d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.238+0000 7f4291a8d700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c10d800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f427c000c00 tx=0x7f427c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 shutdown_connections 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f42780692a0 0x7f427806b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f428c0fbe40 0x7f428c10d800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 --2- 172.21.15.19:0/1194740300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f428c0fcfb0 0x7f428c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 >> 172.21.15.19:0/1194740300 conn(0x7f428c0f74e0 msgr2=0x7f428c0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 shutdown_connections 2024-02-22T15:47:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:30.239+0000 7f4291a8d700 1 -- 172.21.15.19:0/1194740300 wait complete. 2024-02-22T15:47:30.387 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-02-22T15:47:30.387 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:30.387 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:30.387 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:30.387 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:30.387 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:30.388 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:30.388 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:30 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1194740300' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-22T15:47:30.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-02-22T15:47:30.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:30.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:30.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:30.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:30.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:30.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:30.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:30 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1194740300' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: Deploying daemon mds.cephfs.smithi019.fesnwx on smithi019 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] up:boot 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1194740300' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: daemon mds.cephfs.smithi062.oyfagg assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-22T15:47:31.341 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-22T15:47:31.342 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: fsmap cephfs:1 {0=cephfs.smithi019.lkzblb=up:active} 1 up:standby 2024-02-22T15:47:31.342 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:47:31.342 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:creating} 2024-02-22T15:47:31.342 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:31 smithi019 ceph-mon[67418]: daemon mds.cephfs.smithi062.oyfagg is now active in filesystem cephfs as rank 1 2024-02-22T15:47:31.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: Deploying daemon mds.cephfs.smithi019.fesnwx on smithi019 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] up:boot 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1194740300' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: daemon mds.cephfs.smithi062.oyfagg assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: fsmap cephfs:1 {0=cephfs.smithi019.lkzblb=up:active} 1 up:standby 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:creating} 2024-02-22T15:47:31.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:31 smithi062 ceph-mon[48130]: daemon mds.cephfs.smithi062.oyfagg is now active in filesystem cephfs as rank 1 2024-02-22T15:47:31.901 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:47:31.911 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:47:31.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-02-22T15:47:32.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:32 smithi019 ceph-mon[67418]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 6 op/s 2024-02-22T15:47:32.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:32 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] up:active 2024-02-22T15:47:32.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:32 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 2024-02-22T15:47:32.575 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:32.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:32 smithi062 ceph-mon[48130]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 6 op/s 2024-02-22T15:47:32.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:32 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] up:active 2024-02-22T15:47:32.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:32 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: Deploying daemon mds.cephfs.smithi062.eyhruo on smithi062 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-02-22T15:47:33.720 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-22T15:47:33.721 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: Cluster is now healthy 2024-02-22T15:47:33.721 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:boot 2024-02-22T15:47:33.721 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby 2024-02-22T15:47:33.721 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:33 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:47:33.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:33.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:33.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:33.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: Deploying daemon mds.cephfs.smithi062.eyhruo on smithi062 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: Cluster is now healthy 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:boot 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby 2024-02-22T15:47:33.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:33 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 -- 172.21.15.19:0/143237263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 msgr2=0x7ff9200fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 --2- 172.21.15.19:0/143237263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff9200fd430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff914004940 tx=0x7ff914009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 -- 172.21.15.19:0/143237263 shutdown_connections 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 --2- 172.21.15.19:0/143237263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff9200fd430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 --2- 172.21.15.19:0/143237263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9200fbe50 0x7ff9200fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 -- 172.21.15.19:0/143237263 >> 172.21.15.19:0/143237263 conn(0x7ff9200f7510 msgr2=0x7ff9200f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 -- 172.21.15.19:0/143237263 shutdown_connections 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.305+0000 7ff924c8b700 1 -- 172.21.15.19:0/143237263 wait complete. 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff924c8b700 1 Processor -- start 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff924c8b700 1 -- start start 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff924c8b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9200fbe50 0x7ff92010c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff924c8b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff92010cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff924c8b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92010d340 con 0x7ff9200fcfc0 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff924c8b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92010dbe0 con 0x7ff9200fbe50 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff91dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff92010cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff91dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff92010cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43018/0 (socket says 172.21.15.19:43018) 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff91dd9b700 1 -- 172.21.15.19:0/1798422751 learned_addr learned my addr 172.21.15.19:0/1798422751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.306+0000 7ff91e59c700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9200fbe50 0x7ff92010c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff91dd9b700 1 -- 172.21.15.19:0/1798422751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9200fbe50 msgr2=0x7ff92010c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff91dd9b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9200fbe50 0x7ff92010c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff91dd9b700 1 -- 172.21.15.19:0/1798422751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914005040 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff91dd9b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff92010cd40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff914009360 tx=0x7ff914009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91400e040 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff91400bc50 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92010de60 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91401c3f0 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.307+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92010e3a0 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.308+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92004bed0 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.309+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff914013420 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.309+0000 7ff90effd700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff9080692a0 0x7ff90806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.309+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff914088530 con 0x7ff9200fcfc0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.309+0000 7ff91e59c700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff9080692a0 0x7ff90806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.310+0000 7ff91e59c700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff9080692a0 0x7ff90806b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff9100041d0 tx=0x7ff9100043c0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.314+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff914058270 con 0x7ff9200fcfc0 2024-02-22T15:47:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.550+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7ff920104800 con 0x7ff9200fcfc0 2024-02-22T15:47:34.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.996+0000 7ff90effd700 1 -- 172.21.15.19:0/1798422751 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v10) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff920104800 con 0x7ff9200fcfc0 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff9080692a0 msgr2=0x7ff90806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff9080692a0 0x7ff90806b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff9100041d0 tx=0x7ff9100043c0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 msgr2=0x7ff92010cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff92010cd40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff914009360 tx=0x7ff914009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 shutdown_connections 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7ff9080692a0 0x7ff90806b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9200fbe50 0x7ff92010c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 --2- 172.21.15.19:0/1798422751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9200fcfc0 0x7ff92010cd40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 >> 172.21.15.19:0/1798422751 conn(0x7ff9200f7510 msgr2=0x7ff920100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 shutdown_connections 2024-02-22T15:47:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:34.999+0000 7ff924c8b700 1 -- 172.21.15.19:0/1798422751 wait complete. 2024-02-22T15:47:35.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:34 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1798422751' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-22T15:47:35.289 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:34 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1798422751' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-22T15:47:35.755 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:47:35.765 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:47:35.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-02-22T15:47:35.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-22T15:47:35.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1798422751' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-22T15:47:35.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby 2024-02-22T15:47:35.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 2024-02-22T15:47:35.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:35.997 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:35 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1798422751' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.293 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.294 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.294 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:36.294 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:36.294 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:36.687 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:37.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:37 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] up:boot 2024-02-22T15:47:37.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:37 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:37.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:37 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:47:37.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:37 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:47:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:37 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] up:boot 2024-02-22T15:47:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:37 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:37 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:47:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:37 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:47:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-02-22T15:47:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:standby-replay 2024-02-22T15:47:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: Dropping low affinity active daemon mds.cephfs.smithi019.lkzblb in favor of higher affinity standby. 2024-02-22T15:47:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: Replacing daemon mds.cephfs.smithi019.lkzblb as rank 0 with standby daemon mds.cephfs.smithi019.fesnwx 2024-02-22T15:47:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-22T15:47:38.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:47:38.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: osdmap e47: 6 total, 6 up, 6 in 2024-02-22T15:47:38.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:replay,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 2024-02-22T15:47:38.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:38.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:38 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:38.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-02-22T15:47:38.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:standby-replay 2024-02-22T15:47:38.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: Dropping low affinity active daemon mds.cephfs.smithi019.lkzblb in favor of higher affinity standby. 2024-02-22T15:47:38.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: Replacing daemon mds.cephfs.smithi019.lkzblb as rank 0 with standby daemon mds.cephfs.smithi019.fesnwx 2024-02-22T15:47:38.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-22T15:47:38.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:47:38.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: osdmap e47: 6 total, 6 up, 6 in 2024-02-22T15:47:38.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:replay,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 2024-02-22T15:47:38.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:38.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:38 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 -- 172.21.15.19:0/3054453589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 msgr2=0x7fedb0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 --2- 172.21.15.19:0/3054453589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb0102780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed9c004940 tx=0x7fed9c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 -- 172.21.15.19:0/3054453589 shutdown_connections 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 --2- 172.21.15.19:0/3054453589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb0102780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 --2- 172.21.15.19:0/3054453589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb00fda80 0x7fedb00ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 -- 172.21.15.19:0/3054453589 >> 172.21.15.19:0/3054453589 conn(0x7fedb00f74e0 msgr2=0x7fedb00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 -- 172.21.15.19:0/3054453589 shutdown_connections 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.828+0000 7fedb6e48700 1 -- 172.21.15.19:0/3054453589 wait complete. 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb6e48700 1 Processor -- start 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb6e48700 1 -- start start 2024-02-22T15:47:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb6e48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb00fda80 0x7fedb010d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb6e48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb010ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb6e48700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb010e3c0 con 0x7fedb01003a0 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb6e48700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb010ec60 con 0x7fedb00fda80 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedaffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb010ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedaffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb010ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48578/0 (socket says 172.21.15.19:48578) 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedaffff700 1 -- 172.21.15.19:0/4202374987 learned_addr learned my addr 172.21.15.19:0/4202374987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedb4be4700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb00fda80 0x7fedb010d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedaffff700 1 -- 172.21.15.19:0/4202374987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb00fda80 msgr2=0x7fedb010d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedaffff700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb00fda80 0x7fedb010d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.829+0000 7fedaffff700 1 -- 172.21.15.19:0/4202374987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c005040 con 0x7fedb01003a0 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.830+0000 7fedaffff700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb010ddc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009360 tx=0x7fed9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.830+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9c00e040 con 0x7fedb01003a0 2024-02-22T15:47:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.830+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb010eee0 con 0x7fedb01003a0 2024-02-22T15:47:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.830+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed9c00bc50 con 0x7fedb01003a0 2024-02-22T15:47:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.830+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9c01c3f0 con 0x7fedb01003a0 2024-02-22T15:47:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.830+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb010f420 con 0x7fedb01003a0 2024-02-22T15:47:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.831+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb0107910 con 0x7fedb01003a0 2024-02-22T15:47:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.832+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fed9c013420 con 0x7fedb01003a0 2024-02-22T15:47:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.832+0000 7fedad7fa700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feda00692a0 0x7feda006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.832+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fed9c089030 con 0x7fedb01003a0 2024-02-22T15:47:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.832+0000 7fedb4be4700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feda00692a0 0x7feda006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.833+0000 7fedb4be4700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feda00692a0 0x7feda006b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feda4004f40 tx=0x7feda400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:38.836+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fed9c058da0 con 0x7fedb01003a0 2024-02-22T15:47:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:39.074+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fedb01058d0 con 0x7fedb01003a0 2024-02-22T15:47:39.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:39 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] up:boot 2024-02-22T15:47:39.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:39 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:resolve 2024-02-22T15:47:39.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:39 smithi062 ceph-mon[48130]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:resolve,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:39.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:39 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:47:39.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:39 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] up:boot 2024-02-22T15:47:39.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:39 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:resolve 2024-02-22T15:47:39.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:39 smithi019 ceph-mon[67418]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:resolve,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:39.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:39 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:47:40.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.028+0000 7fedad7fa700 1 -- 172.21.15.19:0/4202374987 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7fed9c058930 con 0x7fedb01003a0 2024-02-22T15:47:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feda00692a0 msgr2=0x7feda006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feda00692a0 0x7feda006b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feda4004f40 tx=0x7feda400b040 comp rx=0 tx=0).stop 2024-02-22T15:47:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 msgr2=0x7fedb010ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb010ddc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009360 tx=0x7fed9c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 shutdown_connections 2024-02-22T15:47:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7feda00692a0 0x7feda006b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedb00fda80 0x7fedb010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 --2- 172.21.15.19:0/4202374987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb01003a0 0x7fedb010ddc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 >> 172.21.15.19:0/4202374987 conn(0x7fedb00f74e0 msgr2=0x7fedb00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 shutdown_connections 2024-02-22T15:47:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:40.030+0000 7fedb6e48700 1 -- 172.21.15.19:0/4202374987 wait complete. 2024-02-22T15:47:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:inline data disabled 2024-02-22T15:47:40.437 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:40 smithi019 ceph-mon[67418]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-22T15:47:40.437 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:40 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:reconnect 2024-02-22T15:47:40.437 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:40 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] up:active 2024-02-22T15:47:40.437 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:40 smithi019 ceph-mon[67418]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:reconnect,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:40.437 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:40 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/4202374987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-22T15:47:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:40 smithi062 ceph-mon[48130]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-02-22T15:47:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:40 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:reconnect 2024-02-22T15:47:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:40 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] up:active 2024-02-22T15:47:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:40 smithi062 ceph-mon[48130]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:reconnect,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:40 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/4202374987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-22T15:47:40.955 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T15:47:40.965 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:47:40.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-22T15:47:41.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:41 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:rejoin 2024-02-22T15:47:41.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:41 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/4202374987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-22T15:47:41.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:41 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] up:standby-replay 2024-02-22T15:47:41.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:41 smithi019 ceph-mon[67418]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:rejoin,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:41.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:41 smithi019 ceph-mon[67418]: daemon mds.cephfs.smithi019.fesnwx is now active in filesystem cephfs as rank 0 2024-02-22T15:47:41.330 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:41.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:41 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:rejoin 2024-02-22T15:47:41.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:41 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/4202374987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-22T15:47:41.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:41 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] up:standby-replay 2024-02-22T15:47:41.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:41 smithi062 ceph-mon[48130]: fsmap cephfs:2/2 {0=cephfs.smithi019.fesnwx=up:rejoin,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:41.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:41 smithi062 ceph-mon[48130]: daemon mds.cephfs.smithi019.fesnwx is now active in filesystem cephfs as rank 0 2024-02-22T15:47:42.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:42 smithi019 ceph-mon[67418]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:47:42.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:42 smithi019 ceph-mon[67418]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-22T15:47:42.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:42 smithi019 ceph-mon[67418]: Cluster is now healthy 2024-02-22T15:47:42.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:42 smithi019 ceph-mon[67418]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:active 2024-02-22T15:47:42.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:42 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:42.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:42 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:47:42.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:42 smithi062 ceph-mon[48130]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:47:42.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:42 smithi062 ceph-mon[48130]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-22T15:47:42.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:42 smithi062 ceph-mon[48130]: Cluster is now healthy 2024-02-22T15:47:42.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:42 smithi062 ceph-mon[48130]: mds.? [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] up:active 2024-02-22T15:47:42.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:42 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 1 up:standby-replay 1 up:standby 2024-02-22T15:47:42.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:42 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:47:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 -- 172.21.15.19:0/78389252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fe0e0 msgr2=0x7f45700fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 --2- 172.21.15.19:0/78389252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fe0e0 0x7f45700fe550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4564004940 tx=0x7f4564009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 -- 172.21.15.19:0/78389252 shutdown_connections 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 --2- 172.21.15.19:0/78389252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fe0e0 0x7f45700fe550 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 --2- 172.21.15.19:0/78389252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45700fcef0 0x7f45700fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 -- 172.21.15.19:0/78389252 >> 172.21.15.19:0/78389252 conn(0x7f45700f8620 msgr2=0x7f45700faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 -- 172.21.15.19:0/78389252 shutdown_connections 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 -- 172.21.15.19:0/78389252 wait complete. 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.639+0000 7f4576562700 1 Processor -- start 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f4576562700 1 -- start start 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f4576562700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 0x7f457010d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f4576562700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45700fe0e0 0x7f457010dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:43.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f4576562700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457010e390 con 0x7f45700fcef0 2024-02-22T15:47:43.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f4576562700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457010ec30 con 0x7f45700fe0e0 2024-02-22T15:47:43.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 0x7f457010d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 0x7f457010d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48596/0 (socket says 172.21.15.19:48596) 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456ffff700 1 -- 172.21.15.19:0/983621457 learned_addr learned my addr 172.21.15.19:0/983621457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456f7fe700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45700fe0e0 0x7f457010dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456ffff700 1 -- 172.21.15.19:0/983621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45700fe0e0 msgr2=0x7f457010dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456ffff700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45700fe0e0 0x7f457010dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.640+0000 7f456ffff700 1 -- 172.21.15.19:0/983621457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4564005040 con 0x7f45700fcef0 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.641+0000 7f456ffff700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 0x7f457010d850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4560002a80 tx=0x7f4560004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.641+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4560007cc0 con 0x7f45700fcef0 2024-02-22T15:47:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.641+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f456000f040 con 0x7f45700fcef0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.641+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457010ef10 con 0x7f45700fcef0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.641+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456000d9c0 con 0x7f45700fcef0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.641+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457010f450 con 0x7f45700fcef0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.642+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457004bed0 con 0x7f45700fcef0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.642+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f456000db20 con 0x7f45700fcef0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.643+0000 7f456cff9700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f45580692a0 0x7f455806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.643+0000 7f456f7fe700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f45580692a0 0x7f455806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.643+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4560086ec0 con 0x7f45700fcef0 2024-02-22T15:47:43.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.643+0000 7f456f7fe700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f45580692a0 0x7f455806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f456400b3b0 tx=0x7f4564009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:43.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.648+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4560056b90 con 0x7f45700fcef0 2024-02-22T15:47:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.893+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f457010f880 con 0x7f45700fcef0 2024-02-22T15:47:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.894+0000 7f456cff9700 1 -- 172.21.15.19:0/983621457 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f4560056720 con 0x7f45700fcef0 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:e20 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 20 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:47:43.877 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:47:41.037207+0000 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:47:43.878 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:43.879 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f45580692a0 msgr2=0x7f455806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f45580692a0 0x7f455806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f456400b3b0 tx=0x7f4564009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 msgr2=0x7f457010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 0x7f457010d850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4560002a80 tx=0x7f4560004600 comp rx=0 tx=0).stop 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 shutdown_connections 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f45580692a0 0x7f455806b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45700fcef0 0x7f457010d850 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:43.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 --2- 172.21.15.19:0/983621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45700fe0e0 0x7f457010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 >> 172.21.15.19:0/983621457 conn(0x7f45700f8620 msgr2=0x7f45701013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 shutdown_connections 2024-02-22T15:47:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:43.896+0000 7f4576562700 1 -- 172.21.15.19:0/983621457 wait complete. 2024-02-22T15:47:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-02-22T15:47:44.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 409 B/s wr, 7 op/s 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:44.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:44 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/983621457' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:47:44.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 409 B/s wr, 7 op/s 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:47:44.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:44 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/983621457' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:47:44.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-22T15:47:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:46 smithi062 ceph-mon[48130]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-02-22T15:47:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:46.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:46 smithi019 ceph-mon[67418]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-02-22T15:47:46.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:46.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:47.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 -- 172.21.15.19:0/1848038113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 msgr2=0x7f9c2c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/1848038113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c0fd320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c004940 tx=0x7f9c1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 -- 172.21.15.19:0/1848038113 shutdown_connections 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/1848038113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c0fe0e0 0x7f9c2c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/1848038113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c0fd320 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 -- 172.21.15.19:0/1848038113 >> 172.21.15.19:0/1848038113 conn(0x7f9c2c0f8620 msgr2=0x7f9c2c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 -- 172.21.15.19:0/1848038113 shutdown_connections 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.807+0000 7f9c30c0f700 1 -- 172.21.15.19:0/1848038113 wait complete. 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c30c0f700 1 Processor -- start 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c30c0f700 1 -- start start 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c30c0f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c30c0f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c0fe0e0 0x7f9c2c10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c30c0f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c10e480 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c30c0f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c10ecc0 con 0x7f9c2c0fe0e0 2024-02-22T15:47:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c10d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39798/0 (socket says 172.21.15.19:39798) 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 -- 172.21.15.19:0/2199108328 learned_addr learned my addr 172.21.15.19:0/2199108328 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c29d9b700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c0fe0e0 0x7f9c2c10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 -- 172.21.15.19:0/2199108328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c0fe0e0 msgr2=0x7f9c2c10ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c0fe0e0 0x7f9c2c10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 -- 172.21.15.19:0/2199108328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c005040 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.808+0000 7f9c2a59c700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c10d8b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009fd0 tx=0x7f9c1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.809+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1c00e040 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.809+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c10ef40 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.809+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c1c00bc50 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.809+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1c01c3f0 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.809+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c10f480 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.810+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c04bed0 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.810+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9c1c013420 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.810+0000 7f9c22ffd700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f9c180692a0 0x7f9c1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:47.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.810+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9c1c0886e0 con 0x7f9c2c0fcef0 2024-02-22T15:47:47.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.811+0000 7f9c29d9b700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f9c180692a0 0x7f9c1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:47.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.811+0000 7f9c29d9b700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f9c180692a0 0x7f9c1806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006fd0 tx=0x7f9c14008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:47.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:47.815+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9c1c058450 con 0x7f9c2c0fcef0 2024-02-22T15:47:48.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.057+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c2c10f860 con 0x7f9c2c0fcef0 2024-02-22T15:47:48.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.057+0000 7f9c22ffd700 1 -- 172.21.15.19:0/2199108328 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4716 (secure 0 0 0) 0x7f9c1c057fe0 con 0x7f9c2c0fcef0 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f9c180692a0 msgr2=0x7f9c1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f9c180692a0 0x7f9c1806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006fd0 tx=0x7f9c14008040 comp rx=0 tx=0).stop 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 msgr2=0x7f9c2c10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c10d8b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009fd0 tx=0x7f9c1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 shutdown_connections 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f9c180692a0 0x7f9c1806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2c0fcef0 0x7f9c2c10d8b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 --2- 172.21.15.19:0/2199108328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c2c0fe0e0 0x7f9c2c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 >> 172.21.15.19:0/2199108328 conn(0x7f9c2c0f8620 msgr2=0x7f9c2c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 shutdown_connections 2024-02-22T15:47:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:48.059+0000 7f9c30c0f700 1 -- 172.21.15.19:0/2199108328 wait complete. 2024-02-22T15:47:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-02-22T15:47:48.052 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:47:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:48 smithi019 ceph-mon[67418]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-22T15:47:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:48 smithi062 ceph-mon[48130]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-22T15:47:49.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:49 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2199108328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-22T15:47:49.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-22T15:47:49.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:49 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2199108328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-22T15:47:49.885 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:50.347 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:50 smithi019 ceph-mon[67418]: pgmap v199: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-22T15:47:50.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:50 smithi062 ceph-mon[48130]: pgmap v199: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-22T15:47:51.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:51.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:47:52.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 -- 172.21.15.19:0/3513000933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 msgr2=0x7f8c5c0fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:52.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3513000933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c0fb630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c004940 tx=0x7f8c4c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:52.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 -- 172.21.15.19:0/3513000933 shutdown_connections 2024-02-22T15:47:52.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3513000933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c5c0fe2b0 0x7f8c5c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:52.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3513000933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c0fb630 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:52.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 -- 172.21.15.19:0/3513000933 >> 172.21.15.19:0/3513000933 conn(0x7f8c5c0f6ca0 msgr2=0x7f8c5c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:52.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 -- 172.21.15.19:0/3513000933 shutdown_connections 2024-02-22T15:47:52.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.169+0000 7f8c6295e700 1 -- 172.21.15.19:0/3513000933 wait complete. 2024-02-22T15:47:52.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c6295e700 1 Processor -- start 2024-02-22T15:47:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c6295e700 1 -- start start 2024-02-22T15:47:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c6295e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c6295e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c5c0fe2b0 0x7f8c5c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c6295e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c10e460 con 0x7f8c5c0fb220 2024-02-22T15:47:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c6295e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c10ed00 con 0x7f8c5c0fe2b0 2024-02-22T15:47:52.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39804/0 (socket says 172.21.15.19:39804) 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5bfff700 1 -- 172.21.15.19:0/3401831792 learned_addr learned my addr 172.21.15.19:0/3401831792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5b7fe700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c5c0fe2b0 0x7f8c5c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5bfff700 1 -- 172.21.15.19:0/3401831792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c5c0fe2b0 msgr2=0x7f8c5c10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5bfff700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c5c0fe2b0 0x7f8c5c10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.170+0000 7f8c5bfff700 1 -- 172.21.15.19:0/3401831792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c005040 con 0x7f8c5c0fb220 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.171+0000 7f8c5bfff700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c10d920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009180 tx=0x7f8c4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.171+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4c00e040 con 0x7f8c5c0fb220 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.171+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c4c004290 con 0x7f8c5c0fb220 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.171+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c10ef80 con 0x7f8c5c0fb220 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.171+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4c01c3f0 con 0x7f8c5c0fb220 2024-02-22T15:47:52.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.171+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c10f4c0 con 0x7f8c5c0fb220 2024-02-22T15:47:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.172+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8c4c00bb80 con 0x7f8c5c0fb220 2024-02-22T15:47:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.173+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c107940 con 0x7f8c5c0fb220 2024-02-22T15:47:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.173+0000 7f8c58ff9700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8c440692b0 0x7f8c4406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.173+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8c4c0885c0 con 0x7f8c5c0fb220 2024-02-22T15:47:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.173+0000 7f8c5b7fe700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8c440692b0 0x7f8c4406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.173+0000 7f8c5b7fe700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8c440692b0 0x7f8c4406b760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c50004f40 tx=0x7f8c5000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.178+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8c4c058290 con 0x7f8c5c0fb220 2024-02-22T15:47:52.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:52 smithi019 ceph-mon[67418]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-22T15:47:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.454+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f8c5c02b2a0 con 0x7f8c5c0fb220 2024-02-22T15:47:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.454+0000 7f8c58ff9700 1 -- 172.21.15.19:0/3401831792 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v20) v1 ==== 78+0+99 (secure 0 0 0) 0x7f8c4c057e20 con 0x7f8c5c0fb220 2024-02-22T15:47:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8c440692b0 msgr2=0x7f8c4406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8c440692b0 0x7f8c4406b760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c50004f40 tx=0x7f8c5000b040 comp rx=0 tx=0).stop 2024-02-22T15:47:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 msgr2=0x7f8c5c10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c10d920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009180 tx=0x7f8c4c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 shutdown_connections 2024-02-22T15:47:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f8c440692b0 0x7f8c4406b760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.456+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c5c0fb220 0x7f8c5c10d920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.457+0000 7f8c6295e700 1 --2- 172.21.15.19:0/3401831792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c5c0fe2b0 0x7f8c5c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.457+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 >> 172.21.15.19:0/3401831792 conn(0x7f8c5c0f6ca0 msgr2=0x7f8c5c0f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.457+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 shutdown_connections 2024-02-22T15:47:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:52.457+0000 7f8c6295e700 1 -- 172.21.15.19:0/3401831792 wait complete. 2024-02-22T15:47:52.450 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:47:52.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:52 smithi062 ceph-mon[48130]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-22T15:47:53.130 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-22T15:47:53.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-22T15:47:53.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 -- 172.21.15.19:0/1704682953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 msgr2=0x7f7d980fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 --2- 172.21.15.19:0/1704682953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980fe440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c004940 tx=0x7f7d8c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 -- 172.21.15.19:0/1704682953 shutdown_connections 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 --2- 172.21.15.19:0/1704682953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980fe440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 --2- 172.21.15.19:0/1704682953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d980fcde0 0x7f7d980fd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 -- 172.21.15.19:0/1704682953 >> 172.21.15.19:0/1704682953 conn(0x7f7d980f8480 msgr2=0x7f7d980fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 -- 172.21.15.19:0/1704682953 shutdown_connections 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 -- 172.21.15.19:0/1704682953 wait complete. 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 Processor -- start 2024-02-22T15:47:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.298+0000 7f7d9e361700 1 -- start start 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d9e361700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d980fcde0 0x7f7d98071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d9e361700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d9e361700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98072280 con 0x7f7d980fdff0 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d9e361700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98070850 con 0x7f7d980fcde0 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39814/0 (socket says 172.21.15.19:39814) 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 -- 172.21.15.19:0/2976181412 learned_addr learned my addr 172.21.15.19:0/2976181412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d97fff700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d980fcde0 0x7f7d98071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 -- 172.21.15.19:0/2976181412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d980fcde0 msgr2=0x7f7d98071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d980fcde0 0x7f7d98071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 -- 172.21.15.19:0/2976181412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c005040 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d977fe700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980702e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009fd0 tx=0x7f7d8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d8c00e040 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d98070ad0 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d8c004290 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d8c01c3f0 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.299+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d98071010 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.300+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d981076f0 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.301+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7d8c013420 con 0x7f7d980fdff0 2024-02-22T15:47:53.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.301+0000 7f7d94ff9700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7d800692b0 0x7f7d8006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.301+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7d8c088640 con 0x7f7d980fdff0 2024-02-22T15:47:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.301+0000 7f7d97fff700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7d800692b0 0x7f7d8006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.301+0000 7f7d97fff700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7d800692b0 0x7f7d8006b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d980fde50 tx=0x7f7d8800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.305+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7d8c058330 con 0x7f7d980fdff0 2024-02-22T15:47:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:53 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:53 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3401831792' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-22T15:47:53.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.500+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d98105780 con 0x7f7d980fdff0 2024-02-22T15:47:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.501+0000 7f7d94ff9700 1 -- 172.21.15.19:0/2976181412 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14347 (secure 0 0 0) 0x7f7d8c054020 con 0x7f7d980fdff0 2024-02-22T15:47:53.484 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:53.484 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":47,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:47:37.015470+0000","last_up_change":"2024-02-22T15:45:59.303021+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-22T15:47:20.868526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-22T15:47:21.608692+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6803","nonce":3426164274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6805","nonce":3426164274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6809","nonce":3426164274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3426164274},{"type":"v1","addr":"172.21.15.19:6807","nonce":3426164274}]},"public_addr":"172.21.15.19:6803/3426164274","cluster_addr":"172.21.15.19:6805/3426164274","heartbeat_back_addr":"172.21.15.19:6809/3426164274","heartbeat_front_addr":"172.21.15.19:6807/3426164274","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6811","nonce":1226754076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6813","nonce":1226754076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6817","nonce":1226754076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1226754076},{"type":"v1","addr":"172.21.15.19:6815","nonce":1226754076}]},"public_addr":"172.21.15.19:6811/1226754076","cluster_addr":"172.21.15.19:6813/1226754076","heartbeat_back_addr":"172.21.15.19:6817/1226754076","heartbeat_front_addr":"172.21.15.19:6815/1226754076","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6819","nonce":2801184587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6821","nonce":2801184587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6825","nonce":2801184587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2801184587},{"type":"v1","addr":"172.21.15.19:6823","nonce":2801184587}]},"public_addr":"172.21.15.19:6819/2801184587","cluster_addr":"172.21.15.19:6821/2801184587","heartbeat_back_addr":"172.21.15.19:6825/2801184587","heartbeat_front_addr":"172.21.15.19:6823/2801184587","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6801","nonce":179328076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6803","nonce":179328076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6807","nonce":179328076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":179328076},{"type":"v1","addr":"172.21.15.62:6805","nonce":179328076}]},"public_addr":"172.21.15.62:6801/179328076","cluster_addr":"172.21.15.62:6803/179328076","heartbeat_back_addr":"172.21.15.62:6807/179328076","heartbeat_front_addr":"172.21.15.62:6805/179328076","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6809","nonce":925300074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6811","nonce":925300074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6815","nonce":925300074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":925300074},{"type":"v1","addr":"172.21.15.62:6813","nonce":925300074}]},"public_addr":"172.21.15.62:6809/925300074","cluster_addr":"172.21.15.62:6811/925300074","heartbeat_back_addr":"172.21.15.62:6815/925300074","heartbeat_front_addr":"172.21.15.62:6813/925300074","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6817","nonce":3437650181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6819","nonce":3437650181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6823","nonce":3437650181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3437650181},{"type":"v1","addr":"172.21.15.62:6821","nonce":3437650181}]},"public_addr":"172.21.15.62:6817/3437650181","cluster_addr":"172.21.15.62:6819/3437650181","heartbeat_back_addr":"172.21.15.62:6823/3437650181","heartbeat_front_addr":"172.21.15.62:6821/3437650181","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6826/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6827/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7d800692b0 msgr2=0x7f7d8006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7d800692b0 0x7f7d8006b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d980fde50 tx=0x7f7d8800b040 comp rx=0 tx=0).stop 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 msgr2=0x7f7d980702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980702e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009fd0 tx=0x7f7d8c009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 shutdown_connections 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7d800692b0 0x7f7d8006b760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d980fcde0 0x7f7d98071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 --2- 172.21.15.19:0/2976181412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d980fdff0 0x7f7d980702e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 >> 172.21.15.19:0/2976181412 conn(0x7f7d980f8480 msgr2=0x7f7d980fa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 shutdown_connections 2024-02-22T15:47:53.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.504+0000 7f7d9e361700 1 -- 172.21.15.19:0/2976181412 wait complete. 2024-02-22T15:47:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:47:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:47:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:53 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:47:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:53 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3401831792' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-22T15:47:53.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-22T15:44:43.417715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-22T15:47:20.868526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-22T15:47:21.608692+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-22T15:47:53.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-22T15:47:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 -- 172.21.15.19:0/3650236082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fbcc0 msgr2=0x7f078c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 --2- 172.21.15.19:0/3650236082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fbcc0 0x7f078c0fc0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0780004940 tx=0x7f0780009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 -- 172.21.15.19:0/3650236082 shutdown_connections 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 --2- 172.21.15.19:0/3650236082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fced0 0x7f078c0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 --2- 172.21.15.19:0/3650236082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fbcc0 0x7f078c0fc0d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 -- 172.21.15.19:0/3650236082 >> 172.21.15.19:0/3650236082 conn(0x7f078c0f7360 msgr2=0x7f078c0f97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 -- 172.21.15.19:0/3650236082 shutdown_connections 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 -- 172.21.15.19:0/3650236082 wait complete. 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 Processor -- start 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.642+0000 7f079317b700 1 -- start start 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f079317b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fbcc0 0x7f078c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f079317b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 0x7f078c0fe210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f079317b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1001b0 con 0x7f078c0fced0 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f079317b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0fe780 con 0x7f078c0fbcc0 2024-02-22T15:47:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 0x7f078c0fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 0x7f078c0fe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39840/0 (socket says 172.21.15.19:39840) 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 -- 172.21.15.19:0/1204157998 learned_addr learned my addr 172.21.15.19:0/1204157998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f0790f17700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fbcc0 0x7f078c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 -- 172.21.15.19:0/1204157998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fbcc0 msgr2=0x7f078c0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fbcc0 0x7f078c0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 -- 172.21.15.19:0/1204157998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780005040 con 0x7f078c0fced0 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f0790f17700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fbcc0 0x7f078c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f078bfff700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 0x7f078c0fe210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0778002a50 tx=0x7f07780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0778007cc0 con 0x7f078c0fced0 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f077800f040 con 0x7f078c0fced0 2024-02-22T15:47:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c0fea60 con 0x7f078c0fced0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0778009540 con 0x7f078c0fced0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.643+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c0ff020 con 0x7f078c0fced0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.644+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c104660 con 0x7f078c0fced0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.645+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f07780096a0 con 0x7f078c0fced0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.645+0000 7f07897fa700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f077c0692b0 0x7f077c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.645+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0778086fc0 con 0x7f078c0fced0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.645+0000 7f0790f17700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f077c0692b0 0x7f077c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.646+0000 7f0790f17700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f077c0692b0 0x7f077c06b760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0780004f90 tx=0x7f0780009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.649+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0778056c90 con 0x7f078c0fced0 2024-02-22T15:47:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.844+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f078c115790 con 0x7f078c0fced0 2024-02-22T15:47:53.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.845+0000 7f07897fa700 1 -- 172.21.15.19:0/1204157998 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0778056820 con 0x7f078c0fced0 2024-02-22T15:47:53.839 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-02-22T15:47:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.846+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f077c0692b0 msgr2=0x7f077c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.846+0000 7f079317b700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f077c0692b0 0x7f077c06b760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0780004f90 tx=0x7f0780009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.846+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 msgr2=0x7f078c0fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.846+0000 7f079317b700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 0x7f078c0fe210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0778002a50 tx=0x7f07780046b0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 shutdown_connections 2024-02-22T15:47:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f077c0692b0 0x7f077c06b760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f078c0fbcc0 0x7f078c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 --2- 172.21.15.19:0/1204157998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f078c0fced0 0x7f078c0fe210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 >> 172.21.15.19:0/1204157998 conn(0x7f078c0f7360 msgr2=0x7f078c064ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 shutdown_connections 2024-02-22T15:47:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.847+0000 7f079317b700 1 -- 172.21.15.19:0/1204157998 wait complete. 2024-02-22T15:47:53.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.987+0000 7f86bb729700 1 -- 172.21.15.19:0/667384578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fbcd0 msgr2=0x7f86b40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.987+0000 7f86bb729700 1 --2- 172.21.15.19:0/667384578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fbcd0 0x7f86b40fc0e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f86b0004940 tx=0x7f86b0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.987+0000 7f86bb729700 1 -- 172.21.15.19:0/667384578 shutdown_connections 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.987+0000 7f86bb729700 1 --2- 172.21.15.19:0/667384578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b40fcee0 0x7f86b40fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.987+0000 7f86bb729700 1 --2- 172.21.15.19:0/667384578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fbcd0 0x7f86b40fc0e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.987+0000 7f86bb729700 1 -- 172.21.15.19:0/667384578 >> 172.21.15.19:0/667384578 conn(0x7f86b40f73b0 msgr2=0x7f86b40f97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 -- 172.21.15.19:0/667384578 shutdown_connections 2024-02-22T15:47:53.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 -- 172.21.15.19:0/667384578 wait complete. 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 Processor -- start 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 -- start start 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b40fbcd0 0x7f86b410c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 0x7f86b410ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b410d080 con 0x7f86b40fcee0 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86bb729700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b410d1f0 con 0x7f86b40fbcd0 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86b8cc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 0x7f86b410ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86b8cc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 0x7f86b410ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39856/0 (socket says 172.21.15.19:39856) 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.988+0000 7f86b8cc4700 1 -- 172.21.15.19:0/3966427850 learned_addr learned my addr 172.21.15.19:0/3966427850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86b94c5700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b40fbcd0 0x7f86b410c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86b8cc4700 1 -- 172.21.15.19:0/3966427850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b40fbcd0 msgr2=0x7f86b410c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86b8cc4700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b40fbcd0 0x7f86b410c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:53.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86b8cc4700 1 -- 172.21.15.19:0/3966427850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0005040 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86b8cc4700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 0x7f86b410ca80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86a400fce0 tx=0x7f86a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a400fe90 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86a40090d0 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b4065c80 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a40126a0 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.989+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b4066240 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.990+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b4104670 con 0x7f86b40fcee0 2024-02-22T15:47:53.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.990+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f86a4009240 con 0x7f86b40fcee0 2024-02-22T15:47:53.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.991+0000 7f86a9ffb700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86a00692b0 0x7f86a006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:53.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.991+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f86a4089980 con 0x7f86b40fcee0 2024-02-22T15:47:53.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.991+0000 7f86b94c5700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86a00692b0 0x7f86a006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:53.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.991+0000 7f86b94c5700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86a00692b0 0x7f86a006b760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f86b000b730 tx=0x7f86b0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:53.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:53.995+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f86a4059670 con 0x7f86b40fcee0 2024-02-22T15:47:54.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.188+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f86b4066670 con 0x7f86b40fcee0 2024-02-22T15:47:54.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.188+0000 7f86a9ffb700 1 -- 172.21.15.19:0/3966427850 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f86a4016020 con 0x7f86b40fcee0 2024-02-22T15:47:54.171 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T15:47:54.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86a00692b0 msgr2=0x7f86a006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86a00692b0 0x7f86a006b760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f86b000b730 tx=0x7f86b0009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:54.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 msgr2=0x7f86b410ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 0x7f86b410ca80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86a400fce0 tx=0x7f86a40046b0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 shutdown_connections 2024-02-22T15:47:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f86a00692b0 0x7f86a006b760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86b40fbcd0 0x7f86b410c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 --2- 172.21.15.19:0/3966427850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b40fcee0 0x7f86b410ca80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 >> 172.21.15.19:0/3966427850 conn(0x7f86b40f73b0 msgr2=0x7f86b4100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 shutdown_connections 2024-02-22T15:47:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.191+0000 7f86bb729700 1 -- 172.21.15.19:0/3966427850 wait complete. 2024-02-22T15:47:54.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-22T15:47:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 -- 172.21.15.19:0/1153809818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fbcc0 msgr2=0x7f5d300fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/1153809818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fbcc0 0x7f5d300fc0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d24004940 tx=0x7f5d24009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 -- 172.21.15.19:0/1153809818 shutdown_connections 2024-02-22T15:47:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/1153809818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300fced0 0x7f5d300fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/1153809818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fbcc0 0x7f5d300fc0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 -- 172.21.15.19:0/1153809818 >> 172.21.15.19:0/1153809818 conn(0x7f5d300f7360 msgr2=0x7f5d300f97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.329+0000 7f5d36a8c700 1 -- 172.21.15.19:0/1153809818 shutdown_connections 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 -- 172.21.15.19:0/1153809818 wait complete. 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 Processor -- start 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 -- start start 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 0x7f5d3010db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3010e130 con 0x7f5d300fced0 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d36a8c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3010e2a0 con 0x7f5d300fbcc0 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 0x7f5d3010db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d2ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 0x7f5d3010db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39878/0 (socket says 172.21.15.19:39878) 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.330+0000 7f5d2ffff700 1 -- 172.21.15.19:0/3802059683 learned_addr learned my addr 172.21.15.19:0/3802059683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d34828700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2ffff700 1 -- 172.21.15.19:0/3802059683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300fbcc0 msgr2=0x7f5d3010d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2ffff700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2ffff700 1 -- 172.21.15.19:0/3802059683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24005040 con 0x7f5d300fced0 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2ffff700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 0x7f5d3010db30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c830 tx=0x7f5d1c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1c00a760 con 0x7f5d300fced0 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d1c00ad70 con 0x7f5d300fced0 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1c0113a0 con 0x7f5d300fced0 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d30112d50 con 0x7f5d300fced0 2024-02-22T15:47:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.331+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d30113310 con 0x7f5d300fced0 2024-02-22T15:47:54.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.332+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30104660 con 0x7f5d300fced0 2024-02-22T15:47:54.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.332+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5d1c012400 con 0x7f5d300fced0 2024-02-22T15:47:54.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.333+0000 7f5d2d7fa700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5d200692a0 0x7f5d2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:54.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.333+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5d1c087270 con 0x7f5d300fced0 2024-02-22T15:47:54.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.333+0000 7f5d34828700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5d200692a0 0x7f5d2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:54.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.333+0000 7f5d34828700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5d200692a0 0x7f5d2006b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009fd0 tx=0x7f5d24009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.337+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5d1c056fe0 con 0x7f5d300fced0 2024-02-22T15:47:54.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:54 smithi062 ceph-mon[48130]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-22T15:47:54.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:54 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2976181412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:47:54.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:54 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1204157998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T15:47:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:54 smithi019 ceph-mon[67418]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-22T15:47:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:54 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2976181412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T15:47:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:54 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1204157998' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T15:47:54.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.530+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f5d301137a0 con 0x7f5d300fced0 2024-02-22T15:47:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.530+0000 7f5d2d7fa700 1 -- 172.21.15.19:0/3802059683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5d1c056b70 con 0x7f5d300fced0 2024-02-22T15:47:54.513 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5d200692a0 msgr2=0x7f5d2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5d200692a0 0x7f5d2006b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009fd0 tx=0x7f5d24009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 msgr2=0x7f5d3010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 0x7f5d3010db30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c830 tx=0x7f5d1c004470 comp rx=0 tx=0).stop 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 shutdown_connections 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5d200692a0 0x7f5d2006b750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d300fbcc0 0x7f5d3010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 --2- 172.21.15.19:0/3802059683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d300fced0 0x7f5d3010db30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 >> 172.21.15.19:0/3802059683 conn(0x7f5d300f7360 msgr2=0x7f5d30064ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 shutdown_connections 2024-02-22T15:47:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.533+0000 7f5d36a8c700 1 -- 172.21.15.19:0/3802059683 wait complete. 2024-02-22T15:47:54.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-22T15:47:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/4049771537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 msgr2=0x7fbeb00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/4049771537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb00fc0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbeac004940 tx=0x7fbeac009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/4049771537 shutdown_connections 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/4049771537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb00fcee0 0x7fbeb00fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/4049771537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb00fc0e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/4049771537 >> 172.21.15.19:0/4049771537 conn(0x7fbeb00f73b0 msgr2=0x7fbeb00f97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/4049771537 shutdown_connections 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.667+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/4049771537 wait complete. 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb7f4f700 1 Processor -- start 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb7f4f700 1 -- start start 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb7f4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb010c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb7f4f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb00fcee0 0x7fbeb010c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb7f4f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb010cfd0 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb7f4f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb010d140 con 0x7fbeb00fcee0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb010c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb010c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39886/0 (socket says 172.21.15.19:39886) 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 -- 172.21.15.19:0/3644451762 learned_addr learned my addr 172.21.15.19:0/3644451762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb54ea700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb00fcee0 0x7fbeb010c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 -- 172.21.15.19:0/3644451762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb00fcee0 msgr2=0x7fbeb010c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb00fcee0 0x7fbeb010c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 -- 172.21.15.19:0/3644451762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac005040 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbeb5ceb700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb010c490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbeac008f80 tx=0x7fbeac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.668+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeac021070 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.669+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbeac01c400 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.669+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeac00e070 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.669+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb0111b90 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.669+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb0112070 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.670+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0104670 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.670+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbeac01c9c0 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.670+0000 7fbea27fc700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbe9c0692b0 0x7fbe9c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.670+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbeac0883d0 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.670+0000 7fbeb54ea700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbe9c0692b0 0x7fbe9c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.671+0000 7fbeb54ea700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbe9c0692b0 0x7fbe9c06b760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbea4004600 tx=0x7fbea400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.675+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbeac0581e0 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.915+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeb0112330 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.915+0000 7fbea27fc700 1 -- 172.21.15.19:0/3644451762 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4716 (secure 0 0 0) 0x7fbeac057d70 con 0x7fbeb00fbcd0 2024-02-22T15:47:54.898 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:47:54.898 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:47:41.037207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi019.fesnwx","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6829/3100392983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3100392983},{"type":"v1","addr":"172.21.15.19:6829","nonce":3100392983}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6827/588513461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":588513461},{"type":"v1","addr":"172.21.15.19:6827","nonce":588513461}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi062.oyfagg","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6825/1888029196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1888029196},{"type":"v1","addr":"172.21.15.62:6825","nonce":1888029196}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi062.eyhruo","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.62:6827/3228927024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3228927024},{"type":"v1","addr":"172.21.15.62:6827","nonce":3228927024}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-22T15:47:54.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbe9c0692b0 msgr2=0x7fbe9c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbe9c0692b0 0x7fbe9c06b760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbea4004600 tx=0x7fbea400b040 comp rx=0 tx=0).stop 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 msgr2=0x7fbeb010c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb010c490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbeac008f80 tx=0x7fbeac009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 shutdown_connections 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fbe9c0692b0 0x7fbe9c06b760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbeb00fbcd0 0x7fbeb010c490 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 --2- 172.21.15.19:0/3644451762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbeb00fcee0 0x7fbeb010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 >> 172.21.15.19:0/3644451762 conn(0x7fbeb00f73b0 msgr2=0x7fbeb0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 shutdown_connections 2024-02-22T15:47:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:54.918+0000 7fbeb7f4f700 1 -- 172.21.15.19:0/3644451762 wait complete. 2024-02-22T15:47:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-02-22T15:47:54.913 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 20, 'max_mds': 2, 'flags': 50} 2024-02-22T15:47:54.913 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-22T15:47:54.938 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-22T15:47:54.938 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-02-22T15:47:54.938 INFO:tasks.ceph_fuse:client.0 config is {} 2024-02-22T15:47:54.938 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-22T15:47:54.938 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-22T15:47:54.938 INFO:tasks.ceph_fuse:client.1 config is {} 2024-02-22T15:47:54.939 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-22T15:47:54.939 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-22T15:47:54.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:54.939 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-02-22T15:47:54.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:54.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-02-22T15:47:55.043 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-02-22T15:47:55.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T15:47:55.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:55.045 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-02-22T15:47:55.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:55.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-02-22T15:47:55.133 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-02-22T15:47:55.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T15:47:55.135 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-22T15:47:55.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph fs ls 2024-02-22T15:47:55.201 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:55 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3966427850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T15:47:55.201 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:55 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3802059683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T15:47:55.202 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:55 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3644451762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-22T15:47:55.453 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:47:55.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:55 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3966427850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T15:47:55.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:55 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3802059683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T15:47:55.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:55 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3644451762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-22T15:47:56.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:56 smithi062 ceph-mon[48130]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-22T15:47:56.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:56 smithi019 ceph-mon[67418]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-22T15:47:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 -- 172.21.15.19:0/2231917939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fe8f0 msgr2=0x7fd25c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 --2- 172.21.15.19:0/2231917939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c107050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd258004970 tx=0x7fd258009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 -- 172.21.15.19:0/2231917939 shutdown_connections 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 --2- 172.21.15.19:0/2231917939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c107050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 --2- 172.21.15.19:0/2231917939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 -- 172.21.15.19:0/2231917939 >> 172.21.15.19:0/2231917939 conn(0x7fd25c071d90 msgr2=0x7fd25c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 -- 172.21.15.19:0/2231917939 shutdown_connections 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.579+0000 7fd26360c700 1 -- 172.21.15.19:0/2231917939 wait complete. 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd26360c700 1 Processor -- start 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd26360c700 1 -- start start 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd26360c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd26360c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd26360c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c114500 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd26360c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c10d830 con 0x7fd25c0fe8f0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53822/0 (socket says 172.21.15.19:53822) 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 -- 172.21.15.19:0/1835918776 learned_addr learned my addr 172.21.15.19:0/1835918776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd260ba7700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 -- 172.21.15.19:0/1835918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fe8f0 msgr2=0x7fd25c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 -- 172.21.15.19:0/1835918776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258005040 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd260ba7700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd2613a8700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c113970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00c830 tx=0x7fd24c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.580+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24c00a760 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.581+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c10db10 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.581+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd24c00ad70 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.581+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24c0113a0 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.581+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c10e050 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.582+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c1046c0 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.582+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd24c012400 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.582+0000 7fd251ffb700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd2480692a0 0x7fd24806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.582+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd24c087270 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.582+0000 7fd260ba7700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd2480692a0 0x7fd24806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.583+0000 7fd260ba7700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd2480692a0 0x7fd24806b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd258009fd0 tx=0x7fd258009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:47:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.587+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd24c056fe0 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.824+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd25c04bed0 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.825+0000 7fd251ffb700 1 -- 172.21.15.19:0/1835918776 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+83 (secure 0 0 0) 0x7fd24c056b70 con 0x7fd25c0fdfe0 2024-02-22T15:47:57.808 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-22T15:47:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd2480692a0 msgr2=0x7fd24806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd2480692a0 0x7fd24806b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd258009fd0 tx=0x7fd258009e30 comp rx=0 tx=0).stop 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 msgr2=0x7fd25c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c113970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00c830 tx=0x7fd24c004470 comp rx=0 tx=0).stop 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 shutdown_connections 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd2480692a0 0x7fd24806b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd25c0fdfe0 0x7fd25c113970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 --2- 172.21.15.19:0/1835918776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd25c0fe8f0 0x7fd25c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 >> 172.21.15.19:0/1835918776 conn(0x7fd25c071d90 msgr2=0x7fd25c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 shutdown_connections 2024-02-22T15:47:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:47:57.827+0000 7fd26360c700 1 -- 172.21.15.19:0/1835918776 wait complete. 2024-02-22T15:47:58.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:58 smithi062 ceph-mon[48130]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:47:58.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:47:58 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1835918776' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-22T15:47:58.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:58 smithi019 ceph-mon[67418]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:47:58.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:47:58 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1835918776' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-22T15:47:58.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-22T15:47:58.571 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-22T15:47:58.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:58.571 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-02-22T15:47:58.633 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-22T15:47:58.633 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-02-22T15:47:58.634 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 40304sec preferred_lft 40304sec 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-02-22T15:47:58.635 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-02-22T15:47:58.636 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-22T15:47:58.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-22T15:47:58.636 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-02-22T15:47:58.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:58.813 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-22T15:47:58.841 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-02-22T15:47:58.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:58.843 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-02-22T15:47:58.912 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-02-22T15:47:58.913 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-22T15:47:58.913 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-22T15:47:58.913 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-22T15:47:58.913 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-22T15:47:58.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:58.913 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-02-22T15:47:58.913 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-02-22T15:47:58.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-22T15:47:58.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-22T15:47:58.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-22T15:47:58.914 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-02-22T15:47:59.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:59.191 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-02-22T15:47:59.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:59.210 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-02-22T15:47:59.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:59.268 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-02-22T15:47:59.268 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-02-22T15:47:59.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-22T15:47:59.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-22T15:47:59.268 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-02-22T15:47:59.365 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-22T15:47:59.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-22T15:47:59.366 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-02-22T15:47:59.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:59.566 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec bash -c ' 2024-02-22T15:47:59.566 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-02-22T15:47:59.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-02-22T15:47:59.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-02-22T15:47:59.566 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-02-22T15:47:59.665 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-02-22T15:47:59.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-22T15:47:59.665 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-22T15:47:59.683 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-22T15:47:59.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-22T15:47:59.684 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-22T15:47:59.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-02-22T15:47:59.810 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-22T15:47:59.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-22T15:47:59.878 INFO:teuthology.orchestra.run.smithi019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-22T15:47:59.879 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-22T15:47:59.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:47:59.879 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-02-22T15:47:59.938 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-22T15:47:59.938 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-22T15:47:59.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-02-22T15:48:00.009 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-22T15:48:00.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-22T15:48:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-22T15:48:00.077 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-22T15:48:00.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:00.078 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-02-22T15:48:00.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:2024-02-22T15:48:00.154+0000 7f119da20540 -1 init, newargv = 0x56274be24cc0 newargc=15 2024-02-22T15:48:00.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:ceph-fuse[127608]: starting ceph client 2024-02-22T15:48:00.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:ceph-fuse[127608]: starting fuse 2024-02-22T15:48:00.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:00 smithi062 ceph-mon[48130]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:00.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:00 smithi019 ceph-mon[67418]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:01.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-02-22T15:48:01.274 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-22T15:48:01.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-22T15:48:01.347 INFO:teuthology.orchestra.run.smithi019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-22T15:48:01.348 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-22T15:48:01.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:01.353 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-02-22T15:48:01.413 INFO:teuthology.orchestra.run.smithi019.stdout:161 2024-02-22T15:48:01.414 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [161] 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> import glob 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> import re 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> import subprocess 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> def _find_admin_socket(client_name): 2024-02-22T15:48:01.414 DEBUG:teuthology.orchestra.run.smithi019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> files = glob.glob(asok_path) 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> # Given a non-glob path, it better be there 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> if "*" not in asok_path: 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> assert(len(files) == 1) 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> return files[0] 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> for f in files: 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> contents = proc_f.read() 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> if mountpoint in contents: 2024-02-22T15:48:01.415 DEBUG:teuthology.orchestra.run.smithi019:> return f 2024-02-22T15:48:01.416 DEBUG:teuthology.orchestra.run.smithi019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-22T15:48:01.416 DEBUG:teuthology.orchestra.run.smithi019:> 2024-02-22T15:48:01.416 DEBUG:teuthology.orchestra.run.smithi019:> print(_find_admin_socket("client.0")) 2024-02-22T15:48:01.416 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-02-22T15:48:01.547 INFO:teuthology.orchestra.run.smithi019.stdout:/var/run/ceph/ceph-client.0.127608.asok 2024-02-22T15:48:01.728 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.127608.asok 2024-02-22T15:48:01.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:01.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.127608.asok status 2024-02-22T15:48:01.889 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:48:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "metadata": { 2024-02-22T15:48:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_sha1": "6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9", 2024-02-22T15:48:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)", 2024-02-22T15:48:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "entity_id": "0", 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": "smithi019", 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "pid": "127608", 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "root": "/" 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_count": 0, 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_pinned_count": 0, 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 14528, 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "inst": { 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": { 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "client", 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 14528 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-02-22T15:48:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2316115328 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2316115328 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "inst_str": "client.14528 192.168.0.1:0/2316115328", 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "addr_str": "192.168.0.1:0/2316115328", 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "inode_count": 1, 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_epoch": 20, 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch": 47, 2024-02-22T15:48:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch_barrier": 0, 2024-02-22T15:48:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklisted": false, 2024-02-22T15:48:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs" 2024-02-22T15:48:01.892 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:48:01.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- ceph fs ls 2024-02-22T15:48:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:48:02.423 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:02 smithi019 ceph-mon[67418]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 3 op/s 2024-02-22T15:48:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:02 smithi062 ceph-mon[48130]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 3 op/s 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 -- 172.21.15.19:0/2792368981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de4102f50 msgr2=0x7f4de4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/2792368981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de4102f50 0x7f4de4103320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4004940 tx=0x7f4dd4009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 -- 172.21.15.19:0/2792368981 shutdown_connections 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/2792368981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de41038f0 0x7f4de40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/2792368981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de4102f50 0x7f4de4103320 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 -- 172.21.15.19:0/2792368981 >> 172.21.15.19:0/2792368981 conn(0x7f4de40f8ae0 msgr2=0x7f4de40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 -- 172.21.15.19:0/2792368981 shutdown_connections 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.087+0000 7f4dea9e3700 1 -- 172.21.15.19:0/2792368981 wait complete. 2024-02-22T15:48:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.088+0000 7f4dea9e3700 1 Processor -- start 2024-02-22T15:48:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.088+0000 7f4dea9e3700 1 -- start start 2024-02-22T15:48:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.088+0000 7f4dea9e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de4102f50 0x7f4de4070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.088+0000 7f4dea9e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 0x7f4de40707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.088+0000 7f4dea9e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de4070dd0 con 0x7f4de41038f0 2024-02-22T15:48:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.088+0000 7f4dea9e3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de4070f40 con 0x7f4de4102f50 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 0x7f4de40707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 0x7f4de40707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53848/0 (socket says 172.21.15.19:53848) 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 -- 172.21.15.19:0/243596334 learned_addr learned my addr 172.21.15.19:0/243596334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de3fff700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de4102f50 0x7f4de4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 -- 172.21.15.19:0/243596334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de4102f50 msgr2=0x7f4de4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de4102f50 0x7f4de4070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 -- 172.21.15.19:0/243596334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd4005040 con 0x7f4de41038f0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de37fe700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 0x7f4de40707d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800c800 tx=0x7f4dd8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd800a760 con 0x7f4de41038f0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4dd800ad70 con 0x7f4de41038f0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de4119bb0 con 0x7f4de41038f0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd80113a0 con 0x7f4de41038f0 2024-02-22T15:48:04.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.089+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de411a090 con 0x7f4de41038f0 2024-02-22T15:48:04.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.090+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de4100260 con 0x7f4de41038f0 2024-02-22T15:48:04.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.091+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4dd8012400 con 0x7f4de41038f0 2024-02-22T15:48:04.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.091+0000 7f4de0ff9700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f4dcc0692a0 0x7f4dcc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:04.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.091+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4dd8087140 con 0x7f4de41038f0 2024-02-22T15:48:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.091+0000 7f4de3fff700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f4dcc0692a0 0x7f4dcc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.092+0000 7f4de3fff700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f4dcc0692a0 0x7f4dcc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4004f90 tx=0x7f4dd4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.096+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4dd80534c0 con 0x7f4de41038f0 2024-02-22T15:48:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.336+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4de40719d0 con 0x7f4de41038f0 2024-02-22T15:48:04.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.337+0000 7f4de0ff9700 1 -- 172.21.15.19:0/243596334 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4dd8056ae0 con 0x7f4de41038f0 2024-02-22T15:48:04.320 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f4dcc0692a0 msgr2=0x7f4dcc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f4dcc0692a0 0x7f4dcc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4004f90 tx=0x7f4dd4009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 msgr2=0x7f4de40707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 0x7f4de40707d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800c800 tx=0x7f4dd8004470 comp rx=0 tx=0).stop 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 shutdown_connections 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f4dcc0692a0 0x7f4dcc06b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4de4102f50 0x7f4de4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 --2- 172.21.15.19:0/243596334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4de41038f0 0x7f4de40707d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:04.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 >> 172.21.15.19:0/243596334 conn(0x7f4de40f8ae0 msgr2=0x7f4de40fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 shutdown_connections 2024-02-22T15:48:04.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:04.339+0000 7f4dea9e3700 1 -- 172.21.15.19:0/243596334 wait complete. 2024-02-22T15:48:04.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:04 smithi062 ceph-mon[48130]: pgmap v206: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-02-22T15:48:04.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:04 smithi019 ceph-mon[67418]: pgmap v206: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-22T15:48:04.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-22T15:48:04.978 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-22T15:48:04.978 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-22T15:48:04.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:04.978 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-22T15:48:05.001 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 40304sec preferred_lft 40304sec 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-02-22T15:48:05.002 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-02-22T15:48:05.003 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-22T15:48:05.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-22T15:48:05.003 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-02-22T15:48:05.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.178 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-22T15:48:05.208 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-02-22T15:48:05.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.209 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-02-22T15:48:05.284 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-02-22T15:48:05.284 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-22T15:48:05.284 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-22T15:48:05.284 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-22T15:48:05.285 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-22T15:48:05.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-02-22T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-02-22T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-22T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-22T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-22T15:48:05.285 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-02-22T15:48:05.365 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:05 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/243596334' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-22T15:48:05.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:05 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/243596334' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-22T15:48:05.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.566 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-02-22T15:48:05.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.624 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-02-22T15:48:05.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.683 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-02-22T15:48:05.683 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-02-22T15:48:05.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-22T15:48:05.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-22T15:48:05.683 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-02-22T15:48:05.779 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-22T15:48:05.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-22T15:48:05.780 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-02-22T15:48:05.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:05.985 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-02-22T15:48:05.985 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-02-22T15:48:05.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-02-22T15:48:05.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-02-22T15:48:05.985 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-02-22T15:48:06.084 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-02-22T15:48:06.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-22T15:48:06.084 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:06.102 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-22T15:48:06.102 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-22T15:48:06.103 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:06.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-02-22T15:48:06.230 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-22T15:48:06.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-22T15:48:06.294 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:06 smithi062 ceph-mon[48130]: pgmap v207: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-22T15:48:06.295 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:06.297 INFO:teuthology.orchestra.run.smithi062.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-22T15:48:06.299 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-22T15:48:06.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:06.299 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-02-22T15:48:06.357 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-22T15:48:06.358 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-02-22T15:48:06.400 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-02-22T15:48:06.428 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-22T15:48:06.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-22T15:48:06.496 INFO:teuthology.orchestra.run.smithi062.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-22T15:48:06.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:06 smithi019 ceph-mon[67418]: pgmap v207: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-22T15:48:06.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:06.497 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-22T15:48:06.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:06.498 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-02-22T15:48:06.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi062.stderr:2024-02-22T15:48:06.517+0000 7f0b6533a540 -1 init, newargv = 0x55a0fe2b0b50 newargc=15 2024-02-22T15:48:06.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi062.stderr:ceph-fuse[79114]: starting ceph client 2024-02-22T15:48:06.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi062.stderr:ceph-fuse[79114]: starting fuse 2024-02-22T15:48:06.519 INFO:teuthology.orchestra.run.smithi062.stdout:125 2024-02-22T15:48:06.519 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> import re 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2024-02-22T15:48:06.520 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> return f 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.1")) 2024-02-22T15:48:06.521 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-02-22T15:48:06.645 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.1.79114.asok 2024-02-22T15:48:06.827 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.79114.asok 2024-02-22T15:48:06.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:06.827 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.79114.asok status 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9", 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)", 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "1", 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2024-02-22T15:48:06.985 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "79114", 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/" 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24311, 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 24311 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-02-22T15:48:06.986 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1047278498 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1047278498 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.24311 192.168.0.1:0/1047278498", 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "192.168.0.1:0/1047278498", 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 1, 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 20, 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 47, 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2024-02-22T15:48:06.987 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2024-02-22T15:48:06.988 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2024-02-22T15:48:06.988 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-02-22T15:48:06.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:06.995 DEBUG:teuthology.orchestra.run.smithi019:> stat --file-system '--printf=%T 2024-02-22T15:48:06.995 DEBUG:teuthology.orchestra.run.smithi019:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-22T15:48:07.021 INFO:teuthology.orchestra.run.smithi019.stdout:fuseblk 2024-02-22T15:48:07.022 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-22T15:48:07.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:07.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-22T15:48:07.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:07.100 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2024-02-22T15:48:07.100 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:07.122 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2024-02-22T15:48:07.122 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:07.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T15:48:07.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:07.454 INFO:teuthology.run_tasks:Running task print... 2024-02-22T15:48:07.465 INFO:teuthology.task.print:**** done client 2024-02-22T15:48:07.466 INFO:teuthology.run_tasks:Running task parallel... 2024-02-22T15:48:07.478 INFO:teuthology.task.parallel:starting parallel... 2024-02-22T15:48:07.478 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-22T15:48:07.479 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-22T15:48:07.479 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:48:07.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-22T15:48:07.480 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-22T15:48:07.480 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-22T15:48:07.483 INFO:tasks.workunit:Pulling workunits from ref c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:07.484 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-22T15:48:07.484 INFO:tasks.workunit:timeout=3h 2024-02-22T15:48:07.484 INFO:tasks.workunit:cleanup=True 2024-02-22T15:48:07.484 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-22T15:48:07.510 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-22T15:48:07.510 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-22T15:48:07.510 INFO:teuthology.orchestra.run.smithi019.stdout:Device: a1h/161d Inode: 1 Links: 2 2024-02-22T15:48:07.510 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:48:07.510 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-22T15:48:07.511 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-22T15:48:07.511 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-02-22 15:47:27.219023594 +0000 2024-02-22T15:48:07.511 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-02-22 15:48:07.115048817 +0000 2024-02-22T15:48:07.511 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2024-02-22T15:48:07.511 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-22T15:48:07.511 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-22T15:48:07.589 DEBUG:teuthology.orchestra.run.smithi062:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:07.611 INFO:teuthology.orchestra.run.smithi062.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:07.611 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-22T15:48:07.611 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-02-22T15:48:07.611 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-22T15:48:07.611 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-22T15:48:07.612 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-22T15:48:07.612 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-02-22 15:48:07.599367207 +0000 2024-02-22T15:48:07.612 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-02-22 15:48:07.599367207 +0000 2024-02-22T15:48:07.612 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2024-02-22T15:48:07.612 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-22T15:48:07.612 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-22T15:48:07.687 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:07.688 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:07.707 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-22T15:48:07.746 INFO:tasks.workunit.client.1.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-22T15:48:07.780 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:48:08.296 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:08 smithi019 ceph-mon[67418]: pgmap v208: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-22T15:48:08.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:08 smithi062 ceph-mon[48130]: pgmap v208: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-02-22T15:48:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- 172.21.15.19:0/3376736160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 msgr2=0x7f74dc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 --2- 172.21.15.19:0/3376736160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0fe550 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f74d0004940 tx=0x7f74d0009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- 172.21.15.19:0/3376736160 shutdown_connections 2024-02-22T15:48:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 --2- 172.21.15.19:0/3376736160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0fe550 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:09.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 --2- 172.21.15.19:0/3376736160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74dc0fcef0 0x7f74dc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- 172.21.15.19:0/3376736160 >> 172.21.15.19:0/3376736160 conn(0x7f74dc0f8620 msgr2=0x7f74dc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- 172.21.15.19:0/3376736160 shutdown_connections 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- 172.21.15.19:0/3376736160 wait complete. 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 Processor -- start 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- start start 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74dc0fcef0 0x7f74dc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc0708e0 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.791+0000 7f74e445f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc070a50 con 0x7f74dc0fcef0 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48628/0 (socket says 172.21.15.19:48628) 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 -- 172.21.15.19:0/1510982315 learned_addr learned my addr 172.21.15.19:0/1510982315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:09.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e21fb700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74dc0fcef0 0x7f74dc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 -- 172.21.15.19:0/1510982315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74dc0fcef0 msgr2=0x7f74dc071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74dc0fcef0 0x7f74dc071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 -- 172.21.15.19:0/1510982315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0005040 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e19fa700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0702e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009360 tx=0x7f74d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d000e040 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74d000bc50 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc070cd0 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d001c630 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.792+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc115880 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.793+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74dc04bed0 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.794+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f74d0013420 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.794+0000 7f74ceffd700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f74c80692a0 0x7f74c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.794+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f74d0088560 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.794+0000 7f74e21fb700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f74c80692a0 0x7f74c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:09.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.795+0000 7f74e21fb700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f74c80692a0 0x7f74c806b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74d8007fd0 tx=0x7f74d8009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:09.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.799+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f74d0058370 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.990+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f74dc115a10 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.990+0000 7f74ceffd700 1 -- 172.21.15.19:0/1510982315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f74dc115a10 con 0x7f74dc0fe0e0 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f74c80692a0 msgr2=0x7f74c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f74c80692a0 0x7f74c806b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74d8007fd0 tx=0x7f74d8009040 comp rx=0 tx=0).stop 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 msgr2=0x7f74dc0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0702e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009360 tx=0x7f74d0009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 shutdown_connections 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f74c80692a0 0x7f74c806b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74dc0fcef0 0x7f74dc071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 --2- 172.21.15.19:0/1510982315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74dc0fe0e0 0x7f74dc0702e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 >> 172.21.15.19:0/1510982315 conn(0x7f74dc0f8620 msgr2=0x7f74dc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:09.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 shutdown_connections 2024-02-22T15:48:09.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:09.992+0000 7f74e445f700 1 -- 172.21.15.19:0/1510982315 wait complete. 2024-02-22T15:48:10.398 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:10 smithi019 ceph-mon[67418]: pgmap v209: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-22T15:48:10.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:10 smithi062 ceph-mon[48130]: pgmap v209: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-02-22T15:48:10.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-22T15:48:11.059 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:48:12.395 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:12 smithi019 ceph-mon[67418]: pgmap v210: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-22T15:48:12.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:12 smithi062 ceph-mon[48130]: pgmap v210: 65 pgs: 65 active+clean; 583 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 -- 172.21.15.19:0/3290399964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fcfb0 msgr2=0x7fdd1c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3290399964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fcfb0 0x7fdd1c0fd420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd08004940 tx=0x7fdd08009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 -- 172.21.15.19:0/3290399964 shutdown_connections 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3290399964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fcfb0 0x7fdd1c0fd420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3290399964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 -- 172.21.15.19:0/3290399964 >> 172.21.15.19:0/3290399964 conn(0x7fdd1c0f74e0 msgr2=0x7fdd1c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 -- 172.21.15.19:0/3290399964 shutdown_connections 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 -- 172.21.15.19:0/3290399964 wait complete. 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 Processor -- start 2024-02-22T15:48:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.029+0000 7fdd22f82700 1 -- start start 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd22f82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd22f82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1c0fcfb0 0x7fdd1c0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd22f82700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0fe980 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd22f82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0feaf0 con 0x7fdd1c0fcfb0 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48644/0 (socket says 172.21.15.19:48644) 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 -- 172.21.15.19:0/3024031091 learned_addr learned my addr 172.21.15.19:0/3024031091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd1bfff700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1c0fcfb0 0x7fdd1c0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 -- 172.21.15.19:0/3024031091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1c0fcfb0 msgr2=0x7fdd1c0fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1c0fcfb0 0x7fdd1c0fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 -- 172.21.15.19:0/3024031091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08005040 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd20d1e700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fff00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd10002a50 tx=0x7fdd100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd10007cc0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd1000f040 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0fedd0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1000d9c0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.030+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c1158b0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.031+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c04bed0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.032+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdd1000db20 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.032+0000 7fdd197fa700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fdd0c0692b0 0x7fdd0c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.032+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdd10086f00 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.032+0000 7fdd1bfff700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fdd0c0692b0 0x7fdd0c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.033+0000 7fdd1bfff700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fdd0c0692b0 0x7fdd0c06b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b3b0 tx=0x7fdd08009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.037+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fdd10056bf0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.229+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fdd1c1047f0 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.229+0000 7fdd197fa700 1 -- 172.21.15.19:0/3024031091 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fdd10056780 con 0x7fdd1c0fbe40 2024-02-22T15:48:13.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fdd0c0692b0 msgr2=0x7fdd0c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fdd0c0692b0 0x7fdd0c06b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b3b0 tx=0x7fdd08009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 msgr2=0x7fdd1c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fff00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd10002a50 tx=0x7fdd100046b0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 shutdown_connections 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fdd0c0692b0 0x7fdd0c06b760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c0fbe40 0x7fdd1c0fff00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 --2- 172.21.15.19:0/3024031091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd1c0fcfb0 0x7fdd1c0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 >> 172.21.15.19:0/3024031091 conn(0x7fdd1c0f74e0 msgr2=0x7fdd1c0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 shutdown_connections 2024-02-22T15:48:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:13.231+0000 7fdd22f82700 1 -- 172.21.15.19:0/3024031091 wait complete. 2024-02-22T15:48:14.114 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-22T15:48:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:48:14.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:14 smithi062 ceph-mon[48130]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-22T15:48:14.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:14 smithi019 ceph-mon[67418]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 5 op/s 2024-02-22T15:48:16.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 -- 172.21.15.19:0/2697400558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fcfb0 msgr2=0x7fd0740fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:16.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 --2- 172.21.15.19:0/2697400558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fcfb0 0x7fd0740fd420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd060004940 tx=0x7fd060009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:16.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 -- 172.21.15.19:0/2697400558 shutdown_connections 2024-02-22T15:48:16.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 --2- 172.21.15.19:0/2697400558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fcfb0 0x7fd0740fd420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:16.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 --2- 172.21.15.19:0/2697400558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0740fbe40 0x7fd0740fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:16.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 -- 172.21.15.19:0/2697400558 >> 172.21.15.19:0/2697400558 conn(0x7fd0740f74e0 msgr2=0x7fd0740f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 -- 172.21.15.19:0/2697400558 shutdown_connections 2024-02-22T15:48:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.419+0000 7fd07b01e700 1 -- 172.21.15.19:0/2697400558 wait complete. 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd07b01e700 1 Processor -- start 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd07b01e700 1 -- start start 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd07b01e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 0x7fd0740fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd07b01e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0740fcfb0 0x7fd0740fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd07b01e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740fe980 con 0x7fd0740fbe40 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd07b01e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740feaf0 con 0x7fd0740fcfb0 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd078dba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 0x7fd0740fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:16.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd078dba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 0x7fd0740fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48670/0 (socket says 172.21.15.19:48670) 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd078dba700 1 -- 172.21.15.19:0/1442700635 learned_addr learned my addr 172.21.15.19:0/1442700635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.420+0000 7fd073fff700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0740fcfb0 0x7fd0740fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd078dba700 1 -- 172.21.15.19:0/1442700635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0740fcfb0 msgr2=0x7fd0740fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd078dba700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0740fcfb0 0x7fd0740fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd078dba700 1 -- 172.21.15.19:0/1442700635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060005040 con 0x7fd0740fbe40 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd078dba700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 0x7fd0740fff00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd06800eb40 tx=0x7fd06800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd068007cc0 con 0x7fd0740fbe40 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0740fed70 con 0x7fd0740fbe40 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd06800f040 con 0x7fd0740fbe40 2024-02-22T15:48:16.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.421+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd06800c990 con 0x7fd0740fbe40 2024-02-22T15:48:16.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.422+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd074115830 con 0x7fd0740fbe40 2024-02-22T15:48:16.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.422+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07404bed0 con 0x7fd0740fbe40 2024-02-22T15:48:16.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.423+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd06800cc00 con 0x7fd0740fbe40 2024-02-22T15:48:16.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.423+0000 7fd0717fa700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0640692b0 0x7fd06406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:16.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.423+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd068086da0 con 0x7fd0740fbe40 2024-02-22T15:48:16.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.423+0000 7fd073fff700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0640692b0 0x7fd06406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:16.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.423+0000 7fd073fff700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0640692b0 0x7fd06406b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd060013680 tx=0x7fd060009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:16.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.427+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd06804dbd0 con 0x7fd0740fbe40 2024-02-22T15:48:16.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:16 smithi062 ceph-mon[48130]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 4 op/s 2024-02-22T15:48:16.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:16 smithi019 ceph-mon[67418]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 4 op/s 2024-02-22T15:48:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.627+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd0741047f0 con 0x7fd0740fbe40 2024-02-22T15:48:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.628+0000 7fd0717fa700 1 -- 172.21.15.19:0/1442700635 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd068056b40 con 0x7fd0740fbe40 2024-02-22T15:48:16.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0640692b0 msgr2=0x7fd06406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:16.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0640692b0 0x7fd06406b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd060013680 tx=0x7fd060009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 msgr2=0x7fd0740fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 0x7fd0740fff00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd06800eb40 tx=0x7fd06800b040 comp rx=0 tx=0).stop 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 shutdown_connections 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fd0640692b0 0x7fd06406b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0740fbe40 0x7fd0740fff00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 --2- 172.21.15.19:0/1442700635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0740fcfb0 0x7fd0740fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 >> 172.21.15.19:0/1442700635 conn(0x7fd0740f74e0 msgr2=0x7fd0740656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 shutdown_connections 2024-02-22T15:48:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:16.630+0000 7fd07b01e700 1 -- 172.21.15.19:0/1442700635 wait complete. 2024-02-22T15:48:17.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-02-22T15:48:17.720 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:48:18.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:18 smithi019 ceph-mon[67418]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-22T15:48:18.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:18 smithi062 ceph-mon[48130]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-22T15:48:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3097261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 msgr2=0x7f6bbc0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3097261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0ffe60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004940 tx=0x7f6bac009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3097261478 shutdown_connections 2024-02-22T15:48:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3097261478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 0x7f6bbc102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3097261478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0ffe60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3097261478 >> 172.21.15.19:0/3097261478 conn(0x7f6bbc0f74e0 msgr2=0x7f6bbc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3097261478 shutdown_connections 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3097261478 wait complete. 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.713+0000 7f6bc1f94700 1 Processor -- start 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bc1f94700 1 -- start start 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bc1f94700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bc1f94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 0x7f6bbc0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:19.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bc1f94700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0fbf90 con 0x7f6bbc0fda80 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bc1f94700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0fc100 con 0x7f6bbc1003a0 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 0x7f6bbc0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51292/0 (socket says 172.21.15.19:51292) 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 0x7f6bbc0fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:60522/0 (socket says 172.21.15.19:60522) 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 -- 172.21.15.19:0/3591397947 learned_addr learned my addr 172.21.15.19:0/3591397947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 -- 172.21.15.19:0/3591397947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 msgr2=0x7f6bbc0fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 0x7f6bbc0fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:19.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 -- 172.21.15.19:0/3591397947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac005040 con 0x7f6bbc0fda80 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.714+0000 7f6bbb7fe700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0fd510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009380 tx=0x7f6bac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.715+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bac00e040 con 0x7f6bbc0fda80 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.715+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6bac00bd20 con 0x7f6bbc0fda80 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.715+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc0fc380 con 0x7f6bbc0fda80 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.715+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bac01c630 con 0x7f6bbc0fda80 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.715+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc115880 con 0x7f6bbc0fda80 2024-02-22T15:48:19.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.716+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc107910 con 0x7f6bbc0fda80 2024-02-22T15:48:19.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.716+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6bac013420 con 0x7f6bbc0fda80 2024-02-22T15:48:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.717+0000 7f6bb3fff700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f6ba80692a0 0x7f6ba806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.717+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6bac088590 con 0x7f6bbc0fda80 2024-02-22T15:48:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.717+0000 7f6bbaffd700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f6ba80692a0 0x7f6ba806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.717+0000 7f6bbaffd700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f6ba80692a0 0x7f6ba806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4007fd0 tx=0x7f6ba4009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:19.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.721+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6bac058300 con 0x7f6bbc0fda80 2024-02-22T15:48:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:19.941+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bbc05e3d0 con 0x7f6ba80692a0 2024-02-22T15:48:20.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:20 smithi062 ceph-mon[48130]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-22T15:48:20.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:20 smithi019 ceph-mon[67418]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-22T15:48:21.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:21 smithi062 ceph-mon[48130]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:21.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:21 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:21.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:21 smithi019 ceph-mon[67418]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:21.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:22.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:22 smithi062 ceph-mon[48130]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 938 B/s wr, 4 op/s 2024-02-22T15:48:22.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:22 smithi019 ceph-mon[67418]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 938 B/s wr, 4 op/s 2024-02-22T15:48:24.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:24 smithi062 ceph-mon[48130]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 4 op/s 2024-02-22T15:48:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:24 smithi019 ceph-mon[67418]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 4 op/s 2024-02-22T15:48:26.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:26 smithi062 ceph-mon[48130]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-02-22T15:48:26.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:26 smithi019 ceph-mon[67418]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-02-22T15:48:28.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:28 smithi062 ceph-mon[48130]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-22T15:48:28.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:28 smithi019 ceph-mon[67418]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-22T15:48:30.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:30 smithi062 ceph-mon[48130]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-22T15:48:30.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:30 smithi019 ceph-mon[67418]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-22T15:48:32.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:32 smithi062 ceph-mon[48130]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-02-22T15:48:32.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:32 smithi019 ceph-mon[67418]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-02-22T15:48:34.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:34 smithi062 ceph-mon[48130]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:48:34.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:34 smithi019 ceph-mon[67418]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:48:36.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:36 smithi062 ceph-mon[48130]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:48:36.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:36 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:36.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:36 smithi019 ceph-mon[67418]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:48:36.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:36 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:38.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:38 smithi062 ceph-mon[48130]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:38.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:38 smithi019 ceph-mon[67418]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:40 smithi062 ceph-mon[48130]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:40.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:40 smithi019 ceph-mon[67418]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:42.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:42 smithi062 ceph-mon[48130]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:42.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:42 smithi019 ceph-mon[67418]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:44.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:44 smithi062 ceph-mon[48130]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:44.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:44 smithi019 ceph-mon[67418]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:45.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.895+0000 7f6bb3fff700 1 -- 172.21.15.19:0/3591397947 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f6bbc05e3d0 con 0x7f6ba80692a0 2024-02-22T15:48:45.878 INFO:teuthology.orchestra.run.smithi019.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:45.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f6ba80692a0 msgr2=0x7f6ba806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:45.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f6ba80692a0 0x7f6ba806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4007fd0 tx=0x7f6ba4009040 comp rx=0 tx=0).stop 2024-02-22T15:48:45.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 msgr2=0x7f6bbc0fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:45.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0fd510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009380 tx=0x7f6bac009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 shutdown_connections 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f6ba80692a0 0x7f6ba806b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bbc0fda80 0x7f6bbc0fd510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 --2- 172.21.15.19:0/3591397947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bbc1003a0 0x7f6bbc0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 >> 172.21.15.19:0/3591397947 conn(0x7f6bbc0f74e0 msgr2=0x7f6bbc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 shutdown_connections 2024-02-22T15:48:45.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:45.897+0000 7f6bc1f94700 1 -- 172.21.15.19:0/3591397947 wait complete. 2024-02-22T15:48:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:46 smithi062 ceph-mon[48130]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:48:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:46.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:46 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:48:46.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:46 smithi019 ceph-mon[67418]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:48:46.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:46.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:46 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:48:46.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-02-22T15:48:47.038 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:48:47.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:47 smithi019 ceph-mon[67418]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:47.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:47 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:48:47.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:47 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:48:47.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:47 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:47.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:47 smithi062 ceph-mon[48130]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:47.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:47 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:48:47.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:47 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:48:47.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:47 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:48.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:48 smithi019 ceph-mon[67418]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:48.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:48 smithi019 ceph-mon[67418]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:48.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:48 smithi062 ceph-mon[48130]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:48.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:48 smithi062 ceph-mon[48130]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:48:50.475 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:50 smithi019 ceph-mon[67418]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:50.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:50 smithi062 ceph-mon[48130]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:51.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:51 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:51.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:51 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:48:52.291 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:52 smithi019 ceph-mon[67418]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:52.291 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:52 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:52.291 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:52 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:48:52.291 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:52 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 -- 172.21.15.19:0/2604529394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a41003a0 msgr2=0x7fe8a4102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/2604529394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a41003a0 0x7fe8a4102780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe894004940 tx=0x7fe894009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 -- 172.21.15.19:0/2604529394 shutdown_connections 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/2604529394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a41003a0 0x7fe8a4102780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/2604529394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a40fda80 0x7fe8a40ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 -- 172.21.15.19:0/2604529394 >> 172.21.15.19:0/2604529394 conn(0x7fe8a40f74e0 msgr2=0x7fe8a40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 -- 172.21.15.19:0/2604529394 shutdown_connections 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.589+0000 7fe8ac10d700 1 -- 172.21.15.19:0/2604529394 wait complete. 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8ac10d700 1 Processor -- start 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8ac10d700 1 -- start start 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8ac10d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 0x7fe8a410d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8ac10d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a41003a0 0x7fe8a410dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8ac10d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a410e330 con 0x7fe8a40fda80 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8ac10d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a410ebd0 con 0x7fe8a41003a0 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8a9ea9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 0x7fe8a410d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8a9ea9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 0x7fe8a410d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43488/0 (socket says 172.21.15.19:43488) 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8a9ea9700 1 -- 172.21.15.19:0/1305983805 learned_addr learned my addr 172.21.15.19:0/1305983805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.590+0000 7fe8a96a8700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a41003a0 0x7fe8a410dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8a9ea9700 1 -- 172.21.15.19:0/1305983805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a41003a0 msgr2=0x7fe8a410dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8a9ea9700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a41003a0 0x7fe8a410dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8a9ea9700 1 -- 172.21.15.19:0/1305983805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe894005040 con 0x7fe8a40fda80 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8a96a8700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a41003a0 0x7fe8a410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8a9ea9700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 0x7fe8a410d7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000eb40 tx=0x7fe8a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8a0007cc0 con 0x7fe8a40fda80 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe8a000f040 con 0x7fe8a40fda80 2024-02-22T15:48:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8a000c990 con 0x7fe8a40fda80 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a410ee50 con 0x7fe8a40fda80 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.591+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a410f340 con 0x7fe8a40fda80 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.592+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4107910 con 0x7fe8a40fda80 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.593+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe8a000cc00 con 0x7fe8a40fda80 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.593+0000 7fe89a7fc700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8900692a0 0x7fe89006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.593+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe8a0086da0 con 0x7fe8a40fda80 2024-02-22T15:48:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.594+0000 7fe8a96a8700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8900692a0 0x7fe89006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.594+0000 7fe8a96a8700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8900692a0 0x7fe89006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe894009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.598+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe8a004dbd0 con 0x7fe8a40fda80 2024-02-22T15:48:52.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:52 smithi062 ceph-mon[48130]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:52.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:52 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:52.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:52 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:48:52.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:52 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:48:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.816+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8a402b3b0 con 0x7fe8900692a0 2024-02-22T15:48:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.817+0000 7fe89a7fc700 1 -- 172.21.15.19:0/1305983805 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fe8a402b3b0 con 0x7fe8900692a0 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8900692a0 msgr2=0x7fe89006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8900692a0 0x7fe89006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe894009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 msgr2=0x7fe8a410d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 0x7fe8a410d7f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000eb40 tx=0x7fe8a000b040 comp rx=0 tx=0).stop 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 shutdown_connections 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fe8900692a0 0x7fe89006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a40fda80 0x7fe8a410d7f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 --2- 172.21.15.19:0/1305983805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8a41003a0 0x7fe8a410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 >> 172.21.15.19:0/1305983805 conn(0x7fe8a40f74e0 msgr2=0x7fe8a40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 shutdown_connections 2024-02-22T15:48:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.819+0000 7fe8ac10d700 1 -- 172.21.15.19:0/1305983805 wait complete. 2024-02-22T15:48:52.816 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.942+0000 7f3c2ef05700 1 -- 172.21.15.19:0/611105876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 msgr2=0x7f3c280ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.942+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/611105876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c280ffe60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c004940 tx=0x7f3c1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 -- 172.21.15.19:0/611105876 shutdown_connections 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/611105876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 0x7f3c28102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/611105876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c280ffe60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 -- 172.21.15.19:0/611105876 >> 172.21.15.19:0/611105876 conn(0x7f3c280f74e0 msgr2=0x7f3c280f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:52.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 -- 172.21.15.19:0/611105876 shutdown_connections 2024-02-22T15:48:52.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 -- 172.21.15.19:0/611105876 wait complete. 2024-02-22T15:48:52.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 Processor -- start 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.943+0000 7f3c2ef05700 1 -- start start 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2ef05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c2810d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2ef05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 0x7f3c2810dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2ef05700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2810e320 con 0x7f3c280fda80 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2ef05700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2810ebc0 con 0x7f3c281003a0 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2cca1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c2810d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2cca1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c2810d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43510/0 (socket says 172.21.15.19:43510) 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2cca1700 1 -- 172.21.15.19:0/3381540940 learned_addr learned my addr 172.21.15.19:0/3381540940 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c27fff700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 0x7f3c2810dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2cca1700 1 -- 172.21.15.19:0/3381540940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 msgr2=0x7f3c2810dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2cca1700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 0x7f3c2810dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c2cca1700 1 -- 172.21.15.19:0/3381540940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c005040 con 0x7f3c280fda80 2024-02-22T15:48:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.944+0000 7f3c27fff700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 0x7f3c2810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:48:52.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.945+0000 7f3c2cca1700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c2810d7e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009fd0 tx=0x7f3c1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:52.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.945+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c1c021070 con 0x7f3c280fda80 2024-02-22T15:48:52.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.945+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c1c01c400 con 0x7f3c280fda80 2024-02-22T15:48:52.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.945+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2810ee40 con 0x7f3c280fda80 2024-02-22T15:48:52.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.945+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c1c00e070 con 0x7f3c280fda80 2024-02-22T15:48:52.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.945+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2810f320 con 0x7f3c280fda80 2024-02-22T15:48:52.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.946+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2804bed0 con 0x7f3c280fda80 2024-02-22T15:48:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.947+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3c1c01ca00 con 0x7f3c280fda80 2024-02-22T15:48:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.948+0000 7f3c257fa700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3c180692a0 0x7f3c1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.948+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3c1c088a20 con 0x7f3c280fda80 2024-02-22T15:48:52.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.948+0000 7f3c27fff700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3c180692a0 0x7f3c1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:52.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.948+0000 7f3c27fff700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3c180692a0 0x7f3c1806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009b80 tx=0x7f3c1400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:52.954+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3c1c058830 con 0x7f3c280fda80 2024-02-22T15:48:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.229+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c28101000 con 0x7f3c180692a0 2024-02-22T15:48:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.230+0000 7f3c257fa700 1 -- 172.21.15.19:0/3381540940 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3c28101000 con 0x7f3c180692a0 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.232+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3c180692a0 msgr2=0x7f3c1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.232+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3c180692a0 0x7f3c1806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009b80 tx=0x7f3c1400b040 comp rx=0 tx=0).stop 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 msgr2=0x7f3c2810d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c2810d7e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009fd0 tx=0x7f3c1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 shutdown_connections 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f3c180692a0 0x7f3c1806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c280fda80 0x7f3c2810d7e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 --2- 172.21.15.19:0/3381540940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c281003a0 0x7f3c2810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 >> 172.21.15.19:0/3381540940 conn(0x7f3c280f74e0 msgr2=0x7f3c280f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:53.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 shutdown_connections 2024-02-22T15:48:53.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.233+0000 7f3c2ef05700 1 -- 172.21.15.19:0/3381540940 wait complete. 2024-02-22T15:48:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 -- 172.21.15.19:0/3376942113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 msgr2=0x7fb9500fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 --2- 172.21.15.19:0/3376942113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb9500fd420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb944004940 tx=0x7fb944009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 -- 172.21.15.19:0/3376942113 shutdown_connections 2024-02-22T15:48:53.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 --2- 172.21.15.19:0/3376942113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb9500fd420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 --2- 172.21.15.19:0/3376942113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb9500fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 -- 172.21.15.19:0/3376942113 >> 172.21.15.19:0/3376942113 conn(0x7fb9500f74e0 msgr2=0x7fb9500f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 -- 172.21.15.19:0/3376942113 shutdown_connections 2024-02-22T15:48:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.364+0000 7fb95496f700 1 -- 172.21.15.19:0/3376942113 wait complete. 2024-02-22T15:48:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb95496f700 1 Processor -- start 2024-02-22T15:48:53.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb95496f700 1 -- start start 2024-02-22T15:48:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb95496f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb950110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb95496f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb950112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb95496f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950066460 con 0x7fb9500fcfb0 2024-02-22T15:48:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb95496f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500665a0 con 0x7fb9500fbe40 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb94e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb950110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb94dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb950112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb94e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb950110610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:41548/0 (socket says 172.21.15.19:41548) 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb94e59c700 1 -- 172.21.15.19:0/2229311479 learned_addr learned my addr 172.21.15.19:0/2229311479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.365+0000 7fb94dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb950112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43534/0 (socket says 172.21.15.19:43534) 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb94dd9b700 1 -- 172.21.15.19:0/2229311479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 msgr2=0x7fb950110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb94dd9b700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb950110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb94dd9b700 1 -- 172.21.15.19:0/2229311479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944005040 con 0x7fb9500fcfb0 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb94e59c700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb950110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb94dd9b700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb950112b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb944009fd0 tx=0x7fb944009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94400e040 con 0x7fb9500fcfb0 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb94400bc50 con 0x7fb9500fcfb0 2024-02-22T15:48:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb950113160 con 0x7fb9500fcfb0 2024-02-22T15:48:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94401c650 con 0x7fb9500fcfb0 2024-02-22T15:48:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.366+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9501136a0 con 0x7fb9500fcfb0 2024-02-22T15:48:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.368+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95004bed0 con 0x7fb9500fcfb0 2024-02-22T15:48:53.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.368+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb944013420 con 0x7fb9500fcfb0 2024-02-22T15:48:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.369+0000 7fb93effd700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fb9380692a0 0x7fb93806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.369+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb944088480 con 0x7fb9500fcfb0 2024-02-22T15:48:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.369+0000 7fb94e59c700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fb9380692a0 0x7fb93806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:53.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.369+0000 7fb94e59c700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fb9380692a0 0x7fb93806b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9500660e0 tx=0x7fb94000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:53.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.376+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb944054760 con 0x7fb9500fcfb0 2024-02-22T15:48:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:53 smithi019 ceph-mon[67418]: Upgrade: Target is version 18.2.1-638-gc1dc18f4 (unknown) 2024-02-22T15:48:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:53 smithi019 ceph-mon[67418]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0'] 2024-02-22T15:48:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:53 smithi019 ceph-mon[67418]: Upgrade: Need to upgrade myself (mgr.smithi019.yvjrkp) 2024-02-22T15:48:53.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.656+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb950101990 con 0x7fb9380692a0 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.673+0000 7fb93effd700 1 -- 172.21.15.19:0/2229311479 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fb950101990 con 0x7fb9380692a0 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (5m) 61s ago 7m 26.5M - 0.25.0 c8568f914cd2 f9f67af4f00a 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (7m) 61s ago 7m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (6m) 67s ago 6m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (5m) 61s ago 6m 83.8M - 9.4.7 954c08fa6188 1b6b49b44a2a 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (81s) 61s ago 81s 15.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:48:53.656 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (87s) 61s ago 87s 11.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (78s) 67s ago 78s 15.5M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (84s) 67s ago 84s 16.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:9283 running (8m) 61s ago 8m 480M - 17.2.7-527-g6ea2a55e e2e4ad29904c 1e57e04e2dad 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 running (6m) 67s ago 6m 432M - 17.2.7-527-g6ea2a55e e2e4ad29904c 8bcf1362db0f 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (8m) 61s ago 8m 45.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (6m) 67s ago 6m 38.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (7m) 61s ago 7m 17.9M - 1.5.0 0da6a335fe13 d890c8b7cdd8 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (6m) 67s ago 6m 16.9M - 1.5.0 0da6a335fe13 379a69e65053 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (5m) 61s ago 5m 48.5M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (4m) 61s ago 4m 47.6M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (4m) 61s ago 4m 47.4M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (3m) 67s ago 3m 42.9M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (3m) 67s ago 3m 44.4M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:48:53.657 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (3m) 67s ago 3m 41.9M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:48:53.658 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (5m) 61s ago 6m 53.6M - 2.43.0 a07b618ecd1d 05fe6b300615 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fb9380692a0 msgr2=0x7fb93806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fb9380692a0 0x7fb93806b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9500660e0 tx=0x7fb94000b040 comp rx=0 tx=0).stop 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 msgr2=0x7fb950112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb950112b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb944009fd0 tx=0x7fb944009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 shutdown_connections 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fb9380692a0 0x7fb93806b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9500fbe40 0x7fb950110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 --2- 172.21.15.19:0/2229311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9500fcfb0 0x7fb950112b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.676+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 >> 172.21.15.19:0/2229311479 conn(0x7fb9500f74e0 msgr2=0x7fb950100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.677+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 shutdown_connections 2024-02-22T15:48:53.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.677+0000 7fb95496f700 1 -- 172.21.15.19:0/2229311479 wait complete. 2024-02-22T15:48:53.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:53 smithi062 ceph-mon[48130]: Upgrade: Target is version 18.2.1-638-gc1dc18f4 (unknown) 2024-02-22T15:48:53.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:53 smithi062 ceph-mon[48130]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0'] 2024-02-22T15:48:53.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:53 smithi062 ceph-mon[48130]: Upgrade: Need to upgrade myself (mgr.smithi019.yvjrkp) 2024-02-22T15:48:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/2827994537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fbe40 msgr2=0x7f7e180fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/2827994537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fbe40 0x7f7e180fc250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e14004940 tx=0x7f7e14009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/2827994537 shutdown_connections 2024-02-22T15:48:53.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/2827994537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fcfb0 0x7f7e180fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/2827994537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fbe40 0x7f7e180fc250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/2827994537 >> 172.21.15.19:0/2827994537 conn(0x7f7e180f74e0 msgr2=0x7f7e180f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:53.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/2827994537 shutdown_connections 2024-02-22T15:48:53.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.785+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/2827994537 wait complete. 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1f4b1700 1 Processor -- start 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1f4b1700 1 -- start start 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1f4b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fbe40 0x7f7e1810c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1f4b1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 0x7f7e1810cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1f4b1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1810d170 con 0x7f7e180fcfb0 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1f4b1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1810da10 con 0x7f7e180fbe40 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1ca4c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 0x7f7e1810cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1ca4c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 0x7f7e1810cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43548/0 (socket says 172.21.15.19:43548) 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1ca4c700 1 -- 172.21.15.19:0/1540184525 learned_addr learned my addr 172.21.15.19:0/1540184525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1d24d700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fbe40 0x7f7e1810c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.786+0000 7f7e1ca4c700 1 -- 172.21.15.19:0/1540184525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fbe40 msgr2=0x7f7e1810c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e1ca4c700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fbe40 0x7f7e1810c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e1ca4c700 1 -- 172.21.15.19:0/1540184525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08005040 con 0x7f7e180fcfb0 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e1d24d700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fbe40 0x7f7e1810c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:48:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e1ca4c700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 0x7f7e1810cb70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e080096d0 tx=0x7f7e08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:53.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e08010a50 con 0x7f7e180fcfb0 2024-02-22T15:48:53.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e14005040 con 0x7f7e180fcfb0 2024-02-22T15:48:53.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1810e040 con 0x7f7e180fcfb0 2024-02-22T15:48:53.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e08010bb0 con 0x7f7e180fcfb0 2024-02-22T15:48:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.787+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e0801a640 con 0x7f7e180fcfb0 2024-02-22T15:48:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.788+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc0052e0 con 0x7f7e180fcfb0 2024-02-22T15:48:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.790+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7e08010d20 con 0x7f7e180fcfb0 2024-02-22T15:48:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.790+0000 7f7e0dffb700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e040692f0 0x7f7e0406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.790+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7e08089c20 con 0x7f7e180fcfb0 2024-02-22T15:48:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.790+0000 7f7e1d24d700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e040692f0 0x7f7e0406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.791+0000 7f7e1d24d700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e040692f0 0x7f7e0406b7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009fd0 tx=0x7f7e14009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:53.794+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7e08059a30 con 0x7f7e180fcfb0 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.148+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7dfc005860 con 0x7f7e180fcfb0 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.149+0000 7f7e0dffb700 1 -- 172.21.15.19:0/1540184525 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f7e080595c0 con 0x7f7e180fcfb0 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:48:54.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 14 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:48:54.133 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.151+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e040692f0 msgr2=0x7f7e0406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.151+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e040692f0 0x7f7e0406b7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009fd0 tx=0x7f7e14009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.151+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 msgr2=0x7f7e1810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.151+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 0x7f7e1810cb70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7e080096d0 tx=0x7f7e08009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 shutdown_connections 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7e040692f0 0x7f7e0406b7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e180fbe40 0x7f7e1810c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 --2- 172.21.15.19:0/1540184525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e180fcfb0 0x7f7e1810cb70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 >> 172.21.15.19:0/1540184525 conn(0x7f7e180f74e0 msgr2=0x7f7e18100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 shutdown_connections 2024-02-22T15:48:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.152+0000 7f7e1f4b1700 1 -- 172.21.15.19:0/1540184525 wait complete. 2024-02-22T15:48:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 -- 172.21.15.19:0/4127585438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 msgr2=0x7f63b4104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/4127585438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b4104810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63a4004940 tx=0x7f63a4009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 -- 172.21.15.19:0/4127585438 shutdown_connections 2024-02-22T15:48:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/4127585438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b4104810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/4127585438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40663e0 0x7f63b4101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 -- 172.21.15.19:0/4127585438 >> 172.21.15.19:0/4127585438 conn(0x7f63b40f74e0 msgr2=0x7f63b40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 -- 172.21.15.19:0/4127585438 shutdown_connections 2024-02-22T15:48:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 -- 172.21.15.19:0/4127585438 wait complete. 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.263+0000 7f63bbe6b700 1 Processor -- start 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63bbe6b700 1 -- start start 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63bbe6b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40663e0 0x7f63b410fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63bbe6b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b410fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63bbe6b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4110680 con 0x7f63b40663e0 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63bbe6b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4110ec0 con 0x7f63b4102430 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63b9406700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b410fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63b9406700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b410fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:41602/0 (socket says 172.21.15.19:41602) 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63b9406700 1 -- 172.21.15.19:0/3222731671 learned_addr learned my addr 172.21.15.19:0/3222731671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.264+0000 7f63b9c07700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40663e0 0x7f63b410fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63b9406700 1 -- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40663e0 msgr2=0x7f63b410fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:54.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63b9406700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40663e0 0x7f63b410fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63b9406700 1 -- 172.21.15.19:0/3222731671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4005040 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63b9406700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b410fff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63a4004f90 tx=0x7f63a4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a400e040 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b4111140 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63a400bc50 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.265+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a401c630 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.266+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b4111680 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.266+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b404bed0 con 0x7f63b4102430 2024-02-22T15:48:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.267+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f63a4013420 con 0x7f63b4102430 2024-02-22T15:48:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.268+0000 7f63aa7fc700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f63a00692a0 0x7f63a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:48:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.268+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f63a4088390 con 0x7f63b4102430 2024-02-22T15:48:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.268+0000 7f63b9c07700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f63a00692a0 0x7f63a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:48:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.269+0000 7f63b9c07700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f63a00692a0 0x7f63a006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009bf0 tx=0x7f63b00091a0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:48:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.275+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f63a40580e0 con 0x7f63b4102430 2024-02-22T15:48:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:54 smithi019 ceph-mon[67418]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 on smithi062 2024-02-22T15:48:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:54 smithi019 ceph-mon[67418]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:54 smithi019 ceph-mon[67418]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:54 smithi019 ceph-mon[67418]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:54.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:54 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1540184525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:48:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.665+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63b4111be0 con 0x7f63a00692a0 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.666+0000 7f63aa7fc700 1 -- 172.21.15.19:0/3222731671 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f63b4111be0 con 0x7f63a00692a0 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "0/2 daemons upgraded", 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 image on host smithi062", 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:48:54.649 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f63a00692a0 msgr2=0x7f63a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f63a00692a0 0x7f63a006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009bf0 tx=0x7f63b00091a0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 msgr2=0x7f63b410fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b410fff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63a4004f90 tx=0x7f63a4009e30 comp rx=0 tx=0).stop 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 shutdown_connections 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f63a00692a0 0x7f63a006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b40663e0 0x7f63b410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 --2- 172.21.15.19:0/3222731671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b4102430 0x7f63b410fff0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 >> 172.21.15.19:0/3222731671 conn(0x7f63b40f74e0 msgr2=0x7f63b40f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.668+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 shutdown_connections 2024-02-22T15:48:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:48:54.669+0000 7f63bbe6b700 1 -- 172.21.15.19:0/3222731671 wait complete. 2024-02-22T15:48:54.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:54 smithi062 ceph-mon[48130]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 on smithi062 2024-02-22T15:48:54.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:54 smithi062 ceph-mon[48130]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:54.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:54 smithi062 ceph-mon[48130]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:48:54.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:54 smithi062 ceph-mon[48130]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:54.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:54 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1540184525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:48:55.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:55 smithi019 ceph-mon[67418]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:55.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:55 smithi062 ceph-mon[48130]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:56.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:56 smithi019 ceph-mon[67418]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:56.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:56 smithi019 ceph-mon[67418]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:48:56.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:56 smithi062 ceph-mon[48130]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:48:56.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:56 smithi062 ceph-mon[48130]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:48:58.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:48:58 smithi019 ceph-mon[67418]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:48:58.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:48:58 smithi062 ceph-mon[48130]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:00.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:00 smithi019 ceph-mon[67418]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:00.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:00 smithi062 ceph-mon[48130]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:02 smithi062 ceph-mon[48130]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:02.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:02 smithi019 ceph-mon[67418]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:04.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:04 smithi062 ceph-mon[48130]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:04.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:04 smithi019 ceph-mon[67418]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:06.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:06 smithi062 ceph-mon[48130]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:49:06.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:06 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:06.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:06 smithi019 ceph-mon[67418]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:49:06.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:06 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:08.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:08 smithi062 ceph-mon[48130]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:08.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:08 smithi019 ceph-mon[67418]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:10.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:10 smithi062 ceph-mon[48130]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:10.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:10 smithi019 ceph-mon[67418]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:12.468 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:12 smithi062 ceph-mon[48130]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:12.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:12 smithi019 ceph-mon[67418]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:14.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:14 smithi062 ceph-mon[48130]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:14.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:14 smithi019 ceph-mon[67418]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:16.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:16 smithi062 ceph-mon[48130]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:49:16.501 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:16 smithi019 ceph-mon[67418]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:49:18.425 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:18 smithi062 ceph-mon[48130]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:18.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:18 smithi019 ceph-mon[67418]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:20.418 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:20 smithi062 ceph-mon[48130]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:20.418 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:20.418 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:49:20.418 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:49:20.418 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:20 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:49:20.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:20 smithi019 ceph-mon[67418]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:20.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:20.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:49:20.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:49:20.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:20 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:49:21.185 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:21 smithi062 ceph-mon[48130]: Upgrade: Updating mgr.smithi062.ecubmv 2024-02-22T15:49:21.185 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:21 smithi062 ceph-mon[48130]: Deploying daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:49:21.185 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:21 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:21.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:21 smithi019 ceph-mon[67418]: Upgrade: Updating mgr.smithi062.ecubmv 2024-02-22T15:49:21.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:21 smithi019 ceph-mon[67418]: Deploying daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:49:21.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:21 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:22.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:22 smithi062 ceph-mon[48130]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:22.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:22 smithi019 ceph-mon[67418]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:24.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:24 smithi062 ceph-mon[48130]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:24.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:24 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:24.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:24 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:24.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:24 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:24.361 INFO:tasks.workunit.client.0.smithi019.stderr:Updating files: 90% (10893/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-02-22T15:49:24.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:24 smithi019 ceph-mon[67418]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:49:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:24 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:24 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:24.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:24 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.776+0000 7f897fc65700 1 -- 172.21.15.19:0/3861210498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 msgr2=0x7f89780fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.776+0000 7f897fc65700 1 --2- 172.21.15.19:0/3861210498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f89780fc250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f89740049c0 tx=0x7f8974009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- 172.21.15.19:0/3861210498 shutdown_connections 2024-02-22T15:49:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 --2- 172.21.15.19:0/3861210498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89780fcfb0 0x7f89780fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 --2- 172.21.15.19:0/3861210498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f89780fc250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- 172.21.15.19:0/3861210498 >> 172.21.15.19:0/3861210498 conn(0x7f89780f74e0 msgr2=0x7f89780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- 172.21.15.19:0/3861210498 shutdown_connections 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- 172.21.15.19:0/3861210498 wait complete. 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 Processor -- start 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- start start 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f897810b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89780fcfb0 0x7f897810bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897810c1a0 con 0x7f89780fbe40 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.777+0000 7f897fc65700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897810ca40 con 0x7f89780fcfb0 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f897810b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f897810b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54578/0 (socket says 172.21.15.19:54578) 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 -- 172.21.15.19:0/2919057621 learned_addr learned my addr 172.21.15.19:0/2919057621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897d200700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89780fcfb0 0x7f897810bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 -- 172.21.15.19:0/2919057621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89780fcfb0 msgr2=0x7f897810bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89780fcfb0 0x7f897810bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 -- 172.21.15.19:0/2919057621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974005040 con 0x7f89780fbe40 2024-02-22T15:49:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897da01700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f897810b660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c90 tx=0x7f8974009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897400d9b0 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897810ccc0 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f897400db10 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897401b640 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.778+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897810d200 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.779+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897804bed0 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.780+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8974021070 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.781+0000 7f896e7fc700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f89640692a0 0x7f896406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.781+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f897408ac20 con 0x7f89780fbe40 2024-02-22T15:49:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.781+0000 7f897d200700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f89640692a0 0x7f896406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.781+0000 7f897d200700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f89640692a0 0x7f896406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8968009b80 tx=0x7f896800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:24.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:24.789+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f897405a990 con 0x7f89780fbe40 2024-02-22T15:49:24.976 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to 'c1dc18f444603397768fcb7c34a8488b10284df6'. 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.977 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-02-22T15:49:24.978 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.978 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-22T15:49:24.978 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-02-22T15:49:24.978 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at c1dc18f4446 Merge branch 'wip-64505-reef' of https://github.com/batrick/ceph into wip-yuri10-testing-2024-02-21-1249-reef 2024-02-22T15:49:24.981 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-22T15:49:24.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.000+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f897810d610 con 0x7f89640692a0 2024-02-22T15:49:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.001+0000 7f896e7fc700 1 -- 172.21.15.19:0/2919057621 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f897810d610 con 0x7f89640692a0 2024-02-22T15:49:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f89640692a0 msgr2=0x7f896406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f89640692a0 0x7f896406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8968009b80 tx=0x7f896800b040 comp rx=0 tx=0).stop 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 msgr2=0x7f897810b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f897810b660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c90 tx=0x7f8974009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 shutdown_connections 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f89640692a0 0x7f896406b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89780fbe40 0x7f897810b660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 --2- 172.21.15.19:0/2919057621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89780fcfb0 0x7f897810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 >> 172.21.15.19:0/2919057621 conn(0x7f89780f74e0 msgr2=0x7f89780f97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 shutdown_connections 2024-02-22T15:49:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.003+0000 7f897fc65700 1 -- 172.21.15.19:0/2919057621 wait complete. 2024-02-22T15:49:24.996 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:49:25.019 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-22T15:49:25.021 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-22T15:49:25.021 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-22T15:49:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 -- 172.21.15.19:0/924553570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 msgr2=0x7f2394102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 --2- 172.21.15.19:0/924553570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f2394102780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2388004940 tx=0x7f2388009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 -- 172.21.15.19:0/924553570 shutdown_connections 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 --2- 172.21.15.19:0/924553570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f2394102780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 --2- 172.21.15.19:0/924553570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23940fda80 0x7f23940ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 -- 172.21.15.19:0/924553570 >> 172.21.15.19:0/924553570 conn(0x7f23940f7500 msgr2=0x7f23940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 -- 172.21.15.19:0/924553570 shutdown_connections 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.109+0000 7f239a281700 1 -- 172.21.15.19:0/924553570 wait complete. 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f239a281700 1 Processor -- start 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f239a281700 1 -- start start 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f239a281700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23940fda80 0x7f239410d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f239a281700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f239410dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f239a281700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239410e330 con 0x7f23941003a0 2024-02-22T15:49:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f239a281700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239410ebd0 con 0x7f23940fda80 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f23937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f239410dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f23937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f239410dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54596/0 (socket says 172.21.15.19:54596) 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f23937fe700 1 -- 172.21.15.19:0/1364097859 learned_addr learned my addr 172.21.15.19:0/1364097859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.110+0000 7f2393fff700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23940fda80 0x7f239410d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f23937fe700 1 -- 172.21.15.19:0/1364097859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23940fda80 msgr2=0x7f239410d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f23937fe700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23940fda80 0x7f239410d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f23937fe700 1 -- 172.21.15.19:0/1364097859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388005040 con 0x7f23941003a0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f23937fe700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f239410dd30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2388008f90 tx=0x7f2388009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2388021070 con 0x7f23941003a0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f238801c400 con 0x7f23941003a0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239410ee50 con 0x7f23941003a0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f238800e030 con 0x7f23941003a0 2024-02-22T15:49:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.111+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239410f390 con 0x7f23941003a0 2024-02-22T15:49:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.112+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2394107910 con 0x7f23941003a0 2024-02-22T15:49:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.112+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f238801ca00 con 0x7f23941003a0 2024-02-22T15:49:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.113+0000 7f2390ff9700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f237c0692b0 0x7f237c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.113+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2388088400 con 0x7f23941003a0 2024-02-22T15:49:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.113+0000 7f2393fff700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f237c0692b0 0x7f237c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.113+0000 7f2393fff700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f237c0692b0 0x7f237c06b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2384009b80 tx=0x7f238400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:25.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.118+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2388058210 con 0x7f23941003a0 2024-02-22T15:49:25.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.329+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f239410f600 con 0x7f237c0692b0 2024-02-22T15:49:25.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.330+0000 7f2390ff9700 1 -- 172.21.15.19:0/1364097859 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f239410f600 con 0x7f237c0692b0 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.331+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f237c0692b0 msgr2=0x7f237c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.331+0000 7f239a281700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f237c0692b0 0x7f237c06b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2384009b80 tx=0x7f238400b040 comp rx=0 tx=0).stop 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.331+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 msgr2=0x7f239410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.331+0000 7f239a281700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f239410dd30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2388008f90 tx=0x7f2388009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 shutdown_connections 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f237c0692b0 0x7f237c06b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23940fda80 0x7f239410d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 --2- 172.21.15.19:0/1364097859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23941003a0 0x7f239410dd30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 >> 172.21.15.19:0/1364097859 conn(0x7f23940f7500 msgr2=0x7f23940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 shutdown_connections 2024-02-22T15:49:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.332+0000 7f239a281700 1 -- 172.21.15.19:0/1364097859 wait complete. 2024-02-22T15:49:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1507538850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 msgr2=0x7f5cd80fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1507538850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd80fc250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4004940 tx=0x7f5cd4009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1507538850 shutdown_connections 2024-02-22T15:49:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1507538850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cd80fcfb0 0x7f5cd80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1507538850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd80fc250 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1507538850 >> 172.21.15.19:0/1507538850 conn(0x7f5cd80f74e0 msgr2=0x7f5cd80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1507538850 shutdown_connections 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1507538850 wait complete. 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.435+0000 7f5cdf7ea700 1 Processor -- start 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdf7ea700 1 -- start start 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdf7ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd810fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdf7ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cd80fcfb0 0x7f5cd810ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdf7ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8110540 con 0x7f5cd80fbe40 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdf7ea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8110de0 con 0x7f5cd80fcfb0 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd810fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd810fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54616/0 (socket says 172.21.15.19:54616) 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 -- 172.21.15.19:0/1734558738 learned_addr learned my addr 172.21.15.19:0/1734558738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdcd85700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cd80fcfb0 0x7f5cd810ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 -- 172.21.15.19:0/1734558738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cd80fcfb0 msgr2=0x7f5cd810ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cd80fcfb0 0x7f5cd810ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 -- 172.21.15.19:0/1734558738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4005040 con 0x7f5cd80fbe40 2024-02-22T15:49:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5cdd586700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd810fa00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4000c00 tx=0x7f5cd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.436+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd4021070 con 0x7f5cd80fbe40 2024-02-22T15:49:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.437+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5cd401c400 con 0x7f5cd80fbe40 2024-02-22T15:49:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.437+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd400e070 con 0x7f5cd80fbe40 2024-02-22T15:49:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.437+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd8111060 con 0x7f5cd80fbe40 2024-02-22T15:49:25.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.437+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd8111540 con 0x7f5cd80fbe40 2024-02-22T15:49:25.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.438+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd804bed0 con 0x7f5cd80fbe40 2024-02-22T15:49:25.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.438+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5cd401c9c0 con 0x7f5cd80fbe40 2024-02-22T15:49:25.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.438+0000 7f5ccdffb700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5cc40692a0 0x7f5cc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.439+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5cd40882e0 con 0x7f5cd80fbe40 2024-02-22T15:49:25.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.439+0000 7f5cdcd85700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5cc40692a0 0x7f5cc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.439+0000 7f5cdcd85700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5cc40692a0 0x7f5cc406b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009b80 tx=0x7f5cc800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.443+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5cd40580f0 con 0x7f5cd80fbe40 2024-02-22T15:49:25.567 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-22T15:49:25.619 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-22T15:49:25.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.651+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cd8111c00 con 0x7f5cc40692a0 2024-02-22T15:49:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.661+0000 7f5ccdffb700 1 -- 172.21.15.19:0/1734558738 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f5cd8111c00 con 0x7f5cc40692a0 2024-02-22T15:49:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:49:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (6m) 93s ago 7m 26.5M - 0.25.0 c8568f914cd2 f9f67af4f00a 2024-02-22T15:49:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (7m) 93s ago 7m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:49:25.643 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (6m) 99s ago 6m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (6m) 93s ago 7m 83.8M - 9.4.7 954c08fa6188 1b6b49b44a2a 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (113s) 93s ago 113s 15.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (119s) 93s ago 119s 11.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (110s) 99s ago 110s 15.5M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (116s) 99s ago 116s 16.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:9283 running (9m) 93s ago 9m 480M - 17.2.7-527-g6ea2a55e e2e4ad29904c 1e57e04e2dad 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 starting - - - - 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (9m) 93s ago 9m 45.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (6m) 99s ago 6m 38.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (7m) 93s ago 7m 17.9M - 1.5.0 0da6a335fe13 d890c8b7cdd8 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (6m) 99s ago 6m 16.9M - 1.5.0 0da6a335fe13 379a69e65053 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (5m) 93s ago 5m 48.5M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (5m) 93s ago 5m 47.6M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:49:25.644 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (4m) 93s ago 4m 47.4M 4300M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:49:25.645 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (4m) 99s ago 4m 42.9M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:49:25.645 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (3m) 99s ago 3m 44.4M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:49:25.645 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (3m) 99s ago 3m 41.9M 5323M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:49:25.645 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (5m) 93s ago 7m 53.6M - 2.43.0 a07b618ecd1d 05fe6b300615 2024-02-22T15:49:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5cc40692a0 msgr2=0x7f5cc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5cc40692a0 0x7f5cc406b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009b80 tx=0x7f5cc800b040 comp rx=0 tx=0).stop 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 msgr2=0x7f5cd810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd810fa00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4000c00 tx=0x7f5cd4009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 shutdown_connections 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f5cc40692a0 0x7f5cc406b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cd80fbe40 0x7f5cd810fa00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 --2- 172.21.15.19:0/1734558738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cd80fcfb0 0x7f5cd810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 >> 172.21.15.19:0/1734558738 conn(0x7f5cd80f74e0 msgr2=0x7f5cd80656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 shutdown_connections 2024-02-22T15:49:25.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.663+0000 7f5cdf7ea700 1 -- 172.21.15.19:0/1734558738 wait complete. 2024-02-22T15:49:25.659 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-22T15:49:25.660 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-22T15:49:25.661 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-22T15:49:25.699 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-22T15:49:25.703 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T15:49:25.703 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-22T15:49:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 -- 172.21.15.19:0/731646593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 msgr2=0x7fa8980ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 --2- 172.21.15.19:0/731646593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980ffb30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa888004940 tx=0x7fa888009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 -- 172.21.15.19:0/731646593 shutdown_connections 2024-02-22T15:49:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 --2- 172.21.15.19:0/731646593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898100100 0x7fa898102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 --2- 172.21.15.19:0/731646593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980ffb30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 -- 172.21.15.19:0/731646593 >> 172.21.15.19:0/731646593 conn(0x7fa8980f74e0 msgr2=0x7fa8980f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 -- 172.21.15.19:0/731646593 shutdown_connections 2024-02-22T15:49:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.768+0000 7fa89c934700 1 -- 172.21.15.19:0/731646593 wait complete. 2024-02-22T15:49:25.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89c934700 1 Processor -- start 2024-02-22T15:49:25.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89c934700 1 -- start start 2024-02-22T15:49:25.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89c934700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89c934700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898100100 0x7fa8980fba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89c934700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980fc030 con 0x7fa8980663a0 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89c934700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980fc1a0 con 0x7fa898100100 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980fd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54624/0 (socket says 172.21.15.19:54624) 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89659c700 1 -- 172.21.15.19:0/1747609651 learned_addr learned my addr 172.21.15.19:0/1747609651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa895d9b700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898100100 0x7fa8980fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89659c700 1 -- 172.21.15.19:0/1747609651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898100100 msgr2=0x7fa8980fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89659c700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898100100 0x7fa8980fba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.769+0000 7fa89659c700 1 -- 172.21.15.19:0/1747609651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888005040 con 0x7fa8980663a0 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.770+0000 7fa89659c700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980fd5b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa888009380 tx=0x7fa888009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.770+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa88800e040 con 0x7fa8980663a0 2024-02-22T15:49:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.770+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa88800bd20 con 0x7fa8980663a0 2024-02-22T15:49:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.770+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8980fc420 con 0x7fa8980663a0 2024-02-22T15:49:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.770+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa88801c630 con 0x7fa8980663a0 2024-02-22T15:49:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.770+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8981158e0 con 0x7fa8980663a0 2024-02-22T15:49:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.771+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa898107930 con 0x7fa8980663a0 2024-02-22T15:49:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.772+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa888013420 con 0x7fa8980663a0 2024-02-22T15:49:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.772+0000 7fa886ffd700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8800692a0 0x7fa88006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.772+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa888088510 con 0x7fa8980663a0 2024-02-22T15:49:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.772+0000 7fa895d9b700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8800692a0 0x7fa88006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.773+0000 7fa895d9b700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8800692a0 0x7fa88006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8980660e0 tx=0x7fa88c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:25.777+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa8880582a0 con 0x7fa8980663a0 2024-02-22T15:49:25.763 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-22T15:49:25.764 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-22T15:49:25.765 DEBUG:teuthology.orchestra.run.smithi019:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1dc18f444603397768fcb7c34a8488b10284df6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-22T15:49:25.832 INFO:tasks.workunit.client.0.smithi019.stderr:+ mkdir -p fsstress 2024-02-22T15:49:25.834 INFO:tasks.workunit.client.0.smithi019.stderr:+ pushd fsstress 2024-02-22T15:49:25.835 INFO:tasks.workunit.client.0.smithi019.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-22T15:49:25.836 INFO:tasks.workunit.client.0.smithi019.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-22T15:49:26.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.054+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa89802b3f0 con 0x7fa8980663a0 2024-02-22T15:49:26.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.055+0000 7fa886ffd700 1 -- 172.21.15.19:0/1747609651 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fa888057e30 con 0x7fa8980663a0 2024-02-22T15:49:26.037 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:49:26.037 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:49:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:49:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:49:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:49:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:26.038 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 14 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:49:26.039 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8800692a0 msgr2=0x7fa88006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8800692a0 0x7fa88006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8980660e0 tx=0x7fa88c00b040 comp rx=0 tx=0).stop 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 msgr2=0x7fa8980fd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980fd5b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa888009380 tx=0x7fa888009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 shutdown_connections 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7fa8800692a0 0x7fa88006b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8980663a0 0x7fa8980fd5b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 --2- 172.21.15.19:0/1747609651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa898100100 0x7fa8980fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 >> 172.21.15.19:0/1747609651 conn(0x7fa8980f74e0 msgr2=0x7fa8980f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:26.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 shutdown_connections 2024-02-22T15:49:26.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.057+0000 7fa89c934700 1 -- 172.21.15.19:0/1747609651 wait complete. 2024-02-22T15:49:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 -- 172.21.15.19:0/539298681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fbe40 msgr2=0x7f7b3c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 --2- 172.21.15.19:0/539298681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fbe40 0x7f7b3c0fc250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c004940 tx=0x7f7b2c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:26.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 -- 172.21.15.19:0/539298681 shutdown_connections 2024-02-22T15:49:26.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 --2- 172.21.15.19:0/539298681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 --2- 172.21.15.19:0/539298681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fbe40 0x7f7b3c0fc250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 -- 172.21.15.19:0/539298681 >> 172.21.15.19:0/539298681 conn(0x7f7b3c0f7520 msgr2=0x7f7b3c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:26.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 -- 172.21.15.19:0/539298681 shutdown_connections 2024-02-22T15:49:26.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.163+0000 7f7b41696700 1 -- 172.21.15.19:0/539298681 wait complete. 2024-02-22T15:49:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b41696700 1 Processor -- start 2024-02-22T15:49:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b41696700 1 -- start start 2024-02-22T15:49:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b41696700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3c0fbe40 0x7f7b3c10d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:26.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b41696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c10dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b41696700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c10e350 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b41696700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c10ebf0 con 0x7f7b3c0fbe40 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c10dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3a7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c10dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54648/0 (socket says 172.21.15.19:54648) 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3a7fc700 1 -- 172.21.15.19:0/3851495305 learned_addr learned my addr 172.21.15.19:0/3851495305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3affd700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3c0fbe40 0x7f7b3c10d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3a7fc700 1 -- 172.21.15.19:0/3851495305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3c0fbe40 msgr2=0x7f7b3c10d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3a7fc700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3c0fbe40 0x7f7b3c10d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.164+0000 7f7b3a7fc700 1 -- 172.21.15.19:0/3851495305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c005040 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.165+0000 7f7b3a7fc700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c10dd50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000eb70 tx=0x7f7b3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.165+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b30007cc0 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.165+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c10ee70 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.165+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b3000f040 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.165+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b3000c9f0 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.165+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c10f3b0 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.166+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c04bed0 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.166+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7b3000cb50 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.167+0000 7f7b237fe700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7b240692a0 0x7f7b2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.167+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7b300871b0 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.167+0000 7f7b3affd700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7b240692a0 0x7f7b2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:26.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.167+0000 7f7b3affd700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7b240692a0 0x7f7b2406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:26.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.171+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7b30056e80 con 0x7f7b3c0fcfb0 2024-02-22T15:49:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:26 smithi062 ceph-mon[48130]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:49:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:26 smithi062 ceph-mon[48130]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.388+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 --> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b3c10f870 con 0x7f7b240692a0 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.389+0000 7f7b237fe700 1 -- 172.21.15.19:0/3851495305 <== mgr.14206 v2:172.21.15.19:6800/236956964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f7b3c10f870 con 0x7f7b240692a0 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "", 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mgr daemons", 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:49:26.372 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:49:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.391+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7b240692a0 msgr2=0x7f7b2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.391+0000 7f7b41696700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7b240692a0 0x7f7b2406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c000c00 tx=0x7f7b2c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:26.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.391+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 msgr2=0x7f7b3c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.391+0000 7f7b41696700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c10dd50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000eb70 tx=0x7f7b3000b040 comp rx=0 tx=0).stop 2024-02-22T15:49:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 shutdown_connections 2024-02-22T15:49:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:6800/236956964,v1:172.21.15.19:6801/236956964] conn(0x7f7b240692a0 0x7f7b2406b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3c0fbe40 0x7f7b3c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 --2- 172.21.15.19:0/3851495305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3c0fcfb0 0x7f7b3c10dd50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:26.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 >> 172.21.15.19:0/3851495305 conn(0x7f7b3c0f7520 msgr2=0x7f7b3c065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 shutdown_connections 2024-02-22T15:49:26.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:26.392+0000 7f7b41696700 1 -- 172.21.15.19:0/3851495305 wait complete. 2024-02-22T15:49:26.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:26 smithi019 ceph-mon[67418]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:49:26.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:26 smithi019 ceph-mon[67418]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:26.910 INFO:tasks.workunit.client.0.smithi019.stderr:+ tar xzf ltp-full.tgz 2024-02-22T15:49:27.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:27 smithi062 ceph-mon[48130]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:27.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:27 smithi062 ceph-mon[48130]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:27.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:27 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1747609651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:49:27.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:27 smithi019 ceph-mon[67418]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:27.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:27 smithi019 ceph-mon[67418]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:27.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:27 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1747609651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:49:28.392 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:28 smithi062 ceph-mon[48130]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:28.393 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:28 smithi062 ceph-mon[48130]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:28.393 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:28.393 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:28 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:28.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:28 smithi019 ceph-mon[67418]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:28.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:28 smithi019 ceph-mon[67418]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:49:28.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:28.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:28 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:29.925 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:29.925 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:29 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:29.926 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:29 smithi062 ceph-mon[48130]: pgmap v249: 65 pgs: 65 active+clean; 8.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 709 KiB/s wr, 30 op/s 2024-02-22T15:49:29.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:29.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:29 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:29.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:29 smithi019 ceph-mon[67418]: pgmap v249: 65 pgs: 65 active+clean; 8.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 709 KiB/s wr, 30 op/s 2024-02-22T15:49:32.468 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:32 smithi062 ceph-mon[48130]: pgmap v250: 65 pgs: 65 active+clean; 14 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-02-22T15:49:32.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:32 smithi019 ceph-mon[67418]: pgmap v250: 65 pgs: 65 active+clean; 14 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-02-22T15:49:34.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: pgmap v251: 65 pgs: 65 active+clean; 19 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 343 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-02-22T15:49:34.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv restarted 2024-02-22T15:49:34.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:49:34.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr fail", "who": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: osdmap e48: 6 total, 6 up, 6 in 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "mgr fail", "who": "smithi019.yvjrkp"}]': finished 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: mgrmap e20: smithi062.ecubmv(active, starting, since 0.00648685s) 2024-02-22T15:49:34.209 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:49:34.210 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: Manager daemon smithi062.ecubmv is now available 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/trash_purge_schedule"}]: dispatch 2024-02-22T15:49:34.211 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:34 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/trash_purge_schedule"}]: dispatch 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: pgmap v251: 65 pgs: 65 active+clean; 19 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 343 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: Standby manager daemon smithi062.ecubmv restarted 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:34.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr fail", "who": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: osdmap e48: 6 total, 6 up, 6 in 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.14206 172.21.15.19:0/1542922216' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "mgr fail", "who": "smithi019.yvjrkp"}]': finished 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: mgrmap e20: smithi062.ecubmv(active, starting, since 0.00648685s) 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:49:34.498 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: Manager daemon smithi062.ecubmv is now available 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:34.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:49:34.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:49:34.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/trash_purge_schedule"}]: dispatch 2024-02-22T15:49:34.500 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:34 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi062.ecubmv/trash_purge_schedule"}]: dispatch 2024-02-22T15:49:35.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:35 smithi062 ceph-mon[48130]: Deploying cephadm binary to smithi062 2024-02-22T15:49:35.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:35 smithi062 ceph-mon[48130]: Deploying cephadm binary to smithi019 2024-02-22T15:49:35.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:35 smithi062 ceph-mon[48130]: mgrmap e21: smithi062.ecubmv(active, since 1.00821s) 2024-02-22T15:49:35.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:35 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:35.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:35 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:35.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:35 smithi019 ceph-mon[67418]: Deploying cephadm binary to smithi062 2024-02-22T15:49:35.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:35 smithi019 ceph-mon[67418]: Deploying cephadm binary to smithi019 2024-02-22T15:49:35.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:35 smithi019 ceph-mon[67418]: mgrmap e21: smithi062.ecubmv(active, since 1.00821s) 2024-02-22T15:49:35.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:35 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:35.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:35 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:35.592 INFO:tasks.workunit.client.1.smithi062.stderr:Updating files: 85% (10395/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-02-22T15:49:36.403 INFO:tasks.workunit.client.1.smithi062.stderr:Note: switching to 'c1dc18f444603397768fcb7c34a8488b10284df6'. 2024-02-22T15:49:36.403 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.403 INFO:tasks.workunit.client.1.smithi062.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-22T15:49:36.403 INFO:tasks.workunit.client.1.smithi062.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-22T15:49:36.403 INFO:tasks.workunit.client.1.smithi062.stderr:state without impacting any branches by switching back to a branch. 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: git switch -c 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr:Or undo this operation with: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: git switch - 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr: 2024-02-22T15:49:36.404 INFO:tasks.workunit.client.1.smithi062.stderr:HEAD is now at c1dc18f4446 Merge branch 'wip-64505-reef' of https://github.com/batrick/ceph into wip-yuri10-testing-2024-02-21-1249-reef 2024-02-22T15:49:36.408 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-22T15:49:36.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:36 smithi062 ceph-mon[48130]: pgmap v3: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:49:36.490 INFO:tasks.workunit.client.1.smithi062.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-22T15:49:36.492 INFO:tasks.workunit.client.1.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-22T15:49:36.492 INFO:tasks.workunit.client.1.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-22T15:49:36.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:36 smithi019 ceph-mon[67418]: pgmap v3: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:49:37.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:37 smithi062 ceph-mon[48130]: pgmap v4: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:49:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:37 smithi062 ceph-mon[48130]: mgrmap e22: smithi062.ecubmv(active, since 2s) 2024-02-22T15:49:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:37 smithi062 ceph-mon[48130]: [22/Feb/2024:15:49:36] ENGINE Bus STARTING 2024-02-22T15:49:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:37 smithi062 ceph-mon[48130]: [22/Feb/2024:15:49:36] ENGINE Serving on http://172.21.15.62:8765 2024-02-22T15:49:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:37 smithi062 ceph-mon[48130]: [22/Feb/2024:15:49:36] ENGINE Serving on https://172.21.15.62:7150 2024-02-22T15:49:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:37 smithi062 ceph-mon[48130]: [22/Feb/2024:15:49:36] ENGINE Bus STARTED 2024-02-22T15:49:37.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:37 smithi019 ceph-mon[67418]: pgmap v4: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:49:37.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:37 smithi019 ceph-mon[67418]: mgrmap e22: smithi062.ecubmv(active, since 2s) 2024-02-22T15:49:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:37 smithi019 ceph-mon[67418]: [22/Feb/2024:15:49:36] ENGINE Bus STARTING 2024-02-22T15:49:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:37 smithi019 ceph-mon[67418]: [22/Feb/2024:15:49:36] ENGINE Serving on http://172.21.15.62:8765 2024-02-22T15:49:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:37 smithi019 ceph-mon[67418]: [22/Feb/2024:15:49:36] ENGINE Serving on https://172.21.15.62:7150 2024-02-22T15:49:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:37 smithi019 ceph-mon[67418]: [22/Feb/2024:15:49:36] ENGINE Bus STARTED 2024-02-22T15:49:37.719 INFO:tasks.workunit.client.1.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-22T15:49:37.778 INFO:tasks.workunit.client.1.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-22T15:49:37.817 INFO:tasks.workunit.client.1.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-22T15:49:37.819 INFO:tasks.workunit.client.1.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-22T15:49:37.819 INFO:tasks.workunit.client.1.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-22T15:49:37.858 INFO:tasks.workunit.client.1.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-22T15:49:37.862 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-02-22T15:49:37.862 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-22T15:49:37.923 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-22T15:49:37.927 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-22T15:49:37.927 DEBUG:teuthology.orchestra.run.smithi062:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c1dc18f444603397768fcb7c34a8488b10284df6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-22T15:49:37.999 INFO:tasks.workunit.client.1.smithi062.stderr:+ mkdir -p fsstress 2024-02-22T15:49:38.003 INFO:tasks.workunit.client.1.smithi062.stderr:+ pushd fsstress 2024-02-22T15:49:38.004 INFO:tasks.workunit.client.1.smithi062.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-22T15:49:38.004 INFO:tasks.workunit.client.1.smithi062.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-22T15:49:38.674 INFO:tasks.workunit.client.1.smithi062.stderr:+ tar xzf ltp-full.tgz 2024-02-22T15:49:39.352 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:39 smithi019 ceph-mon[67418]: pgmap v5: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:49:39.353 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:39 smithi019 ceph-mon[67418]: mgrmap e23: smithi062.ecubmv(active, since 4s) 2024-02-22T15:49:39.353 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:39 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:49:39.353 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:39 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:39.353 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:39 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:39.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:39 smithi062 ceph-mon[48130]: pgmap v5: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:49:39.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:39 smithi062 ceph-mon[48130]: mgrmap e23: smithi062.ecubmv(active, since 4s) 2024-02-22T15:49:39.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:39 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:49:39.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:39 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:39.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:39 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:40.889 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:40 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:40.889 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:40 smithi062 ceph-mon[48130]: pgmap v6: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 230 op/s 2024-02-22T15:49:40.889 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:40 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:40.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:40 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:40.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:40 smithi019 ceph-mon[67418]: pgmap v6: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 230 op/s 2024-02-22T15:49:40.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:40 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:42.985 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:42 smithi062 ceph-mon[48130]: pgmap v7: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 889 KiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-02-22T15:49:42.985 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:42 smithi062 ceph-mon[48130]: Standby manager daemon smithi019.yvjrkp started 2024-02-22T15:49:42.985 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:42 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/crt"}]: dispatch 2024-02-22T15:49:42.985 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:42 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:49:42.985 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:42 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/key"}]: dispatch 2024-02-22T15:49:42.986 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:42 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:49:42.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:42 smithi019 ceph-mon[67418]: pgmap v7: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 889 KiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-02-22T15:49:42.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:42 smithi019 ceph-mon[67418]: Standby manager daemon smithi019.yvjrkp started 2024-02-22T15:49:42.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:42 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/crt"}]: dispatch 2024-02-22T15:49:42.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:42 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:49:42.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:42 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/key"}]: dispatch 2024-02-22T15:49:42.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:42 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/1946152852' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:49:43.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:43 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:49:43.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:43 smithi019 ceph-mon[67418]: mgrmap e24: smithi062.ecubmv(active, since 9s), standbys: smithi019.yvjrkp 2024-02-22T15:49:44.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:43 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:49:44.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:43 smithi062 ceph-mon[48130]: mgrmap e24: smithi062.ecubmv(active, since 9s), standbys: smithi019.yvjrkp 2024-02-22T15:49:45.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:45 smithi062 ceph-mon[48130]: pgmap v8: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 692 KiB/s rd, 3.2 MiB/s wr, 184 op/s 2024-02-22T15:49:45.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:45 smithi019 ceph-mon[67418]: pgmap v8: 65 pgs: 65 active+clean; 54 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 692 KiB/s rd, 3.2 MiB/s wr, 184 op/s 2024-02-22T15:49:47.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:47 smithi019 ceph-mon[67418]: pgmap v9: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-02-22T15:49:47.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:47 smithi062 ceph-mon[48130]: pgmap v9: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-02-22T15:49:49.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: pgmap v10: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-02-22T15:49:49.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:48 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: pgmap v10: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:48 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:49:50.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi019 to 1570M 2024-02-22T15:49:50.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:50.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:50.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: Adjusting osd_memory_target on smithi062 to 2592M 2024-02-22T15:49:50.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:50.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:49:50.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:49:50.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:49:50.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:50 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:49:50.731 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi019 to 1570M 2024-02-22T15:49:50.731 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:50.731 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:50.731 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.731 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: Adjusting osd_memory_target on smithi062 to 2592M 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:49:50.732 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:50 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:49:51.546 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: pgmap v11: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.547 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.548 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.548 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:51 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:49:51.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: pgmap v11: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:51.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:51 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:52.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:52 smithi062 ceph-mon[48130]: Reconfiguring node-exporter.smithi019 (dependencies changed)... 2024-02-22T15:49:52.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:52 smithi062 ceph-mon[48130]: Reconfiguring daemon node-exporter.smithi019 on smithi019 2024-02-22T15:49:52.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:52 smithi019 ceph-mon[67418]: Reconfiguring node-exporter.smithi019 (dependencies changed)... 2024-02-22T15:49:52.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:52 smithi019 ceph-mon[67418]: Reconfiguring daemon node-exporter.smithi019 on smithi019 2024-02-22T15:49:53.842 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:53 smithi019 ceph-mon[67418]: pgmap v12: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 371 op/s 2024-02-22T15:49:53.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:53 smithi062 ceph-mon[48130]: pgmap v12: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 371 op/s 2024-02-22T15:49:55.053 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:54 smithi019 ceph-mon[67418]: pgmap v13: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 329 op/s 2024-02-22T15:49:55.053 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:54 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:55.053 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:54 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:55.053 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:54 smithi019 ceph-mon[67418]: Reconfiguring alertmanager.smithi019 (dependencies changed)... 2024-02-22T15:49:55.053 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:54 smithi019 ceph-mon[67418]: Reconfiguring daemon alertmanager.smithi019 on smithi019 2024-02-22T15:49:55.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:54 smithi062 ceph-mon[48130]: pgmap v13: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 329 op/s 2024-02-22T15:49:55.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:54 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:55.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:54 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:49:55.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:54 smithi062 ceph-mon[48130]: Reconfiguring alertmanager.smithi019 (dependencies changed)... 2024-02-22T15:49:55.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:54 smithi062 ceph-mon[48130]: Reconfiguring daemon alertmanager.smithi019 on smithi019 2024-02-22T15:49:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.534+0000 7f3a902d4700 1 -- 172.21.15.19:0/2403884639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a881003a0 msgr2=0x7f3a88102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.534+0000 7f3a902d4700 1 --2- 172.21.15.19:0/2403884639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a881003a0 0x7f3a88102780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3a78004940 tx=0x7f3a78009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.534+0000 7f3a902d4700 1 -- 172.21.15.19:0/2403884639 shutdown_connections 2024-02-22T15:49:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.534+0000 7f3a902d4700 1 --2- 172.21.15.19:0/2403884639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a881003a0 0x7f3a88102780 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3a78004940 tx=0x7f3a78009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.534+0000 7f3a902d4700 1 --2- 172.21.15.19:0/2403884639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a880fda80 0x7f3a880ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.534+0000 7f3a902d4700 1 -- 172.21.15.19:0/2403884639 >> 172.21.15.19:0/2403884639 conn(0x7f3a880f74e0 msgr2=0x7f3a880f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.535+0000 7f3a902d4700 1 -- 172.21.15.19:0/2403884639 shutdown_connections 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.535+0000 7f3a902d4700 1 -- 172.21.15.19:0/2403884639 wait complete. 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.536+0000 7f3a902d4700 1 Processor -- start 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.536+0000 7f3a902d4700 1 -- start start 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.536+0000 7f3a902d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a880fda80 0x7f3a8810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.536+0000 7f3a902d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 0x7f3a8810ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.536+0000 7f3a902d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8810e370 con 0x7f3a8810deb0 2024-02-22T15:49:56.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.536+0000 7f3a902d4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8810e4b0 con 0x7f3a880fda80 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 0x7f3a8810ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 0x7f3a8810ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43284/0 (socket says 172.21.15.19:43284) 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 -- 172.21.15.19:0/4041669202 learned_addr learned my addr 172.21.15.19:0/4041669202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8e070700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a880fda80 0x7f3a8810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 -- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a880fda80 msgr2=0x7f3a8810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a880fda80 0x7f3a8810d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 -- 172.21.15.19:0/4041669202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84005040 con 0x7f3a8810deb0 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a8d86f700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 0x7f3a8810ed50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b0d0 tx=0x7f3a78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.537+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a78021070 con 0x7f3a8810deb0 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.538+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a78005040 con 0x7f3a8810deb0 2024-02-22T15:49:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.538+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8810f6d0 con 0x7f3a8810deb0 2024-02-22T15:49:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.538+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a7801c400 con 0x7f3a8810deb0 2024-02-22T15:49:56.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.538+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a7800e070 con 0x7f3a8810deb0 2024-02-22T15:49:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.539+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0052e0 con 0x7f3a8810deb0 2024-02-22T15:49:56.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.540+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f3a78013a50 con 0x7f3a8810deb0 2024-02-22T15:49:56.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.541+0000 7f3a7effd700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a7406aec0 0x7f3a7406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:56.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.541+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3a7808a8e0 con 0x7f3a8810deb0 2024-02-22T15:49:56.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.541+0000 7f3a8e070700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a7406aec0 0x7f3a7406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:56.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.542+0000 7f3a8e070700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a7406aec0 0x7f3a7406d370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009fd0 tx=0x7f3a8400b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:56.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.548+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f3a78058880 con 0x7f3a8810deb0 2024-02-22T15:49:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.840+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a6c000bf0 con 0x7f3a7406aec0 2024-02-22T15:49:56.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.842+0000 7f3a7effd700 1 -- 172.21.15.19:0/4041669202 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3a6c000bf0 con 0x7f3a7406aec0 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.845+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a7406aec0 msgr2=0x7f3a7406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.845+0000 7f3a902d4700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a7406aec0 0x7f3a7406d370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009fd0 tx=0x7f3a8400b040 comp rx=0 tx=0).stop 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.845+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 msgr2=0x7f3a8810ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.845+0000 7f3a902d4700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 0x7f3a8810ed50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b0d0 tx=0x7f3a78009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 shutdown_connections 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a880fda80 0x7f3a8810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a7406aec0 0x7f3a7406d370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 --2- 172.21.15.19:0/4041669202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a8810deb0 0x7f3a8810ed50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 >> 172.21.15.19:0/4041669202 conn(0x7f3a880f74e0 msgr2=0x7f3a880f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 shutdown_connections 2024-02-22T15:49:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.846+0000 7f3a902d4700 1 -- 172.21.15.19:0/4041669202 wait complete. 2024-02-22T15:49:56.843 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:49:56.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.985+0000 7f8f18021700 1 -- 172.21.15.19:0/230421743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 msgr2=0x7f8f100fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.985+0000 7f8f18021700 1 --2- 172.21.15.19:0/230421743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f100fd320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c004940 tx=0x7f8f0c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.985+0000 7f8f18021700 1 -- 172.21.15.19:0/230421743 shutdown_connections 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.985+0000 7f8f18021700 1 --2- 172.21.15.19:0/230421743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f100fe0e0 0x7f8f100fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.985+0000 7f8f18021700 1 --2- 172.21.15.19:0/230421743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f100fd320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.985+0000 7f8f18021700 1 -- 172.21.15.19:0/230421743 >> 172.21.15.19:0/230421743 conn(0x7f8f100f8620 msgr2=0x7f8f100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 -- 172.21.15.19:0/230421743 shutdown_connections 2024-02-22T15:49:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 -- 172.21.15.19:0/230421743 wait complete. 2024-02-22T15:49:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 Processor -- start 2024-02-22T15:49:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 -- start start 2024-02-22T15:49:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f1010d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f100fe0e0 0x7f8f1010dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1010e360 con 0x7f8f100fcef0 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.986+0000 7f8f18021700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1010ec00 con 0x7f8f100fe0e0 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f1010d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f1010d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43306/0 (socket says 172.21.15.19:43306) 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 -- 172.21.15.19:0/3571054228 learned_addr learned my addr 172.21.15.19:0/3571054228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f155bc700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f100fe0e0 0x7f8f1010dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 -- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f100fe0e0 msgr2=0x7f8f1010dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f100fe0e0 0x7f8f1010dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 -- 172.21.15.19:0/3571054228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c005040 con 0x7f8f100fcef0 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.987+0000 7f8f15dbd700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f1010d820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0043c0 tx=0x7f8f0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.988+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0c00e070 con 0x7f8f100fcef0 2024-02-22T15:49:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.988+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f0c009400 con 0x7f8f100fcef0 2024-02-22T15:49:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.988+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0c00b410 con 0x7f8f100fcef0 2024-02-22T15:49:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.988+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1010ee80 con 0x7f8f100fcef0 2024-02-22T15:49:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.988+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1010f360 con 0x7f8f100fcef0 2024-02-22T15:49:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.989+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1004bed0 con 0x7f8f100fcef0 2024-02-22T15:49:56.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.990+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f8f0c02d410 con 0x7f8f100fcef0 2024-02-22T15:49:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.990+0000 7f8f067fc700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f8efc06aec0 0x7f8efc06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.990+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8f0c09c060 con 0x7f8f100fcef0 2024-02-22T15:49:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.991+0000 7f8f155bc700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f8efc06aec0 0x7f8efc06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:56.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.991+0000 7f8f155bc700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f8efc06aec0 0x7f8efc06d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f00004600 tx=0x7f8f0000b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:56.999+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f8f0c069f40 con 0x7f8f100fcef0 2024-02-22T15:49:57.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:56 smithi019 ceph-mon[67418]: pgmap v14: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 537 op/s 2024-02-22T15:49:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.323+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f1010faa0 con 0x7f8efc06aec0 2024-02-22T15:49:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.325+0000 7f8f067fc700 1 -- 172.21.15.19:0/3571054228 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8f1010faa0 con 0x7f8efc06aec0 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f8efc06aec0 msgr2=0x7f8efc06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f8efc06aec0 0x7f8efc06d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f00004600 tx=0x7f8f0000b040 comp rx=0 tx=0).stop 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 msgr2=0x7f8f1010d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f1010d820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0043c0 tx=0x7f8f0c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 shutdown_connections 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f8efc06aec0 0x7f8efc06d370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f100fcef0 0x7f8f1010d820 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 --2- 172.21.15.19:0/3571054228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f100fe0e0 0x7f8f1010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 >> 172.21.15.19:0/3571054228 conn(0x7f8f100f8620 msgr2=0x7f8f101013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 shutdown_connections 2024-02-22T15:49:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.329+0000 7f8f18021700 1 -- 172.21.15.19:0/3571054228 wait complete. 2024-02-22T15:49:57.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:56 smithi062 ceph-mon[48130]: pgmap v14: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 537 op/s 2024-02-22T15:49:57.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.485+0000 7f9978367700 1 -- 172.21.15.19:0/261206471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 msgr2=0x7f9970108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.485+0000 7f9978367700 1 --2- 172.21.15.19:0/261206471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970108fb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f996c004940 tx=0x7f996c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:57.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.485+0000 7f9978367700 1 -- 172.21.15.19:0/261206471 shutdown_connections 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.485+0000 7f9978367700 1 --2- 172.21.15.19:0/261206471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970108fb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.485+0000 7f9978367700 1 --2- 172.21.15.19:0/261206471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f997006e560 0x7f997006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.485+0000 7f9978367700 1 -- 172.21.15.19:0/261206471 >> 172.21.15.19:0/261206471 conn(0x7f9970069e70 msgr2=0x7f997006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.486+0000 7f9978367700 1 -- 172.21.15.19:0/261206471 shutdown_connections 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.486+0000 7f9978367700 1 -- 172.21.15.19:0/261206471 wait complete. 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.486+0000 7f9978367700 1 Processor -- start 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.486+0000 7f9978367700 1 -- start start 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.487+0000 7f9978367700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f997006e560 0x7f9970112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.487+0000 7f9978367700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.487+0000 7f9978367700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970113860 con 0x7f997006eeb0 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.487+0000 7f9978367700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970126380 con 0x7f997006e560 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.488+0000 7f9975902700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.488+0000 7f9975902700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41156/0 (socket says 172.21.15.19:41156) 2024-02-22T15:49:57.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.488+0000 7f9975902700 1 -- 172.21.15.19:0/4034981557 learned_addr learned my addr 172.21.15.19:0/4034981557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.489+0000 7f9976103700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f997006e560 0x7f9970112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.489+0000 7f9975902700 1 -- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f997006e560 msgr2=0x7f9970112c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.489+0000 7f9975902700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f997006e560 0x7f9970112c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.489+0000 7f9975902700 1 -- 172.21.15.19:0/4034981557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c005040 con 0x7f997006eeb0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.490+0000 7f9975902700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970113180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f996c008f80 tx=0x7f996c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.491+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996c00e070 con 0x7f997006eeb0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.491+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f996c013420 con 0x7f997006eeb0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.491+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996c01c640 con 0x7f997006eeb0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.491+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9970126600 con 0x7f997006eeb0 2024-02-22T15:49:57.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.491+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9970126a90 con 0x7f997006eeb0 2024-02-22T15:49:57.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.492+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997010cd10 con 0x7f997006eeb0 2024-02-22T15:49:57.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.494+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f996c01c8b0 con 0x7f997006eeb0 2024-02-22T15:49:57.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.494+0000 7f9962ffd700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f995c06aec0 0x7f995c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:57.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.494+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f996c08b080 con 0x7f997006eeb0 2024-02-22T15:49:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.497+0000 7f9976103700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f995c06aec0 0x7f995c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.497+0000 7f9976103700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f995c06aec0 0x7f995c06d370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9964006fd0 tx=0x7f9964008040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.501+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f996c055510 con 0x7f997006eeb0 2024-02-22T15:49:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.823+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9970126ed0 con 0x7f995c06aec0 2024-02-22T15:49:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.842+0000 7f9962ffd700 1 -- 172.21.15.19:0/4034981557 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f9970126ed0 con 0x7f995c06aec0 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (6m) 10s ago 8m 26.5M - 0.25.0 c8568f914cd2 f9f67af4f00a 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (8m) 10s ago 8m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (7m) 18s ago 7m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (6m) 10s ago 7m 82.9M - 9.4.7 954c08fa6188 1b6b49b44a2a 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (2m) 10s ago 2m 62.7M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (2m) 10s ago 2m 36.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (2m) 18s ago 2m 20.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (2m) 18s ago 2m 20.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:9283 running (9m) 10s ago 9m 169M - 17.2.7-527-g6ea2a55e e2e4ad29904c 1e57e04e2dad 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 running (34s) 18s ago 7m 475M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 60b7b18ac805 2024-02-22T15:49:57.826 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (9m) 10s ago 9m 49.5M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (7m) 18s ago 7m 41.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 starting - - - - 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (7m) 18s ago 7m 16.4M - 1.5.0 0da6a335fe13 379a69e65053 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (6m) 10s ago 6m 69.5M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (5m) 10s ago 5m 71.3M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (5m) 10s ago 5m 62.8M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (4m) 18s ago 4m 61.7M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (4m) 18s ago 4m 63.4M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (4m) 18s ago 4m 61.9M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:49:57.827 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (6m) 10s ago 7m 57.9M - 2.43.0 a07b618ecd1d 05fe6b300615 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.845+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f995c06aec0 msgr2=0x7f995c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.845+0000 7f9978367700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f995c06aec0 0x7f995c06d370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9964006fd0 tx=0x7f9964008040 comp rx=0 tx=0).stop 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.845+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 msgr2=0x7f9970113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.845+0000 7f9978367700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970113180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f996c008f80 tx=0x7f996c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.846+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 shutdown_connections 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.846+0000 7f9978367700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f997006e560 0x7f9970112c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.846+0000 7f9978367700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f995c06aec0 0x7f995c06d370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.846+0000 7f9978367700 1 --2- 172.21.15.19:0/4034981557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f997006eeb0 0x7f9970113180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.846+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 >> 172.21.15.19:0/4034981557 conn(0x7f9970069e70 msgr2=0x7f9970107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.850+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 shutdown_connections 2024-02-22T15:49:57.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:57.850+0000 7f9978367700 1 -- 172.21.15.19:0/4034981557 wait complete. 2024-02-22T15:49:57.977 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:57 smithi019 ceph-mon[67418]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:57.977 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:57 smithi019 ceph-mon[67418]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.002+0000 7f605239b700 1 -- 172.21.15.19:0/1364539821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c0fbe40 msgr2=0x7f604c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.002+0000 7f605239b700 1 --2- 172.21.15.19:0/1364539821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c0fbe40 0x7f604c0fc250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f603c004940 tx=0x7f603c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.002+0000 7f605239b700 1 -- 172.21.15.19:0/1364539821 shutdown_connections 2024-02-22T15:49:57.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.002+0000 7f605239b700 1 --2- 172.21.15.19:0/1364539821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.002+0000 7f605239b700 1 --2- 172.21.15.19:0/1364539821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c0fbe40 0x7f604c0fc250 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f603c004940 tx=0x7f603c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.002+0000 7f605239b700 1 -- 172.21.15.19:0/1364539821 >> 172.21.15.19:0/1364539821 conn(0x7f604c0f74e0 msgr2=0x7f604c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.003+0000 7f605239b700 1 -- 172.21.15.19:0/1364539821 shutdown_connections 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.003+0000 7f605239b700 1 -- 172.21.15.19:0/1364539821 wait complete. 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.003+0000 7f605239b700 1 Processor -- start 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.003+0000 7f605239b700 1 -- start start 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f605239b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c10b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f605239b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c10bcb0 0x7f604c10cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f605239b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c10c170 con 0x7f604c0fcfb0 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f605239b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c10c2e0 con 0x7f604c10bcb0 2024-02-22T15:49:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c10b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c10b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41174/0 (socket says 172.21.15.19:41174) 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604bfff700 1 -- 172.21.15.19:0/3255979185 learned_addr learned my addr 172.21.15.19:0/3255979185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604b7fe700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c10bcb0 0x7f604c10cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604bfff700 1 -- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c10bcb0 msgr2=0x7f604c10cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604bfff700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c10bcb0 0x7f604c10cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.004+0000 7f604bfff700 1 -- 172.21.15.19:0/3255979185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c005040 con 0x7f604c0fcfb0 2024-02-22T15:49:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.005+0000 7f604bfff700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c10b770 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f603c000c00 tx=0x7f603c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.005+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603c021070 con 0x7f604c0fcfb0 2024-02-22T15:49:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.005+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f603c01c400 con 0x7f604c0fcfb0 2024-02-22T15:49:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.005+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c10d1a0 con 0x7f604c0fcfb0 2024-02-22T15:49:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.005+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c10d690 con 0x7f604c0fcfb0 2024-02-22T15:49:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.005+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603c00e030 con 0x7f604c0fcfb0 2024-02-22T15:49:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.006+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60380052e0 con 0x7f604c0fcfb0 2024-02-22T15:49:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.007+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f603c01c9c0 con 0x7f604c0fcfb0 2024-02-22T15:49:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.008+0000 7f6048ff9700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f603406aec0 0x7f603406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.008+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f603c08a8f0 con 0x7f604c0fcfb0 2024-02-22T15:49:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.008+0000 7f604b7fe700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f603406aec0 0x7f603406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.009+0000 7f604b7fe700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f603406aec0 0x7f603406d370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f604c10c7e0 tx=0x7f604000b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.015+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f603c054e60 con 0x7f604c0fcfb0 2024-02-22T15:49:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.448+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6038005860 con 0x7f604c0fcfb0 2024-02-22T15:49:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.448+0000 7f6048ff9700 1 -- 172.21.15.19:0/3255979185 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+848 (secure 0 0 0) 0x7f603c058480 con 0x7f604c0fcfb0 2024-02-22T15:49:58.430 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:49:58.430 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 1, 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:49:58.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 13, 2024-02-22T15:49:58.432 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:49:58.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:49:58.432 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f603406aec0 msgr2=0x7f603406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f603406aec0 0x7f603406d370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f604c10c7e0 tx=0x7f604000b040 comp rx=0 tx=0).stop 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 msgr2=0x7f604c10b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c10b770 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f603c000c00 tx=0x7f603c009e30 comp rx=0 tx=0).stop 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 shutdown_connections 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f603406aec0 0x7f603406d370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f604c0fcfb0 0x7f604c10b770 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 --2- 172.21.15.19:0/3255979185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604c10bcb0 0x7f604c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 >> 172.21.15.19:0/3255979185 conn(0x7f604c0f74e0 msgr2=0x7f604c0f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 shutdown_connections 2024-02-22T15:49:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.451+0000 7f605239b700 1 -- 172.21.15.19:0/3255979185 wait complete. 2024-02-22T15:49:58.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:57 smithi062 ceph-mon[48130]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:58.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:57 smithi062 ceph-mon[48130]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 -- 172.21.15.19:0/2509280803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 msgr2=0x7f0dc80fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2509280803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc80fcc40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0db8004a50 tx=0x7f0db800af40 comp rx=0 tx=0).stop 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 -- 172.21.15.19:0/2509280803 shutdown_connections 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2509280803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc80fcc40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2509280803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc80fb5e0 0x7f0dc80fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 -- 172.21.15.19:0/2509280803 >> 172.21.15.19:0/2509280803 conn(0x7f0dc80f6ca0 msgr2=0x7f0dc80f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 -- 172.21.15.19:0/2509280803 shutdown_connections 2024-02-22T15:49:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.588+0000 7f0dd0636700 1 -- 172.21.15.19:0/2509280803 wait complete. 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dd0636700 1 Processor -- start 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dd0636700 1 -- start start 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dd0636700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc80fb5e0 0x7f0dc810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dd0636700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc8110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dd0636700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8110690 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dd0636700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8110f30 con 0x7f0dc80fb5e0 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dcdbd1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc8110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dcdbd1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc8110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41200/0 (socket says 172.21.15.19:41200) 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dcdbd1700 1 -- 172.21.15.19:0/2381765965 learned_addr learned my addr 172.21.15.19:0/2381765965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dce3d2700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc80fb5e0 0x7f0dc810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dcdbd1700 1 -- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc80fb5e0 msgr2=0x7f0dc810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dcdbd1700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc80fb5e0 0x7f0dc810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.589+0000 7f0dcdbd1700 1 -- 172.21.15.19:0/2381765965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8005040 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.590+0000 7f0dcdbd1700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc8110090 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0db80049a0 tx=0x7f0db800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.590+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db800f050 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.590+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0db800a9f0 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.590+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc81111b0 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.590+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db8014770 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.590+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc8111690 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.591+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc8109b50 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.592+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f0db801db30 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.592+0000 7f0dbeffd700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f0db406aec0 0x7f0db406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:49:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.592+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0db808b4a0 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.592+0000 7f0dce3d2700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f0db406aec0 0x7f0db406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:49:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.593+0000 7f0dce3d2700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f0db406aec0 0x7f0db406d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009b80 tx=0x7f0dc400b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:49:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.598+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f0db8059380 con 0x7f0dc80fc7f0 2024-02-22T15:49:58.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.874+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dc8111970 con 0x7f0db406aec0 2024-02-22T15:49:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.876+0000 7f0dbeffd700 1 -- 172.21.15.19:0/2381765965 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0dc8111970 con 0x7f0db406aec0 2024-02-22T15:49:58.858 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:49:58.858 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:49:58.858 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:49:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:49:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T15:49:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "1/2 daemons upgraded", 2024-02-22T15:49:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-02-22T15:49:58.859 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:49:58.859 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:49:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.879+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f0db406aec0 msgr2=0x7f0db406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.879+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f0db406aec0 0x7f0db406d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009b80 tx=0x7f0dc400b040 comp rx=0 tx=0).stop 2024-02-22T15:49:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.879+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 msgr2=0x7f0dc8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:49:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.879+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc8110090 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0db80049a0 tx=0x7f0db800af40 comp rx=0 tx=0).stop 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 shutdown_connections 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc80fb5e0 0x7f0dc810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f0db406aec0 0x7f0db406d370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 --2- 172.21.15.19:0/2381765965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc80fc7f0 0x7f0dc8110090 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 >> 172.21.15.19:0/2381765965 conn(0x7f0dc80f6ca0 msgr2=0x7f0dc80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 shutdown_connections 2024-02-22T15:49:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:49:58.880+0000 7f0dd0636700 1 -- 172.21.15.19:0/2381765965 wait complete. 2024-02-22T15:49:59.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:58 smithi019 ceph-mon[67418]: pgmap v15: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 398 op/s 2024-02-22T15:49:59.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:58 smithi019 ceph-mon[67418]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:59.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:49:58 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3255979185' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:49:59.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:58 smithi062 ceph-mon[48130]: pgmap v15: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 398 op/s 2024-02-22T15:49:59.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:58 smithi062 ceph-mon[48130]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:49:59.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:49:58 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3255979185' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:50:00.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:00 smithi062 ceph-mon[48130]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:00.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:00 smithi062 ceph-mon[48130]: overall HEALTH_OK 2024-02-22T15:50:00.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:00 smithi019 ceph-mon[67418]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:00.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:00 smithi019 ceph-mon[67418]: overall HEALTH_OK 2024-02-22T15:50:01.354 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:01 smithi019 ceph-mon[67418]: pgmap v16: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 624 op/s 2024-02-22T15:50:01.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:01 smithi062 ceph-mon[48130]: pgmap v16: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 624 op/s 2024-02-22T15:50:03.328 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:03 smithi019 ceph-mon[67418]: pgmap v17: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-02-22T15:50:03.328 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:03 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:03.328 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:03 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:03.328 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:03 smithi019 ceph-mon[67418]: Reconfiguring grafana.smithi019 (dependencies changed)... 2024-02-22T15:50:03.328 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:03 smithi019 ceph-mon[67418]: Reconfiguring daemon grafana.smithi019 on smithi019 2024-02-22T15:50:03.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:03 smithi062 ceph-mon[48130]: pgmap v17: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 498 op/s 2024-02-22T15:50:03.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:03 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:03.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:03 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:03.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:03 smithi062 ceph-mon[48130]: Reconfiguring grafana.smithi019 (dependencies changed)... 2024-02-22T15:50:03.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:03 smithi062 ceph-mon[48130]: Reconfiguring daemon grafana.smithi019 on smithi019 2024-02-22T15:50:04.369 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:04 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:04.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:04 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:05.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:05 smithi019 ceph-mon[67418]: pgmap v18: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 434 op/s 2024-02-22T15:50:05.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:05 smithi062 ceph-mon[48130]: pgmap v18: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 434 op/s 2024-02-22T15:50:07.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:07 smithi062 ceph-mon[48130]: pgmap v19: 65 pgs: 65 active+clean; 147 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 649 op/s 2024-02-22T15:50:07.978 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:07 smithi019 ceph-mon[67418]: pgmap v19: 65 pgs: 65 active+clean; 147 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 649 op/s 2024-02-22T15:50:09.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:09 smithi062 ceph-mon[48130]: pgmap v20: 65 pgs: 65 active+clean; 147 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 441 op/s 2024-02-22T15:50:09.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:09 smithi019 ceph-mon[67418]: pgmap v20: 65 pgs: 65 active+clean; 147 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 441 op/s 2024-02-22T15:50:10.990 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:10 smithi019 ceph-mon[67418]: pgmap v21: 65 pgs: 65 active+clean; 173 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 647 op/s 2024-02-22T15:50:10.991 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:10 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:10.991 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:10 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:10.991 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:10 smithi019 ceph-mon[67418]: Reconfiguring prometheus.smithi019 (dependencies changed)... 2024-02-22T15:50:10.991 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:10 smithi019 ceph-mon[67418]: Reconfiguring daemon prometheus.smithi019 on smithi019 2024-02-22T15:50:11.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:10 smithi062 ceph-mon[48130]: pgmap v21: 65 pgs: 65 active+clean; 173 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 647 op/s 2024-02-22T15:50:11.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:10 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:11.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:10 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:11.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:10 smithi062 ceph-mon[48130]: Reconfiguring prometheus.smithi019 (dependencies changed)... 2024-02-22T15:50:11.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:10 smithi062 ceph-mon[48130]: Reconfiguring daemon prometheus.smithi019 on smithi019 2024-02-22T15:50:12.013 INFO:tasks.workunit.client.0.smithi019.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-22T15:50:12.020 INFO:tasks.workunit.client.0.smithi019.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-22T15:50:12.021 INFO:tasks.workunit.client.0.smithi019.stderr:+ make 2024-02-22T15:50:12.358 INFO:tasks.workunit.client.0.smithi019.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-22T15:50:12.897 INFO:tasks.workunit.client.0.smithi019.stderr:++ readlink -f fsstress 2024-02-22T15:50:12.898 INFO:tasks.workunit.client.0.smithi019.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-22T15:50:12.899 INFO:tasks.workunit.client.0.smithi019.stderr:+ popd 2024-02-22T15:50:12.899 INFO:tasks.workunit.client.0.smithi019.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-22T15:50:12.900 INFO:tasks.workunit.client.0.smithi019.stderr:+ popd 2024-02-22T15:50:12.900 INFO:tasks.workunit.client.0.smithi019.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-22T15:50:12.900 INFO:tasks.workunit.client.0.smithi019.stderr:++ mktemp -d -p . 2024-02-22T15:50:12.903 INFO:tasks.workunit.client.0.smithi019.stderr:+ T=./tmp.rgMABlLemQ 2024-02-22T15:50:12.903 INFO:tasks.workunit.client.0.smithi019.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.rgMABlLemQ -l 1 -n 1000 -p 10 -v 2024-02-22T15:50:12.905 INFO:tasks.workunit.client.0.smithi019.stdout:seed = 1708749599 2024-02-22T15:50:12.911 INFO:tasks.workunit.client.0.smithi019.stdout:1/0: dread - no filename 2024-02-22T15:50:12.911 INFO:tasks.workunit.client.0.smithi019.stdout:1/1: dread - no filename 2024-02-22T15:50:12.911 INFO:tasks.workunit.client.0.smithi019.stdout:1/2: truncate - no filename 2024-02-22T15:50:12.911 INFO:tasks.workunit.client.0.smithi019.stdout:1/3: unlink - no file 2024-02-22T15:50:12.911 INFO:tasks.workunit.client.0.smithi019.stdout:0/0: symlink l0 0 2024-02-22T15:50:12.912 INFO:tasks.workunit.client.0.smithi019.stdout:2/0: dread - no filename 2024-02-22T15:50:12.912 INFO:tasks.workunit.client.0.smithi019.stdout:2/1: rename - no filename 2024-02-22T15:50:12.913 INFO:tasks.workunit.client.0.smithi019.stdout:0/1: creat f1 x:0 0 0 2024-02-22T15:50:12.913 INFO:tasks.workunit.client.0.smithi019.stdout:1/4: symlink l0 0 2024-02-22T15:50:12.913 INFO:tasks.workunit.client.0.smithi019.stdout:1/5: write - no filename 2024-02-22T15:50:12.913 INFO:tasks.workunit.client.0.smithi019.stdout:1/6: dread - no filename 2024-02-22T15:50:12.913 INFO:tasks.workunit.client.0.smithi019.stdout:1/7: dread - no filename 2024-02-22T15:50:12.914 INFO:tasks.workunit.client.0.smithi019.stdout:3/0: stat - no entries 2024-02-22T15:50:12.914 INFO:tasks.workunit.client.0.smithi019.stdout:3/1: write - no filename 2024-02-22T15:50:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:1/8: mknod c1 0 2024-02-22T15:50:12.916 INFO:tasks.workunit.client.0.smithi019.stdout:2/2: mknod c0 0 2024-02-22T15:50:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:2/3: chown c0 7227575 1 2024-02-22T15:50:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/0: dwrite - no filename 2024-02-22T15:50:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/1: dwrite - no filename 2024-02-22T15:50:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/2: chown . 2256 1 2024-02-22T15:50:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/3: truncate - no filename 2024-02-22T15:50:12.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/4: write - no filename 2024-02-22T15:50:12.918 INFO:tasks.workunit.client.0.smithi019.stdout:1/9: unlink l0 0 2024-02-22T15:50:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:1/10: chown c1 131 1 2024-02-22T15:50:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:1/11: dread - no filename 2024-02-22T15:50:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:1/12: write - no filename 2024-02-22T15:50:12.919 INFO:tasks.workunit.client.0.smithi019.stdout:1/13: write - no filename 2024-02-22T15:50:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:5/0: chown . 82680 1 2024-02-22T15:50:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:5/1: write - no filename 2024-02-22T15:50:12.920 INFO:tasks.workunit.client.0.smithi019.stdout:3/2: symlink l0 0 2024-02-22T15:50:12.921 INFO:tasks.workunit.client.0.smithi019.stdout:3/3: stat l0 0 2024-02-22T15:50:12.922 INFO:tasks.workunit.client.0.smithi019.stdout:2/4: mkdir d1 0 2024-02-22T15:50:12.922 INFO:tasks.workunit.client.0.smithi019.stdout:2/5: chown c0 52073 1 2024-02-22T15:50:12.924 INFO:tasks.workunit.client.0.smithi019.stdout:1/14: rename c1 to c2 0 2024-02-22T15:50:12.925 INFO:tasks.workunit.client.0.smithi019.stdout:1/15: read - no filename 2024-02-22T15:50:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:6/0: stat - no entries 2024-02-22T15:50:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:6/1: rename - no filename 2024-02-22T15:50:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:6/2: rmdir - no directory 2024-02-22T15:50:12.927 INFO:tasks.workunit.client.0.smithi019.stdout:0/2: fsync f1 0 2024-02-22T15:50:12.928 INFO:tasks.workunit.client.0.smithi019.stdout:3/4: rename l0 to l1 0 2024-02-22T15:50:12.929 INFO:tasks.workunit.client.0.smithi019.stdout:4/5: symlink l0 0 2024-02-22T15:50:12.931 INFO:tasks.workunit.client.0.smithi019.stdout:1/16: creat f3 x:0 0 0 2024-02-22T15:50:12.932 INFO:tasks.workunit.client.0.smithi019.stdout:1/17: write f3 [348465,124048] 0 2024-02-22T15:50:12.932 INFO:tasks.workunit.client.0.smithi019.stdout:5/2: creat f0 x:0 0 0 2024-02-22T15:50:12.934 INFO:tasks.workunit.client.0.smithi019.stdout:1/18: truncate f3 845952 0 2024-02-22T15:50:12.934 INFO:tasks.workunit.client.0.smithi019.stdout:5/3: truncate f0 18001 0 2024-02-22T15:50:12.934 INFO:tasks.workunit.client.0.smithi019.stdout:4/6: mknod c1 0 2024-02-22T15:50:12.935 INFO:tasks.workunit.client.0.smithi019.stdout:2/6: symlink d1/l2 0 2024-02-22T15:50:12.935 INFO:tasks.workunit.client.0.smithi019.stdout:4/7: chown c1 15 1 2024-02-22T15:50:12.937 INFO:tasks.workunit.client.0.smithi019.stdout:8/0: stat - no entries 2024-02-22T15:50:12.937 INFO:tasks.workunit.client.0.smithi019.stdout:8/1: write - no filename 2024-02-22T15:50:12.937 INFO:tasks.workunit.client.0.smithi019.stdout:0/3: creat f2 x:0 0 0 2024-02-22T15:50:12.937 INFO:tasks.workunit.client.0.smithi019.stdout:0/4: rmdir - no directory 2024-02-22T15:50:12.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:12 smithi019 ceph-mon[67418]: pgmap v22: 65 pgs: 65 active+clean; 173 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-02-22T15:50:12.938 INFO:tasks.workunit.client.0.smithi019.stdout:0/5: fdatasync f2 0 2024-02-22T15:50:12.939 INFO:tasks.workunit.client.0.smithi019.stdout:6/3: creat f0 x:0 0 0 2024-02-22T15:50:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:1/19: mknod c4 0 2024-02-22T15:50:12.940 INFO:tasks.workunit.client.0.smithi019.stdout:2/7: mknod d1/c3 0 2024-02-22T15:50:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:4/8: mkdir d2 0 2024-02-22T15:50:12.941 INFO:tasks.workunit.client.0.smithi019.stdout:4/9: truncate - no filename 2024-02-22T15:50:12.945 INFO:tasks.workunit.client.0.smithi019.stdout:7/0: symlink l0 0 2024-02-22T15:50:12.947 INFO:tasks.workunit.client.0.smithi019.stdout:0/6: mknod c3 0 2024-02-22T15:50:12.950 INFO:tasks.workunit.client.0.smithi019.stdout:7/1: chown l0 14360 1 2024-02-22T15:50:12.951 INFO:tasks.workunit.client.0.smithi019.stdout:8/2: creat f0 x:0 0 0 2024-02-22T15:50:12.951 INFO:tasks.workunit.client.0.smithi019.stdout:0/7: dread - f1 zero size 2024-02-22T15:50:12.951 INFO:tasks.workunit.client.0.smithi019.stdout:5/4: dwrite f0 [0,4194304] 0 2024-02-22T15:50:12.956 INFO:tasks.workunit.client.0.smithi019.stdout:7/2: chown l0 59210920 1 2024-02-22T15:50:12.956 INFO:tasks.workunit.client.0.smithi019.stdout:7/3: dwrite - no filename 2024-02-22T15:50:12.959 INFO:tasks.workunit.client.0.smithi019.stdout:3/5: readlink l1 0 2024-02-22T15:50:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:4/10: getdents d2 0 2024-02-22T15:50:12.960 INFO:tasks.workunit.client.0.smithi019.stdout:1/20: dwrite f3 [0,4194304] 0 2024-02-22T15:50:12.962 INFO:tasks.workunit.client.0.smithi019.stdout:0/8: mkdir d4 0 2024-02-22T15:50:12.965 INFO:tasks.workunit.client.0.smithi019.stdout:7/4: creat f1 x:0 0 0 2024-02-22T15:50:12.967 INFO:tasks.workunit.client.0.smithi019.stdout:6/4: dwrite f0 [0,4194304] 0 2024-02-22T15:50:12.967 INFO:tasks.workunit.client.0.smithi019.stdout:9/0: creat f0 x:0 0 0 2024-02-22T15:50:12.967 INFO:tasks.workunit.client.0.smithi019.stdout:9/1: readlink - no filename 2024-02-22T15:50:12.967 INFO:tasks.workunit.client.0.smithi019.stdout:9/2: rmdir - no directory 2024-02-22T15:50:12.968 INFO:tasks.workunit.client.0.smithi019.stdout:6/5: readlink - no filename 2024-02-22T15:50:12.968 INFO:tasks.workunit.client.0.smithi019.stdout:3/6: mknod c2 0 2024-02-22T15:50:12.968 INFO:tasks.workunit.client.0.smithi019.stdout:3/7: truncate - no filename 2024-02-22T15:50:12.969 INFO:tasks.workunit.client.0.smithi019.stdout:6/6: write f0 [2177913,68196] 0 2024-02-22T15:50:12.970 INFO:tasks.workunit.client.0.smithi019.stdout:2/8: link d1/l2 d1/l4 0 2024-02-22T15:50:12.971 INFO:tasks.workunit.client.0.smithi019.stdout:7/5: mknod c2 0 2024-02-22T15:50:12.975 INFO:tasks.workunit.client.0.smithi019.stdout:7/6: fdatasync f1 0 2024-02-22T15:50:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:4/11: mknod d2/c3 0 2024-02-22T15:50:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:0/9: mkdir d4/d5 0 2024-02-22T15:50:12.978 INFO:tasks.workunit.client.0.smithi019.stdout:6/7: symlink l1 0 2024-02-22T15:50:12.980 INFO:tasks.workunit.client.0.smithi019.stdout:9/3: mkdir d1 0 2024-02-22T15:50:12.980 INFO:tasks.workunit.client.0.smithi019.stdout:3/8: chown l1 0 1 2024-02-22T15:50:12.980 INFO:tasks.workunit.client.0.smithi019.stdout:3/9: dread - no filename 2024-02-22T15:50:12.981 INFO:tasks.workunit.client.0.smithi019.stdout:0/10: dread - f1 zero size 2024-02-22T15:50:12.981 INFO:tasks.workunit.client.0.smithi019.stdout:8/3: dwrite f0 [0,4194304] 0 2024-02-22T15:50:12.982 INFO:tasks.workunit.client.0.smithi019.stdout:4/12: mknod d2/c4 0 2024-02-22T15:50:12.982 INFO:tasks.workunit.client.0.smithi019.stdout:6/8: mknod c2 0 2024-02-22T15:50:12.984 INFO:tasks.workunit.client.0.smithi019.stdout:0/11: chown d4 94426 1 2024-02-22T15:50:12.985 INFO:tasks.workunit.client.0.smithi019.stdout:6/9: stat l1 0 2024-02-22T15:50:12.988 INFO:tasks.workunit.client.0.smithi019.stdout:2/9: rename d1/l2 to d1/l5 0 2024-02-22T15:50:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:2/10: write - no filename 2024-02-22T15:50:12.989 INFO:tasks.workunit.client.0.smithi019.stdout:2/11: dread - no filename 2024-02-22T15:50:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:1/21: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:1/22: readlink - no filename 2024-02-22T15:50:13.131 INFO:tasks.workunit.client.0.smithi019.stdout:3/10: creat f3 x:0 0 0 2024-02-22T15:50:13.132 INFO:tasks.workunit.client.0.smithi019.stdout:4/13: mknod d2/c5 0 2024-02-22T15:50:13.132 INFO:tasks.workunit.client.0.smithi019.stdout:6/10: creat f3 x:0 0 0 2024-02-22T15:50:13.135 INFO:tasks.workunit.client.0.smithi019.stdout:6/11: fdatasync f0 0 2024-02-22T15:50:13.137 INFO:tasks.workunit.client.0.smithi019.stdout:7/7: dwrite f1 [0,4194304] 0 2024-02-22T15:50:13.140 INFO:tasks.workunit.client.0.smithi019.stdout:6/12: truncate f3 488640 0 2024-02-22T15:50:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:3/11: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.149 INFO:tasks.workunit.client.0.smithi019.stdout:8/4: dwrite f0 [4194304,4194304] 0 2024-02-22T15:50:13.158 INFO:tasks.workunit.client.0.smithi019.stdout:8/5: dwrite f0 [0,4194304] 0 2024-02-22T15:50:13.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:12 smithi062 ceph-mon[48130]: pgmap v22: 65 pgs: 65 active+clean; 173 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-02-22T15:50:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:9/4: rmdir d1 0 2024-02-22T15:50:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:9/5: rmdir - no directory 2024-02-22T15:50:13.239 INFO:tasks.workunit.client.0.smithi019.stdout:9/6: fsync f0 0 2024-02-22T15:50:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:9/7: write f0 [892796,105409] 0 2024-02-22T15:50:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:4/14: creat d2/f6 x:0 0 0 2024-02-22T15:50:13.241 INFO:tasks.workunit.client.0.smithi019.stdout:0/12: getdents d4/d5 0 2024-02-22T15:50:13.243 INFO:tasks.workunit.client.0.smithi019.stdout:7/8: unlink l0 0 2024-02-22T15:50:13.244 INFO:tasks.workunit.client.0.smithi019.stdout:6/13: mknod c4 0 2024-02-22T15:50:13.247 INFO:tasks.workunit.client.0.smithi019.stdout:7/9: chown f1 1 1 2024-02-22T15:50:13.248 INFO:tasks.workunit.client.0.smithi019.stdout:8/6: rename f0 to f1 0 2024-02-22T15:50:13.251 INFO:tasks.workunit.client.0.smithi019.stdout:4/15: symlink d2/l7 0 2024-02-22T15:50:13.253 INFO:tasks.workunit.client.0.smithi019.stdout:9/8: dwrite f0 [0,4194304] 0 2024-02-22T15:50:13.256 INFO:tasks.workunit.client.0.smithi019.stdout:7/10: creat f3 x:0 0 0 2024-02-22T15:50:13.258 INFO:tasks.workunit.client.0.smithi019.stdout:0/13: dwrite f2 [0,4194304] 0 2024-02-22T15:50:13.262 INFO:tasks.workunit.client.0.smithi019.stdout:0/14: write f1 [499608,59528] 0 2024-02-22T15:50:13.264 INFO:tasks.workunit.client.0.smithi019.stdout:0/15: readlink l0 0 2024-02-22T15:50:13.265 INFO:tasks.workunit.client.0.smithi019.stdout:8/7: dread f1 [4194304,4194304] 0 2024-02-22T15:50:13.266 INFO:tasks.workunit.client.0.smithi019.stdout:7/11: unlink c2 0 2024-02-22T15:50:13.267 INFO:tasks.workunit.client.0.smithi019.stdout:2/12: link d1/l5 d1/l6 0 2024-02-22T15:50:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:7/12: read f1 [1239864,88346] 0 2024-02-22T15:50:13.269 INFO:tasks.workunit.client.0.smithi019.stdout:1/23: getdents . 0 2024-02-22T15:50:13.270 INFO:tasks.workunit.client.0.smithi019.stdout:4/16: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:13.272 INFO:tasks.workunit.client.0.smithi019.stdout:7/13: read f1 [1560933,76298] 0 2024-02-22T15:50:13.275 INFO:tasks.workunit.client.0.smithi019.stdout:8/8: mknod c2 0 2024-02-22T15:50:13.275 INFO:tasks.workunit.client.0.smithi019.stdout:3/12: getdents . 0 2024-02-22T15:50:13.277 INFO:tasks.workunit.client.0.smithi019.stdout:2/13: rmdir d1 39 2024-02-22T15:50:13.277 INFO:tasks.workunit.client.0.smithi019.stdout:2/14: fdatasync - no filename 2024-02-22T15:50:13.278 INFO:tasks.workunit.client.0.smithi019.stdout:3/13: chown f3 9 1 2024-02-22T15:50:13.280 INFO:tasks.workunit.client.0.smithi019.stdout:1/24: creat f5 x:0 0 0 2024-02-22T15:50:13.282 INFO:tasks.workunit.client.0.smithi019.stdout:0/16: symlink d4/d5/l6 0 2024-02-22T15:50:13.284 INFO:tasks.workunit.client.0.smithi019.stdout:4/17: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:13.290 INFO:tasks.workunit.client.0.smithi019.stdout:1/25: mknod c6 0 2024-02-22T15:50:13.294 INFO:tasks.workunit.client.0.smithi019.stdout:1/26: write f5 [668259,108357] 0 2024-02-22T15:50:13.295 INFO:tasks.workunit.client.0.smithi019.stdout:4/18: mknod d2/c8 0 2024-02-22T15:50:13.300 INFO:tasks.workunit.client.0.smithi019.stdout:2/15: unlink d1/l4 0 2024-02-22T15:50:13.300 INFO:tasks.workunit.client.0.smithi019.stdout:4/19: chown d2/c4 8 1 2024-02-22T15:50:13.300 INFO:tasks.workunit.client.0.smithi019.stdout:8/9: dwrite f1 [4194304,4194304] 0 2024-02-22T15:50:13.306 INFO:tasks.workunit.client.0.smithi019.stdout:0/17: dwrite f1 [0,4194304] 0 2024-02-22T15:50:13.309 INFO:tasks.workunit.client.0.smithi019.stdout:8/10: creat f3 x:0 0 0 2024-02-22T15:50:13.310 INFO:tasks.workunit.client.0.smithi019.stdout:0/18: chown d4/d5/l6 111614 1 2024-02-22T15:50:13.314 INFO:tasks.workunit.client.0.smithi019.stdout:1/27: dread f3 [0,4194304] 0 2024-02-22T15:50:13.321 INFO:tasks.workunit.client.0.smithi019.stdout:2/16: symlink d1/l7 0 2024-02-22T15:50:13.323 INFO:tasks.workunit.client.0.smithi019.stdout:0/19: dwrite f2 [0,4194304] 0 2024-02-22T15:50:13.323 INFO:tasks.workunit.client.0.smithi019.stdout:8/11: mknod c4 0 2024-02-22T15:50:13.324 INFO:tasks.workunit.client.0.smithi019.stdout:4/20: link d2/c3 d2/c9 0 2024-02-22T15:50:13.325 INFO:tasks.workunit.client.0.smithi019.stdout:8/12: write f1 [7892306,104087] 0 2024-02-22T15:50:13.326 INFO:tasks.workunit.client.0.smithi019.stdout:1/28: fdatasync f3 0 2024-02-22T15:50:13.327 INFO:tasks.workunit.client.0.smithi019.stdout:2/17: mknod d1/c8 0 2024-02-22T15:50:13.327 INFO:tasks.workunit.client.0.smithi019.stdout:2/18: dread - no filename 2024-02-22T15:50:13.327 INFO:tasks.workunit.client.0.smithi019.stdout:2/19: dread - no filename 2024-02-22T15:50:13.327 INFO:tasks.workunit.client.0.smithi019.stdout:1/29: write f5 [287309,65011] 0 2024-02-22T15:50:13.328 INFO:tasks.workunit.client.0.smithi019.stdout:2/20: chown d1/c8 1 1 2024-02-22T15:50:13.331 INFO:tasks.workunit.client.0.smithi019.stdout:0/20: mkdir d4/d5/d7 0 2024-02-22T15:50:13.336 INFO:tasks.workunit.client.0.smithi019.stdout:4/21: stat d2/c9 0 2024-02-22T15:50:13.338 INFO:tasks.workunit.client.0.smithi019.stdout:1/30: unlink c6 0 2024-02-22T15:50:13.339 INFO:tasks.workunit.client.0.smithi019.stdout:8/13: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.340 INFO:tasks.workunit.client.0.smithi019.stdout:2/21: creat d1/f9 x:0 0 0 2024-02-22T15:50:13.341 INFO:tasks.workunit.client.0.smithi019.stdout:8/14: chown c2 759 1 2024-02-22T15:50:13.341 INFO:tasks.workunit.client.0.smithi019.stdout:0/21: symlink d4/l8 0 2024-02-22T15:50:13.344 INFO:tasks.workunit.client.0.smithi019.stdout:1/31: symlink l7 0 2024-02-22T15:50:13.345 INFO:tasks.workunit.client.0.smithi019.stdout:1/32: fdatasync f5 0 2024-02-22T15:50:13.346 INFO:tasks.workunit.client.0.smithi019.stdout:2/22: unlink d1/c3 0 2024-02-22T15:50:13.352 INFO:tasks.workunit.client.0.smithi019.stdout:8/15: dread f3 [0,4194304] 0 2024-02-22T15:50:13.352 INFO:tasks.workunit.client.0.smithi019.stdout:8/16: stat f3 0 2024-02-22T15:50:13.352 INFO:tasks.workunit.client.0.smithi019.stdout:1/33: mkdir d8 0 2024-02-22T15:50:13.354 INFO:tasks.workunit.client.0.smithi019.stdout:8/17: creat f5 x:0 0 0 2024-02-22T15:50:13.357 INFO:tasks.workunit.client.0.smithi019.stdout:1/34: creat d8/f9 x:0 0 0 2024-02-22T15:50:13.357 INFO:tasks.workunit.client.0.smithi019.stdout:8/18: truncate f5 751109 0 2024-02-22T15:50:13.357 INFO:tasks.workunit.client.0.smithi019.stdout:0/22: dwrite f1 [0,4194304] 0 2024-02-22T15:50:13.358 INFO:tasks.workunit.client.0.smithi019.stdout:1/35: chown f5 53 1 2024-02-22T15:50:13.359 INFO:tasks.workunit.client.0.smithi019.stdout:8/19: chown c2 354381 1 2024-02-22T15:50:13.360 INFO:tasks.workunit.client.0.smithi019.stdout:0/23: write f1 [4018359,58862] 0 2024-02-22T15:50:13.360 INFO:tasks.workunit.client.0.smithi019.stdout:8/20: write f5 [1746436,69753] 0 2024-02-22T15:50:13.366 INFO:tasks.workunit.client.0.smithi019.stdout:1/36: mkdir d8/da 0 2024-02-22T15:50:13.369 INFO:tasks.workunit.client.0.smithi019.stdout:0/24: dwrite f1 [0,4194304] 0 2024-02-22T15:50:13.370 INFO:tasks.workunit.client.0.smithi019.stdout:8/21: creat f6 x:0 0 0 2024-02-22T15:50:13.376 INFO:tasks.workunit.client.0.smithi019.stdout:1/37: dread f3 [0,4194304] 0 2024-02-22T15:50:13.376 INFO:tasks.workunit.client.0.smithi019.stdout:8/22: mknod c7 0 2024-02-22T15:50:13.377 INFO:tasks.workunit.client.0.smithi019.stdout:0/25: dread f2 [0,4194304] 0 2024-02-22T15:50:13.382 INFO:tasks.workunit.client.0.smithi019.stdout:8/23: dread f1 [4194304,4194304] 0 2024-02-22T15:50:13.383 INFO:tasks.workunit.client.0.smithi019.stdout:1/38: rmdir d8/da 0 2024-02-22T15:50:13.390 INFO:tasks.workunit.client.0.smithi019.stdout:0/26: dwrite f1 [0,4194304] 0 2024-02-22T15:50:13.393 INFO:tasks.workunit.client.0.smithi019.stdout:0/27: fdatasync f2 0 2024-02-22T15:50:13.395 INFO:tasks.workunit.client.0.smithi019.stdout:8/24: dread f1 [4194304,4194304] 0 2024-02-22T15:50:13.397 INFO:tasks.workunit.client.0.smithi019.stdout:1/39: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.400 INFO:tasks.workunit.client.0.smithi019.stdout:0/28: mkdir d4/d9 0 2024-02-22T15:50:13.401 INFO:tasks.workunit.client.0.smithi019.stdout:1/40: truncate d8/f9 718845 0 2024-02-22T15:50:13.411 INFO:tasks.workunit.client.0.smithi019.stdout:0/29: dwrite f2 [0,4194304] 0 2024-02-22T15:50:13.412 INFO:tasks.workunit.client.0.smithi019.stdout:1/41: dread f3 [0,4194304] 0 2024-02-22T15:50:13.416 INFO:tasks.workunit.client.0.smithi019.stdout:1/42: creat d8/fb x:0 0 0 2024-02-22T15:50:13.418 INFO:tasks.workunit.client.0.smithi019.stdout:0/30: creat d4/d9/fa x:0 0 0 2024-02-22T15:50:13.420 INFO:tasks.workunit.client.0.smithi019.stdout:1/43: creat d8/fc x:0 0 0 2024-02-22T15:50:13.422 INFO:tasks.workunit.client.0.smithi019.stdout:1/44: write d8/fc [110786,20256] 0 2024-02-22T15:50:13.424 INFO:tasks.workunit.client.0.smithi019.stdout:1/45: read f5 [215007,24074] 0 2024-02-22T15:50:13.425 INFO:tasks.workunit.client.0.smithi019.stdout:6/14: getdents . 0 2024-02-22T15:50:13.426 INFO:tasks.workunit.client.0.smithi019.stdout:0/31: dread f2 [0,4194304] 0 2024-02-22T15:50:13.430 INFO:tasks.workunit.client.0.smithi019.stdout:1/46: rename f5 to d8/fd 0 2024-02-22T15:50:13.432 INFO:tasks.workunit.client.0.smithi019.stdout:0/32: write d4/d9/fa [375000,107157] 0 2024-02-22T15:50:13.433 INFO:tasks.workunit.client.0.smithi019.stdout:6/15: symlink l5 0 2024-02-22T15:50:13.435 INFO:tasks.workunit.client.0.smithi019.stdout:3/14: dwrite f3 [4194304,4194304] 0 2024-02-22T15:50:13.437 INFO:tasks.workunit.client.0.smithi019.stdout:1/47: chown d8/f9 2296 1 2024-02-22T15:50:13.451 INFO:tasks.workunit.client.0.smithi019.stdout:6/16: truncate f0 3490962 0 2024-02-22T15:50:13.454 INFO:tasks.workunit.client.0.smithi019.stdout:2/23: rmdir d1 39 2024-02-22T15:50:13.455 INFO:tasks.workunit.client.0.smithi019.stdout:6/17: mkdir d6 0 2024-02-22T15:50:13.461 INFO:tasks.workunit.client.0.smithi019.stdout:1/48: dwrite d8/fc [0,4194304] 0 2024-02-22T15:50:13.465 INFO:tasks.workunit.client.0.smithi019.stdout:9/9: dwrite f0 [4194304,4194304] 0 2024-02-22T15:50:13.466 INFO:tasks.workunit.client.0.smithi019.stdout:8/25: getdents . 0 2024-02-22T15:50:13.468 INFO:tasks.workunit.client.0.smithi019.stdout:0/33: truncate f2 4153664 0 2024-02-22T15:50:13.470 INFO:tasks.workunit.client.0.smithi019.stdout:2/24: creat d1/fa x:0 0 0 2024-02-22T15:50:13.471 INFO:tasks.workunit.client.0.smithi019.stdout:6/18: rename l5 to d6/l7 0 2024-02-22T15:50:13.473 INFO:tasks.workunit.client.0.smithi019.stdout:9/10: mkdir d2 0 2024-02-22T15:50:13.474 INFO:tasks.workunit.client.0.smithi019.stdout:3/15: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.476 INFO:tasks.workunit.client.0.smithi019.stdout:1/49: dread d8/fd [0,4194304] 0 2024-02-22T15:50:13.477 INFO:tasks.workunit.client.0.smithi019.stdout:1/50: stat f3 0 2024-02-22T15:50:13.478 INFO:tasks.workunit.client.0.smithi019.stdout:1/51: truncate d8/fb 950002 0 2024-02-22T15:50:13.479 INFO:tasks.workunit.client.0.smithi019.stdout:8/26: rename c7 to c8 0 2024-02-22T15:50:13.482 INFO:tasks.workunit.client.0.smithi019.stdout:2/25: rmdir d1 39 2024-02-22T15:50:13.484 INFO:tasks.workunit.client.0.smithi019.stdout:6/19: creat d6/f8 x:0 0 0 2024-02-22T15:50:13.485 INFO:tasks.workunit.client.0.smithi019.stdout:6/20: chown f3 574 1 2024-02-22T15:50:13.486 INFO:tasks.workunit.client.0.smithi019.stdout:0/34: read f2 [2785532,119922] 0 2024-02-22T15:50:13.487 INFO:tasks.workunit.client.0.smithi019.stdout:1/52: dread d8/fc [0,4194304] 0 2024-02-22T15:50:13.489 INFO:tasks.workunit.client.0.smithi019.stdout:8/27: mkdir d9 0 2024-02-22T15:50:13.489 INFO:tasks.workunit.client.0.smithi019.stdout:0/35: chown d4/l8 198555112 1 2024-02-22T15:50:13.492 INFO:tasks.workunit.client.0.smithi019.stdout:1/53: mkdir d8/de 0 2024-02-22T15:50:13.493 INFO:tasks.workunit.client.0.smithi019.stdout:2/26: rmdir d1 39 2024-02-22T15:50:13.496 INFO:tasks.workunit.client.0.smithi019.stdout:0/36: mkdir d4/db 0 2024-02-22T15:50:13.497 INFO:tasks.workunit.client.0.smithi019.stdout:6/21: fsync f0 0 2024-02-22T15:50:13.500 INFO:tasks.workunit.client.0.smithi019.stdout:0/37: readlink l0 0 2024-02-22T15:50:13.500 INFO:tasks.workunit.client.0.smithi019.stdout:3/16: fdatasync f3 0 2024-02-22T15:50:13.501 INFO:tasks.workunit.client.0.smithi019.stdout:8/28: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.501 INFO:tasks.workunit.client.0.smithi019.stdout:0/38: stat c3 0 2024-02-22T15:50:13.502 INFO:tasks.workunit.client.0.smithi019.stdout:6/22: read - d6/f8 zero size 2024-02-22T15:50:13.505 INFO:tasks.workunit.client.0.smithi019.stdout:9/11: truncate f0 7474382 0 2024-02-22T15:50:13.506 INFO:tasks.workunit.client.0.smithi019.stdout:2/27: write d1/f9 [934060,121487] 0 2024-02-22T15:50:13.507 INFO:tasks.workunit.client.0.smithi019.stdout:0/39: read f1 [3012245,92346] 0 2024-02-22T15:50:13.510 INFO:tasks.workunit.client.0.smithi019.stdout:6/23: rename f3 to d6/f9 0 2024-02-22T15:50:13.511 INFO:tasks.workunit.client.0.smithi019.stdout:8/29: getdents d9 0 2024-02-22T15:50:13.514 INFO:tasks.workunit.client.0.smithi019.stdout:0/40: unlink d4/d9/fa 0 2024-02-22T15:50:13.515 INFO:tasks.workunit.client.0.smithi019.stdout:1/54: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.516 INFO:tasks.workunit.client.0.smithi019.stdout:3/17: dread f3 [4194304,4194304] 0 2024-02-22T15:50:13.516 INFO:tasks.workunit.client.0.smithi019.stdout:0/41: chown c3 54360738 1 2024-02-22T15:50:13.519 INFO:tasks.workunit.client.0.smithi019.stdout:6/24: symlink d6/la 0 2024-02-22T15:50:13.519 INFO:tasks.workunit.client.0.smithi019.stdout:2/28: rename d1/l6 to d1/lb 0 2024-02-22T15:50:13.522 INFO:tasks.workunit.client.0.smithi019.stdout:8/30: write f1 [9301512,92175] 0 2024-02-22T15:50:13.522 INFO:tasks.workunit.client.0.smithi019.stdout:9/12: read f0 [5647961,103389] 0 2024-02-22T15:50:13.524 INFO:tasks.workunit.client.0.smithi019.stdout:8/31: chown d9 2 1 2024-02-22T15:50:13.526 INFO:tasks.workunit.client.0.smithi019.stdout:0/42: mkdir d4/d5/dc 0 2024-02-22T15:50:13.527 INFO:tasks.workunit.client.0.smithi019.stdout:3/18: link f3 f4 0 2024-02-22T15:50:13.531 INFO:tasks.workunit.client.0.smithi019.stdout:8/32: creat d9/fa x:0 0 0 2024-02-22T15:50:13.532 INFO:tasks.workunit.client.0.smithi019.stdout:2/29: rename d1/l5 to d1/lc 0 2024-02-22T15:50:13.534 INFO:tasks.workunit.client.0.smithi019.stdout:0/43: rename d4/l8 to d4/d5/dc/ld 0 2024-02-22T15:50:13.534 INFO:tasks.workunit.client.0.smithi019.stdout:1/55: dwrite f3 [0,4194304] 0 2024-02-22T15:50:13.536 INFO:tasks.workunit.client.0.smithi019.stdout:2/30: truncate d1/fa 221571 0 2024-02-22T15:50:13.537 INFO:tasks.workunit.client.0.smithi019.stdout:6/25: dread f0 [0,4194304] 0 2024-02-22T15:50:13.538 INFO:tasks.workunit.client.0.smithi019.stdout:8/33: symlink d9/lb 0 2024-02-22T15:50:13.540 INFO:tasks.workunit.client.0.smithi019.stdout:9/13: dread f0 [4194304,4194304] 0 2024-02-22T15:50:13.540 INFO:tasks.workunit.client.0.smithi019.stdout:2/31: dread d1/fa [0,4194304] 0 2024-02-22T15:50:13.542 INFO:tasks.workunit.client.0.smithi019.stdout:0/44: write f1 [2704502,3412] 0 2024-02-22T15:50:13.546 INFO:tasks.workunit.client.0.smithi019.stdout:6/26: unlink f0 0 2024-02-22T15:50:13.547 INFO:tasks.workunit.client.0.smithi019.stdout:3/19: link c2 c5 0 2024-02-22T15:50:13.547 INFO:tasks.workunit.client.0.smithi019.stdout:1/56: dwrite d8/f9 [0,4194304] 0 2024-02-22T15:50:13.550 INFO:tasks.workunit.client.0.smithi019.stdout:1/57: chown d8/fc 349 1 2024-02-22T15:50:13.551 INFO:tasks.workunit.client.0.smithi019.stdout:9/14: mknod d2/c3 0 2024-02-22T15:50:13.552 INFO:tasks.workunit.client.0.smithi019.stdout:8/34: dread f5 [0,4194304] 0 2024-02-22T15:50:13.553 INFO:tasks.workunit.client.0.smithi019.stdout:3/20: creat f6 x:0 0 0 2024-02-22T15:50:13.554 INFO:tasks.workunit.client.0.smithi019.stdout:1/58: read d8/f9 [538076,54022] 0 2024-02-22T15:50:13.555 INFO:tasks.workunit.client.0.smithi019.stdout:6/27: rename d6/f8 to d6/fb 0 2024-02-22T15:50:13.555 INFO:tasks.workunit.client.0.smithi019.stdout:0/45: dread f1 [0,4194304] 0 2024-02-22T15:50:13.556 INFO:tasks.workunit.client.0.smithi019.stdout:8/35: write f6 [962880,24464] 0 2024-02-22T15:50:13.558 INFO:tasks.workunit.client.0.smithi019.stdout:8/36: chown c2 1 1 2024-02-22T15:50:13.559 INFO:tasks.workunit.client.0.smithi019.stdout:9/15: fsync f0 0 2024-02-22T15:50:13.563 INFO:tasks.workunit.client.0.smithi019.stdout:3/21: creat f7 x:0 0 0 2024-02-22T15:50:13.566 INFO:tasks.workunit.client.0.smithi019.stdout:2/32: link d1/lb d1/ld 0 2024-02-22T15:50:13.567 INFO:tasks.workunit.client.0.smithi019.stdout:6/28: write d6/fb [1005955,102200] 0 2024-02-22T15:50:13.570 INFO:tasks.workunit.client.0.smithi019.stdout:8/37: dwrite f6 [0,4194304] 0 2024-02-22T15:50:13.570 INFO:tasks.workunit.client.0.smithi019.stdout:9/16: rename f0 to d2/f4 0 2024-02-22T15:50:13.570 INFO:tasks.workunit.client.0.smithi019.stdout:1/59: mknod d8/de/cf 0 2024-02-22T15:50:13.575 INFO:tasks.workunit.client.0.smithi019.stdout:1/60: chown f3 12400 1 2024-02-22T15:50:13.575 INFO:tasks.workunit.client.0.smithi019.stdout:3/22: mknod c8 0 2024-02-22T15:50:13.575 INFO:tasks.workunit.client.0.smithi019.stdout:3/23: chown f4 182164798 1 2024-02-22T15:50:13.576 INFO:tasks.workunit.client.0.smithi019.stdout:3/24: rmdir - no directory 2024-02-22T15:50:13.576 INFO:tasks.workunit.client.0.smithi019.stdout:3/25: stat f7 0 2024-02-22T15:50:13.580 INFO:tasks.workunit.client.0.smithi019.stdout:9/17: mknod d2/c5 0 2024-02-22T15:50:13.580 INFO:tasks.workunit.client.0.smithi019.stdout:1/61: mknod d8/c10 0 2024-02-22T15:50:13.580 INFO:tasks.workunit.client.0.smithi019.stdout:2/33: dread d1/f9 [0,4194304] 0 2024-02-22T15:50:13.584 INFO:tasks.workunit.client.0.smithi019.stdout:9/18: write d2/f4 [7713637,40290] 0 2024-02-22T15:50:13.584 INFO:tasks.workunit.client.0.smithi019.stdout:1/62: creat d8/de/f11 x:0 0 0 2024-02-22T15:50:13.584 INFO:tasks.workunit.client.0.smithi019.stdout:0/46: truncate f1 3564655 0 2024-02-22T15:50:13.585 INFO:tasks.workunit.client.0.smithi019.stdout:3/26: dwrite f7 [0,4194304] 0 2024-02-22T15:50:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:2/34: dread d1/f9 [0,4194304] 0 2024-02-22T15:50:13.587 INFO:tasks.workunit.client.0.smithi019.stdout:9/19: stat d2/f4 0 2024-02-22T15:50:13.589 INFO:tasks.workunit.client.0.smithi019.stdout:0/47: chown d4/d5/l6 26 1 2024-02-22T15:50:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:1/63: rename f3 to d8/de/f12 0 2024-02-22T15:50:13.591 INFO:tasks.workunit.client.0.smithi019.stdout:1/64: readlink l7 0 2024-02-22T15:50:13.592 INFO:tasks.workunit.client.0.smithi019.stdout:2/35: mkdir d1/de 0 2024-02-22T15:50:13.594 INFO:tasks.workunit.client.0.smithi019.stdout:9/20: creat d2/f6 x:0 0 0 2024-02-22T15:50:13.594 INFO:tasks.workunit.client.0.smithi019.stdout:9/21: readlink - no filename 2024-02-22T15:50:13.595 INFO:tasks.workunit.client.0.smithi019.stdout:9/22: read d2/f4 [6927042,56489] 0 2024-02-22T15:50:13.596 INFO:tasks.workunit.client.0.smithi019.stdout:9/23: chown d2/f4 0 1 2024-02-22T15:50:13.597 INFO:tasks.workunit.client.0.smithi019.stdout:0/48: unlink d4/d5/l6 0 2024-02-22T15:50:13.598 INFO:tasks.workunit.client.0.smithi019.stdout:1/65: mknod d8/de/c13 0 2024-02-22T15:50:13.599 INFO:tasks.workunit.client.0.smithi019.stdout:2/36: read d1/fa [111720,47420] 0 2024-02-22T15:50:13.602 INFO:tasks.workunit.client.0.smithi019.stdout:9/24: mkdir d2/d7 0 2024-02-22T15:50:13.602 INFO:tasks.workunit.client.0.smithi019.stdout:9/25: chown d2 1220953 1 2024-02-22T15:50:13.603 INFO:tasks.workunit.client.0.smithi019.stdout:1/66: creat d8/f14 x:0 0 0 2024-02-22T15:50:13.605 INFO:tasks.workunit.client.0.smithi019.stdout:3/27: fdatasync f7 0 2024-02-22T15:50:13.607 INFO:tasks.workunit.client.0.smithi019.stdout:2/37: creat d1/de/ff x:0 0 0 2024-02-22T15:50:13.610 INFO:tasks.workunit.client.0.smithi019.stdout:9/26: mknod d2/d7/c8 0 2024-02-22T15:50:13.611 INFO:tasks.workunit.client.0.smithi019.stdout:2/38: creat d1/f10 x:0 0 0 2024-02-22T15:50:13.615 INFO:tasks.workunit.client.0.smithi019.stdout:1/67: rename d8/de to d8/d15 0 2024-02-22T15:50:13.616 INFO:tasks.workunit.client.0.smithi019.stdout:2/39: truncate d1/fa 1117314 0 2024-02-22T15:50:13.618 INFO:tasks.workunit.client.0.smithi019.stdout:2/40: dread - d1/f10 zero size 2024-02-22T15:50:13.619 INFO:tasks.workunit.client.0.smithi019.stdout:2/41: chown d1/de/ff 576363 1 2024-02-22T15:50:13.620 INFO:tasks.workunit.client.0.smithi019.stdout:1/68: rename c4 to d8/c16 0 2024-02-22T15:50:13.621 INFO:tasks.workunit.client.0.smithi019.stdout:0/49: truncate f1 1707358 0 2024-02-22T15:50:13.622 INFO:tasks.workunit.client.0.smithi019.stdout:9/27: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:13.624 INFO:tasks.workunit.client.0.smithi019.stdout:2/42: readlink d1/ld 0 2024-02-22T15:50:13.625 INFO:tasks.workunit.client.0.smithi019.stdout:1/69: fsync d8/fc 0 2024-02-22T15:50:13.627 INFO:tasks.workunit.client.0.smithi019.stdout:1/70: read - d8/f14 zero size 2024-02-22T15:50:13.629 INFO:tasks.workunit.client.0.smithi019.stdout:0/50: creat d4/d5/fe x:0 0 0 2024-02-22T15:50:13.629 INFO:tasks.workunit.client.0.smithi019.stdout:2/43: chown d1/l7 31861 1 2024-02-22T15:50:13.633 INFO:tasks.workunit.client.0.smithi019.stdout:3/28: dwrite f4 [0,4194304] 0 2024-02-22T15:50:13.643 INFO:tasks.workunit.client.0.smithi019.stdout:3/29: getdents . 0 2024-02-22T15:50:13.644 INFO:tasks.workunit.client.0.smithi019.stdout:1/71: dwrite d8/fb [0,4194304] 0 2024-02-22T15:50:13.644 INFO:tasks.workunit.client.0.smithi019.stdout:0/51: dread f1 [0,4194304] 0 2024-02-22T15:50:13.647 INFO:tasks.workunit.client.0.smithi019.stdout:9/28: truncate d2/f6 1231842 0 2024-02-22T15:50:13.650 INFO:tasks.workunit.client.0.smithi019.stdout:9/29: read d2/f4 [6170894,117582] 0 2024-02-22T15:50:13.654 INFO:tasks.workunit.client.0.smithi019.stdout:0/52: rename d4/d5/dc/ld to d4/d5/lf 0 2024-02-22T15:50:13.654 INFO:tasks.workunit.client.0.smithi019.stdout:0/53: readlink l0 0 2024-02-22T15:50:13.655 INFO:tasks.workunit.client.0.smithi019.stdout:9/30: creat d2/d7/f9 x:0 0 0 2024-02-22T15:50:13.659 INFO:tasks.workunit.client.0.smithi019.stdout:0/54: creat d4/f10 x:0 0 0 2024-02-22T15:50:13.659 INFO:tasks.workunit.client.0.smithi019.stdout:2/44: dwrite d1/fa [0,4194304] 0 2024-02-22T15:50:13.659 INFO:tasks.workunit.client.0.smithi019.stdout:3/30: dread f3 [0,4194304] 0 2024-02-22T15:50:13.660 INFO:tasks.workunit.client.0.smithi019.stdout:9/31: truncate d2/d7/f9 656745 0 2024-02-22T15:50:13.661 INFO:tasks.workunit.client.0.smithi019.stdout:3/31: stat l1 0 2024-02-22T15:50:13.662 INFO:tasks.workunit.client.0.smithi019.stdout:2/45: fdatasync d1/f9 0 2024-02-22T15:50:13.662 INFO:tasks.workunit.client.0.smithi019.stdout:1/72: dread d8/fc [0,4194304] 0 2024-02-22T15:50:13.663 INFO:tasks.workunit.client.0.smithi019.stdout:1/73: readlink l7 0 2024-02-22T15:50:13.664 INFO:tasks.workunit.client.0.smithi019.stdout:1/74: dread - d8/f14 zero size 2024-02-22T15:50:13.664 INFO:tasks.workunit.client.0.smithi019.stdout:3/32: mkdir d9 0 2024-02-22T15:50:13.665 INFO:tasks.workunit.client.0.smithi019.stdout:9/32: fsync d2/f4 0 2024-02-22T15:50:13.669 INFO:tasks.workunit.client.0.smithi019.stdout:3/33: creat d9/fa x:0 0 0 2024-02-22T15:50:13.671 INFO:tasks.workunit.client.0.smithi019.stdout:0/55: getdents d4/d5 0 2024-02-22T15:50:13.675 INFO:tasks.workunit.client.0.smithi019.stdout:3/34: dread f4 [0,4194304] 0 2024-02-22T15:50:13.675 INFO:tasks.workunit.client.0.smithi019.stdout:0/56: link f2 d4/d9/f11 0 2024-02-22T15:50:13.679 INFO:tasks.workunit.client.0.smithi019.stdout:1/75: truncate d8/f9 1849678 0 2024-02-22T15:50:13.682 INFO:tasks.workunit.client.0.smithi019.stdout:3/35: dread f3 [0,4194304] 0 2024-02-22T15:50:13.683 INFO:tasks.workunit.client.0.smithi019.stdout:9/33: dwrite d2/f4 [0,4194304] 0 2024-02-22T15:50:13.685 INFO:tasks.workunit.client.0.smithi019.stdout:3/36: chown c2 189 1 2024-02-22T15:50:13.686 INFO:tasks.workunit.client.0.smithi019.stdout:9/34: symlink d2/d7/la 0 2024-02-22T15:50:13.687 INFO:tasks.workunit.client.0.smithi019.stdout:3/37: creat d9/fb x:0 0 0 2024-02-22T15:50:13.697 INFO:tasks.workunit.client.0.smithi019.stdout:9/35: dwrite d2/d7/f9 [0,4194304] 0 2024-02-22T15:50:13.701 INFO:tasks.workunit.client.0.smithi019.stdout:9/36: write d2/f4 [364843,53726] 0 2024-02-22T15:50:13.701 INFO:tasks.workunit.client.0.smithi019.stdout:3/38: dwrite f4 [8388608,4194304] 0 2024-02-22T15:50:13.709 INFO:tasks.workunit.client.0.smithi019.stdout:9/37: write d2/d7/f9 [4271003,32094] 0 2024-02-22T15:50:13.710 INFO:tasks.workunit.client.0.smithi019.stdout:3/39: getdents d9 0 2024-02-22T15:50:13.712 INFO:tasks.workunit.client.0.smithi019.stdout:3/40: write f6 [253577,102312] 0 2024-02-22T15:50:13.714 INFO:tasks.workunit.client.0.smithi019.stdout:3/41: mkdir d9/dc 0 2024-02-22T15:50:13.722 INFO:tasks.workunit.client.0.smithi019.stdout:9/38: dwrite d2/f4 [8388608,4194304] 0 2024-02-22T15:50:13.724 INFO:tasks.workunit.client.0.smithi019.stdout:3/42: dwrite f4 [8388608,4194304] 0 2024-02-22T15:50:13.725 INFO:tasks.workunit.client.0.smithi019.stdout:3/43: fsync d9/fa 0 2024-02-22T15:50:13.726 INFO:tasks.workunit.client.0.smithi019.stdout:9/39: creat d2/fb x:0 0 0 2024-02-22T15:50:13.729 INFO:tasks.workunit.client.0.smithi019.stdout:3/44: link d9/fa d9/dc/fd 0 2024-02-22T15:50:13.732 INFO:tasks.workunit.client.0.smithi019.stdout:3/45: creat d9/dc/fe x:0 0 0 2024-02-22T15:50:13.733 INFO:tasks.workunit.client.0.smithi019.stdout:9/40: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:13.735 INFO:tasks.workunit.client.0.smithi019.stdout:3/46: write f6 [1047837,18446] 0 2024-02-22T15:50:13.735 INFO:tasks.workunit.client.0.smithi019.stdout:3/47: fdatasync f7 0 2024-02-22T15:50:13.741 INFO:tasks.workunit.client.0.smithi019.stdout:9/41: dread d2/d7/f9 [0,4194304] 0 2024-02-22T15:50:13.746 INFO:tasks.workunit.client.0.smithi019.stdout:3/48: dwrite f3 [8388608,4194304] 0 2024-02-22T15:50:13.752 INFO:tasks.workunit.client.0.smithi019.stdout:9/42: dwrite d2/fb [0,4194304] 0 2024-02-22T15:50:13.753 INFO:tasks.workunit.client.0.smithi019.stdout:9/43: readlink d2/d7/la 0 2024-02-22T15:50:13.754 INFO:tasks.workunit.client.0.smithi019.stdout:9/44: chown d2 28 1 2024-02-22T15:50:13.755 INFO:tasks.workunit.client.0.smithi019.stdout:3/49: getdents d9/dc 0 2024-02-22T15:50:13.757 INFO:tasks.workunit.client.0.smithi019.stdout:9/45: mkdir d2/d7/dc 0 2024-02-22T15:50:13.758 INFO:tasks.workunit.client.0.smithi019.stdout:3/50: symlink d9/lf 0 2024-02-22T15:50:13.760 INFO:tasks.workunit.client.0.smithi019.stdout:3/51: write d9/dc/fe [961271,102648] 0 2024-02-22T15:50:13.765 INFO:tasks.workunit.client.0.smithi019.stdout:3/52: rename l1 to d9/dc/l10 0 2024-02-22T15:50:13.768 INFO:tasks.workunit.client.0.smithi019.stdout:9/46: dwrite d2/d7/f9 [0,4194304] 0 2024-02-22T15:50:13.769 INFO:tasks.workunit.client.0.smithi019.stdout:9/47: fsync d2/f4 0 2024-02-22T15:50:13.772 INFO:tasks.workunit.client.0.smithi019.stdout:9/48: write d2/f6 [3171381,91326] 0 2024-02-22T15:50:13.774 INFO:tasks.workunit.client.0.smithi019.stdout:3/53: rename d9/dc to d9/d11 0 2024-02-22T15:50:13.776 INFO:tasks.workunit.client.0.smithi019.stdout:9/49: mkdir d2/d7/dd 0 2024-02-22T15:50:13.780 INFO:tasks.workunit.client.0.smithi019.stdout:9/50: dread d2/d7/f9 [0,4194304] 0 2024-02-22T15:50:13.788 INFO:tasks.workunit.client.0.smithi019.stdout:9/51: dread d2/fb [0,4194304] 0 2024-02-22T15:50:13.788 INFO:tasks.workunit.client.0.smithi019.stdout:9/52: creat d2/d7/fe x:0 0 0 2024-02-22T15:50:13.788 INFO:tasks.workunit.client.0.smithi019.stdout:3/54: truncate f4 12165239 0 2024-02-22T15:50:13.788 INFO:tasks.workunit.client.0.smithi019.stdout:9/53: stat d2/d7/f9 0 2024-02-22T15:50:13.789 INFO:tasks.workunit.client.0.smithi019.stdout:9/54: chown d2/d7/fe 385977382 1 2024-02-22T15:50:13.790 INFO:tasks.workunit.client.0.smithi019.stdout:3/55: mkdir d9/d12 0 2024-02-22T15:50:13.790 INFO:tasks.workunit.client.0.smithi019.stdout:9/55: chown d2/f6 10 1 2024-02-22T15:50:13.791 INFO:tasks.workunit.client.0.smithi019.stdout:9/56: write d2/d7/fe [212113,59240] 0 2024-02-22T15:50:13.791 INFO:tasks.workunit.client.0.smithi019.stdout:3/56: readlink d9/d11/l10 0 2024-02-22T15:50:13.792 INFO:tasks.workunit.client.0.smithi019.stdout:3/57: rmdir d9 39 2024-02-22T15:50:13.794 INFO:tasks.workunit.client.0.smithi019.stdout:9/57: symlink d2/d7/dc/lf 0 2024-02-22T15:50:13.796 INFO:tasks.workunit.client.0.smithi019.stdout:9/58: symlink d2/d7/l10 0 2024-02-22T15:50:13.799 INFO:tasks.workunit.client.0.smithi019.stdout:9/59: creat d2/d7/dc/f11 x:0 0 0 2024-02-22T15:50:13.808 INFO:tasks.workunit.client.0.smithi019.stdout:3/58: dwrite f3 [8388608,4194304] 0 2024-02-22T15:50:13.809 INFO:tasks.workunit.client.0.smithi019.stdout:9/60: dwrite d2/d7/dc/f11 [0,4194304] 0 2024-02-22T15:50:13.813 INFO:tasks.workunit.client.0.smithi019.stdout:3/59: chown d9 6 1 2024-02-22T15:50:13.816 INFO:tasks.workunit.client.0.smithi019.stdout:9/61: getdents d2 0 2024-02-22T15:50:13.817 INFO:tasks.workunit.client.0.smithi019.stdout:9/62: write d2/f4 [13418900,119320] 0 2024-02-22T15:50:13.822 INFO:tasks.workunit.client.0.smithi019.stdout:9/63: link d2/f6 d2/d7/f12 0 2024-02-22T15:50:13.829 INFO:tasks.workunit.client.0.smithi019.stdout:9/64: dwrite d2/d7/fe [0,4194304] 0 2024-02-22T15:50:13.834 INFO:tasks.workunit.client.0.smithi019.stdout:9/65: dread d2/d7/fe [0,4194304] 0 2024-02-22T15:50:13.838 INFO:tasks.workunit.client.0.smithi019.stdout:9/66: rmdir d2/d7/dd 0 2024-02-22T15:50:13.840 INFO:tasks.workunit.client.0.smithi019.stdout:9/67: symlink d2/l13 0 2024-02-22T15:50:13.841 INFO:tasks.workunit.client.0.smithi019.stdout:9/68: write d2/f4 [10080290,104549] 0 2024-02-22T15:50:13.843 INFO:tasks.workunit.client.0.smithi019.stdout:9/69: mkdir d2/d7/d14 0 2024-02-22T15:50:13.844 INFO:tasks.workunit.client.0.smithi019.stdout:9/70: chown d2/f4 97354236 1 2024-02-22T15:50:13.845 INFO:tasks.workunit.client.0.smithi019.stdout:9/71: chown d2/d7/fe 69 1 2024-02-22T15:50:13.849 INFO:tasks.workunit.client.0.smithi019.stdout:9/72: dread d2/f4 [8388608,4194304] 0 2024-02-22T15:50:13.850 INFO:tasks.workunit.client.0.smithi019.stdout:9/73: mknod d2/c15 0 2024-02-22T15:50:13.851 INFO:tasks.workunit.client.0.smithi019.stdout:9/74: fsync d2/fb 0 2024-02-22T15:50:13.855 INFO:tasks.workunit.client.0.smithi019.stdout:9/75: dread d2/f4 [4194304,4194304] 0 2024-02-22T15:50:13.856 INFO:tasks.workunit.client.0.smithi019.stdout:9/76: chown d2/d7 7 1 2024-02-22T15:50:13.861 INFO:tasks.workunit.client.0.smithi019.stdout:9/77: write d2/f6 [4307853,95047] 0 2024-02-22T15:50:13.863 INFO:tasks.workunit.client.0.smithi019.stdout:9/78: mkdir d2/d16 0 2024-02-22T15:50:13.864 INFO:tasks.workunit.client.0.smithi019.stdout:9/79: fdatasync d2/d7/fe 0 2024-02-22T15:50:13.871 INFO:tasks.workunit.client.0.smithi019.stdout:9/80: dwrite d2/d7/dc/f11 [0,4194304] 0 2024-02-22T15:50:13.877 INFO:tasks.workunit.client.0.smithi019.stdout:9/81: dread d2/d7/f12 [0,4194304] 0 2024-02-22T15:50:13.879 INFO:tasks.workunit.client.0.smithi019.stdout:9/82: symlink d2/d7/l17 0 2024-02-22T15:50:13.879 INFO:tasks.workunit.client.0.smithi019.stdout:9/83: stat d2/c15 0 2024-02-22T15:50:13.880 INFO:tasks.workunit.client.0.smithi019.stdout:9/84: write d2/f6 [4292163,12292] 0 2024-02-22T15:50:13.888 INFO:tasks.workunit.client.0.smithi019.stdout:9/85: truncate d2/f6 2306673 0 2024-02-22T15:50:13.890 INFO:tasks.workunit.client.0.smithi019.stdout:9/86: symlink d2/d7/d14/l18 0 2024-02-22T15:50:13.891 INFO:tasks.workunit.client.0.smithi019.stdout:9/87: chown d2/c5 69 1 2024-02-22T15:50:13.900 INFO:tasks.workunit.client.0.smithi019.stdout:9/88: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:13.901 INFO:tasks.workunit.client.0.smithi019.stdout:9/89: chown d2/d7/l17 298813 1 2024-02-22T15:50:13.907 INFO:tasks.workunit.client.0.smithi019.stdout:9/90: link d2/d7/f9 d2/f19 0 2024-02-22T15:50:13.909 INFO:tasks.workunit.client.0.smithi019.stdout:9/91: mknod d2/d7/c1a 0 2024-02-22T15:50:13.910 INFO:tasks.workunit.client.0.smithi019.stdout:9/92: creat d2/d7/f1b x:0 0 0 2024-02-22T15:50:13.918 INFO:tasks.workunit.client.0.smithi019.stdout:9/93: getdents d2/d7 0 2024-02-22T15:50:13.918 INFO:tasks.workunit.client.0.smithi019.stdout:9/94: chown d2/d7/fe 1362 1 2024-02-22T15:50:13.922 INFO:tasks.workunit.client.0.smithi019.stdout:9/95: truncate d2/f4 12725586 0 2024-02-22T15:50:15.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:15 smithi062 ceph-mon[48130]: pgmap v23: 65 pgs: 65 active+clean; 173 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-02-22T15:50:15.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:15 smithi019 ceph-mon[67418]: pgmap v23: 65 pgs: 65 active+clean; 173 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-02-22T15:50:17.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:17 smithi062 ceph-mon[48130]: pgmap v24: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 23 MiB/s wr, 598 op/s 2024-02-22T15:50:17.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:17 smithi019 ceph-mon[67418]: pgmap v24: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 23 MiB/s wr, 598 op/s 2024-02-22T15:50:19.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:19 smithi062 ceph-mon[48130]: pgmap v25: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 21 MiB/s wr, 382 op/s 2024-02-22T15:50:19.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:19 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:19.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:19 smithi019 ceph-mon[67418]: pgmap v25: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 21 MiB/s wr, 382 op/s 2024-02-22T15:50:19.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:19 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:21.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:21 smithi062 ceph-mon[48130]: pgmap v26: 65 pgs: 65 active+clean; 317 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 501 op/s 2024-02-22T15:50:21.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:21 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:50:21.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:21 smithi019 ceph-mon[67418]: pgmap v26: 65 pgs: 65 active+clean; 317 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 22 MiB/s wr, 501 op/s 2024-02-22T15:50:21.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:21 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:50:23.370 INFO:tasks.workunit.client.1.smithi062.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-22T15:50:23.378 INFO:tasks.workunit.client.1.smithi062.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-22T15:50:23.378 INFO:tasks.workunit.client.1.smithi062.stderr:+ make 2024-02-22T15:50:23.449 INFO:tasks.workunit.client.1.smithi062.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-22T15:50:23.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:23 smithi062 ceph-mon[48130]: pgmap v27: 65 pgs: 65 active+clean; 317 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 296 op/s 2024-02-22T15:50:23.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:23 smithi019 ceph-mon[67418]: pgmap v27: 65 pgs: 65 active+clean; 317 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 296 op/s 2024-02-22T15:50:23.819 INFO:tasks.workunit.client.1.smithi062.stderr:++ readlink -f fsstress 2024-02-22T15:50:23.820 INFO:tasks.workunit.client.1.smithi062.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-22T15:50:23.820 INFO:tasks.workunit.client.1.smithi062.stderr:+ popd 2024-02-22T15:50:23.821 INFO:tasks.workunit.client.1.smithi062.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-22T15:50:23.821 INFO:tasks.workunit.client.1.smithi062.stderr:+ popd 2024-02-22T15:50:23.822 INFO:tasks.workunit.client.1.smithi062.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-22T15:50:23.822 INFO:tasks.workunit.client.1.smithi062.stderr:++ mktemp -d -p . 2024-02-22T15:50:23.825 INFO:tasks.workunit.client.1.smithi062.stderr:+ T=./tmp.WKoMFo77vo 2024-02-22T15:50:23.825 INFO:tasks.workunit.client.1.smithi062.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.WKoMFo77vo -l 1 -n 1000 -p 10 -v 2024-02-22T15:50:23.828 INFO:tasks.workunit.client.1.smithi062.stdout:seed = 1708884671 2024-02-22T15:50:23.831 INFO:tasks.workunit.client.1.smithi062.stdout:0/0: fdatasync - no filename 2024-02-22T15:50:23.832 INFO:tasks.workunit.client.1.smithi062.stdout:1/0: write - no filename 2024-02-22T15:50:23.833 INFO:tasks.workunit.client.1.smithi062.stdout:1/1: rename - no filename 2024-02-22T15:50:23.833 INFO:tasks.workunit.client.1.smithi062.stdout:1/2: rmdir - no directory 2024-02-22T15:50:23.833 INFO:tasks.workunit.client.1.smithi062.stdout:1/3: chown . 243 1 2024-02-22T15:50:23.833 INFO:tasks.workunit.client.1.smithi062.stdout:0/1: creat f0 x:0 0 0 2024-02-22T15:50:23.834 INFO:tasks.workunit.client.1.smithi062.stdout:0/2: mkdir d1 0 2024-02-22T15:50:23.835 INFO:tasks.workunit.client.1.smithi062.stdout:1/4: symlink l0 0 2024-02-22T15:50:23.837 INFO:tasks.workunit.client.1.smithi062.stdout:1/5: mkdir d1 0 2024-02-22T15:50:23.837 INFO:tasks.workunit.client.1.smithi062.stdout:1/6: dread - no filename 2024-02-22T15:50:23.837 INFO:tasks.workunit.client.1.smithi062.stdout:1/7: chown d1 297234 1 2024-02-22T15:50:23.838 INFO:tasks.workunit.client.1.smithi062.stdout:2/0: symlink l0 0 2024-02-22T15:50:23.838 INFO:tasks.workunit.client.1.smithi062.stdout:2/1: fdatasync - no filename 2024-02-22T15:50:23.838 INFO:tasks.workunit.client.1.smithi062.stdout:2/2: read - no filename 2024-02-22T15:50:23.838 INFO:tasks.workunit.client.1.smithi062.stdout:2/3: readlink l0 0 2024-02-22T15:50:23.839 INFO:tasks.workunit.client.1.smithi062.stdout:0/3: creat d1/f2 x:0 0 0 2024-02-22T15:50:23.840 INFO:tasks.workunit.client.1.smithi062.stdout:2/4: mkdir d1 0 2024-02-22T15:50:23.841 INFO:tasks.workunit.client.1.smithi062.stdout:0/4: mkdir d1/d3 0 2024-02-22T15:50:23.842 INFO:tasks.workunit.client.1.smithi062.stdout:3/0: symlink l0 0 2024-02-22T15:50:23.842 INFO:tasks.workunit.client.1.smithi062.stdout:1/8: mknod d1/c2 0 2024-02-22T15:50:23.843 INFO:tasks.workunit.client.1.smithi062.stdout:1/9: chown d1 0 1 2024-02-22T15:50:23.843 INFO:tasks.workunit.client.1.smithi062.stdout:3/1: chown l0 1101677291 1 2024-02-22T15:50:23.843 INFO:tasks.workunit.client.1.smithi062.stdout:3/2: write - no filename 2024-02-22T15:50:23.843 INFO:tasks.workunit.client.1.smithi062.stdout:3/3: dread - no filename 2024-02-22T15:50:23.845 INFO:tasks.workunit.client.1.smithi062.stdout:3/4: mkdir d1 0 2024-02-22T15:50:23.845 INFO:tasks.workunit.client.1.smithi062.stdout:3/5: dread - no filename 2024-02-22T15:50:23.845 INFO:tasks.workunit.client.1.smithi062.stdout:3/6: dread - no filename 2024-02-22T15:50:23.845 INFO:tasks.workunit.client.1.smithi062.stdout:1/10: chown d1/c2 3617 1 2024-02-22T15:50:23.845 INFO:tasks.workunit.client.1.smithi062.stdout:1/11: dread - no filename 2024-02-22T15:50:23.845 INFO:tasks.workunit.client.1.smithi062.stdout:4/0: symlink l0 0 2024-02-22T15:50:23.848 INFO:tasks.workunit.client.1.smithi062.stdout:2/5: link l0 d1/l2 0 2024-02-22T15:50:23.849 INFO:tasks.workunit.client.1.smithi062.stdout:4/1: creat f1 x:0 0 0 2024-02-22T15:50:23.851 INFO:tasks.workunit.client.1.smithi062.stdout:5/0: mkdir d0 0 2024-02-22T15:50:23.851 INFO:tasks.workunit.client.1.smithi062.stdout:5/1: unlink - no file 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/0: write - no filename 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/1: chown . 1715 1 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/2: rename - no filename 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/3: write - no filename 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/4: dread - no filename 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/5: write - no filename 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/6: truncate - no filename 2024-02-22T15:50:23.852 INFO:tasks.workunit.client.1.smithi062.stdout:7/7: dwrite - no filename 2024-02-22T15:50:23.853 INFO:tasks.workunit.client.1.smithi062.stdout:0/5: dwrite d1/f2 [0,4194304] 0 2024-02-22T15:50:23.855 INFO:tasks.workunit.client.1.smithi062.stdout:2/6: rmdir d1 39 2024-02-22T15:50:23.855 INFO:tasks.workunit.client.1.smithi062.stdout:2/7: dread - no filename 2024-02-22T15:50:23.856 INFO:tasks.workunit.client.1.smithi062.stdout:3/7: rmdir d1 0 2024-02-22T15:50:23.857 INFO:tasks.workunit.client.1.smithi062.stdout:4/2: creat f2 x:0 0 0 2024-02-22T15:50:23.858 INFO:tasks.workunit.client.1.smithi062.stdout:4/3: chown l0 559 1 2024-02-22T15:50:23.861 INFO:tasks.workunit.client.1.smithi062.stdout:8/0: readlink - no filename 2024-02-22T15:50:23.862 INFO:tasks.workunit.client.1.smithi062.stdout:1/12: link l0 d1/l3 0 2024-02-22T15:50:23.864 INFO:tasks.workunit.client.1.smithi062.stdout:6/0: creat f0 x:0 0 0 2024-02-22T15:50:23.865 INFO:tasks.workunit.client.1.smithi062.stdout:3/8: symlink l2 0 2024-02-22T15:50:23.866 INFO:tasks.workunit.client.1.smithi062.stdout:5/2: creat d0/f1 x:0 0 0 2024-02-22T15:50:23.868 INFO:tasks.workunit.client.1.smithi062.stdout:7/8: mkdir d0 0 2024-02-22T15:50:23.868 INFO:tasks.workunit.client.1.smithi062.stdout:9/0: dread - no filename 2024-02-22T15:50:23.869 INFO:tasks.workunit.client.1.smithi062.stdout:5/3: read - d0/f1 zero size 2024-02-22T15:50:23.869 INFO:tasks.workunit.client.1.smithi062.stdout:9/1: chown . 986679 1 2024-02-22T15:50:23.869 INFO:tasks.workunit.client.1.smithi062.stdout:9/2: rename - no filename 2024-02-22T15:50:23.869 INFO:tasks.workunit.client.1.smithi062.stdout:9/3: write - no filename 2024-02-22T15:50:23.871 INFO:tasks.workunit.client.1.smithi062.stdout:4/4: dwrite f2 [0,4194304] 0 2024-02-22T15:50:23.871 INFO:tasks.workunit.client.1.smithi062.stdout:6/1: creat f1 x:0 0 0 2024-02-22T15:50:23.872 INFO:tasks.workunit.client.1.smithi062.stdout:5/4: dread - d0/f1 zero size 2024-02-22T15:50:23.872 INFO:tasks.workunit.client.1.smithi062.stdout:6/2: read - f1 zero size 2024-02-22T15:50:23.873 INFO:tasks.workunit.client.1.smithi062.stdout:6/3: dread - f1 zero size 2024-02-22T15:50:23.873 INFO:tasks.workunit.client.1.smithi062.stdout:3/9: creat f3 x:0 0 0 2024-02-22T15:50:23.875 INFO:tasks.workunit.client.1.smithi062.stdout:8/1: creat f0 x:0 0 0 2024-02-22T15:50:23.876 INFO:tasks.workunit.client.1.smithi062.stdout:3/10: write f3 [325177,125846] 0 2024-02-22T15:50:23.876 INFO:tasks.workunit.client.1.smithi062.stdout:3/11: stat l2 0 2024-02-22T15:50:23.879 INFO:tasks.workunit.client.1.smithi062.stdout:5/5: creat d0/f2 x:0 0 0 2024-02-22T15:50:23.879 INFO:tasks.workunit.client.1.smithi062.stdout:3/12: write f3 [578606,67321] 0 2024-02-22T15:50:23.880 INFO:tasks.workunit.client.1.smithi062.stdout:6/4: creat f2 x:0 0 0 2024-02-22T15:50:23.881 INFO:tasks.workunit.client.1.smithi062.stdout:7/9: creat d0/f1 x:0 0 0 2024-02-22T15:50:23.883 INFO:tasks.workunit.client.1.smithi062.stdout:7/10: chown d0/f1 108 1 2024-02-22T15:50:23.884 INFO:tasks.workunit.client.1.smithi062.stdout:9/4: creat f0 x:0 0 0 2024-02-22T15:50:23.886 INFO:tasks.workunit.client.1.smithi062.stdout:7/11: chown d0/f1 13421446 1 2024-02-22T15:50:23.886 INFO:tasks.workunit.client.1.smithi062.stdout:3/13: creat f4 x:0 0 0 2024-02-22T15:50:23.888 INFO:tasks.workunit.client.1.smithi062.stdout:3/14: fdatasync f3 0 2024-02-22T15:50:23.889 INFO:tasks.workunit.client.1.smithi062.stdout:9/5: rename f0 to f1 0 2024-02-22T15:50:23.889 INFO:tasks.workunit.client.1.smithi062.stdout:9/6: rmdir - no directory 2024-02-22T15:50:23.890 INFO:tasks.workunit.client.1.smithi062.stdout:7/12: write d0/f1 [777635,108132] 0 2024-02-22T15:50:23.893 INFO:tasks.workunit.client.1.smithi062.stdout:9/7: write f1 [663378,43619] 0 2024-02-22T15:50:23.894 INFO:tasks.workunit.client.1.smithi062.stdout:6/5: dwrite f0 [0,4194304] 0 2024-02-22T15:50:23.898 INFO:tasks.workunit.client.1.smithi062.stdout:9/8: link f1 f2 0 2024-02-22T15:50:23.901 INFO:tasks.workunit.client.1.smithi062.stdout:9/9: mknod c3 0 2024-02-22T15:50:23.903 INFO:tasks.workunit.client.1.smithi062.stdout:9/10: mkdir d4 0 2024-02-22T15:50:23.906 INFO:tasks.workunit.client.1.smithi062.stdout:9/11: chown c3 9 1 2024-02-22T15:50:23.909 INFO:tasks.workunit.client.1.smithi062.stdout:9/12: creat d4/f5 x:0 0 0 2024-02-22T15:50:23.909 INFO:tasks.workunit.client.1.smithi062.stdout:9/13: read - d4/f5 zero size 2024-02-22T15:50:23.911 INFO:tasks.workunit.client.1.smithi062.stdout:9/14: rename c3 to d4/c6 0 2024-02-22T15:50:23.969 INFO:tasks.workunit.client.1.smithi062.stdout:0/6: fsync f0 0 2024-02-22T15:50:23.976 INFO:tasks.workunit.client.1.smithi062.stdout:2/8: mknod d1/c3 0 2024-02-22T15:50:23.976 INFO:tasks.workunit.client.1.smithi062.stdout:2/9: stat d1 0 2024-02-22T15:50:23.976 INFO:tasks.workunit.client.1.smithi062.stdout:2/10: dread - no filename 2024-02-22T15:50:23.976 INFO:tasks.workunit.client.1.smithi062.stdout:2/11: dread - no filename 2024-02-22T15:50:23.977 INFO:tasks.workunit.client.1.smithi062.stdout:2/12: write - no filename 2024-02-22T15:50:23.977 INFO:tasks.workunit.client.1.smithi062.stdout:2/13: dwrite - no filename 2024-02-22T15:50:23.977 INFO:tasks.workunit.client.1.smithi062.stdout:2/14: dread - no filename 2024-02-22T15:50:23.977 INFO:tasks.workunit.client.1.smithi062.stdout:0/7: dwrite d1/f2 [0,4194304] 0 2024-02-22T15:50:23.981 INFO:tasks.workunit.client.1.smithi062.stdout:2/15: unlink d1/l2 0 2024-02-22T15:50:23.982 INFO:tasks.workunit.client.1.smithi062.stdout:0/8: mkdir d1/d3/d4 0 2024-02-22T15:50:23.985 INFO:tasks.workunit.client.1.smithi062.stdout:0/9: rename f0 to d1/f5 0 2024-02-22T15:50:23.986 INFO:tasks.workunit.client.1.smithi062.stdout:0/10: write d1/f2 [2102801,90373] 0 2024-02-22T15:50:23.987 INFO:tasks.workunit.client.1.smithi062.stdout:0/11: truncate d1/f2 4971686 0 2024-02-22T15:50:23.988 INFO:tasks.workunit.client.1.smithi062.stdout:1/13: getdents d1 0 2024-02-22T15:50:23.988 INFO:tasks.workunit.client.1.smithi062.stdout:1/14: dread - no filename 2024-02-22T15:50:23.990 INFO:tasks.workunit.client.1.smithi062.stdout:2/16: link l0 d1/l4 0 2024-02-22T15:50:23.990 INFO:tasks.workunit.client.1.smithi062.stdout:2/17: dread - no filename 2024-02-22T15:50:23.993 INFO:tasks.workunit.client.1.smithi062.stdout:1/15: mkdir d1/d4 0 2024-02-22T15:50:23.993 INFO:tasks.workunit.client.1.smithi062.stdout:1/16: dwrite - no filename 2024-02-22T15:50:23.994 INFO:tasks.workunit.client.1.smithi062.stdout:2/18: rmdir d1 39 2024-02-22T15:50:23.994 INFO:tasks.workunit.client.1.smithi062.stdout:2/19: dread - no filename 2024-02-22T15:50:23.996 INFO:tasks.workunit.client.1.smithi062.stdout:0/12: dwrite d1/f2 [0,4194304] 0 2024-02-22T15:50:23.999 INFO:tasks.workunit.client.1.smithi062.stdout:6/6: fsync f1 0 2024-02-22T15:50:23.999 INFO:tasks.workunit.client.1.smithi062.stdout:1/17: creat d1/d4/f5 x:0 0 0 2024-02-22T15:50:24.000 INFO:tasks.workunit.client.1.smithi062.stdout:0/13: fdatasync d1/f5 0 2024-02-22T15:50:24.000 INFO:tasks.workunit.client.1.smithi062.stdout:2/20: chown d1/c3 16968 1 2024-02-22T15:50:24.000 INFO:tasks.workunit.client.1.smithi062.stdout:2/21: write - no filename 2024-02-22T15:50:24.000 INFO:tasks.workunit.client.1.smithi062.stdout:5/6: rmdir d0 39 2024-02-22T15:50:24.001 INFO:tasks.workunit.client.1.smithi062.stdout:5/7: readlink - no filename 2024-02-22T15:50:24.002 INFO:tasks.workunit.client.1.smithi062.stdout:6/7: mknod c3 0 2024-02-22T15:50:24.002 INFO:tasks.workunit.client.1.smithi062.stdout:6/8: dread - f1 zero size 2024-02-22T15:50:24.003 INFO:tasks.workunit.client.1.smithi062.stdout:6/9: readlink - no filename 2024-02-22T15:50:24.004 INFO:tasks.workunit.client.1.smithi062.stdout:7/13: rmdir d0 39 2024-02-22T15:50:24.005 INFO:tasks.workunit.client.1.smithi062.stdout:1/18: mkdir d1/d6 0 2024-02-22T15:50:24.006 INFO:tasks.workunit.client.1.smithi062.stdout:2/22: creat d1/f5 x:0 0 0 2024-02-22T15:50:24.007 INFO:tasks.workunit.client.1.smithi062.stdout:0/14: rename d1/f2 to d1/d3/f6 0 2024-02-22T15:50:24.009 INFO:tasks.workunit.client.1.smithi062.stdout:6/10: rename c3 to c4 0 2024-02-22T15:50:24.010 INFO:tasks.workunit.client.1.smithi062.stdout:3/15: getdents . 0 2024-02-22T15:50:24.010 INFO:tasks.workunit.client.1.smithi062.stdout:6/11: dread - f1 zero size 2024-02-22T15:50:24.011 INFO:tasks.workunit.client.1.smithi062.stdout:5/8: mkdir d0/d3 0 2024-02-22T15:50:24.013 INFO:tasks.workunit.client.1.smithi062.stdout:7/14: read d0/f1 [805867,68225] 0 2024-02-22T15:50:24.019 INFO:tasks.workunit.client.1.smithi062.stdout:3/16: mkdir d5 0 2024-02-22T15:50:24.019 INFO:tasks.workunit.client.1.smithi062.stdout:3/17: write f3 [654686,95794] 0 2024-02-22T15:50:24.021 INFO:tasks.workunit.client.1.smithi062.stdout:5/9: mknod d0/c4 0 2024-02-22T15:50:24.021 INFO:tasks.workunit.client.1.smithi062.stdout:6/12: mkdir d5 0 2024-02-22T15:50:24.022 INFO:tasks.workunit.client.1.smithi062.stdout:5/10: dread - d0/f2 zero size 2024-02-22T15:50:24.025 INFO:tasks.workunit.client.1.smithi062.stdout:1/19: unlink l0 0 2024-02-22T15:50:24.026 INFO:tasks.workunit.client.1.smithi062.stdout:2/23: chown d1/l4 5772724 1 2024-02-22T15:50:24.028 INFO:tasks.workunit.client.1.smithi062.stdout:0/15: creat d1/d3/d4/f7 x:0 0 0 2024-02-22T15:50:24.028 INFO:tasks.workunit.client.1.smithi062.stdout:9/15: chown d4/c6 291718 1 2024-02-22T15:50:24.033 INFO:tasks.workunit.client.1.smithi062.stdout:3/18: dwrite f4 [0,4194304] 0 2024-02-22T15:50:24.033 INFO:tasks.workunit.client.1.smithi062.stdout:9/16: fsync d4/f5 0 2024-02-22T15:50:24.033 INFO:tasks.workunit.client.1.smithi062.stdout:7/15: rename d0/f1 to d0/f2 0 2024-02-22T15:50:24.033 INFO:tasks.workunit.client.1.smithi062.stdout:3/19: chown d5 0 1 2024-02-22T15:50:24.033 INFO:tasks.workunit.client.1.smithi062.stdout:2/24: readlink l0 0 2024-02-22T15:50:24.034 INFO:tasks.workunit.client.1.smithi062.stdout:6/13: creat d5/f6 x:0 0 0 2024-02-22T15:50:24.035 INFO:tasks.workunit.client.1.smithi062.stdout:0/16: mknod d1/d3/d4/c8 0 2024-02-22T15:50:24.035 INFO:tasks.workunit.client.1.smithi062.stdout:2/25: chown d1 3 1 2024-02-22T15:50:24.035 INFO:tasks.workunit.client.1.smithi062.stdout:5/11: getdents d0/d3 0 2024-02-22T15:50:24.036 INFO:tasks.workunit.client.1.smithi062.stdout:6/14: dread - f2 zero size 2024-02-22T15:50:24.037 INFO:tasks.workunit.client.1.smithi062.stdout:5/12: dread - d0/f1 zero size 2024-02-22T15:50:24.037 INFO:tasks.workunit.client.1.smithi062.stdout:9/17: creat d4/f7 x:0 0 0 2024-02-22T15:50:24.040 INFO:tasks.workunit.client.1.smithi062.stdout:1/20: mkdir d1/d6/d7 0 2024-02-22T15:50:24.041 INFO:tasks.workunit.client.1.smithi062.stdout:3/20: dread f3 [0,4194304] 0 2024-02-22T15:50:24.041 INFO:tasks.workunit.client.1.smithi062.stdout:7/16: dread d0/f2 [0,4194304] 0 2024-02-22T15:50:24.043 INFO:tasks.workunit.client.1.smithi062.stdout:7/17: dread d0/f2 [0,4194304] 0 2024-02-22T15:50:24.043 INFO:tasks.workunit.client.1.smithi062.stdout:2/26: creat d1/f6 x:0 0 0 2024-02-22T15:50:24.044 INFO:tasks.workunit.client.1.smithi062.stdout:0/17: creat d1/d3/d4/f9 x:0 0 0 2024-02-22T15:50:24.046 INFO:tasks.workunit.client.1.smithi062.stdout:6/15: creat d5/f7 x:0 0 0 2024-02-22T15:50:24.047 INFO:tasks.workunit.client.1.smithi062.stdout:9/18: rename f1 to d4/f8 0 2024-02-22T15:50:24.049 INFO:tasks.workunit.client.1.smithi062.stdout:1/21: readlink d1/l3 0 2024-02-22T15:50:24.050 INFO:tasks.workunit.client.1.smithi062.stdout:1/22: chown d1 1149914446 1 2024-02-22T15:50:24.050 INFO:tasks.workunit.client.1.smithi062.stdout:1/23: write d1/d4/f5 [779184,24225] 0 2024-02-22T15:50:24.057 INFO:tasks.workunit.client.1.smithi062.stdout:2/27: creat d1/f7 x:0 0 0 2024-02-22T15:50:24.058 INFO:tasks.workunit.client.1.smithi062.stdout:2/28: dread - d1/f5 zero size 2024-02-22T15:50:24.058 INFO:tasks.workunit.client.1.smithi062.stdout:1/24: dwrite d1/d4/f5 [0,4194304] 0 2024-02-22T15:50:24.059 INFO:tasks.workunit.client.1.smithi062.stdout:7/18: creat d0/f3 x:0 0 0 2024-02-22T15:50:24.060 INFO:tasks.workunit.client.1.smithi062.stdout:0/18: write d1/d3/f6 [2221183,58530] 0 2024-02-22T15:50:24.062 INFO:tasks.workunit.client.1.smithi062.stdout:6/16: symlink d5/l8 0 2024-02-22T15:50:24.063 INFO:tasks.workunit.client.1.smithi062.stdout:0/19: write d1/d3/d4/f9 [884545,34081] 0 2024-02-22T15:50:24.063 INFO:tasks.workunit.client.1.smithi062.stdout:5/13: mknod d0/d3/c5 0 2024-02-22T15:50:24.065 INFO:tasks.workunit.client.1.smithi062.stdout:0/20: write d1/d3/d4/f9 [1320131,121547] 0 2024-02-22T15:50:24.066 INFO:tasks.workunit.client.1.smithi062.stdout:5/14: truncate d0/f1 1002688 0 2024-02-22T15:50:24.067 INFO:tasks.workunit.client.1.smithi062.stdout:9/19: creat d4/f9 x:0 0 0 2024-02-22T15:50:24.069 INFO:tasks.workunit.client.1.smithi062.stdout:3/21: creat d5/f6 x:0 0 0 2024-02-22T15:50:24.069 INFO:tasks.workunit.client.1.smithi062.stdout:3/22: stat l2 0 2024-02-22T15:50:24.070 INFO:tasks.workunit.client.1.smithi062.stdout:2/29: mkdir d1/d8 0 2024-02-22T15:50:24.072 INFO:tasks.workunit.client.1.smithi062.stdout:1/25: readlink d1/l3 0 2024-02-22T15:50:24.078 INFO:tasks.workunit.client.1.smithi062.stdout:0/21: symlink d1/d3/la 0 2024-02-22T15:50:24.079 INFO:tasks.workunit.client.1.smithi062.stdout:0/22: read d1/d3/f6 [602683,62322] 0 2024-02-22T15:50:24.081 INFO:tasks.workunit.client.1.smithi062.stdout:9/20: rename d4/c6 to d4/ca 0 2024-02-22T15:50:24.082 INFO:tasks.workunit.client.1.smithi062.stdout:3/23: write f3 [634661,14933] 0 2024-02-22T15:50:24.083 INFO:tasks.workunit.client.1.smithi062.stdout:3/24: readlink l2 0 2024-02-22T15:50:24.091 INFO:tasks.workunit.client.1.smithi062.stdout:5/15: link d0/f1 d0/f6 0 2024-02-22T15:50:24.092 INFO:tasks.workunit.client.1.smithi062.stdout:0/23: symlink d1/d3/d4/lb 0 2024-02-22T15:50:24.095 INFO:tasks.workunit.client.1.smithi062.stdout:2/30: rename d1/l4 to d1/d8/l9 0 2024-02-22T15:50:24.096 INFO:tasks.workunit.client.1.smithi062.stdout:3/25: symlink d5/l7 0 2024-02-22T15:50:24.097 INFO:tasks.workunit.client.1.smithi062.stdout:7/19: link d0/f2 d0/f4 0 2024-02-22T15:50:24.102 INFO:tasks.workunit.client.1.smithi062.stdout:6/17: truncate f0 2106257 0 2024-02-22T15:50:24.104 INFO:tasks.workunit.client.1.smithi062.stdout:5/16: dwrite d0/f2 [0,4194304] 0 2024-02-22T15:50:24.107 INFO:tasks.workunit.client.1.smithi062.stdout:2/31: unlink d1/c3 0 2024-02-22T15:50:24.108 INFO:tasks.workunit.client.1.smithi062.stdout:0/24: dwrite d1/f5 [0,4194304] 0 2024-02-22T15:50:24.110 INFO:tasks.workunit.client.1.smithi062.stdout:3/26: creat d5/f8 x:0 0 0 2024-02-22T15:50:24.112 INFO:tasks.workunit.client.1.smithi062.stdout:0/25: write d1/f5 [597488,44506] 0 2024-02-22T15:50:24.115 INFO:tasks.workunit.client.1.smithi062.stdout:2/32: read - d1/f6 zero size 2024-02-22T15:50:24.119 INFO:tasks.workunit.client.1.smithi062.stdout:9/21: write d4/f8 [837564,95721] 0 2024-02-22T15:50:24.119 INFO:tasks.workunit.client.1.smithi062.stdout:6/18: dwrite f1 [0,4194304] 0 2024-02-22T15:50:24.119 INFO:tasks.workunit.client.1.smithi062.stdout:5/17: creat d0/f7 x:0 0 0 2024-02-22T15:50:24.120 INFO:tasks.workunit.client.1.smithi062.stdout:5/18: rename d0 to d0/d8 22 2024-02-22T15:50:24.125 INFO:tasks.workunit.client.1.smithi062.stdout:0/26: unlink d1/d3/d4/f9 0 2024-02-22T15:50:24.126 INFO:tasks.workunit.client.1.smithi062.stdout:0/27: chown d1/d3/d4/lb 26087967 1 2024-02-22T15:50:24.132 INFO:tasks.workunit.client.1.smithi062.stdout:2/33: creat d1/d8/fa x:0 0 0 2024-02-22T15:50:24.133 INFO:tasks.workunit.client.1.smithi062.stdout:3/27: rmdir d5 39 2024-02-22T15:50:24.133 INFO:tasks.workunit.client.1.smithi062.stdout:2/34: read - d1/d8/fa zero size 2024-02-22T15:50:24.138 INFO:tasks.workunit.client.1.smithi062.stdout:9/22: mknod d4/cb 0 2024-02-22T15:50:24.141 INFO:tasks.workunit.client.1.smithi062.stdout:1/26: mknod d1/d6/c8 0 2024-02-22T15:50:24.142 INFO:tasks.workunit.client.1.smithi062.stdout:0/28: mkdir d1/dc 0 2024-02-22T15:50:24.146 INFO:tasks.workunit.client.1.smithi062.stdout:2/35: dwrite d1/d8/fa [0,4194304] 0 2024-02-22T15:50:24.148 INFO:tasks.workunit.client.1.smithi062.stdout:7/20: getdents d0 0 2024-02-22T15:50:24.149 INFO:tasks.workunit.client.1.smithi062.stdout:9/23: dread d4/f8 [0,4194304] 0 2024-02-22T15:50:24.149 INFO:tasks.workunit.client.1.smithi062.stdout:2/36: read d1/d8/fa [1333405,13945] 0 2024-02-22T15:50:24.153 INFO:tasks.workunit.client.1.smithi062.stdout:3/28: mknod d5/c9 0 2024-02-22T15:50:24.155 INFO:tasks.workunit.client.1.smithi062.stdout:0/29: unlink d1/d3/la 0 2024-02-22T15:50:24.156 INFO:tasks.workunit.client.1.smithi062.stdout:7/21: creat d0/f5 x:0 0 0 2024-02-22T15:50:24.157 INFO:tasks.workunit.client.1.smithi062.stdout:0/30: chown d1/d3/d4 365969 1 2024-02-22T15:50:24.158 INFO:tasks.workunit.client.1.smithi062.stdout:5/19: dwrite d0/f6 [0,4194304] 0 2024-02-22T15:50:24.159 INFO:tasks.workunit.client.1.smithi062.stdout:6/19: chown f0 1 1 2024-02-22T15:50:24.161 INFO:tasks.workunit.client.1.smithi062.stdout:5/20: stat d0/f2 0 2024-02-22T15:50:24.163 INFO:tasks.workunit.client.1.smithi062.stdout:6/20: dread - f2 zero size 2024-02-22T15:50:24.167 INFO:tasks.workunit.client.1.smithi062.stdout:5/21: read d0/f6 [1848922,21489] 0 2024-02-22T15:50:24.168 INFO:tasks.workunit.client.1.smithi062.stdout:2/37: link d1/f6 d1/d8/fb 0 2024-02-22T15:50:24.171 INFO:tasks.workunit.client.1.smithi062.stdout:5/22: dread - d0/f7 zero size 2024-02-22T15:50:24.171 INFO:tasks.workunit.client.1.smithi062.stdout:7/22: dread d0/f4 [0,4194304] 0 2024-02-22T15:50:24.172 INFO:tasks.workunit.client.1.smithi062.stdout:6/21: symlink d5/l9 0 2024-02-22T15:50:24.172 INFO:tasks.workunit.client.1.smithi062.stdout:9/24: truncate d4/f8 555014 0 2024-02-22T15:50:24.172 INFO:tasks.workunit.client.1.smithi062.stdout:2/38: dread - d1/f7 zero size 2024-02-22T15:50:24.173 INFO:tasks.workunit.client.1.smithi062.stdout:3/29: dwrite d5/f8 [0,4194304] 0 2024-02-22T15:50:24.175 INFO:tasks.workunit.client.1.smithi062.stdout:9/25: dread - d4/f7 zero size 2024-02-22T15:50:24.175 INFO:tasks.workunit.client.1.smithi062.stdout:0/31: write d1/d3/f6 [4944427,77394] 0 2024-02-22T15:50:24.177 INFO:tasks.workunit.client.1.smithi062.stdout:6/22: rmdir d5 39 2024-02-22T15:50:24.180 INFO:tasks.workunit.client.1.smithi062.stdout:7/23: mkdir d0/d6 0 2024-02-22T15:50:24.182 INFO:tasks.workunit.client.1.smithi062.stdout:2/39: symlink d1/lc 0 2024-02-22T15:50:24.184 INFO:tasks.workunit.client.1.smithi062.stdout:3/30: rename l0 to d5/la 0 2024-02-22T15:50:24.187 INFO:tasks.workunit.client.1.smithi062.stdout:5/23: dread d0/f2 [0,4194304] 0 2024-02-22T15:50:24.187 INFO:tasks.workunit.client.1.smithi062.stdout:0/32: mkdir d1/dd 0 2024-02-22T15:50:24.191 INFO:tasks.workunit.client.1.smithi062.stdout:6/23: rmdir d5 39 2024-02-22T15:50:24.191 INFO:tasks.workunit.client.1.smithi062.stdout:7/24: chown d0/f3 1 1 2024-02-22T15:50:24.192 INFO:tasks.workunit.client.1.smithi062.stdout:1/27: dwrite d1/d4/f5 [4194304,4194304] 0 2024-02-22T15:50:24.192 INFO:tasks.workunit.client.1.smithi062.stdout:0/33: chown d1/d3/d4/lb 0 1 2024-02-22T15:50:24.197 INFO:tasks.workunit.client.1.smithi062.stdout:0/34: write d1/f5 [4928667,8802] 0 2024-02-22T15:50:24.198 INFO:tasks.workunit.client.1.smithi062.stdout:1/28: read d1/d4/f5 [3739658,28523] 0 2024-02-22T15:50:24.198 INFO:tasks.workunit.client.1.smithi062.stdout:1/29: chown d1 58289866 1 2024-02-22T15:50:24.198 INFO:tasks.workunit.client.1.smithi062.stdout:5/24: creat d0/f9 x:0 0 0 2024-02-22T15:50:24.198 INFO:tasks.workunit.client.1.smithi062.stdout:1/30: chown d1/d4 12 1 2024-02-22T15:50:24.198 INFO:tasks.workunit.client.1.smithi062.stdout:3/31: fdatasync d5/f8 0 2024-02-22T15:50:24.198 INFO:tasks.workunit.client.1.smithi062.stdout:7/25: dread d0/f4 [0,4194304] 0 2024-02-22T15:50:24.199 INFO:tasks.workunit.client.1.smithi062.stdout:0/35: mkdir d1/d3/de 0 2024-02-22T15:50:24.199 INFO:tasks.workunit.client.1.smithi062.stdout:3/32: dread - d5/f6 zero size 2024-02-22T15:50:24.200 INFO:tasks.workunit.client.1.smithi062.stdout:0/36: read - d1/d3/d4/f7 zero size 2024-02-22T15:50:24.200 INFO:tasks.workunit.client.1.smithi062.stdout:3/33: read d5/f8 [4181484,90056] 0 2024-02-22T15:50:24.201 INFO:tasks.workunit.client.1.smithi062.stdout:5/25: unlink d0/f6 0 2024-02-22T15:50:24.202 INFO:tasks.workunit.client.1.smithi062.stdout:2/40: write d1/f6 [908852,11960] 0 2024-02-22T15:50:24.208 INFO:tasks.workunit.client.1.smithi062.stdout:1/31: dread d1/d4/f5 [0,4194304] 0 2024-02-22T15:50:24.215 INFO:tasks.workunit.client.1.smithi062.stdout:7/26: symlink d0/l7 0 2024-02-22T15:50:24.217 INFO:tasks.workunit.client.1.smithi062.stdout:3/34: dwrite f4 [4194304,4194304] 0 2024-02-22T15:50:24.218 INFO:tasks.workunit.client.1.smithi062.stdout:0/37: dwrite d1/d3/f6 [0,4194304] 0 2024-02-22T15:50:24.222 INFO:tasks.workunit.client.1.smithi062.stdout:7/27: chown d0/f3 2 1 2024-02-22T15:50:24.226 INFO:tasks.workunit.client.1.smithi062.stdout:3/35: dread f4 [0,4194304] 0 2024-02-22T15:50:24.228 INFO:tasks.workunit.client.1.smithi062.stdout:6/24: link d5/l8 d5/la 0 2024-02-22T15:50:24.229 INFO:tasks.workunit.client.1.smithi062.stdout:5/26: rmdir d0 39 2024-02-22T15:50:24.234 INFO:tasks.workunit.client.1.smithi062.stdout:1/32: unlink d1/d4/f5 0 2024-02-22T15:50:24.238 INFO:tasks.workunit.client.1.smithi062.stdout:7/28: unlink d0/f3 0 2024-02-22T15:50:24.240 INFO:tasks.workunit.client.1.smithi062.stdout:7/29: truncate d0/f5 1006501 0 2024-02-22T15:50:24.243 INFO:tasks.workunit.client.1.smithi062.stdout:6/25: creat d5/fb x:0 0 0 2024-02-22T15:50:24.247 INFO:tasks.workunit.client.1.smithi062.stdout:1/33: mkdir d1/d4/d9 0 2024-02-22T15:50:24.247 INFO:tasks.workunit.client.1.smithi062.stdout:1/34: dwrite - no filename 2024-02-22T15:50:24.247 INFO:tasks.workunit.client.1.smithi062.stdout:1/35: dwrite - no filename 2024-02-22T15:50:24.250 INFO:tasks.workunit.client.1.smithi062.stdout:0/38: creat d1/d3/de/ff x:0 0 0 2024-02-22T15:50:24.251 INFO:tasks.workunit.client.1.smithi062.stdout:6/26: readlink d5/l8 0 2024-02-22T15:50:24.253 INFO:tasks.workunit.client.1.smithi062.stdout:7/30: mkdir d0/d6/d8 0 2024-02-22T15:50:24.254 INFO:tasks.workunit.client.1.smithi062.stdout:6/27: truncate d5/f7 150132 0 2024-02-22T15:50:24.254 INFO:tasks.workunit.client.1.smithi062.stdout:5/27: dwrite d0/f7 [0,4194304] 0 2024-02-22T15:50:24.255 INFO:tasks.workunit.client.1.smithi062.stdout:2/41: getdents d1/d8 0 2024-02-22T15:50:24.257 INFO:tasks.workunit.client.1.smithi062.stdout:1/36: mknod d1/ca 0 2024-02-22T15:50:24.257 INFO:tasks.workunit.client.1.smithi062.stdout:1/37: dread - no filename 2024-02-22T15:50:24.257 INFO:tasks.workunit.client.1.smithi062.stdout:3/36: link d5/la d5/lb 0 2024-02-22T15:50:24.257 INFO:tasks.workunit.client.1.smithi062.stdout:1/38: chown d1 663310 1 2024-02-22T15:50:24.257 INFO:tasks.workunit.client.1.smithi062.stdout:1/39: write - no filename 2024-02-22T15:50:24.259 INFO:tasks.workunit.client.1.smithi062.stdout:3/37: truncate d5/f6 600443 0 2024-02-22T15:50:24.259 INFO:tasks.workunit.client.1.smithi062.stdout:5/28: mkdir d0/d3/da 0 2024-02-22T15:50:24.259 INFO:tasks.workunit.client.1.smithi062.stdout:3/38: write d5/f8 [2089263,19374] 0 2024-02-22T15:50:24.261 INFO:tasks.workunit.client.1.smithi062.stdout:2/42: mknod d1/cd 0 2024-02-22T15:50:24.262 INFO:tasks.workunit.client.1.smithi062.stdout:6/28: symlink d5/lc 0 2024-02-22T15:50:24.263 INFO:tasks.workunit.client.1.smithi062.stdout:1/40: symlink d1/d4/lb 0 2024-02-22T15:50:24.264 INFO:tasks.workunit.client.1.smithi062.stdout:3/39: symlink d5/lc 0 2024-02-22T15:50:24.265 INFO:tasks.workunit.client.1.smithi062.stdout:0/39: rmdir d1/dc 0 2024-02-22T15:50:24.267 INFO:tasks.workunit.client.1.smithi062.stdout:2/43: write d1/d8/fa [2316638,85143] 0 2024-02-22T15:50:24.268 INFO:tasks.workunit.client.1.smithi062.stdout:6/29: unlink f1 0 2024-02-22T15:50:24.270 INFO:tasks.workunit.client.1.smithi062.stdout:3/40: rename d5/lc to d5/ld 0 2024-02-22T15:50:24.271 INFO:tasks.workunit.client.1.smithi062.stdout:5/29: rename d0/f1 to d0/d3/da/fb 0 2024-02-22T15:50:24.273 INFO:tasks.workunit.client.1.smithi062.stdout:2/44: creat d1/fe x:0 0 0 2024-02-22T15:50:24.274 INFO:tasks.workunit.client.1.smithi062.stdout:1/41: mkdir d1/dc 0 2024-02-22T15:50:24.275 INFO:tasks.workunit.client.1.smithi062.stdout:6/30: mkdir d5/dd 0 2024-02-22T15:50:24.276 INFO:tasks.workunit.client.1.smithi062.stdout:5/30: dread d0/f7 [0,4194304] 0 2024-02-22T15:50:24.276 INFO:tasks.workunit.client.1.smithi062.stdout:3/41: creat d5/fe x:0 0 0 2024-02-22T15:50:24.277 INFO:tasks.workunit.client.1.smithi062.stdout:3/42: dread - d5/fe zero size 2024-02-22T15:50:24.279 INFO:tasks.workunit.client.1.smithi062.stdout:1/42: creat d1/d4/d9/fd x:0 0 0 2024-02-22T15:50:24.280 INFO:tasks.workunit.client.1.smithi062.stdout:6/31: creat d5/dd/fe x:0 0 0 2024-02-22T15:50:24.281 INFO:tasks.workunit.client.1.smithi062.stdout:1/43: fdatasync d1/d4/d9/fd 0 2024-02-22T15:50:24.286 INFO:tasks.workunit.client.1.smithi062.stdout:5/31: dread d0/f7 [0,4194304] 0 2024-02-22T15:50:24.291 INFO:tasks.workunit.client.1.smithi062.stdout:5/32: symlink d0/d3/lc 0 2024-02-22T15:50:24.292 INFO:tasks.workunit.client.1.smithi062.stdout:3/43: dwrite d5/fe [0,4194304] 0 2024-02-22T15:50:24.295 INFO:tasks.workunit.client.1.smithi062.stdout:3/44: mknod d5/cf 0 2024-02-22T15:50:24.296 INFO:tasks.workunit.client.1.smithi062.stdout:6/32: dwrite d5/f7 [0,4194304] 0 2024-02-22T15:50:24.298 INFO:tasks.workunit.client.1.smithi062.stdout:6/33: mkdir d5/df 0 2024-02-22T15:50:24.303 INFO:tasks.workunit.client.1.smithi062.stdout:6/34: link f0 d5/df/f10 0 2024-02-22T15:50:24.305 INFO:tasks.workunit.client.1.smithi062.stdout:6/35: creat d5/dd/f11 x:0 0 0 2024-02-22T15:50:24.307 INFO:tasks.workunit.client.1.smithi062.stdout:7/31: truncate d0/f5 610201 0 2024-02-22T15:50:24.315 INFO:tasks.workunit.client.1.smithi062.stdout:6/36: dwrite f2 [0,4194304] 0 2024-02-22T15:50:24.316 INFO:tasks.workunit.client.1.smithi062.stdout:6/37: chown d5/la 297572 1 2024-02-22T15:50:24.316 INFO:tasks.workunit.client.1.smithi062.stdout:6/38: chown d5/dd/fe 1738108130 1 2024-02-22T15:50:24.317 INFO:tasks.workunit.client.1.smithi062.stdout:1/44: rename d1/d4 to d1/d6/de 0 2024-02-22T15:50:24.319 INFO:tasks.workunit.client.1.smithi062.stdout:6/39: creat d5/dd/f12 x:0 0 0 2024-02-22T15:50:24.324 INFO:tasks.workunit.client.1.smithi062.stdout:1/45: rename d1/d6/c8 to d1/d6/de/d9/cf 0 2024-02-22T15:50:24.328 INFO:tasks.workunit.client.1.smithi062.stdout:7/32: write d0/f4 [582584,118354] 0 2024-02-22T15:50:24.330 INFO:tasks.workunit.client.1.smithi062.stdout:1/46: write d1/d6/de/d9/fd [901320,89027] 0 2024-02-22T15:50:24.331 INFO:tasks.workunit.client.1.smithi062.stdout:6/40: dwrite d5/f7 [4194304,4194304] 0 2024-02-22T15:50:24.334 INFO:tasks.workunit.client.1.smithi062.stdout:7/33: dread d0/f4 [0,4194304] 0 2024-02-22T15:50:24.338 INFO:tasks.workunit.client.1.smithi062.stdout:6/41: rename d5/dd/f11 to d5/dd/f13 0 2024-02-22T15:50:24.339 INFO:tasks.workunit.client.1.smithi062.stdout:6/42: fsync d5/f6 0 2024-02-22T15:50:24.340 INFO:tasks.workunit.client.1.smithi062.stdout:1/47: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:24.341 INFO:tasks.workunit.client.1.smithi062.stdout:6/43: chown f0 6 1 2024-02-22T15:50:24.342 INFO:tasks.workunit.client.1.smithi062.stdout:1/48: mkdir d1/d6/d10 0 2024-02-22T15:50:24.344 INFO:tasks.workunit.client.1.smithi062.stdout:6/44: symlink d5/df/l14 0 2024-02-22T15:50:24.344 INFO:tasks.workunit.client.1.smithi062.stdout:7/34: dread d0/f5 [0,4194304] 0 2024-02-22T15:50:24.346 INFO:tasks.workunit.client.1.smithi062.stdout:6/45: fsync f0 0 2024-02-22T15:50:24.348 INFO:tasks.workunit.client.1.smithi062.stdout:1/49: unlink d1/d6/de/d9/cf 0 2024-02-22T15:50:24.348 INFO:tasks.workunit.client.1.smithi062.stdout:1/50: readlink d1/d6/de/lb 0 2024-02-22T15:50:24.349 INFO:tasks.workunit.client.1.smithi062.stdout:1/51: stat d1/d6/d10 0 2024-02-22T15:50:24.351 INFO:tasks.workunit.client.1.smithi062.stdout:6/46: mkdir d5/d15 0 2024-02-22T15:50:24.353 INFO:tasks.workunit.client.1.smithi062.stdout:6/47: symlink d5/d15/l16 0 2024-02-22T15:50:24.354 INFO:tasks.workunit.client.1.smithi062.stdout:1/52: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:24.356 INFO:tasks.workunit.client.1.smithi062.stdout:6/48: write d5/f7 [5570734,122593] 0 2024-02-22T15:50:24.357 INFO:tasks.workunit.client.1.smithi062.stdout:7/35: dwrite d0/f4 [0,4194304] 0 2024-02-22T15:50:24.359 INFO:tasks.workunit.client.1.smithi062.stdout:6/49: readlink d5/la 0 2024-02-22T15:50:24.359 INFO:tasks.workunit.client.1.smithi062.stdout:1/53: fdatasync d1/d6/de/d9/fd 0 2024-02-22T15:50:24.361 INFO:tasks.workunit.client.1.smithi062.stdout:7/36: dread d0/f5 [0,4194304] 0 2024-02-22T15:50:24.362 INFO:tasks.workunit.client.1.smithi062.stdout:1/54: mknod d1/d6/c11 0 2024-02-22T15:50:24.365 INFO:tasks.workunit.client.1.smithi062.stdout:1/55: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:24.366 INFO:tasks.workunit.client.1.smithi062.stdout:6/50: link d5/dd/f12 d5/df/f17 0 2024-02-22T15:50:24.367 INFO:tasks.workunit.client.1.smithi062.stdout:1/56: symlink d1/d6/d10/l12 0 2024-02-22T15:50:24.370 INFO:tasks.workunit.client.1.smithi062.stdout:6/51: dread d5/f7 [4194304,4194304] 0 2024-02-22T15:50:24.371 INFO:tasks.workunit.client.1.smithi062.stdout:1/57: rmdir d1/dc 0 2024-02-22T15:50:24.372 INFO:tasks.workunit.client.1.smithi062.stdout:7/37: write d0/f5 [1258289,63093] 0 2024-02-22T15:50:24.373 INFO:tasks.workunit.client.1.smithi062.stdout:6/52: write d5/f7 [4994948,113995] 0 2024-02-22T15:50:24.374 INFO:tasks.workunit.client.1.smithi062.stdout:6/53: dread - d5/dd/f12 zero size 2024-02-22T15:50:24.375 INFO:tasks.workunit.client.1.smithi062.stdout:1/58: unlink d1/d6/d10/l12 0 2024-02-22T15:50:24.376 INFO:tasks.workunit.client.1.smithi062.stdout:1/59: read d1/d6/de/d9/fd [947155,27832] 0 2024-02-22T15:50:24.378 INFO:tasks.workunit.client.1.smithi062.stdout:6/54: dread d5/df/f10 [0,4194304] 0 2024-02-22T15:50:24.378 INFO:tasks.workunit.client.1.smithi062.stdout:1/60: fsync d1/d6/de/d9/fd 0 2024-02-22T15:50:24.379 INFO:tasks.workunit.client.1.smithi062.stdout:7/38: getdents d0/d6 0 2024-02-22T15:50:24.379 INFO:tasks.workunit.client.1.smithi062.stdout:6/55: dread - d5/dd/fe zero size 2024-02-22T15:50:24.382 INFO:tasks.workunit.client.1.smithi062.stdout:6/56: rmdir d5/d15 39 2024-02-22T15:50:24.383 INFO:tasks.workunit.client.1.smithi062.stdout:6/57: read - d5/df/f17 zero size 2024-02-22T15:50:24.383 INFO:tasks.workunit.client.1.smithi062.stdout:1/61: mknod d1/d6/d7/c13 0 2024-02-22T15:50:24.384 INFO:tasks.workunit.client.1.smithi062.stdout:6/58: dread - d5/dd/f13 zero size 2024-02-22T15:50:24.384 INFO:tasks.workunit.client.1.smithi062.stdout:7/39: dread d0/f2 [0,4194304] 0 2024-02-22T15:50:24.386 INFO:tasks.workunit.client.1.smithi062.stdout:1/62: write d1/d6/de/d9/fd [1456876,20499] 0 2024-02-22T15:50:24.386 INFO:tasks.workunit.client.1.smithi062.stdout:6/59: truncate d5/fb 186492 0 2024-02-22T15:50:24.387 INFO:tasks.workunit.client.1.smithi062.stdout:7/40: mkdir d0/d6/d9 0 2024-02-22T15:50:24.388 INFO:tasks.workunit.client.1.smithi062.stdout:1/63: rmdir d1 39 2024-02-22T15:50:24.389 INFO:tasks.workunit.client.1.smithi062.stdout:7/41: symlink d0/d6/la 0 2024-02-22T15:50:24.392 INFO:tasks.workunit.client.1.smithi062.stdout:1/64: mkdir d1/d6/d14 0 2024-02-22T15:50:24.393 INFO:tasks.workunit.client.1.smithi062.stdout:7/42: mkdir d0/db 0 2024-02-22T15:50:24.393 INFO:tasks.workunit.client.1.smithi062.stdout:6/60: link d5/d15/l16 d5/l18 0 2024-02-22T15:50:24.394 INFO:tasks.workunit.client.1.smithi062.stdout:1/65: symlink d1/d6/l15 0 2024-02-22T15:50:24.395 INFO:tasks.workunit.client.1.smithi062.stdout:7/43: dread d0/f5 [0,4194304] 0 2024-02-22T15:50:24.396 INFO:tasks.workunit.client.1.smithi062.stdout:6/61: mkdir d5/d19 0 2024-02-22T15:50:24.396 INFO:tasks.workunit.client.1.smithi062.stdout:7/44: read d0/f2 [3028336,111748] 0 2024-02-22T15:50:24.400 INFO:tasks.workunit.client.1.smithi062.stdout:6/62: creat d5/f1a x:0 0 0 2024-02-22T15:50:24.401 INFO:tasks.workunit.client.1.smithi062.stdout:7/45: rename d0/f2 to d0/d6/d8/fc 0 2024-02-22T15:50:24.403 INFO:tasks.workunit.client.1.smithi062.stdout:6/63: mknod d5/c1b 0 2024-02-22T15:50:24.404 INFO:tasks.workunit.client.1.smithi062.stdout:1/66: dwrite d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:24.404 INFO:tasks.workunit.client.1.smithi062.stdout:6/64: read - d5/f6 zero size 2024-02-22T15:50:24.405 INFO:tasks.workunit.client.1.smithi062.stdout:1/67: chown d1/d6/d10 58772 1 2024-02-22T15:50:24.406 INFO:tasks.workunit.client.1.smithi062.stdout:6/65: fdatasync d5/df/f10 0 2024-02-22T15:50:24.409 INFO:tasks.workunit.client.1.smithi062.stdout:7/46: dread d0/d6/d8/fc [0,4194304] 0 2024-02-22T15:50:24.409 INFO:tasks.workunit.client.1.smithi062.stdout:6/66: write d5/f1a [835684,79885] 0 2024-02-22T15:50:24.417 INFO:tasks.workunit.client.1.smithi062.stdout:1/68: link d1/l3 d1/d6/d10/l16 0 2024-02-22T15:50:24.420 INFO:tasks.workunit.client.1.smithi062.stdout:7/47: dwrite d0/f4 [0,4194304] 0 2024-02-22T15:50:24.420 INFO:tasks.workunit.client.1.smithi062.stdout:1/69: write d1/d6/de/d9/fd [3976588,62520] 0 2024-02-22T15:50:24.422 INFO:tasks.workunit.client.1.smithi062.stdout:6/67: write d5/dd/f12 [357262,48282] 0 2024-02-22T15:50:24.427 INFO:tasks.workunit.client.1.smithi062.stdout:1/70: creat d1/d6/de/f17 x:0 0 0 2024-02-22T15:50:24.428 INFO:tasks.workunit.client.1.smithi062.stdout:6/68: stat d5/l18 0 2024-02-22T15:50:24.430 INFO:tasks.workunit.client.1.smithi062.stdout:6/69: creat d5/d19/f1c x:0 0 0 2024-02-22T15:50:24.433 INFO:tasks.workunit.client.1.smithi062.stdout:7/48: dwrite d0/f5 [0,4194304] 0 2024-02-22T15:50:24.438 INFO:tasks.workunit.client.1.smithi062.stdout:7/49: getdents d0/d6/d8 0 2024-02-22T15:50:24.440 INFO:tasks.workunit.client.1.smithi062.stdout:6/70: dwrite d5/f1a [0,4194304] 0 2024-02-22T15:50:24.475 INFO:tasks.workunit.client.1.smithi062.stdout:7/50: dwrite d0/f5 [0,4194304] 0 2024-02-22T15:50:24.607 INFO:tasks.workunit.client.1.smithi062.stdout:6/71: creat d5/f1d x:0 0 0 2024-02-22T15:50:24.608 INFO:tasks.workunit.client.1.smithi062.stdout:7/51: rmdir d0/d6/d9 0 2024-02-22T15:50:24.610 INFO:tasks.workunit.client.1.smithi062.stdout:7/52: creat d0/d6/fd x:0 0 0 2024-02-22T15:50:24.611 INFO:tasks.workunit.client.1.smithi062.stdout:6/72: rename d5/f1d to d5/f1e 0 2024-02-22T15:50:24.611 INFO:tasks.workunit.client.1.smithi062.stdout:6/73: chown d5/d15 31296 1 2024-02-22T15:50:24.612 INFO:tasks.workunit.client.1.smithi062.stdout:6/74: dread - d5/f1e zero size 2024-02-22T15:50:24.613 INFO:tasks.workunit.client.1.smithi062.stdout:7/53: unlink d0/d6/la 0 2024-02-22T15:50:24.613 INFO:tasks.workunit.client.1.smithi062.stdout:6/75: write d5/f7 [3214030,104649] 0 2024-02-22T15:50:24.615 INFO:tasks.workunit.client.1.smithi062.stdout:7/54: symlink d0/db/le 0 2024-02-22T15:50:24.723 INFO:tasks.workunit.client.1.smithi062.stdout:7/55: rmdir d0 39 2024-02-22T15:50:24.726 INFO:tasks.workunit.client.1.smithi062.stdout:7/56: getdents d0/db 0 2024-02-22T15:50:24.728 INFO:tasks.workunit.client.1.smithi062.stdout:7/57: mknod d0/cf 0 2024-02-22T15:50:24.730 INFO:tasks.workunit.client.1.smithi062.stdout:7/58: creat d0/f10 x:0 0 0 2024-02-22T15:50:24.731 INFO:tasks.workunit.client.1.smithi062.stdout:7/59: getdents d0/d6/d8 0 2024-02-22T15:50:24.733 INFO:tasks.workunit.client.1.smithi062.stdout:7/60: mkdir d0/d11 0 2024-02-22T15:50:24.734 INFO:tasks.workunit.client.1.smithi062.stdout:7/61: symlink d0/d6/d8/l12 0 2024-02-22T15:50:24.735 INFO:tasks.workunit.client.1.smithi062.stdout:7/62: dread - d0/f10 zero size 2024-02-22T15:50:24.735 INFO:tasks.workunit.client.1.smithi062.stdout:7/63: read d0/d6/d8/fc [1306194,2197] 0 2024-02-22T15:50:24.738 INFO:tasks.workunit.client.1.smithi062.stdout:7/64: dread d0/f4 [0,4194304] 0 2024-02-22T15:50:24.743 INFO:tasks.workunit.client.1.smithi062.stdout:7/65: rename d0/d6/d8 to d0/d11/d13 0 2024-02-22T15:50:24.748 INFO:tasks.workunit.client.1.smithi062.stdout:7/66: rename d0/d11 to d0/d14 0 2024-02-22T15:50:25.079 INFO:tasks.workunit.client.0.smithi019.stdout:5/5: sync 2024-02-22T15:50:25.082 INFO:tasks.workunit.client.0.smithi019.stdout:5/6: fdatasync f0 0 2024-02-22T15:50:25.085 INFO:tasks.workunit.client.0.smithi019.stdout:5/7: link f0 f1 0 2024-02-22T15:50:25.090 INFO:tasks.workunit.client.0.smithi019.stdout:5/8: truncate f1 3014568 0 2024-02-22T15:50:25.092 INFO:tasks.workunit.client.0.smithi019.stdout:5/9: creat f2 x:0 0 0 2024-02-22T15:50:25.093 INFO:tasks.workunit.client.0.smithi019.stdout:5/10: creat f3 x:0 0 0 2024-02-22T15:50:25.100 INFO:tasks.workunit.client.0.smithi019.stdout:5/11: dread f1 [0,4194304] 0 2024-02-22T15:50:25.102 INFO:tasks.workunit.client.0.smithi019.stdout:5/12: mknod c4 0 2024-02-22T15:50:25.103 INFO:tasks.workunit.client.0.smithi019.stdout:5/13: write f0 [1377153,84072] 0 2024-02-22T15:50:25.103 INFO:tasks.workunit.client.0.smithi019.stdout:5/14: rmdir - no directory 2024-02-22T15:50:25.104 INFO:tasks.workunit.client.0.smithi019.stdout:5/15: chown f0 406216 1 2024-02-22T15:50:25.112 INFO:tasks.workunit.client.0.smithi019.stdout:5/16: dwrite f0 [0,4194304] 0 2024-02-22T15:50:25.112 INFO:tasks.workunit.client.0.smithi019.stdout:5/17: readlink - no filename 2024-02-22T15:50:25.115 INFO:tasks.workunit.client.0.smithi019.stdout:5/18: link f0 f5 0 2024-02-22T15:50:25.124 INFO:tasks.workunit.client.0.smithi019.stdout:5/19: dwrite f3 [0,4194304] 0 2024-02-22T15:50:25.125 INFO:tasks.workunit.client.0.smithi019.stdout:5/20: stat c4 0 2024-02-22T15:50:25.127 INFO:tasks.workunit.client.0.smithi019.stdout:5/21: truncate f1 2065595 0 2024-02-22T15:50:25.134 INFO:tasks.workunit.client.0.smithi019.stdout:5/22: dread f1 [0,4194304] 0 2024-02-22T15:50:25.137 INFO:tasks.workunit.client.0.smithi019.stdout:5/23: getdents . 0 2024-02-22T15:50:25.141 INFO:tasks.workunit.client.0.smithi019.stdout:5/24: dread f3 [0,4194304] 0 2024-02-22T15:50:25.143 INFO:tasks.workunit.client.0.smithi019.stdout:5/25: symlink l6 0 2024-02-22T15:50:25.150 INFO:tasks.workunit.client.0.smithi019.stdout:5/26: dwrite f3 [0,4194304] 0 2024-02-22T15:50:25.152 INFO:tasks.workunit.client.0.smithi019.stdout:5/27: write f1 [1309606,60132] 0 2024-02-22T15:50:25.154 INFO:tasks.workunit.client.0.smithi019.stdout:5/28: write f5 [1525199,86104] 0 2024-02-22T15:50:25.156 INFO:tasks.workunit.client.0.smithi019.stdout:5/29: symlink l7 0 2024-02-22T15:50:25.156 INFO:tasks.workunit.client.0.smithi019.stdout:5/30: chown l6 53604832 1 2024-02-22T15:50:25.163 INFO:tasks.workunit.client.0.smithi019.stdout:5/31: dwrite f3 [0,4194304] 0 2024-02-22T15:50:25.166 INFO:tasks.workunit.client.0.smithi019.stdout:5/32: mkdir d8 0 2024-02-22T15:50:25.167 INFO:tasks.workunit.client.0.smithi019.stdout:5/33: truncate f5 2192225 0 2024-02-22T15:50:25.169 INFO:tasks.workunit.client.0.smithi019.stdout:5/34: mknod d8/c9 0 2024-02-22T15:50:25.178 INFO:tasks.workunit.client.0.smithi019.stdout:5/35: fsync f3 0 2024-02-22T15:50:25.179 INFO:tasks.workunit.client.0.smithi019.stdout:5/36: write f2 [763541,17817] 0 2024-02-22T15:50:25.181 INFO:tasks.workunit.client.0.smithi019.stdout:5/37: rename f0 to d8/fa 0 2024-02-22T15:50:25.183 INFO:tasks.workunit.client.0.smithi019.stdout:5/38: rename l7 to d8/lb 0 2024-02-22T15:50:25.187 INFO:tasks.workunit.client.0.smithi019.stdout:5/39: dread f2 [0,4194304] 0 2024-02-22T15:50:25.189 INFO:tasks.workunit.client.0.smithi019.stdout:5/40: creat d8/fc x:0 0 0 2024-02-22T15:50:25.190 INFO:tasks.workunit.client.0.smithi019.stdout:5/41: fsync f2 0 2024-02-22T15:50:25.190 INFO:tasks.workunit.client.0.smithi019.stdout:5/42: read f5 [354067,11850] 0 2024-02-22T15:50:25.196 INFO:tasks.workunit.client.0.smithi019.stdout:5/43: truncate f1 1901849 0 2024-02-22T15:50:25.199 INFO:tasks.workunit.client.0.smithi019.stdout:5/44: rmdir d8 39 2024-02-22T15:50:25.202 INFO:tasks.workunit.client.0.smithi019.stdout:5/45: read f1 [1448477,85955] 0 2024-02-22T15:50:25.203 INFO:tasks.workunit.client.0.smithi019.stdout:5/46: write f2 [1540996,122874] 0 2024-02-22T15:50:25.205 INFO:tasks.workunit.client.0.smithi019.stdout:5/47: mkdir d8/dd 0 2024-02-22T15:50:25.206 INFO:tasks.workunit.client.0.smithi019.stdout:5/48: write d8/fa [389511,47913] 0 2024-02-22T15:50:25.208 INFO:tasks.workunit.client.0.smithi019.stdout:5/49: creat d8/fe x:0 0 0 2024-02-22T15:50:25.209 INFO:tasks.workunit.client.0.smithi019.stdout:5/50: creat d8/ff x:0 0 0 2024-02-22T15:50:25.211 INFO:tasks.workunit.client.0.smithi019.stdout:5/51: unlink d8/fa 0 2024-02-22T15:50:25.212 INFO:tasks.workunit.client.0.smithi019.stdout:5/52: read - d8/ff zero size 2024-02-22T15:50:25.217 INFO:tasks.workunit.client.0.smithi019.stdout:5/53: rmdir d8 39 2024-02-22T15:50:25.222 INFO:tasks.workunit.client.0.smithi019.stdout:5/54: dwrite f3 [0,4194304] 0 2024-02-22T15:50:25.224 INFO:tasks.workunit.client.0.smithi019.stdout:5/55: fsync f2 0 2024-02-22T15:50:25.234 INFO:tasks.workunit.client.0.smithi019.stdout:5/56: dwrite f5 [0,4194304] 0 2024-02-22T15:50:25.235 INFO:tasks.workunit.client.0.smithi019.stdout:5/57: write f2 [484528,88973] 0 2024-02-22T15:50:25.238 INFO:tasks.workunit.client.0.smithi019.stdout:5/58: write f1 [2883477,83071] 0 2024-02-22T15:50:25.243 INFO:tasks.workunit.client.0.smithi019.stdout:5/59: dread f2 [0,4194304] 0 2024-02-22T15:50:25.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:25 smithi062 ceph-mon[48130]: pgmap v28: 65 pgs: 65 active+clean; 317 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 296 op/s 2024-02-22T15:50:25.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:25 smithi019 ceph-mon[67418]: pgmap v28: 65 pgs: 65 active+clean; 317 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 20 MiB/s wr, 296 op/s 2024-02-22T15:50:25.617 INFO:tasks.workunit.client.0.smithi019.stdout:4/22: sync 2024-02-22T15:50:25.617 INFO:tasks.workunit.client.0.smithi019.stdout:5/60: sync 2024-02-22T15:50:25.617 INFO:tasks.workunit.client.0.smithi019.stdout:7/14: sync 2024-02-22T15:50:25.617 INFO:tasks.workunit.client.0.smithi019.stdout:1/76: sync 2024-02-22T15:50:25.617 INFO:tasks.workunit.client.0.smithi019.stdout:9/96: sync 2024-02-22T15:50:25.617 INFO:tasks.workunit.client.0.smithi019.stdout:3/60: sync 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:0/57: sync 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:2/46: sync 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:8/38: sync 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:6/29: sync 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:0/58: chown d4 3873679 1 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:7/15: stat f1 0 2024-02-22T15:50:25.618 INFO:tasks.workunit.client.0.smithi019.stdout:8/39: readlink d9/lb 0 2024-02-22T15:50:25.620 INFO:tasks.workunit.client.0.smithi019.stdout:4/23: creat d2/fa x:0 0 0 2024-02-22T15:50:25.621 INFO:tasks.workunit.client.0.smithi019.stdout:2/47: rename d1/f9 to d1/f11 0 2024-02-22T15:50:25.622 INFO:tasks.workunit.client.0.smithi019.stdout:4/24: fdatasync d2/f6 0 2024-02-22T15:50:25.622 INFO:tasks.workunit.client.0.smithi019.stdout:2/48: chown d1/ld 13 1 2024-02-22T15:50:25.622 INFO:tasks.workunit.client.0.smithi019.stdout:2/49: readlink d1/ld 0 2024-02-22T15:50:25.623 INFO:tasks.workunit.client.0.smithi019.stdout:2/50: chown d1/de/ff 7647 1 2024-02-22T15:50:25.627 INFO:tasks.workunit.client.0.smithi019.stdout:0/59: chown f2 502229936 1 2024-02-22T15:50:25.627 INFO:tasks.workunit.client.0.smithi019.stdout:5/61: dread - d8/fe zero size 2024-02-22T15:50:25.628 INFO:tasks.workunit.client.0.smithi019.stdout:3/61: chown c8 46482292 1 2024-02-22T15:50:25.629 INFO:tasks.workunit.client.0.smithi019.stdout:6/30: dread d6/f9 [0,4194304] 0 2024-02-22T15:50:25.634 INFO:tasks.workunit.client.0.smithi019.stdout:0/60: dread f1 [0,4194304] 0 2024-02-22T15:50:25.634 INFO:tasks.workunit.client.0.smithi019.stdout:1/77: creat d8/d15/f17 x:0 0 0 2024-02-22T15:50:25.642 INFO:tasks.workunit.client.0.smithi019.stdout:5/62: chown l6 1064534277 1 2024-02-22T15:50:25.642 INFO:tasks.workunit.client.0.smithi019.stdout:6/31: creat d6/fc x:0 0 0 2024-02-22T15:50:25.646 INFO:tasks.workunit.client.0.smithi019.stdout:1/78: mkdir d8/d18 0 2024-02-22T15:50:25.649 INFO:tasks.workunit.client.0.smithi019.stdout:3/62: write d9/d11/fd [827802,25332] 0 2024-02-22T15:50:25.649 INFO:tasks.workunit.client.0.smithi019.stdout:2/51: dwrite d1/de/ff [0,4194304] 0 2024-02-22T15:50:25.652 INFO:tasks.workunit.client.0.smithi019.stdout:5/63: write f1 [4782883,58566] 0 2024-02-22T15:50:25.658 INFO:tasks.workunit.client.0.smithi019.stdout:6/32: symlink d6/ld 0 2024-02-22T15:50:25.660 INFO:tasks.workunit.client.0.smithi019.stdout:2/52: stat d1/f11 0 2024-02-22T15:50:25.662 INFO:tasks.workunit.client.0.smithi019.stdout:8/40: dwrite f1 [0,4194304] 0 2024-02-22T15:50:25.662 INFO:tasks.workunit.client.0.smithi019.stdout:6/33: fsync d6/fc 0 2024-02-22T15:50:25.663 INFO:tasks.workunit.client.0.smithi019.stdout:4/25: getdents d2 0 2024-02-22T15:50:25.663 INFO:tasks.workunit.client.0.smithi019.stdout:7/16: dwrite f3 [0,4194304] 0 2024-02-22T15:50:25.664 INFO:tasks.workunit.client.0.smithi019.stdout:5/64: symlink d8/l10 0 2024-02-22T15:50:25.665 INFO:tasks.workunit.client.0.smithi019.stdout:5/65: fdatasync f1 0 2024-02-22T15:50:25.669 INFO:tasks.workunit.client.0.smithi019.stdout:7/17: chown f3 658150 1 2024-02-22T15:50:25.670 INFO:tasks.workunit.client.0.smithi019.stdout:2/53: creat d1/f12 x:0 0 0 2024-02-22T15:50:25.670 INFO:tasks.workunit.client.0.smithi019.stdout:3/63: link c2 d9/c13 0 2024-02-22T15:50:25.670 INFO:tasks.workunit.client.0.smithi019.stdout:9/97: dwrite d2/d7/fe [0,4194304] 0 2024-02-22T15:50:25.671 INFO:tasks.workunit.client.0.smithi019.stdout:1/79: truncate d8/f9 521391 0 2024-02-22T15:50:25.671 INFO:tasks.workunit.client.0.smithi019.stdout:0/61: dread d4/d9/f11 [0,4194304] 0 2024-02-22T15:50:25.672 INFO:tasks.workunit.client.0.smithi019.stdout:8/41: creat d9/fc x:0 0 0 2024-02-22T15:50:25.672 INFO:tasks.workunit.client.0.smithi019.stdout:2/54: dread - d1/f12 zero size 2024-02-22T15:50:25.674 INFO:tasks.workunit.client.0.smithi019.stdout:4/26: mkdir d2/db 0 2024-02-22T15:50:25.675 INFO:tasks.workunit.client.0.smithi019.stdout:5/66: mkdir d8/d11 0 2024-02-22T15:50:25.678 INFO:tasks.workunit.client.0.smithi019.stdout:5/67: write d8/ff [126844,124108] 0 2024-02-22T15:50:25.678 INFO:tasks.workunit.client.0.smithi019.stdout:1/80: write d8/d15/f17 [767933,24257] 0 2024-02-22T15:50:25.680 INFO:tasks.workunit.client.0.smithi019.stdout:0/62: symlink d4/l12 0 2024-02-22T15:50:25.681 INFO:tasks.workunit.client.0.smithi019.stdout:8/42: symlink d9/ld 0 2024-02-22T15:50:25.681 INFO:tasks.workunit.client.0.smithi019.stdout:9/98: rename d2/d7/l10 to d2/d7/l1c 0 2024-02-22T15:50:25.682 INFO:tasks.workunit.client.0.smithi019.stdout:5/68: write f5 [913139,11407] 0 2024-02-22T15:50:25.683 INFO:tasks.workunit.client.0.smithi019.stdout:2/55: symlink d1/de/l13 0 2024-02-22T15:50:25.684 INFO:tasks.workunit.client.0.smithi019.stdout:8/43: chown f5 727234 1 2024-02-22T15:50:25.687 INFO:tasks.workunit.client.0.smithi019.stdout:0/63: truncate d4/d9/f11 4904092 0 2024-02-22T15:50:25.689 INFO:tasks.workunit.client.0.smithi019.stdout:8/44: rename c2 to d9/ce 0 2024-02-22T15:50:25.690 INFO:tasks.workunit.client.0.smithi019.stdout:4/27: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:25.691 INFO:tasks.workunit.client.0.smithi019.stdout:9/99: rmdir d2/d16 0 2024-02-22T15:50:25.692 INFO:tasks.workunit.client.0.smithi019.stdout:2/56: truncate d1/f11 510231 0 2024-02-22T15:50:25.694 INFO:tasks.workunit.client.0.smithi019.stdout:1/81: getdents d8/d15 0 2024-02-22T15:50:25.695 INFO:tasks.workunit.client.0.smithi019.stdout:0/64: creat d4/d9/f13 x:0 0 0 2024-02-22T15:50:25.695 INFO:tasks.workunit.client.0.smithi019.stdout:5/69: dwrite f1 [4194304,4194304] 0 2024-02-22T15:50:25.700 INFO:tasks.workunit.client.0.smithi019.stdout:8/45: dread f5 [0,4194304] 0 2024-02-22T15:50:25.702 INFO:tasks.workunit.client.0.smithi019.stdout:9/100: rmdir d2/d7/dc 39 2024-02-22T15:50:25.704 INFO:tasks.workunit.client.0.smithi019.stdout:2/57: creat d1/f14 x:0 0 0 2024-02-22T15:50:25.706 INFO:tasks.workunit.client.0.smithi019.stdout:2/58: readlink d1/ld 0 2024-02-22T15:50:25.707 INFO:tasks.workunit.client.0.smithi019.stdout:1/82: creat d8/d15/f19 x:0 0 0 2024-02-22T15:50:25.709 INFO:tasks.workunit.client.0.smithi019.stdout:0/65: mknod d4/d5/dc/c14 0 2024-02-22T15:50:25.711 INFO:tasks.workunit.client.0.smithi019.stdout:3/64: write d9/fb [226400,7633] 0 2024-02-22T15:50:25.716 INFO:tasks.workunit.client.0.smithi019.stdout:5/70: mknod d8/d11/c12 0 2024-02-22T15:50:25.717 INFO:tasks.workunit.client.0.smithi019.stdout:6/34: dwrite d6/fb [0,4194304] 0 2024-02-22T15:50:25.717 INFO:tasks.workunit.client.0.smithi019.stdout:2/59: mkdir d1/de/d15 0 2024-02-22T15:50:25.721 INFO:tasks.workunit.client.0.smithi019.stdout:8/46: stat d9/ce 0 2024-02-22T15:50:25.726 INFO:tasks.workunit.client.0.smithi019.stdout:8/47: truncate d9/fc 680979 0 2024-02-22T15:50:25.730 INFO:tasks.workunit.client.0.smithi019.stdout:7/18: dwrite f1 [0,4194304] 0 2024-02-22T15:50:25.730 INFO:tasks.workunit.client.0.smithi019.stdout:0/66: write d4/d9/f11 [1691058,11496] 0 2024-02-22T15:50:25.731 INFO:tasks.workunit.client.0.smithi019.stdout:3/65: mkdir d9/d14 0 2024-02-22T15:50:25.732 INFO:tasks.workunit.client.0.smithi019.stdout:1/83: truncate d8/f14 541020 0 2024-02-22T15:50:25.736 INFO:tasks.workunit.client.0.smithi019.stdout:6/35: write d6/f9 [112174,88583] 0 2024-02-22T15:50:25.740 INFO:tasks.workunit.client.0.smithi019.stdout:6/36: chown c2 20 1 2024-02-22T15:50:25.740 INFO:tasks.workunit.client.0.smithi019.stdout:3/66: rmdir d9/d11 39 2024-02-22T15:50:25.742 INFO:tasks.workunit.client.0.smithi019.stdout:0/67: mknod d4/d5/d7/c15 0 2024-02-22T15:50:25.743 INFO:tasks.workunit.client.0.smithi019.stdout:2/60: rename d1/f11 to d1/de/f16 0 2024-02-22T15:50:25.744 INFO:tasks.workunit.client.0.smithi019.stdout:5/71: link d8/d11/c12 d8/d11/c13 0 2024-02-22T15:50:25.745 INFO:tasks.workunit.client.0.smithi019.stdout:6/37: creat d6/fe x:0 0 0 2024-02-22T15:50:25.746 INFO:tasks.workunit.client.0.smithi019.stdout:2/61: stat d1/f14 0 2024-02-22T15:50:25.747 INFO:tasks.workunit.client.0.smithi019.stdout:4/28: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:25.747 INFO:tasks.workunit.client.0.smithi019.stdout:7/19: getdents . 0 2024-02-22T15:50:25.749 INFO:tasks.workunit.client.0.smithi019.stdout:7/20: stat f3 0 2024-02-22T15:50:25.750 INFO:tasks.workunit.client.0.smithi019.stdout:6/38: creat d6/ff x:0 0 0 2024-02-22T15:50:25.751 INFO:tasks.workunit.client.0.smithi019.stdout:2/62: mkdir d1/d17 0 2024-02-22T15:50:25.752 INFO:tasks.workunit.client.0.smithi019.stdout:7/21: unlink f3 0 2024-02-22T15:50:25.756 INFO:tasks.workunit.client.0.smithi019.stdout:9/101: dwrite d2/f4 [0,4194304] 0 2024-02-22T15:50:25.756 INFO:tasks.workunit.client.0.smithi019.stdout:2/63: creat d1/f18 x:0 0 0 2024-02-22T15:50:25.757 INFO:tasks.workunit.client.0.smithi019.stdout:4/29: getdents d2/db 0 2024-02-22T15:50:25.759 INFO:tasks.workunit.client.0.smithi019.stdout:7/22: symlink l4 0 2024-02-22T15:50:25.764 INFO:tasks.workunit.client.0.smithi019.stdout:4/30: chown d2/c9 1007 1 2024-02-22T15:50:25.764 INFO:tasks.workunit.client.0.smithi019.stdout:5/72: dwrite d8/fc [0,4194304] 0 2024-02-22T15:50:25.765 INFO:tasks.workunit.client.0.smithi019.stdout:2/64: mkdir d1/d17/d19 0 2024-02-22T15:50:25.768 INFO:tasks.workunit.client.0.smithi019.stdout:4/31: truncate d2/fa 316289 0 2024-02-22T15:50:25.768 INFO:tasks.workunit.client.0.smithi019.stdout:6/39: dread d6/fb [0,4194304] 0 2024-02-22T15:50:25.769 INFO:tasks.workunit.client.0.smithi019.stdout:5/73: fdatasync d8/ff 0 2024-02-22T15:50:25.775 INFO:tasks.workunit.client.0.smithi019.stdout:4/32: symlink d2/lc 0 2024-02-22T15:50:25.776 INFO:tasks.workunit.client.0.smithi019.stdout:5/74: rename l6 to d8/dd/l14 0 2024-02-22T15:50:25.777 INFO:tasks.workunit.client.0.smithi019.stdout:4/33: truncate d2/fa 431118 0 2024-02-22T15:50:25.779 INFO:tasks.workunit.client.0.smithi019.stdout:2/65: link d1/ld d1/l1a 0 2024-02-22T15:50:25.779 INFO:tasks.workunit.client.0.smithi019.stdout:4/34: chown c1 33627942 1 2024-02-22T15:50:25.783 INFO:tasks.workunit.client.0.smithi019.stdout:5/75: rename f3 to d8/d11/f15 0 2024-02-22T15:50:25.784 INFO:tasks.workunit.client.0.smithi019.stdout:2/66: rename d1 to d1/d17/d1b 22 2024-02-22T15:50:25.785 INFO:tasks.workunit.client.0.smithi019.stdout:6/40: dwrite d6/fc [0,4194304] 0 2024-02-22T15:50:25.786 INFO:tasks.workunit.client.0.smithi019.stdout:2/67: dread - d1/f18 zero size 2024-02-22T15:50:25.787 INFO:tasks.workunit.client.0.smithi019.stdout:5/76: mknod d8/dd/c16 0 2024-02-22T15:50:25.788 INFO:tasks.workunit.client.0.smithi019.stdout:2/68: chown d1/de/ff 874 1 2024-02-22T15:50:25.791 INFO:tasks.workunit.client.0.smithi019.stdout:6/41: creat d6/f10 x:0 0 0 2024-02-22T15:50:25.791 INFO:tasks.workunit.client.0.smithi019.stdout:1/84: write d8/d15/f11 [218190,81285] 0 2024-02-22T15:50:25.800 INFO:tasks.workunit.client.0.smithi019.stdout:6/42: truncate d6/ff 821535 0 2024-02-22T15:50:25.807 INFO:tasks.workunit.client.0.smithi019.stdout:7/23: getdents . 0 2024-02-22T15:50:25.811 INFO:tasks.workunit.client.0.smithi019.stdout:8/48: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:25.814 INFO:tasks.workunit.client.0.smithi019.stdout:4/35: getdents d2 0 2024-02-22T15:50:25.818 INFO:tasks.workunit.client.0.smithi019.stdout:5/77: dread f1 [4194304,4194304] 0 2024-02-22T15:50:25.819 INFO:tasks.workunit.client.0.smithi019.stdout:7/24: mknod c5 0 2024-02-22T15:50:25.824 INFO:tasks.workunit.client.0.smithi019.stdout:8/49: creat d9/ff x:0 0 0 2024-02-22T15:50:25.827 INFO:tasks.workunit.client.0.smithi019.stdout:3/67: dwrite f6 [0,4194304] 0 2024-02-22T15:50:25.832 INFO:tasks.workunit.client.0.smithi019.stdout:0/68: dwrite f1 [0,4194304] 0 2024-02-22T15:50:25.832 INFO:tasks.workunit.client.0.smithi019.stdout:2/69: dwrite d1/f14 [0,4194304] 0 2024-02-22T15:50:25.834 INFO:tasks.workunit.client.0.smithi019.stdout:8/50: creat d9/f10 x:0 0 0 2024-02-22T15:50:25.837 INFO:tasks.workunit.client.0.smithi019.stdout:4/36: dread d2/fa [0,4194304] 0 2024-02-22T15:50:25.847 INFO:tasks.workunit.client.0.smithi019.stdout:9/102: dwrite d2/d7/f1b [0,4194304] 0 2024-02-22T15:50:25.847 INFO:tasks.workunit.client.0.smithi019.stdout:5/78: creat d8/d11/f17 x:0 0 0 2024-02-22T15:50:25.852 INFO:tasks.workunit.client.0.smithi019.stdout:4/37: creat d2/fd x:0 0 0 2024-02-22T15:50:25.853 INFO:tasks.workunit.client.0.smithi019.stdout:5/79: chown d8/l10 1549872 1 2024-02-22T15:50:25.854 INFO:tasks.workunit.client.0.smithi019.stdout:2/70: mkdir d1/d17/d1c 0 2024-02-22T15:50:25.859 INFO:tasks.workunit.client.0.smithi019.stdout:5/80: dread f2 [0,4194304] 0 2024-02-22T15:50:25.860 INFO:tasks.workunit.client.0.smithi019.stdout:7/25: truncate f1 3435214 0 2024-02-22T15:50:25.861 INFO:tasks.workunit.client.0.smithi019.stdout:4/38: write d2/fd [103676,37917] 0 2024-02-22T15:50:25.864 INFO:tasks.workunit.client.0.smithi019.stdout:3/68: link f7 d9/d12/f15 0 2024-02-22T15:50:25.864 INFO:tasks.workunit.client.0.smithi019.stdout:8/51: dwrite d9/f10 [0,4194304] 0 2024-02-22T15:50:25.867 INFO:tasks.workunit.client.0.smithi019.stdout:4/39: write d2/f6 [4963429,44110] 0 2024-02-22T15:50:25.867 INFO:tasks.workunit.client.0.smithi019.stdout:0/69: getdents d4/d9 0 2024-02-22T15:50:25.869 INFO:tasks.workunit.client.0.smithi019.stdout:5/81: mknod d8/d11/c18 0 2024-02-22T15:50:25.869 INFO:tasks.workunit.client.0.smithi019.stdout:2/71: mknod d1/de/d15/c1d 0 2024-02-22T15:50:25.871 INFO:tasks.workunit.client.0.smithi019.stdout:8/52: fdatasync f5 0 2024-02-22T15:50:25.871 INFO:tasks.workunit.client.0.smithi019.stdout:4/40: dread d2/fa [0,4194304] 0 2024-02-22T15:50:25.872 INFO:tasks.workunit.client.0.smithi019.stdout:7/26: unlink f1 0 2024-02-22T15:50:25.873 INFO:tasks.workunit.client.0.smithi019.stdout:2/72: creat d1/de/f1e x:0 0 0 2024-02-22T15:50:25.877 INFO:tasks.workunit.client.0.smithi019.stdout:5/82: mkdir d8/dd/d19 0 2024-02-22T15:50:25.879 INFO:tasks.workunit.client.0.smithi019.stdout:0/70: mknod d4/db/c16 0 2024-02-22T15:50:25.881 INFO:tasks.workunit.client.0.smithi019.stdout:8/53: rename d9/ld to d9/l11 0 2024-02-22T15:50:25.881 INFO:tasks.workunit.client.0.smithi019.stdout:7/27: creat f6 x:0 0 0 2024-02-22T15:50:25.882 INFO:tasks.workunit.client.0.smithi019.stdout:5/83: chown f1 1271 1 2024-02-22T15:50:25.883 INFO:tasks.workunit.client.0.smithi019.stdout:3/69: dread d9/d12/f15 [0,4194304] 0 2024-02-22T15:50:25.884 INFO:tasks.workunit.client.0.smithi019.stdout:7/28: dread - f6 zero size 2024-02-22T15:50:25.884 INFO:tasks.workunit.client.0.smithi019.stdout:3/70: stat f6 0 2024-02-22T15:50:25.884 INFO:tasks.workunit.client.0.smithi019.stdout:9/103: getdents d2/d7/dc 0 2024-02-22T15:50:25.886 INFO:tasks.workunit.client.0.smithi019.stdout:5/84: stat d8/d11/c12 0 2024-02-22T15:50:25.886 INFO:tasks.workunit.client.0.smithi019.stdout:8/54: truncate f6 1024997 0 2024-02-22T15:50:25.888 INFO:tasks.workunit.client.0.smithi019.stdout:7/29: write f6 [575133,35538] 0 2024-02-22T15:50:25.892 INFO:tasks.workunit.client.0.smithi019.stdout:4/41: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:25.892 INFO:tasks.workunit.client.0.smithi019.stdout:4/42: read d2/fa [176141,36459] 0 2024-02-22T15:50:25.898 INFO:tasks.workunit.client.0.smithi019.stdout:7/30: rename l4 to l7 0 2024-02-22T15:50:25.900 INFO:tasks.workunit.client.0.smithi019.stdout:3/71: write d9/d12/f15 [2955549,118471] 0 2024-02-22T15:50:25.901 INFO:tasks.workunit.client.0.smithi019.stdout:9/104: write d2/d7/dc/f11 [1422641,32739] 0 2024-02-22T15:50:25.903 INFO:tasks.workunit.client.0.smithi019.stdout:5/85: truncate f1 6519076 0 2024-02-22T15:50:25.903 INFO:tasks.workunit.client.0.smithi019.stdout:7/31: creat f8 x:0 0 0 2024-02-22T15:50:25.910 INFO:tasks.workunit.client.0.smithi019.stdout:9/105: symlink d2/d7/dc/l1d 0 2024-02-22T15:50:25.918 INFO:tasks.workunit.client.0.smithi019.stdout:9/106: fsync d2/d7/fe 0 2024-02-22T15:50:25.922 INFO:tasks.workunit.client.0.smithi019.stdout:5/86: rename d8/dd/l14 to d8/dd/l1a 0 2024-02-22T15:50:25.922 INFO:tasks.workunit.client.0.smithi019.stdout:0/71: dwrite d4/d5/fe [0,4194304] 0 2024-02-22T15:50:25.923 INFO:tasks.workunit.client.0.smithi019.stdout:4/43: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:25.925 INFO:tasks.workunit.client.0.smithi019.stdout:8/55: dwrite f5 [0,4194304] 0 2024-02-22T15:50:25.926 INFO:tasks.workunit.client.0.smithi019.stdout:2/73: dwrite d1/fa [0,4194304] 0 2024-02-22T15:50:25.930 INFO:tasks.workunit.client.0.smithi019.stdout:9/107: mknod d2/c1e 0 2024-02-22T15:50:25.936 INFO:tasks.workunit.client.0.smithi019.stdout:0/72: creat d4/d9/f17 x:0 0 0 2024-02-22T15:50:25.936 INFO:tasks.workunit.client.0.smithi019.stdout:7/32: dwrite f6 [0,4194304] 0 2024-02-22T15:50:25.938 INFO:tasks.workunit.client.0.smithi019.stdout:0/73: fdatasync d4/d9/f17 0 2024-02-22T15:50:25.941 INFO:tasks.workunit.client.0.smithi019.stdout:8/56: creat d9/f12 x:0 0 0 2024-02-22T15:50:25.941 INFO:tasks.workunit.client.0.smithi019.stdout:9/108: mkdir d2/d1f 0 2024-02-22T15:50:25.945 INFO:tasks.workunit.client.0.smithi019.stdout:0/74: write d4/d9/f17 [735934,94211] 0 2024-02-22T15:50:25.948 INFO:tasks.workunit.client.0.smithi019.stdout:2/74: link d1/f12 d1/f1f 0 2024-02-22T15:50:25.948 INFO:tasks.workunit.client.0.smithi019.stdout:7/33: dread - f8 zero size 2024-02-22T15:50:25.948 INFO:tasks.workunit.client.0.smithi019.stdout:8/57: chown d9/fa 0 1 2024-02-22T15:50:25.952 INFO:tasks.workunit.client.0.smithi019.stdout:7/34: mkdir d9 0 2024-02-22T15:50:25.953 INFO:tasks.workunit.client.0.smithi019.stdout:3/72: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:25.957 INFO:tasks.workunit.client.0.smithi019.stdout:8/58: creat d9/f13 x:0 0 0 2024-02-22T15:50:25.958 INFO:tasks.workunit.client.0.smithi019.stdout:2/75: write d1/f12 [184864,33731] 0 2024-02-22T15:50:25.959 INFO:tasks.workunit.client.0.smithi019.stdout:9/109: creat d2/d7/f20 x:0 0 0 2024-02-22T15:50:25.970 INFO:tasks.workunit.client.0.smithi019.stdout:4/44: dwrite d2/fa [0,4194304] 0 2024-02-22T15:50:25.974 INFO:tasks.workunit.client.0.smithi019.stdout:5/87: dwrite d8/fe [0,4194304] 0 2024-02-22T15:50:25.974 INFO:tasks.workunit.client.0.smithi019.stdout:7/35: creat d9/fa x:0 0 0 2024-02-22T15:50:25.974 INFO:tasks.workunit.client.0.smithi019.stdout:2/76: rmdir d1 39 2024-02-22T15:50:25.975 INFO:tasks.workunit.client.0.smithi019.stdout:8/59: mkdir d9/d14 0 2024-02-22T15:50:25.975 INFO:tasks.workunit.client.0.smithi019.stdout:4/45: chown d2/c5 19583318 1 2024-02-22T15:50:25.979 INFO:tasks.workunit.client.0.smithi019.stdout:7/36: mknod d9/cb 0 2024-02-22T15:50:25.981 INFO:tasks.workunit.client.0.smithi019.stdout:3/73: rename d9/d12 to d9/d16 0 2024-02-22T15:50:25.981 INFO:tasks.workunit.client.0.smithi019.stdout:9/110: dread d2/d7/f12 [0,4194304] 0 2024-02-22T15:50:25.982 INFO:tasks.workunit.client.0.smithi019.stdout:0/75: getdents d4/d5/d7 0 2024-02-22T15:50:25.986 INFO:tasks.workunit.client.0.smithi019.stdout:1/85: sync 2024-02-22T15:50:25.986 INFO:tasks.workunit.client.0.smithi019.stdout:6/43: sync 2024-02-22T15:50:25.987 INFO:tasks.workunit.client.0.smithi019.stdout:4/46: creat d2/db/fe x:0 0 0 2024-02-22T15:50:25.988 INFO:tasks.workunit.client.0.smithi019.stdout:5/88: rename d8/lb to d8/dd/d19/l1b 0 2024-02-22T15:50:25.989 INFO:tasks.workunit.client.0.smithi019.stdout:0/76: mknod d4/db/c18 0 2024-02-22T15:50:25.993 INFO:tasks.workunit.client.0.smithi019.stdout:1/86: creat d8/f1a x:0 0 0 2024-02-22T15:50:25.994 INFO:tasks.workunit.client.0.smithi019.stdout:5/89: dread d8/ff [0,4194304] 0 2024-02-22T15:50:25.999 INFO:tasks.workunit.client.0.smithi019.stdout:8/60: getdents d9 0 2024-02-22T15:50:26.002 INFO:tasks.workunit.client.0.smithi019.stdout:3/74: rename d9/d11/fe to d9/d11/f17 0 2024-02-22T15:50:26.004 INFO:tasks.workunit.client.0.smithi019.stdout:2/77: rename d1/l1a to d1/de/l20 0 2024-02-22T15:50:26.004 INFO:tasks.workunit.client.0.smithi019.stdout:7/37: dwrite f6 [0,4194304] 0 2024-02-22T15:50:26.006 INFO:tasks.workunit.client.0.smithi019.stdout:9/111: dwrite d2/d7/dc/f11 [0,4194304] 0 2024-02-22T15:50:26.006 INFO:tasks.workunit.client.0.smithi019.stdout:7/38: chown c5 20882705 1 2024-02-22T15:50:26.011 INFO:tasks.workunit.client.0.smithi019.stdout:7/39: truncate f8 1024808 0 2024-02-22T15:50:26.011 INFO:tasks.workunit.client.0.smithi019.stdout:3/75: write d9/d16/f15 [3404797,86927] 0 2024-02-22T15:50:26.013 INFO:tasks.workunit.client.0.smithi019.stdout:4/47: getdents d2 0 2024-02-22T15:50:26.014 INFO:tasks.workunit.client.0.smithi019.stdout:5/90: creat d8/d11/f1c x:0 0 0 2024-02-22T15:50:26.014 INFO:tasks.workunit.client.0.smithi019.stdout:4/48: readlink d2/l7 0 2024-02-22T15:50:26.017 INFO:tasks.workunit.client.0.smithi019.stdout:0/77: truncate d4/f10 837983 0 2024-02-22T15:50:26.019 INFO:tasks.workunit.client.0.smithi019.stdout:2/78: read - d1/f10 zero size 2024-02-22T15:50:26.020 INFO:tasks.workunit.client.0.smithi019.stdout:2/79: write d1/f18 [185310,13239] 0 2024-02-22T15:50:26.021 INFO:tasks.workunit.client.0.smithi019.stdout:2/80: read d1/de/ff [2054026,56527] 0 2024-02-22T15:50:26.021 INFO:tasks.workunit.client.0.smithi019.stdout:6/44: rename l1 to d6/l11 0 2024-02-22T15:50:26.022 INFO:tasks.workunit.client.0.smithi019.stdout:8/61: write f3 [1034626,32021] 0 2024-02-22T15:50:26.023 INFO:tasks.workunit.client.0.smithi019.stdout:8/62: readlink d9/l11 0 2024-02-22T15:50:26.024 INFO:tasks.workunit.client.0.smithi019.stdout:7/40: rmdir d9 39 2024-02-22T15:50:26.026 INFO:tasks.workunit.client.0.smithi019.stdout:5/91: mknod d8/d11/c1d 0 2024-02-22T15:50:26.027 INFO:tasks.workunit.client.0.smithi019.stdout:4/49: mknod d2/db/cf 0 2024-02-22T15:50:26.028 INFO:tasks.workunit.client.0.smithi019.stdout:5/92: chown d8/c9 1 1 2024-02-22T15:50:26.029 INFO:tasks.workunit.client.0.smithi019.stdout:4/50: read - d2/db/fe zero size 2024-02-22T15:50:26.029 INFO:tasks.workunit.client.0.smithi019.stdout:5/93: stat d8/ff 0 2024-02-22T15:50:26.030 INFO:tasks.workunit.client.0.smithi019.stdout:0/78: creat d4/f19 x:0 0 0 2024-02-22T15:50:26.032 INFO:tasks.workunit.client.0.smithi019.stdout:4/51: readlink l0 0 2024-02-22T15:50:26.033 INFO:tasks.workunit.client.0.smithi019.stdout:2/81: dread d1/f18 [0,4194304] 0 2024-02-22T15:50:26.035 INFO:tasks.workunit.client.0.smithi019.stdout:5/94: dread - d8/d11/f1c zero size 2024-02-22T15:50:26.035 INFO:tasks.workunit.client.0.smithi019.stdout:2/82: chown d1 946 1 2024-02-22T15:50:26.035 INFO:tasks.workunit.client.0.smithi019.stdout:1/87: link d8/fd d8/d15/f1b 0 2024-02-22T15:50:26.038 INFO:tasks.workunit.client.0.smithi019.stdout:0/79: creat d4/d9/f1a x:0 0 0 2024-02-22T15:50:26.040 INFO:tasks.workunit.client.0.smithi019.stdout:8/63: dwrite f5 [4194304,4194304] 0 2024-02-22T15:50:26.040 INFO:tasks.workunit.client.0.smithi019.stdout:2/83: mkdir d1/d17/d21 0 2024-02-22T15:50:26.042 INFO:tasks.workunit.client.0.smithi019.stdout:9/112: write d2/f19 [4174401,93555] 0 2024-02-22T15:50:26.043 INFO:tasks.workunit.client.0.smithi019.stdout:1/88: rmdir d8 39 2024-02-22T15:50:26.043 INFO:tasks.workunit.client.0.smithi019.stdout:2/84: chown d1/de 683 1 2024-02-22T15:50:26.046 INFO:tasks.workunit.client.0.smithi019.stdout:2/85: chown d1/de 696 1 2024-02-22T15:50:26.047 INFO:tasks.workunit.client.0.smithi019.stdout:5/95: readlink d8/dd/l1a 0 2024-02-22T15:50:26.049 INFO:tasks.workunit.client.0.smithi019.stdout:3/76: link c5 d9/d11/c18 0 2024-02-22T15:50:26.050 INFO:tasks.workunit.client.0.smithi019.stdout:0/80: mkdir d4/db/d1b 0 2024-02-22T15:50:26.050 INFO:tasks.workunit.client.0.smithi019.stdout:7/41: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:26.052 INFO:tasks.workunit.client.0.smithi019.stdout:0/81: stat d4/d5/d7 0 2024-02-22T15:50:26.053 INFO:tasks.workunit.client.0.smithi019.stdout:9/113: fdatasync d2/fb 0 2024-02-22T15:50:26.053 INFO:tasks.workunit.client.0.smithi019.stdout:2/86: rmdir d1/de 39 2024-02-22T15:50:26.054 INFO:tasks.workunit.client.0.smithi019.stdout:1/89: chown d8/d15/f19 67068541 1 2024-02-22T15:50:26.054 INFO:tasks.workunit.client.0.smithi019.stdout:2/87: stat d1/l7 0 2024-02-22T15:50:26.055 INFO:tasks.workunit.client.0.smithi019.stdout:8/64: dread f5 [4194304,4194304] 0 2024-02-22T15:50:26.057 INFO:tasks.workunit.client.0.smithi019.stdout:8/65: fdatasync d9/f10 0 2024-02-22T15:50:26.057 INFO:tasks.workunit.client.0.smithi019.stdout:9/114: chown d2/f19 12950989 1 2024-02-22T15:50:26.058 INFO:tasks.workunit.client.0.smithi019.stdout:8/66: stat d9/lb 0 2024-02-22T15:50:26.064 INFO:tasks.workunit.client.0.smithi019.stdout:5/96: rename d8/dd/d19/l1b to d8/d11/l1e 0 2024-02-22T15:50:26.064 INFO:tasks.workunit.client.0.smithi019.stdout:4/52: truncate d2/fa 1880664 0 2024-02-22T15:50:26.066 INFO:tasks.workunit.client.0.smithi019.stdout:1/90: truncate d8/fc 2561283 0 2024-02-22T15:50:26.066 INFO:tasks.workunit.client.0.smithi019.stdout:4/53: readlink l0 0 2024-02-22T15:50:26.067 INFO:tasks.workunit.client.0.smithi019.stdout:8/67: stat c8 0 2024-02-22T15:50:26.068 INFO:tasks.workunit.client.0.smithi019.stdout:5/97: chown d8/fc 61433 1 2024-02-22T15:50:26.070 INFO:tasks.workunit.client.0.smithi019.stdout:1/91: write d8/f1a [794580,125879] 0 2024-02-22T15:50:26.070 INFO:tasks.workunit.client.0.smithi019.stdout:4/54: unlink l0 0 2024-02-22T15:50:26.071 INFO:tasks.workunit.client.0.smithi019.stdout:8/68: write d9/ff [394646,117956] 0 2024-02-22T15:50:26.073 INFO:tasks.workunit.client.0.smithi019.stdout:9/115: mknod d2/d7/c21 0 2024-02-22T15:50:26.075 INFO:tasks.workunit.client.0.smithi019.stdout:3/77: getdents d9/d11 0 2024-02-22T15:50:26.077 INFO:tasks.workunit.client.0.smithi019.stdout:2/88: dwrite d1/de/f1e [0,4194304] 0 2024-02-22T15:50:26.077 INFO:tasks.workunit.client.0.smithi019.stdout:9/116: write d2/f19 [1944117,63714] 0 2024-02-22T15:50:26.080 INFO:tasks.workunit.client.0.smithi019.stdout:0/82: rename d4/f10 to d4/db/d1b/f1c 0 2024-02-22T15:50:26.082 INFO:tasks.workunit.client.0.smithi019.stdout:7/42: getdents d9 0 2024-02-22T15:50:26.082 INFO:tasks.workunit.client.0.smithi019.stdout:8/69: mknod d9/c15 0 2024-02-22T15:50:26.082 INFO:tasks.workunit.client.0.smithi019.stdout:4/55: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:26.084 INFO:tasks.workunit.client.0.smithi019.stdout:3/78: fsync d9/d11/f17 0 2024-02-22T15:50:26.086 INFO:tasks.workunit.client.0.smithi019.stdout:3/79: chown d9 105814109 1 2024-02-22T15:50:26.086 INFO:tasks.workunit.client.0.smithi019.stdout:4/56: chown d2/c8 1 1 2024-02-22T15:50:26.091 INFO:tasks.workunit.client.0.smithi019.stdout:4/57: truncate d2/db/fe 75330 0 2024-02-22T15:50:26.092 INFO:tasks.workunit.client.0.smithi019.stdout:4/58: readlink d2/lc 0 2024-02-22T15:50:26.093 INFO:tasks.workunit.client.0.smithi019.stdout:7/43: dread f6 [0,4194304] 0 2024-02-22T15:50:26.095 INFO:tasks.workunit.client.0.smithi019.stdout:5/98: truncate d8/fc 1975024 0 2024-02-22T15:50:26.096 INFO:tasks.workunit.client.0.smithi019.stdout:8/70: unlink d9/f13 0 2024-02-22T15:50:26.096 INFO:tasks.workunit.client.0.smithi019.stdout:7/44: chown f6 123 1 2024-02-22T15:50:26.097 INFO:tasks.workunit.client.0.smithi019.stdout:2/89: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:26.099 INFO:tasks.workunit.client.0.smithi019.stdout:1/92: rename d8/f9 to d8/d18/f1c 0 2024-02-22T15:50:26.102 INFO:tasks.workunit.client.0.smithi019.stdout:2/90: dread d1/f14 [0,4194304] 0 2024-02-22T15:50:26.103 INFO:tasks.workunit.client.0.smithi019.stdout:4/59: fdatasync d2/f6 0 2024-02-22T15:50:26.105 INFO:tasks.workunit.client.0.smithi019.stdout:2/91: dread - d1/f10 zero size 2024-02-22T15:50:26.106 INFO:tasks.workunit.client.0.smithi019.stdout:3/80: mkdir d9/d14/d19 0 2024-02-22T15:50:26.106 INFO:tasks.workunit.client.0.smithi019.stdout:3/81: chown f7 10843958 1 2024-02-22T15:50:26.107 INFO:tasks.workunit.client.0.smithi019.stdout:0/83: truncate d4/db/d1b/f1c 1108569 0 2024-02-22T15:50:26.108 INFO:tasks.workunit.client.0.smithi019.stdout:5/99: mknod d8/d11/c1f 0 2024-02-22T15:50:26.108 INFO:tasks.workunit.client.0.smithi019.stdout:0/84: truncate d4/d9/f11 4905409 0 2024-02-22T15:50:26.109 INFO:tasks.workunit.client.0.smithi019.stdout:5/100: read f2 [565600,121659] 0 2024-02-22T15:50:26.109 INFO:tasks.workunit.client.0.smithi019.stdout:9/117: link d2/d7/c1a d2/d7/dc/c22 0 2024-02-22T15:50:26.110 INFO:tasks.workunit.client.0.smithi019.stdout:9/118: fsync d2/d7/f9 0 2024-02-22T15:50:26.113 INFO:tasks.workunit.client.0.smithi019.stdout:1/93: symlink d8/d15/l1d 0 2024-02-22T15:50:26.114 INFO:tasks.workunit.client.0.smithi019.stdout:4/60: mknod d2/c10 0 2024-02-22T15:50:26.116 INFO:tasks.workunit.client.0.smithi019.stdout:0/85: mkdir d4/db/d1d 0 2024-02-22T15:50:26.119 INFO:tasks.workunit.client.0.smithi019.stdout:7/45: dwrite f6 [0,4194304] 0 2024-02-22T15:50:26.121 INFO:tasks.workunit.client.0.smithi019.stdout:1/94: symlink d8/d18/l1e 0 2024-02-22T15:50:26.122 INFO:tasks.workunit.client.0.smithi019.stdout:2/92: creat d1/d17/d1c/f22 x:0 0 0 2024-02-22T15:50:26.124 INFO:tasks.workunit.client.0.smithi019.stdout:3/82: mkdir d9/d14/d19/d1a 0 2024-02-22T15:50:26.125 INFO:tasks.workunit.client.0.smithi019.stdout:0/86: mknod d4/d9/c1e 0 2024-02-22T15:50:26.125 INFO:tasks.workunit.client.0.smithi019.stdout:8/71: link d9/ce d9/d14/c16 0 2024-02-22T15:50:26.126 INFO:tasks.workunit.client.0.smithi019.stdout:0/87: dread - d4/d9/f13 zero size 2024-02-22T15:50:26.128 INFO:tasks.workunit.client.0.smithi019.stdout:7/46: creat d9/fc x:0 0 0 2024-02-22T15:50:26.128 INFO:tasks.workunit.client.0.smithi019.stdout:7/47: stat d9/cb 0 2024-02-22T15:50:26.129 INFO:tasks.workunit.client.0.smithi019.stdout:9/119: rename d2/d7/l1c to d2/d7/dc/l23 0 2024-02-22T15:50:26.132 INFO:tasks.workunit.client.0.smithi019.stdout:9/120: write d2/f19 [2056061,118401] 0 2024-02-22T15:50:26.133 INFO:tasks.workunit.client.0.smithi019.stdout:3/83: creat d9/d14/f1b x:0 0 0 2024-02-22T15:50:26.133 INFO:tasks.workunit.client.0.smithi019.stdout:8/72: dread f3 [0,4194304] 0 2024-02-22T15:50:26.136 INFO:tasks.workunit.client.0.smithi019.stdout:0/88: dread f2 [0,4194304] 0 2024-02-22T15:50:26.136 INFO:tasks.workunit.client.0.smithi019.stdout:1/95: dread d8/f14 [0,4194304] 0 2024-02-22T15:50:26.137 INFO:tasks.workunit.client.0.smithi019.stdout:0/89: readlink l0 0 2024-02-22T15:50:26.137 INFO:tasks.workunit.client.0.smithi019.stdout:7/48: mkdir d9/dd 0 2024-02-22T15:50:26.138 INFO:tasks.workunit.client.0.smithi019.stdout:9/121: write d2/d7/f9 [512406,121521] 0 2024-02-22T15:50:26.140 INFO:tasks.workunit.client.0.smithi019.stdout:5/101: link c4 d8/c20 0 2024-02-22T15:50:26.143 INFO:tasks.workunit.client.0.smithi019.stdout:3/84: dread d9/d16/f15 [0,4194304] 0 2024-02-22T15:50:26.144 INFO:tasks.workunit.client.0.smithi019.stdout:7/49: dread f6 [0,4194304] 0 2024-02-22T15:50:26.145 INFO:tasks.workunit.client.0.smithi019.stdout:7/50: fdatasync d9/fa 0 2024-02-22T15:50:26.147 INFO:tasks.workunit.client.0.smithi019.stdout:3/85: dread f7 [0,4194304] 0 2024-02-22T15:50:26.149 INFO:tasks.workunit.client.0.smithi019.stdout:1/96: symlink d8/l1f 0 2024-02-22T15:50:26.152 INFO:tasks.workunit.client.0.smithi019.stdout:5/102: mkdir d8/d11/d21 0 2024-02-22T15:50:26.153 INFO:tasks.workunit.client.0.smithi019.stdout:8/73: mkdir d9/d14/d17 0 2024-02-22T15:50:26.156 INFO:tasks.workunit.client.0.smithi019.stdout:8/74: write f1 [5972788,44641] 0 2024-02-22T15:50:26.159 INFO:tasks.workunit.client.0.smithi019.stdout:3/86: chown c5 82 1 2024-02-22T15:50:26.159 INFO:tasks.workunit.client.0.smithi019.stdout:4/61: dwrite d2/fa [0,4194304] 0 2024-02-22T15:50:26.159 INFO:tasks.workunit.client.0.smithi019.stdout:8/75: chown d9/fc 6162721 1 2024-02-22T15:50:26.161 INFO:tasks.workunit.client.0.smithi019.stdout:9/122: link d2/f4 d2/d7/dc/f24 0 2024-02-22T15:50:26.162 INFO:tasks.workunit.client.0.smithi019.stdout:1/97: rmdir d8/d18 39 2024-02-22T15:50:26.163 INFO:tasks.workunit.client.0.smithi019.stdout:0/90: dwrite d4/d9/f11 [4194304,4194304] 0 2024-02-22T15:50:26.169 INFO:tasks.workunit.client.0.smithi019.stdout:5/103: creat d8/d11/d21/f22 x:0 0 0 2024-02-22T15:50:26.171 INFO:tasks.workunit.client.0.smithi019.stdout:4/62: unlink d2/db/fe 0 2024-02-22T15:50:26.172 INFO:tasks.workunit.client.0.smithi019.stdout:8/76: dwrite f1 [0,4194304] 0 2024-02-22T15:50:26.172 INFO:tasks.workunit.client.0.smithi019.stdout:9/123: mknod d2/c25 0 2024-02-22T15:50:26.173 INFO:tasks.workunit.client.0.smithi019.stdout:4/63: chown d2/db/cf 1522044137 1 2024-02-22T15:50:26.174 INFO:tasks.workunit.client.0.smithi019.stdout:7/51: rmdir d9/dd 0 2024-02-22T15:50:26.175 INFO:tasks.workunit.client.0.smithi019.stdout:0/91: creat d4/db/d1b/f1f x:0 0 0 2024-02-22T15:50:26.178 INFO:tasks.workunit.client.0.smithi019.stdout:1/98: chown d8/d18/f1c 0 1 2024-02-22T15:50:26.179 INFO:tasks.workunit.client.0.smithi019.stdout:9/124: mkdir d2/d7/d14/d26 0 2024-02-22T15:50:26.180 INFO:tasks.workunit.client.0.smithi019.stdout:4/64: creat d2/db/f11 x:0 0 0 2024-02-22T15:50:26.182 INFO:tasks.workunit.client.0.smithi019.stdout:3/87: symlink d9/d14/d19/d1a/l1c 0 2024-02-22T15:50:26.188 INFO:tasks.workunit.client.0.smithi019.stdout:9/125: mkdir d2/d7/dc/d27 0 2024-02-22T15:50:26.188 INFO:tasks.workunit.client.0.smithi019.stdout:1/99: truncate d8/d15/f19 4862 0 2024-02-22T15:50:26.189 INFO:tasks.workunit.client.0.smithi019.stdout:4/65: mknod d2/c12 0 2024-02-22T15:50:26.196 INFO:tasks.workunit.client.0.smithi019.stdout:5/104: dwrite f2 [0,4194304] 0 2024-02-22T15:50:26.196 INFO:tasks.workunit.client.0.smithi019.stdout:0/92: dwrite d4/d9/f1a [0,4194304] 0 2024-02-22T15:50:26.197 INFO:tasks.workunit.client.0.smithi019.stdout:1/100: rmdir d8/d18 39 2024-02-22T15:50:26.198 INFO:tasks.workunit.client.0.smithi019.stdout:4/66: mknod d2/c13 0 2024-02-22T15:50:26.199 INFO:tasks.workunit.client.0.smithi019.stdout:7/52: dread f6 [0,4194304] 0 2024-02-22T15:50:26.199 INFO:tasks.workunit.client.0.smithi019.stdout:9/126: unlink d2/d7/f20 0 2024-02-22T15:50:26.199 INFO:tasks.workunit.client.0.smithi019.stdout:4/67: chown d2/c4 1 1 2024-02-22T15:50:26.200 INFO:tasks.workunit.client.0.smithi019.stdout:8/77: truncate d9/ff 228839 0 2024-02-22T15:50:26.201 INFO:tasks.workunit.client.0.smithi019.stdout:8/78: readlink d9/lb 0 2024-02-22T15:50:26.202 INFO:tasks.workunit.client.0.smithi019.stdout:3/88: link d9/d16/f15 d9/d16/f1d 0 2024-02-22T15:50:26.204 INFO:tasks.workunit.client.0.smithi019.stdout:1/101: read d8/fd [441652,75887] 0 2024-02-22T15:50:26.206 INFO:tasks.workunit.client.0.smithi019.stdout:7/53: mknod d9/ce 0 2024-02-22T15:50:26.207 INFO:tasks.workunit.client.0.smithi019.stdout:4/68: mkdir d2/db/d14 0 2024-02-22T15:50:26.207 INFO:tasks.workunit.client.0.smithi019.stdout:9/127: truncate d2/fb 4830205 0 2024-02-22T15:50:26.211 INFO:tasks.workunit.client.0.smithi019.stdout:5/105: fdatasync d8/fc 0 2024-02-22T15:50:26.213 INFO:tasks.workunit.client.0.smithi019.stdout:1/102: creat d8/d15/f20 x:0 0 0 2024-02-22T15:50:26.215 INFO:tasks.workunit.client.0.smithi019.stdout:4/69: symlink d2/db/l15 0 2024-02-22T15:50:26.218 INFO:tasks.workunit.client.0.smithi019.stdout:3/89: dwrite f7 [0,4194304] 0 2024-02-22T15:50:26.218 INFO:tasks.workunit.client.0.smithi019.stdout:9/128: symlink d2/d7/d14/d26/l28 0 2024-02-22T15:50:26.220 INFO:tasks.workunit.client.0.smithi019.stdout:3/90: chown d9/d11/c18 50511190 1 2024-02-22T15:50:26.222 INFO:tasks.workunit.client.0.smithi019.stdout:4/70: symlink d2/db/d14/l16 0 2024-02-22T15:50:26.222 INFO:tasks.workunit.client.0.smithi019.stdout:9/129: creat d2/d7/dc/f29 x:0 0 0 2024-02-22T15:50:26.223 INFO:tasks.workunit.client.0.smithi019.stdout:5/106: getdents d8/dd 0 2024-02-22T15:50:26.224 INFO:tasks.workunit.client.0.smithi019.stdout:1/103: dread d8/fc [0,4194304] 0 2024-02-22T15:50:26.224 INFO:tasks.workunit.client.0.smithi019.stdout:4/71: chown d2/fa 2350 1 2024-02-22T15:50:26.226 INFO:tasks.workunit.client.0.smithi019.stdout:5/107: chown d8/fe 1 1 2024-02-22T15:50:26.226 INFO:tasks.workunit.client.0.smithi019.stdout:8/79: rmdir d9 39 2024-02-22T15:50:26.227 INFO:tasks.workunit.client.0.smithi019.stdout:1/104: rmdir d8 39 2024-02-22T15:50:26.231 INFO:tasks.workunit.client.0.smithi019.stdout:7/54: rmdir d9 39 2024-02-22T15:50:26.232 INFO:tasks.workunit.client.0.smithi019.stdout:5/108: readlink d8/d11/l1e 0 2024-02-22T15:50:26.233 INFO:tasks.workunit.client.0.smithi019.stdout:8/80: write d9/fc [1374592,109445] 0 2024-02-22T15:50:26.233 INFO:tasks.workunit.client.0.smithi019.stdout:3/91: dwrite d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:26.235 INFO:tasks.workunit.client.0.smithi019.stdout:3/92: chown f6 216406105 1 2024-02-22T15:50:26.235 INFO:tasks.workunit.client.0.smithi019.stdout:8/81: truncate d9/f12 489590 0 2024-02-22T15:50:26.236 INFO:tasks.workunit.client.0.smithi019.stdout:8/82: readlink d9/lb 0 2024-02-22T15:50:26.237 INFO:tasks.workunit.client.0.smithi019.stdout:8/83: stat f1 0 2024-02-22T15:50:26.241 INFO:tasks.workunit.client.0.smithi019.stdout:9/130: creat d2/d7/f2a x:0 0 0 2024-02-22T15:50:26.242 INFO:tasks.workunit.client.0.smithi019.stdout:9/131: chown d2/d7/d14/d26 1996 1 2024-02-22T15:50:26.243 INFO:tasks.workunit.client.0.smithi019.stdout:3/93: dread f6 [0,4194304] 0 2024-02-22T15:50:26.245 INFO:tasks.workunit.client.0.smithi019.stdout:1/105: fsync d8/d15/f1b 0 2024-02-22T15:50:26.245 INFO:tasks.workunit.client.0.smithi019.stdout:5/109: dread f5 [4194304,4194304] 0 2024-02-22T15:50:26.248 INFO:tasks.workunit.client.0.smithi019.stdout:4/72: getdents d2/db 0 2024-02-22T15:50:26.250 INFO:tasks.workunit.client.0.smithi019.stdout:3/94: dread d9/d16/f15 [0,4194304] 0 2024-02-22T15:50:26.252 INFO:tasks.workunit.client.0.smithi019.stdout:8/84: symlink d9/d14/d17/l18 0 2024-02-22T15:50:26.253 INFO:tasks.workunit.client.0.smithi019.stdout:5/110: write d8/d11/f15 [3157749,108350] 0 2024-02-22T15:50:26.254 INFO:tasks.workunit.client.0.smithi019.stdout:4/73: unlink d2/c9 0 2024-02-22T15:50:26.255 INFO:tasks.workunit.client.0.smithi019.stdout:9/132: symlink d2/d7/l2b 0 2024-02-22T15:50:26.257 INFO:tasks.workunit.client.0.smithi019.stdout:3/95: creat d9/d11/f1e x:0 0 0 2024-02-22T15:50:26.258 INFO:tasks.workunit.client.0.smithi019.stdout:1/106: creat d8/d18/f21 x:0 0 0 2024-02-22T15:50:26.258 INFO:tasks.workunit.client.0.smithi019.stdout:3/96: read d9/d11/fd [1953932,124503] 0 2024-02-22T15:50:26.260 INFO:tasks.workunit.client.0.smithi019.stdout:3/97: truncate f7 4968694 0 2024-02-22T15:50:26.260 INFO:tasks.workunit.client.0.smithi019.stdout:3/98: dread - d9/d11/f1e zero size 2024-02-22T15:50:26.260 INFO:tasks.workunit.client.0.smithi019.stdout:8/85: creat d9/d14/f19 x:0 0 0 2024-02-22T15:50:26.261 INFO:tasks.workunit.client.0.smithi019.stdout:4/74: rmdir d2/db 39 2024-02-22T15:50:26.264 INFO:tasks.workunit.client.0.smithi019.stdout:1/107: symlink d8/d15/l22 0 2024-02-22T15:50:26.264 INFO:tasks.workunit.client.0.smithi019.stdout:7/55: link l7 d9/lf 0 2024-02-22T15:50:26.266 INFO:tasks.workunit.client.0.smithi019.stdout:4/75: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:26.267 INFO:tasks.workunit.client.0.smithi019.stdout:8/86: symlink d9/l1a 0 2024-02-22T15:50:26.268 INFO:tasks.workunit.client.0.smithi019.stdout:9/133: getdents d2/d1f 0 2024-02-22T15:50:26.271 INFO:tasks.workunit.client.0.smithi019.stdout:9/134: write d2/d7/f9 [3490565,42141] 0 2024-02-22T15:50:26.272 INFO:tasks.workunit.client.0.smithi019.stdout:7/56: creat d9/f10 x:0 0 0 2024-02-22T15:50:26.274 INFO:tasks.workunit.client.0.smithi019.stdout:5/111: write f1 [6394017,76624] 0 2024-02-22T15:50:26.274 INFO:tasks.workunit.client.0.smithi019.stdout:8/87: symlink d9/d14/d17/l1b 0 2024-02-22T15:50:26.275 INFO:tasks.workunit.client.0.smithi019.stdout:1/108: mkdir d8/d23 0 2024-02-22T15:50:26.276 INFO:tasks.workunit.client.0.smithi019.stdout:7/57: write f8 [1204843,59242] 0 2024-02-22T15:50:26.276 INFO:tasks.workunit.client.0.smithi019.stdout:5/112: stat d8/fc 0 2024-02-22T15:50:26.278 INFO:tasks.workunit.client.0.smithi019.stdout:8/88: stat d9/fa 0 2024-02-22T15:50:26.278 INFO:tasks.workunit.client.0.smithi019.stdout:4/76: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:26.278 INFO:tasks.workunit.client.0.smithi019.stdout:3/99: getdents d9/d14/d19 0 2024-02-22T15:50:26.280 INFO:tasks.workunit.client.0.smithi019.stdout:7/58: fsync d9/fc 0 2024-02-22T15:50:26.280 INFO:tasks.workunit.client.0.smithi019.stdout:1/109: chown c2 428021665 1 2024-02-22T15:50:26.281 INFO:tasks.workunit.client.0.smithi019.stdout:7/59: chown d9 110367996 1 2024-02-22T15:50:26.285 INFO:tasks.workunit.client.0.smithi019.stdout:9/135: dread d2/d7/f9 [0,4194304] 0 2024-02-22T15:50:26.286 INFO:tasks.workunit.client.0.smithi019.stdout:8/89: mkdir d9/d14/d17/d1c 0 2024-02-22T15:50:26.286 INFO:tasks.workunit.client.0.smithi019.stdout:3/100: write f7 [110392,27638] 0 2024-02-22T15:50:26.288 INFO:tasks.workunit.client.0.smithi019.stdout:7/60: creat d9/f11 x:0 0 0 2024-02-22T15:50:26.290 INFO:tasks.workunit.client.0.smithi019.stdout:5/113: dread f2 [0,4194304] 0 2024-02-22T15:50:26.290 INFO:tasks.workunit.client.0.smithi019.stdout:4/77: creat d2/db/f17 x:0 0 0 2024-02-22T15:50:26.293 INFO:tasks.workunit.client.0.smithi019.stdout:9/136: mkdir d2/d7/d14/d2c 0 2024-02-22T15:50:26.296 INFO:tasks.workunit.client.0.smithi019.stdout:5/114: write d8/d11/d21/f22 [476336,39130] 0 2024-02-22T15:50:26.297 INFO:tasks.workunit.client.0.smithi019.stdout:7/61: dread - d9/f10 zero size 2024-02-22T15:50:26.300 INFO:tasks.workunit.client.0.smithi019.stdout:1/110: dwrite d8/d18/f21 [0,4194304] 0 2024-02-22T15:50:26.304 INFO:tasks.workunit.client.0.smithi019.stdout:7/62: write d9/fa [410572,58052] 0 2024-02-22T15:50:26.305 INFO:tasks.workunit.client.0.smithi019.stdout:4/78: rename d2/db/f11 to d2/db/d14/f18 0 2024-02-22T15:50:26.306 INFO:tasks.workunit.client.0.smithi019.stdout:5/115: truncate d8/d11/d21/f22 988498 0 2024-02-22T15:50:26.307 INFO:tasks.workunit.client.0.smithi019.stdout:9/137: symlink d2/d7/dc/d27/l2d 0 2024-02-22T15:50:26.308 INFO:tasks.workunit.client.0.smithi019.stdout:5/116: fdatasync d8/ff 0 2024-02-22T15:50:26.309 INFO:tasks.workunit.client.0.smithi019.stdout:8/90: dwrite f5 [0,4194304] 0 2024-02-22T15:50:26.310 INFO:tasks.workunit.client.0.smithi019.stdout:4/79: creat d2/db/d14/f19 x:0 0 0 2024-02-22T15:50:26.310 INFO:tasks.workunit.client.0.smithi019.stdout:3/101: dread d9/fa [0,4194304] 0 2024-02-22T15:50:26.311 INFO:tasks.workunit.client.0.smithi019.stdout:1/111: rename d8/d18/f1c to d8/d15/f24 0 2024-02-22T15:50:26.313 INFO:tasks.workunit.client.0.smithi019.stdout:5/117: truncate f2 5068204 0 2024-02-22T15:50:26.314 INFO:tasks.workunit.client.0.smithi019.stdout:1/112: dread d8/f14 [0,4194304] 0 2024-02-22T15:50:26.315 INFO:tasks.workunit.client.0.smithi019.stdout:5/118: dread d8/ff [0,4194304] 0 2024-02-22T15:50:26.315 INFO:tasks.workunit.client.0.smithi019.stdout:9/138: mkdir d2/d1f/d2e 0 2024-02-22T15:50:26.315 INFO:tasks.workunit.client.0.smithi019.stdout:4/80: mknod d2/c1a 0 2024-02-22T15:50:26.316 INFO:tasks.workunit.client.0.smithi019.stdout:5/119: chown d8/dd/l1a 959 1 2024-02-22T15:50:26.317 INFO:tasks.workunit.client.0.smithi019.stdout:3/102: rename f7 to d9/d11/f1f 0 2024-02-22T15:50:26.317 INFO:tasks.workunit.client.0.smithi019.stdout:7/63: link d9/ce d9/c12 0 2024-02-22T15:50:26.318 INFO:tasks.workunit.client.0.smithi019.stdout:8/91: getdents d9/d14/d17/d1c 0 2024-02-22T15:50:26.321 INFO:tasks.workunit.client.0.smithi019.stdout:1/113: rename d8/d15/cf to d8/d23/c25 0 2024-02-22T15:50:26.325 INFO:tasks.workunit.client.0.smithi019.stdout:7/64: dread f8 [0,4194304] 0 2024-02-22T15:50:26.325 INFO:tasks.workunit.client.0.smithi019.stdout:1/114: creat d8/d15/f26 x:0 0 0 2024-02-22T15:50:26.329 INFO:tasks.workunit.client.0.smithi019.stdout:7/65: write d9/fa [3861850,88541] 0 2024-02-22T15:50:26.329 INFO:tasks.workunit.client.0.smithi019.stdout:4/81: dwrite d2/db/f17 [0,4194304] 0 2024-02-22T15:50:26.329 INFO:tasks.workunit.client.0.smithi019.stdout:3/103: dread d9/d11/f17 [0,4194304] 0 2024-02-22T15:50:26.332 INFO:tasks.workunit.client.0.smithi019.stdout:7/66: stat l7 0 2024-02-22T15:50:26.333 INFO:tasks.workunit.client.0.smithi019.stdout:7/67: read - d9/f11 zero size 2024-02-22T15:50:26.335 INFO:tasks.workunit.client.0.smithi019.stdout:7/68: stat d9/cb 0 2024-02-22T15:50:26.338 INFO:tasks.workunit.client.0.smithi019.stdout:9/139: write d2/d7/dc/f24 [767692,611] 0 2024-02-22T15:50:26.339 INFO:tasks.workunit.client.0.smithi019.stdout:4/82: dread d2/fa [0,4194304] 0 2024-02-22T15:50:26.342 INFO:tasks.workunit.client.0.smithi019.stdout:3/104: dwrite d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:26.343 INFO:tasks.workunit.client.0.smithi019.stdout:3/105: dread - d9/d11/f1e zero size 2024-02-22T15:50:26.345 INFO:tasks.workunit.client.0.smithi019.stdout:8/92: write f6 [1699382,34100] 0 2024-02-22T15:50:26.345 INFO:tasks.workunit.client.0.smithi019.stdout:1/115: write d8/f14 [1345500,113894] 0 2024-02-22T15:50:26.346 INFO:tasks.workunit.client.0.smithi019.stdout:4/83: chown d2/c3 482088 1 2024-02-22T15:50:26.349 INFO:tasks.workunit.client.0.smithi019.stdout:7/69: getdents d9 0 2024-02-22T15:50:26.349 INFO:tasks.workunit.client.0.smithi019.stdout:1/116: creat d8/d18/f27 x:0 0 0 2024-02-22T15:50:26.349 INFO:tasks.workunit.client.0.smithi019.stdout:3/106: write f3 [9455325,106878] 0 2024-02-22T15:50:26.352 INFO:tasks.workunit.client.0.smithi019.stdout:4/84: mkdir d2/db/d14/d1b 0 2024-02-22T15:50:26.353 INFO:tasks.workunit.client.0.smithi019.stdout:9/140: creat d2/d7/f2f x:0 0 0 2024-02-22T15:50:26.355 INFO:tasks.workunit.client.0.smithi019.stdout:1/117: symlink d8/d18/l28 0 2024-02-22T15:50:26.357 INFO:tasks.workunit.client.0.smithi019.stdout:9/141: write d2/d7/dc/f11 [4218680,14070] 0 2024-02-22T15:50:26.357 INFO:tasks.workunit.client.0.smithi019.stdout:3/107: creat d9/d14/d19/d1a/f20 x:0 0 0 2024-02-22T15:50:26.358 INFO:tasks.workunit.client.0.smithi019.stdout:1/118: mknod d8/d23/c29 0 2024-02-22T15:50:26.362 INFO:tasks.workunit.client.0.smithi019.stdout:7/70: link d9/c12 d9/c13 0 2024-02-22T15:50:26.362 INFO:tasks.workunit.client.0.smithi019.stdout:7/71: stat c5 0 2024-02-22T15:50:26.364 INFO:tasks.workunit.client.0.smithi019.stdout:9/142: creat d2/f30 x:0 0 0 2024-02-22T15:50:26.367 INFO:tasks.workunit.client.0.smithi019.stdout:7/72: symlink d9/l14 0 2024-02-22T15:50:26.368 INFO:tasks.workunit.client.0.smithi019.stdout:8/93: dwrite d9/f10 [0,4194304] 0 2024-02-22T15:50:26.371 INFO:tasks.workunit.client.0.smithi019.stdout:9/143: mkdir d2/d7/dc/d27/d31 0 2024-02-22T15:50:26.372 INFO:tasks.workunit.client.0.smithi019.stdout:1/119: dwrite d8/d18/f21 [0,4194304] 0 2024-02-22T15:50:26.372 INFO:tasks.workunit.client.0.smithi019.stdout:8/94: write d9/f12 [389027,93316] 0 2024-02-22T15:50:26.375 INFO:tasks.workunit.client.0.smithi019.stdout:1/120: write d8/f14 [783787,59786] 0 2024-02-22T15:50:26.378 INFO:tasks.workunit.client.0.smithi019.stdout:3/108: dwrite d9/d16/f1d [0,4194304] 0 2024-02-22T15:50:26.380 INFO:tasks.workunit.client.0.smithi019.stdout:7/73: dread f6 [0,4194304] 0 2024-02-22T15:50:26.384 INFO:tasks.workunit.client.0.smithi019.stdout:9/144: dwrite d2/f4 [0,4194304] 0 2024-02-22T15:50:26.385 INFO:tasks.workunit.client.0.smithi019.stdout:7/74: creat d9/f15 x:0 0 0 2024-02-22T15:50:26.387 INFO:tasks.workunit.client.0.smithi019.stdout:9/145: write d2/d7/f1b [3052644,104493] 0 2024-02-22T15:50:26.387 INFO:tasks.workunit.client.0.smithi019.stdout:7/75: truncate d9/fc 832049 0 2024-02-22T15:50:26.389 INFO:tasks.workunit.client.0.smithi019.stdout:3/109: dread d9/d11/f1f [0,4194304] 0 2024-02-22T15:50:26.391 INFO:tasks.workunit.client.0.smithi019.stdout:3/110: stat d9/d14/d19/d1a 0 2024-02-22T15:50:26.394 INFO:tasks.workunit.client.0.smithi019.stdout:8/95: read d9/f12 [265722,83565] 0 2024-02-22T15:50:26.403 INFO:tasks.workunit.client.0.smithi019.stdout:9/146: dwrite d2/d7/fe [0,4194304] 0 2024-02-22T15:50:26.404 INFO:tasks.workunit.client.0.smithi019.stdout:7/76: dwrite d9/f10 [0,4194304] 0 2024-02-22T15:50:26.406 INFO:tasks.workunit.client.0.smithi019.stdout:7/77: dread - d9/f15 zero size 2024-02-22T15:50:26.408 INFO:tasks.workunit.client.0.smithi019.stdout:3/111: dwrite d9/fb [0,4194304] 0 2024-02-22T15:50:26.408 INFO:tasks.workunit.client.0.smithi019.stdout:9/147: mknod d2/d7/d14/c32 0 2024-02-22T15:50:26.409 INFO:tasks.workunit.client.0.smithi019.stdout:3/112: fsync d9/d14/f1b 0 2024-02-22T15:50:26.410 INFO:tasks.workunit.client.0.smithi019.stdout:9/148: stat d2/d7/d14/c32 0 2024-02-22T15:50:26.411 INFO:tasks.workunit.client.0.smithi019.stdout:7/78: rename d9/f10 to d9/f16 0 2024-02-22T15:50:26.418 INFO:tasks.workunit.client.0.smithi019.stdout:3/113: mknod d9/c21 0 2024-02-22T15:50:26.419 INFO:tasks.workunit.client.0.smithi019.stdout:9/149: dwrite d2/f4 [0,4194304] 0 2024-02-22T15:50:26.421 INFO:tasks.workunit.client.0.smithi019.stdout:3/114: chown d9/d14 3833 1 2024-02-22T15:50:26.423 INFO:tasks.workunit.client.0.smithi019.stdout:9/150: symlink d2/d7/d14/d26/l33 0 2024-02-22T15:50:26.426 INFO:tasks.workunit.client.0.smithi019.stdout:3/115: mkdir d9/d22 0 2024-02-22T15:50:26.428 INFO:tasks.workunit.client.0.smithi019.stdout:1/121: dwrite d8/d15/f1b [0,4194304] 0 2024-02-22T15:50:26.429 INFO:tasks.workunit.client.0.smithi019.stdout:9/151: rename d2/d7/c1a to d2/d7/d14/d26/c34 0 2024-02-22T15:50:26.430 INFO:tasks.workunit.client.0.smithi019.stdout:3/116: symlink d9/d14/d19/l23 0 2024-02-22T15:50:26.432 INFO:tasks.workunit.client.0.smithi019.stdout:9/152: symlink d2/l35 0 2024-02-22T15:50:26.436 INFO:tasks.workunit.client.0.smithi019.stdout:9/153: unlink d2/d7/d14/d26/l28 0 2024-02-22T15:50:26.437 INFO:tasks.workunit.client.0.smithi019.stdout:1/122: dwrite d8/d15/f12 [0,4194304] 0 2024-02-22T15:50:26.439 INFO:tasks.workunit.client.0.smithi019.stdout:9/154: truncate d2/d7/dc/f29 641298 0 2024-02-22T15:50:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:1/123: dread d8/f14 [0,4194304] 0 2024-02-22T15:50:26.442 INFO:tasks.workunit.client.0.smithi019.stdout:1/124: chown l7 129 1 2024-02-22T15:50:26.444 INFO:tasks.workunit.client.0.smithi019.stdout:9/155: mkdir d2/d7/d14/d26/d36 0 2024-02-22T15:50:26.446 INFO:tasks.workunit.client.0.smithi019.stdout:1/125: mkdir d8/d2a 0 2024-02-22T15:50:26.447 INFO:tasks.workunit.client.0.smithi019.stdout:9/156: mkdir d2/d7/dc/d27/d37 0 2024-02-22T15:50:26.447 INFO:tasks.workunit.client.0.smithi019.stdout:8/96: write d9/ff [1123171,121260] 0 2024-02-22T15:50:26.449 INFO:tasks.workunit.client.0.smithi019.stdout:9/157: read d2/f19 [3331551,1983] 0 2024-02-22T15:50:26.452 INFO:tasks.workunit.client.0.smithi019.stdout:7/79: truncate f8 337513 0 2024-02-22T15:50:26.452 INFO:tasks.workunit.client.0.smithi019.stdout:1/126: dread d8/fb [0,4194304] 0 2024-02-22T15:50:26.453 INFO:tasks.workunit.client.0.smithi019.stdout:7/80: dread - d9/f15 zero size 2024-02-22T15:50:26.454 INFO:tasks.workunit.client.0.smithi019.stdout:7/81: stat d9/cb 0 2024-02-22T15:50:26.454 INFO:tasks.workunit.client.0.smithi019.stdout:1/127: chown d8/d15/f12 385858 1 2024-02-22T15:50:26.455 INFO:tasks.workunit.client.0.smithi019.stdout:1/128: truncate d8/f1a 1444091 0 2024-02-22T15:50:26.455 INFO:tasks.workunit.client.0.smithi019.stdout:9/158: symlink d2/d7/dc/d27/d37/l38 0 2024-02-22T15:50:26.458 INFO:tasks.workunit.client.0.smithi019.stdout:8/97: getdents d9 0 2024-02-22T15:50:26.459 INFO:tasks.workunit.client.0.smithi019.stdout:8/98: chown d9/f10 40 1 2024-02-22T15:50:26.460 INFO:tasks.workunit.client.0.smithi019.stdout:1/129: creat d8/d2a/f2b x:0 0 0 2024-02-22T15:50:26.461 INFO:tasks.workunit.client.0.smithi019.stdout:8/99: dread - d9/d14/f19 zero size 2024-02-22T15:50:26.463 INFO:tasks.workunit.client.0.smithi019.stdout:7/82: getdents d9 0 2024-02-22T15:50:26.465 INFO:tasks.workunit.client.0.smithi019.stdout:1/130: creat d8/d2a/f2c x:0 0 0 2024-02-22T15:50:26.465 INFO:tasks.workunit.client.0.smithi019.stdout:9/159: dwrite d2/d7/f1b [4194304,4194304] 0 2024-02-22T15:50:26.466 INFO:tasks.workunit.client.0.smithi019.stdout:8/100: symlink d9/l1d 0 2024-02-22T15:50:26.467 INFO:tasks.workunit.client.0.smithi019.stdout:7/83: creat d9/f17 x:0 0 0 2024-02-22T15:50:26.469 INFO:tasks.workunit.client.0.smithi019.stdout:1/131: creat d8/d18/f2d x:0 0 0 2024-02-22T15:50:26.475 INFO:tasks.workunit.client.0.smithi019.stdout:1/132: dread d8/fd [0,4194304] 0 2024-02-22T15:50:26.477 INFO:tasks.workunit.client.0.smithi019.stdout:8/101: creat d9/d14/d17/d1c/f1e x:0 0 0 2024-02-22T15:50:26.479 INFO:tasks.workunit.client.0.smithi019.stdout:1/133: symlink d8/d15/l2e 0 2024-02-22T15:50:26.480 INFO:tasks.workunit.client.0.smithi019.stdout:8/102: mkdir d9/d14/d1f 0 2024-02-22T15:50:26.481 INFO:tasks.workunit.client.0.smithi019.stdout:8/103: chown d9/d14/d17 1 1 2024-02-22T15:50:26.482 INFO:tasks.workunit.client.0.smithi019.stdout:1/134: truncate d8/fc 3122879 0 2024-02-22T15:50:26.482 INFO:tasks.workunit.client.0.smithi019.stdout:1/135: chown d8/d18/f27 176 1 2024-02-22T15:50:26.483 INFO:tasks.workunit.client.0.smithi019.stdout:1/136: dread - d8/d18/f2d zero size 2024-02-22T15:50:26.486 INFO:tasks.workunit.client.0.smithi019.stdout:1/137: write d8/d15/f26 [432517,121625] 0 2024-02-22T15:50:26.490 INFO:tasks.workunit.client.0.smithi019.stdout:8/104: write d9/f12 [496779,112987] 0 2024-02-22T15:50:26.490 INFO:tasks.workunit.client.0.smithi019.stdout:1/138: symlink d8/d23/l2f 0 2024-02-22T15:50:26.493 INFO:tasks.workunit.client.0.smithi019.stdout:9/160: dwrite d2/f6 [4194304,4194304] 0 2024-02-22T15:50:26.494 INFO:tasks.workunit.client.0.smithi019.stdout:8/105: mkdir d9/d14/d1f/d20 0 2024-02-22T15:50:26.495 INFO:tasks.workunit.client.0.smithi019.stdout:7/84: dwrite f6 [0,4194304] 0 2024-02-22T15:50:26.499 INFO:tasks.workunit.client.0.smithi019.stdout:8/106: fsync d9/fa 0 2024-02-22T15:50:26.503 INFO:tasks.workunit.client.0.smithi019.stdout:8/107: chown d9/fc 357 1 2024-02-22T15:50:26.504 INFO:tasks.workunit.client.0.smithi019.stdout:7/85: rename d9/lf to d9/l18 0 2024-02-22T15:50:26.505 INFO:tasks.workunit.client.0.smithi019.stdout:8/108: write d9/d14/f19 [479800,83005] 0 2024-02-22T15:50:26.506 INFO:tasks.workunit.client.0.smithi019.stdout:1/139: dwrite d8/d2a/f2c [0,4194304] 0 2024-02-22T15:50:26.507 INFO:tasks.workunit.client.0.smithi019.stdout:8/109: chown d9/d14/d17/l1b 4 1 2024-02-22T15:50:26.508 INFO:tasks.workunit.client.0.smithi019.stdout:7/86: creat d9/f19 x:0 0 0 2024-02-22T15:50:26.508 INFO:tasks.workunit.client.0.smithi019.stdout:9/161: dread d2/d7/f12 [4194304,4194304] 0 2024-02-22T15:50:26.508 INFO:tasks.workunit.client.0.smithi019.stdout:8/110: readlink d9/l1d 0 2024-02-22T15:50:26.509 INFO:tasks.workunit.client.0.smithi019.stdout:6/45: sync 2024-02-22T15:50:26.514 INFO:tasks.workunit.client.0.smithi019.stdout:9/162: dread d2/f19 [0,4194304] 0 2024-02-22T15:50:26.516 INFO:tasks.workunit.client.0.smithi019.stdout:6/46: unlink d6/l7 0 2024-02-22T15:50:26.518 INFO:tasks.workunit.client.0.smithi019.stdout:9/163: mknod d2/d7/dc/d27/d37/c39 0 2024-02-22T15:50:26.525 INFO:tasks.workunit.client.0.smithi019.stdout:7/87: getdents d9 0 2024-02-22T15:50:26.525 INFO:tasks.workunit.client.0.smithi019.stdout:6/47: dwrite d6/f10 [0,4194304] 0 2024-02-22T15:50:26.526 INFO:tasks.workunit.client.0.smithi019.stdout:8/111: truncate f6 389528 0 2024-02-22T15:50:26.528 INFO:tasks.workunit.client.0.smithi019.stdout:7/88: creat d9/f1a x:0 0 0 2024-02-22T15:50:26.533 INFO:tasks.workunit.client.0.smithi019.stdout:9/164: write d2/fb [5822476,55390] 0 2024-02-22T15:50:26.533 INFO:tasks.workunit.client.0.smithi019.stdout:7/89: creat d9/f1b x:0 0 0 2024-02-22T15:50:26.535 INFO:tasks.workunit.client.0.smithi019.stdout:9/165: symlink d2/d7/d14/l3a 0 2024-02-22T15:50:26.538 INFO:tasks.workunit.client.0.smithi019.stdout:7/90: rename d9/l14 to d9/l1c 0 2024-02-22T15:50:26.539 INFO:tasks.workunit.client.0.smithi019.stdout:8/112: dwrite d9/f12 [0,4194304] 0 2024-02-22T15:50:26.541 INFO:tasks.workunit.client.0.smithi019.stdout:7/91: dread - d9/f1b zero size 2024-02-22T15:50:26.542 INFO:tasks.workunit.client.0.smithi019.stdout:6/48: dwrite d6/fb [0,4194304] 0 2024-02-22T15:50:26.544 INFO:tasks.workunit.client.0.smithi019.stdout:8/113: rmdir d9/d14 39 2024-02-22T15:50:26.548 INFO:tasks.workunit.client.0.smithi019.stdout:9/166: link d2/d7/f9 d2/d7/d14/d26/d36/f3b 0 2024-02-22T15:50:26.550 INFO:tasks.workunit.client.0.smithi019.stdout:6/49: symlink d6/l12 0 2024-02-22T15:50:26.551 INFO:tasks.workunit.client.0.smithi019.stdout:6/50: write d6/f10 [991199,17045] 0 2024-02-22T15:50:26.554 INFO:tasks.workunit.client.0.smithi019.stdout:8/114: unlink d9/l1a 0 2024-02-22T15:50:26.554 INFO:tasks.workunit.client.0.smithi019.stdout:9/167: dread d2/d7/f12 [4194304,4194304] 0 2024-02-22T15:50:26.555 INFO:tasks.workunit.client.0.smithi019.stdout:8/115: chown f1 4744 1 2024-02-22T15:50:26.556 INFO:tasks.workunit.client.0.smithi019.stdout:6/51: creat d6/f13 x:0 0 0 2024-02-22T15:50:26.558 INFO:tasks.workunit.client.0.smithi019.stdout:9/168: symlink d2/d7/d14/l3c 0 2024-02-22T15:50:26.561 INFO:tasks.workunit.client.0.smithi019.stdout:6/52: unlink d6/l11 0 2024-02-22T15:50:26.561 INFO:tasks.workunit.client.0.smithi019.stdout:6/53: chown d6 60552304 1 2024-02-22T15:50:26.562 INFO:tasks.workunit.client.0.smithi019.stdout:6/54: truncate d6/f13 701492 0 2024-02-22T15:50:26.562 INFO:tasks.workunit.client.0.smithi019.stdout:6/55: stat d6/l12 0 2024-02-22T15:50:26.565 INFO:tasks.workunit.client.0.smithi019.stdout:6/56: creat d6/f14 x:0 0 0 2024-02-22T15:50:26.566 INFO:tasks.workunit.client.0.smithi019.stdout:7/92: write f8 [1282054,90123] 0 2024-02-22T15:50:26.566 INFO:tasks.workunit.client.0.smithi019.stdout:9/169: symlink d2/d7/dc/d27/d31/l3d 0 2024-02-22T15:50:26.568 INFO:tasks.workunit.client.0.smithi019.stdout:9/170: readlink d2/d7/dc/d27/d37/l38 0 2024-02-22T15:50:26.569 INFO:tasks.workunit.client.0.smithi019.stdout:8/116: getdents d9/d14 0 2024-02-22T15:50:26.571 INFO:tasks.workunit.client.0.smithi019.stdout:7/93: rename d9/fc to d9/f1d 0 2024-02-22T15:50:26.572 INFO:tasks.workunit.client.0.smithi019.stdout:8/117: write d9/d14/f19 [1464372,61776] 0 2024-02-22T15:50:26.574 INFO:tasks.workunit.client.0.smithi019.stdout:7/94: mkdir d9/d1e 0 2024-02-22T15:50:26.576 INFO:tasks.workunit.client.0.smithi019.stdout:6/57: dwrite d6/f14 [0,4194304] 0 2024-02-22T15:50:26.577 INFO:tasks.workunit.client.0.smithi019.stdout:8/118: chown d9/ce 5873388 1 2024-02-22T15:50:26.577 INFO:tasks.workunit.client.0.smithi019.stdout:8/119: stat d9/ce 0 2024-02-22T15:50:26.582 INFO:tasks.workunit.client.0.smithi019.stdout:7/95: write d9/f1d [902550,129916] 0 2024-02-22T15:50:26.583 INFO:tasks.workunit.client.0.smithi019.stdout:8/120: creat d9/d14/d1f/f21 x:0 0 0 2024-02-22T15:50:26.585 INFO:tasks.workunit.client.0.smithi019.stdout:7/96: mknod d9/c1f 0 2024-02-22T15:50:26.586 INFO:tasks.workunit.client.0.smithi019.stdout:6/58: dwrite d6/fb [0,4194304] 0 2024-02-22T15:50:26.587 INFO:tasks.workunit.client.0.smithi019.stdout:7/97: write d9/f17 [107054,123402] 0 2024-02-22T15:50:26.590 INFO:tasks.workunit.client.0.smithi019.stdout:9/171: truncate d2/d7/f1b 6382262 0 2024-02-22T15:50:26.590 INFO:tasks.workunit.client.0.smithi019.stdout:9/172: chown d2 5 1 2024-02-22T15:50:26.592 INFO:tasks.workunit.client.0.smithi019.stdout:8/121: rename d9/lb to d9/l22 0 2024-02-22T15:50:26.594 INFO:tasks.workunit.client.0.smithi019.stdout:9/173: mknod d2/d7/d14/d2c/c3e 0 2024-02-22T15:50:26.595 INFO:tasks.workunit.client.0.smithi019.stdout:6/59: dread d6/fb [0,4194304] 0 2024-02-22T15:50:26.596 INFO:tasks.workunit.client.0.smithi019.stdout:7/98: rmdir d9/d1e 0 2024-02-22T15:50:26.597 INFO:tasks.workunit.client.0.smithi019.stdout:6/60: mknod d6/c15 0 2024-02-22T15:50:26.597 INFO:tasks.workunit.client.0.smithi019.stdout:7/99: stat d9/c13 0 2024-02-22T15:50:26.604 INFO:tasks.workunit.client.0.smithi019.stdout:6/61: dwrite d6/f9 [0,4194304] 0 2024-02-22T15:50:26.605 INFO:tasks.workunit.client.0.smithi019.stdout:6/62: write d6/f10 [2351686,46794] 0 2024-02-22T15:50:26.606 INFO:tasks.workunit.client.0.smithi019.stdout:6/63: read - d6/fe zero size 2024-02-22T15:50:26.609 INFO:tasks.workunit.client.0.smithi019.stdout:7/100: getdents d9 0 2024-02-22T15:50:26.610 INFO:tasks.workunit.client.0.smithi019.stdout:6/64: symlink d6/l16 0 2024-02-22T15:50:26.612 INFO:tasks.workunit.client.0.smithi019.stdout:8/122: rename d9/l22 to d9/d14/d1f/l23 0 2024-02-22T15:50:26.613 INFO:tasks.workunit.client.0.smithi019.stdout:8/123: readlink d9/d14/d17/l1b 0 2024-02-22T15:50:26.613 INFO:tasks.workunit.client.0.smithi019.stdout:7/101: symlink d9/l20 0 2024-02-22T15:50:26.614 INFO:tasks.workunit.client.0.smithi019.stdout:8/124: dread - d9/d14/d1f/f21 zero size 2024-02-22T15:50:26.614 INFO:tasks.workunit.client.0.smithi019.stdout:6/65: creat d6/f17 x:0 0 0 2024-02-22T15:50:26.616 INFO:tasks.workunit.client.0.smithi019.stdout:9/174: rename d2/d7/f2f to d2/d7/d14/f3f 0 2024-02-22T15:50:26.618 INFO:tasks.workunit.client.0.smithi019.stdout:6/66: dread d6/f13 [0,4194304] 0 2024-02-22T15:50:26.619 INFO:tasks.workunit.client.0.smithi019.stdout:7/102: rename d9/f1a to d9/f21 0 2024-02-22T15:50:26.620 INFO:tasks.workunit.client.0.smithi019.stdout:8/125: dread f5 [4194304,4194304] 0 2024-02-22T15:50:26.625 INFO:tasks.workunit.client.0.smithi019.stdout:9/175: link d2/d7/dc/l1d d2/d7/d14/d26/d36/l40 0 2024-02-22T15:50:26.632 INFO:tasks.workunit.client.0.smithi019.stdout:9/176: dwrite d2/d7/fe [0,4194304] 0 2024-02-22T15:50:26.633 INFO:tasks.workunit.client.0.smithi019.stdout:6/67: truncate d6/f10 3249406 0 2024-02-22T15:50:26.635 INFO:tasks.workunit.client.0.smithi019.stdout:9/177: fsync d2/f6 0 2024-02-22T15:50:26.636 INFO:tasks.workunit.client.0.smithi019.stdout:2/93: sync 2024-02-22T15:50:26.637 INFO:tasks.workunit.client.0.smithi019.stdout:6/68: creat d6/f18 x:0 0 0 2024-02-22T15:50:26.637 INFO:tasks.workunit.client.0.smithi019.stdout:9/178: truncate d2/f30 2274 0 2024-02-22T15:50:26.638 INFO:tasks.workunit.client.0.smithi019.stdout:8/126: dwrite f3 [0,4194304] 0 2024-02-22T15:50:26.639 INFO:tasks.workunit.client.0.smithi019.stdout:6/69: fsync d6/fc 0 2024-02-22T15:50:26.639 INFO:tasks.workunit.client.0.smithi019.stdout:2/94: read - d1/d17/d1c/f22 zero size 2024-02-22T15:50:26.642 INFO:tasks.workunit.client.0.smithi019.stdout:6/70: symlink d6/l19 0 2024-02-22T15:50:26.644 INFO:tasks.workunit.client.0.smithi019.stdout:8/127: creat d9/d14/d17/d1c/f24 x:0 0 0 2024-02-22T15:50:26.646 INFO:tasks.workunit.client.0.smithi019.stdout:2/95: rmdir d1/de/d15 39 2024-02-22T15:50:26.647 INFO:tasks.workunit.client.0.smithi019.stdout:9/179: getdents d2/d1f/d2e 0 2024-02-22T15:50:26.648 INFO:tasks.workunit.client.0.smithi019.stdout:2/96: truncate d1/d17/d1c/f22 440969 0 2024-02-22T15:50:26.649 INFO:tasks.workunit.client.0.smithi019.stdout:8/128: creat d9/d14/d17/f25 x:0 0 0 2024-02-22T15:50:26.650 INFO:tasks.workunit.client.0.smithi019.stdout:8/129: readlink d9/l11 0 2024-02-22T15:50:26.652 INFO:tasks.workunit.client.0.smithi019.stdout:8/130: dread - d9/d14/d17/d1c/f1e zero size 2024-02-22T15:50:26.652 INFO:tasks.workunit.client.0.smithi019.stdout:2/97: unlink d1/fa 0 2024-02-22T15:50:26.653 INFO:tasks.workunit.client.0.smithi019.stdout:9/180: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:26.655 INFO:tasks.workunit.client.0.smithi019.stdout:8/131: mknod d9/d14/d17/c26 0 2024-02-22T15:50:26.657 INFO:tasks.workunit.client.0.smithi019.stdout:6/71: getdents d6 0 2024-02-22T15:50:26.658 INFO:tasks.workunit.client.0.smithi019.stdout:2/98: symlink d1/d17/d19/l23 0 2024-02-22T15:50:26.659 INFO:tasks.workunit.client.0.smithi019.stdout:6/72: write d6/fe [83891,30932] 0 2024-02-22T15:50:26.664 INFO:tasks.workunit.client.0.smithi019.stdout:9/181: link d2/d7/d14/l3a d2/l41 0 2024-02-22T15:50:26.666 INFO:tasks.workunit.client.0.smithi019.stdout:9/182: stat d2/d7/f2a 0 2024-02-22T15:50:26.666 INFO:tasks.workunit.client.0.smithi019.stdout:8/132: dwrite d9/f10 [0,4194304] 0 2024-02-22T15:50:26.669 INFO:tasks.workunit.client.0.smithi019.stdout:2/99: dwrite d1/f12 [0,4194304] 0 2024-02-22T15:50:26.671 INFO:tasks.workunit.client.0.smithi019.stdout:8/133: symlink d9/d14/d1f/d20/l27 0 2024-02-22T15:50:26.673 INFO:tasks.workunit.client.0.smithi019.stdout:9/183: rename d2/d7/f9 to d2/d1f/d2e/f42 0 2024-02-22T15:50:26.674 INFO:tasks.workunit.client.0.smithi019.stdout:2/100: rename d1/d17/d21 to d1/d17/d21/d24 22 2024-02-22T15:50:26.676 INFO:tasks.workunit.client.0.smithi019.stdout:2/101: fsync d1/f18 0 2024-02-22T15:50:26.676 INFO:tasks.workunit.client.0.smithi019.stdout:6/73: fsync d6/f10 0 2024-02-22T15:50:26.677 INFO:tasks.workunit.client.0.smithi019.stdout:9/184: chown d2/d7/d14/d26/l33 4 1 2024-02-22T15:50:26.679 INFO:tasks.workunit.client.0.smithi019.stdout:6/74: chown d6/fe 12 1 2024-02-22T15:50:26.681 INFO:tasks.workunit.client.0.smithi019.stdout:6/75: chown d6/l16 2112239 1 2024-02-22T15:50:26.682 INFO:tasks.workunit.client.0.smithi019.stdout:9/185: creat d2/d7/d14/d26/d36/f43 x:0 0 0 2024-02-22T15:50:26.685 INFO:tasks.workunit.client.0.smithi019.stdout:9/186: dread - d2/d7/d14/f3f zero size 2024-02-22T15:50:26.686 INFO:tasks.workunit.client.0.smithi019.stdout:2/102: dwrite d1/f1f [0,4194304] 0 2024-02-22T15:50:26.687 INFO:tasks.workunit.client.0.smithi019.stdout:8/134: truncate f3 2899592 0 2024-02-22T15:50:26.689 INFO:tasks.workunit.client.0.smithi019.stdout:9/187: readlink d2/d7/dc/d27/l2d 0 2024-02-22T15:50:26.691 INFO:tasks.workunit.client.0.smithi019.stdout:6/76: dwrite d6/f9 [0,4194304] 0 2024-02-22T15:50:26.694 INFO:tasks.workunit.client.0.smithi019.stdout:2/103: symlink d1/de/d15/l25 0 2024-02-22T15:50:26.698 INFO:tasks.workunit.client.0.smithi019.stdout:9/188: rename d2/d7/fe to d2/d7/f44 0 2024-02-22T15:50:26.700 INFO:tasks.workunit.client.0.smithi019.stdout:2/104: creat d1/f26 x:0 0 0 2024-02-22T15:50:26.703 INFO:tasks.workunit.client.0.smithi019.stdout:6/77: dwrite d6/f9 [0,4194304] 0 2024-02-22T15:50:26.706 INFO:tasks.workunit.client.0.smithi019.stdout:6/78: mknod d6/c1a 0 2024-02-22T15:50:26.707 INFO:tasks.workunit.client.0.smithi019.stdout:9/189: dread d2/d7/dc/f24 [0,4194304] 0 2024-02-22T15:50:26.708 INFO:tasks.workunit.client.0.smithi019.stdout:6/79: rmdir d6 39 2024-02-22T15:50:26.712 INFO:tasks.workunit.client.0.smithi019.stdout:9/190: getdents d2/d7/d14/d26 0 2024-02-22T15:50:26.713 INFO:tasks.workunit.client.0.smithi019.stdout:6/80: dread d6/f10 [0,4194304] 0 2024-02-22T15:50:26.714 INFO:tasks.workunit.client.0.smithi019.stdout:6/81: stat d6/ld 0 2024-02-22T15:50:26.714 INFO:tasks.workunit.client.0.smithi019.stdout:6/82: fdatasync d6/f14 0 2024-02-22T15:50:26.715 INFO:tasks.workunit.client.0.smithi019.stdout:6/83: chown d6/c1a 0 1 2024-02-22T15:50:26.717 INFO:tasks.workunit.client.0.smithi019.stdout:8/135: write f6 [1146442,38489] 0 2024-02-22T15:50:26.719 INFO:tasks.workunit.client.0.smithi019.stdout:9/191: getdents d2/d7/d14/d26 0 2024-02-22T15:50:26.721 INFO:tasks.workunit.client.0.smithi019.stdout:8/136: dread - d9/d14/d17/d1c/f24 zero size 2024-02-22T15:50:26.721 INFO:tasks.workunit.client.0.smithi019.stdout:9/192: chown d2/fb 1 1 2024-02-22T15:50:26.722 INFO:tasks.workunit.client.0.smithi019.stdout:2/105: truncate d1/de/f1e 579211 0 2024-02-22T15:50:26.723 INFO:tasks.workunit.client.0.smithi019.stdout:2/106: write d1/f12 [1920692,87814] 0 2024-02-22T15:50:26.725 INFO:tasks.workunit.client.0.smithi019.stdout:8/137: creat d9/d14/d17/f28 x:0 0 0 2024-02-22T15:50:26.726 INFO:tasks.workunit.client.0.smithi019.stdout:2/107: unlink d1/f10 0 2024-02-22T15:50:26.732 INFO:tasks.workunit.client.0.smithi019.stdout:2/108: symlink d1/l27 0 2024-02-22T15:50:26.732 INFO:tasks.workunit.client.0.smithi019.stdout:8/138: creat d9/f29 x:0 0 0 2024-02-22T15:50:26.732 INFO:tasks.workunit.client.0.smithi019.stdout:9/193: dwrite d2/f30 [0,4194304] 0 2024-02-22T15:50:26.738 INFO:tasks.workunit.client.0.smithi019.stdout:6/84: dwrite d6/f13 [0,4194304] 0 2024-02-22T15:50:26.739 INFO:tasks.workunit.client.0.smithi019.stdout:2/109: write d1/f14 [3994507,108514] 0 2024-02-22T15:50:26.740 INFO:tasks.workunit.client.0.smithi019.stdout:2/110: chown d1/f12 121 1 2024-02-22T15:50:26.740 INFO:tasks.workunit.client.0.smithi019.stdout:6/85: write d6/f18 [817098,1523] 0 2024-02-22T15:50:26.741 INFO:tasks.workunit.client.0.smithi019.stdout:9/194: mkdir d2/d7/d14/d26/d45 0 2024-02-22T15:50:26.744 INFO:tasks.workunit.client.0.smithi019.stdout:2/111: chown d1/lc 797021 1 2024-02-22T15:50:26.744 INFO:tasks.workunit.client.0.smithi019.stdout:9/195: read d2/d7/d14/d26/d36/f3b [3530786,78752] 0 2024-02-22T15:50:26.750 INFO:tasks.workunit.client.0.smithi019.stdout:9/196: link d2/d7/l2b d2/d7/d14/d26/l46 0 2024-02-22T15:50:26.752 INFO:tasks.workunit.client.0.smithi019.stdout:9/197: creat d2/d1f/f47 x:0 0 0 2024-02-22T15:50:26.752 INFO:tasks.workunit.client.0.smithi019.stdout:9/198: chown d2/f6 1769 1 2024-02-22T15:50:26.754 INFO:tasks.workunit.client.0.smithi019.stdout:9/199: fdatasync d2/d7/d14/f3f 0 2024-02-22T15:50:26.758 INFO:tasks.workunit.client.0.smithi019.stdout:2/112: truncate d1/f14 1640718 0 2024-02-22T15:50:26.760 INFO:tasks.workunit.client.0.smithi019.stdout:9/200: rename d2/d7/d14/d26 to d2/d1f/d2e/d48 0 2024-02-22T15:50:26.763 INFO:tasks.workunit.client.0.smithi019.stdout:2/113: link d1/lb d1/de/d15/l28 0 2024-02-22T15:50:26.768 INFO:tasks.workunit.client.0.smithi019.stdout:2/114: link d1/ld d1/d17/d21/l29 0 2024-02-22T15:50:26.768 INFO:tasks.workunit.client.0.smithi019.stdout:2/115: chown d1/c8 378 1 2024-02-22T15:50:26.771 INFO:tasks.workunit.client.0.smithi019.stdout:2/116: mknod d1/c2a 0 2024-02-22T15:50:26.778 INFO:tasks.workunit.client.0.smithi019.stdout:2/117: write d1/de/f1e [1275719,55319] 0 2024-02-22T15:50:26.783 INFO:tasks.workunit.client.0.smithi019.stdout:2/118: dread d1/f1f [0,4194304] 0 2024-02-22T15:50:26.783 INFO:tasks.workunit.client.0.smithi019.stdout:2/119: chown d1 0 1 2024-02-22T15:50:26.783 INFO:tasks.workunit.client.0.smithi019.stdout:2/120: read - d1/f26 zero size 2024-02-22T15:50:26.788 INFO:tasks.workunit.client.0.smithi019.stdout:2/121: write d1/de/f16 [1475324,15649] 0 2024-02-22T15:50:26.793 INFO:tasks.workunit.client.0.smithi019.stdout:2/122: link d1/d17/d21/l29 d1/d17/d19/l2b 0 2024-02-22T15:50:26.799 INFO:tasks.workunit.client.0.smithi019.stdout:2/123: dwrite d1/f14 [0,4194304] 0 2024-02-22T15:50:26.804 INFO:tasks.workunit.client.0.smithi019.stdout:2/124: dread d1/f12 [0,4194304] 0 2024-02-22T15:50:26.809 INFO:tasks.workunit.client.0.smithi019.stdout:2/125: truncate d1/de/f16 1677512 0 2024-02-22T15:50:26.810 INFO:tasks.workunit.client.0.smithi019.stdout:2/126: read d1/de/ff [506273,80319] 0 2024-02-22T15:50:26.814 INFO:tasks.workunit.client.0.smithi019.stdout:2/127: rmdir d1 39 2024-02-22T15:50:26.818 INFO:tasks.workunit.client.0.smithi019.stdout:2/128: dread d1/f18 [0,4194304] 0 2024-02-22T15:50:26.819 INFO:tasks.workunit.client.0.smithi019.stdout:2/129: rmdir d1/de 39 2024-02-22T15:50:26.822 INFO:tasks.workunit.client.0.smithi019.stdout:2/130: write d1/de/ff [885811,18272] 0 2024-02-22T15:50:26.868 INFO:tasks.workunit.client.0.smithi019.stdout:5/120: sync 2024-02-22T15:50:26.868 INFO:tasks.workunit.client.0.smithi019.stdout:0/93: sync 2024-02-22T15:50:26.869 INFO:tasks.workunit.client.0.smithi019.stdout:5/121: write f2 [1919080,32981] 0 2024-02-22T15:50:26.870 INFO:tasks.workunit.client.0.smithi019.stdout:0/94: symlink d4/l20 0 2024-02-22T15:50:26.872 INFO:tasks.workunit.client.0.smithi019.stdout:5/122: creat d8/f23 x:0 0 0 2024-02-22T15:50:26.874 INFO:tasks.workunit.client.0.smithi019.stdout:5/123: truncate d8/fc 815851 0 2024-02-22T15:50:26.875 INFO:tasks.workunit.client.0.smithi019.stdout:0/95: dread f2 [0,4194304] 0 2024-02-22T15:50:26.875 INFO:tasks.workunit.client.0.smithi019.stdout:5/124: chown d8/d11/f15 3 1 2024-02-22T15:50:26.876 INFO:tasks.workunit.client.0.smithi019.stdout:5/125: stat d8/d11/d21 0 2024-02-22T15:50:26.876 INFO:tasks.workunit.client.0.smithi019.stdout:5/126: fsync d8/d11/f17 0 2024-02-22T15:50:26.877 INFO:tasks.workunit.client.0.smithi019.stdout:5/127: write d8/ff [313589,37721] 0 2024-02-22T15:50:26.878 INFO:tasks.workunit.client.0.smithi019.stdout:5/128: rename d8 to d8/d11/d21/d24 22 2024-02-22T15:50:26.882 INFO:tasks.workunit.client.0.smithi019.stdout:5/129: dread d8/ff [0,4194304] 0 2024-02-22T15:50:26.882 INFO:tasks.workunit.client.0.smithi019.stdout:0/96: dwrite d4/d9/f11 [0,4194304] 0 2024-02-22T15:50:26.884 INFO:tasks.workunit.client.0.smithi019.stdout:5/130: creat d8/f25 x:0 0 0 2024-02-22T15:50:26.885 INFO:tasks.workunit.client.0.smithi019.stdout:5/131: dread - d8/d11/f1c zero size 2024-02-22T15:50:26.886 INFO:tasks.workunit.client.0.smithi019.stdout:5/132: chown d8/d11 1968 1 2024-02-22T15:50:26.887 INFO:tasks.workunit.client.0.smithi019.stdout:0/97: dread d4/db/d1b/f1c [0,4194304] 0 2024-02-22T15:50:26.889 INFO:tasks.workunit.client.0.smithi019.stdout:5/133: dread f5 [4194304,4194304] 0 2024-02-22T15:50:26.892 INFO:tasks.workunit.client.0.smithi019.stdout:5/134: symlink d8/dd/d19/l26 0 2024-02-22T15:50:26.893 INFO:tasks.workunit.client.0.smithi019.stdout:5/135: dread - d8/d11/f17 zero size 2024-02-22T15:50:26.894 INFO:tasks.workunit.client.0.smithi019.stdout:0/98: dwrite f2 [0,4194304] 0 2024-02-22T15:50:26.896 INFO:tasks.workunit.client.0.smithi019.stdout:5/136: symlink d8/dd/d19/l27 0 2024-02-22T15:50:26.897 INFO:tasks.workunit.client.0.smithi019.stdout:0/99: creat d4/f21 x:0 0 0 2024-02-22T15:50:26.899 INFO:tasks.workunit.client.0.smithi019.stdout:5/137: rename d8/d11/f17 to d8/f28 0 2024-02-22T15:50:26.900 INFO:tasks.workunit.client.0.smithi019.stdout:5/138: read - d8/d11/f1c zero size 2024-02-22T15:50:26.902 INFO:tasks.workunit.client.0.smithi019.stdout:5/139: creat d8/dd/d19/f29 x:0 0 0 2024-02-22T15:50:26.903 INFO:tasks.workunit.client.0.smithi019.stdout:5/140: truncate d8/d11/f1c 857503 0 2024-02-22T15:50:26.915 INFO:tasks.workunit.client.0.smithi019.stdout:5/141: getdents d8/dd/d19 0 2024-02-22T15:50:26.917 INFO:tasks.workunit.client.0.smithi019.stdout:5/142: symlink d8/dd/d19/l2a 0 2024-02-22T15:50:26.922 INFO:tasks.workunit.client.0.smithi019.stdout:5/143: link d8/dd/d19/l2a d8/d11/d21/l2b 0 2024-02-22T15:50:26.922 INFO:tasks.workunit.client.0.smithi019.stdout:5/144: chown d8/d11/c1f 271593 1 2024-02-22T15:50:26.924 INFO:tasks.workunit.client.0.smithi019.stdout:5/145: mkdir d8/d11/d21/d2c 0 2024-02-22T15:50:26.926 INFO:tasks.workunit.client.0.smithi019.stdout:5/146: creat d8/f2d x:0 0 0 2024-02-22T15:50:26.927 INFO:tasks.workunit.client.0.smithi019.stdout:5/147: symlink d8/d11/l2e 0 2024-02-22T15:50:26.932 INFO:tasks.workunit.client.0.smithi019.stdout:5/148: dwrite d8/d11/f15 [0,4194304] 0 2024-02-22T15:50:27.015 INFO:tasks.workunit.client.0.smithi019.stdout:5/149: link f5 d8/d11/f2f 0 2024-02-22T15:50:27.021 INFO:tasks.workunit.client.0.smithi019.stdout:5/150: dwrite d8/f28 [0,4194304] 0 2024-02-22T15:50:27.030 INFO:tasks.workunit.client.0.smithi019.stdout:5/151: dwrite d8/f25 [0,4194304] 0 2024-02-22T15:50:27.033 INFO:tasks.workunit.client.0.smithi019.stdout:5/152: rmdir d8 39 2024-02-22T15:50:27.045 INFO:tasks.workunit.client.0.smithi019.stdout:5/153: write d8/fc [1554432,81543] 0 2024-02-22T15:50:27.048 INFO:tasks.workunit.client.0.smithi019.stdout:5/154: dread f5 [4194304,4194304] 0 2024-02-22T15:50:27.053 INFO:tasks.workunit.client.0.smithi019.stdout:5/155: dwrite f2 [0,4194304] 0 2024-02-22T15:50:27.055 INFO:tasks.workunit.client.0.smithi019.stdout:5/156: mkdir d8/d11/d21/d30 0 2024-02-22T15:50:27.061 INFO:tasks.workunit.client.0.smithi019.stdout:5/157: truncate d8/f25 3274391 0 2024-02-22T15:50:27.065 INFO:tasks.workunit.client.0.smithi019.stdout:5/158: dread d8/d11/f15 [0,4194304] 0 2024-02-22T15:50:27.065 INFO:tasks.workunit.client.0.smithi019.stdout:5/159: stat d8/f23 0 2024-02-22T15:50:27.066 INFO:tasks.workunit.client.0.smithi019.stdout:5/160: fdatasync d8/ff 0 2024-02-22T15:50:27.068 INFO:tasks.workunit.client.0.smithi019.stdout:5/161: creat d8/d11/d21/d2c/f31 x:0 0 0 2024-02-22T15:50:27.070 INFO:tasks.workunit.client.0.smithi019.stdout:4/85: sync 2024-02-22T15:50:27.070 INFO:tasks.workunit.client.0.smithi019.stdout:5/162: mknod d8/c32 0 2024-02-22T15:50:27.072 INFO:tasks.workunit.client.0.smithi019.stdout:5/163: mkdir d8/d11/d33 0 2024-02-22T15:50:27.075 INFO:tasks.workunit.client.0.smithi019.stdout:5/164: mknod d8/d11/d21/d2c/c34 0 2024-02-22T15:50:27.078 INFO:tasks.workunit.client.0.smithi019.stdout:4/86: dwrite d2/db/f17 [0,4194304] 0 2024-02-22T15:50:27.082 INFO:tasks.workunit.client.0.smithi019.stdout:5/165: dwrite d8/fe [0,4194304] 0 2024-02-22T15:50:27.084 INFO:tasks.workunit.client.0.smithi019.stdout:5/166: truncate d8/d11/d21/f22 1431549 0 2024-02-22T15:50:27.084 INFO:tasks.workunit.client.0.smithi019.stdout:5/167: dread - d8/f23 zero size 2024-02-22T15:50:27.086 INFO:tasks.workunit.client.0.smithi019.stdout:5/168: symlink d8/l35 0 2024-02-22T15:50:27.089 INFO:tasks.workunit.client.0.smithi019.stdout:5/169: rename f2 to d8/d11/d21/f36 0 2024-02-22T15:50:27.090 INFO:tasks.workunit.client.0.smithi019.stdout:5/170: dread - d8/f23 zero size 2024-02-22T15:50:27.094 INFO:tasks.workunit.client.0.smithi019.stdout:5/171: dread d8/d11/d21/f22 [0,4194304] 0 2024-02-22T15:50:27.097 INFO:tasks.workunit.client.0.smithi019.stdout:5/172: rename d8/d11/c13 to d8/d11/d21/d2c/c37 0 2024-02-22T15:50:27.097 INFO:tasks.workunit.client.0.smithi019.stdout:5/173: dread - d8/dd/d19/f29 zero size 2024-02-22T15:50:27.099 INFO:tasks.workunit.client.0.smithi019.stdout:5/174: mkdir d8/dd/d19/d38 0 2024-02-22T15:50:27.100 INFO:tasks.workunit.client.0.smithi019.stdout:5/175: dread d8/ff [0,4194304] 0 2024-02-22T15:50:27.101 INFO:tasks.workunit.client.0.smithi019.stdout:5/176: dread - d8/f2d zero size 2024-02-22T15:50:27.102 INFO:tasks.workunit.client.0.smithi019.stdout:5/177: mknod d8/d11/d21/d2c/c39 0 2024-02-22T15:50:27.103 INFO:tasks.workunit.client.0.smithi019.stdout:5/178: write d8/dd/d19/f29 [495033,88959] 0 2024-02-22T15:50:27.109 INFO:tasks.workunit.client.0.smithi019.stdout:5/179: getdents d8 0 2024-02-22T15:50:27.111 INFO:tasks.workunit.client.0.smithi019.stdout:5/180: mkdir d8/dd/d19/d3a 0 2024-02-22T15:50:27.114 INFO:tasks.workunit.client.0.smithi019.stdout:5/181: dread d8/dd/d19/f29 [0,4194304] 0 2024-02-22T15:50:27.115 INFO:tasks.workunit.client.0.smithi019.stdout:5/182: chown d8/dd/d19/l2a 52188905 1 2024-02-22T15:50:27.118 INFO:tasks.workunit.client.0.smithi019.stdout:5/183: dread d8/d11/f15 [0,4194304] 0 2024-02-22T15:50:27.119 INFO:tasks.workunit.client.0.smithi019.stdout:5/184: stat d8/dd/d19/l27 0 2024-02-22T15:50:27.121 INFO:tasks.workunit.client.0.smithi019.stdout:5/185: creat d8/d11/f3b x:0 0 0 2024-02-22T15:50:27.121 INFO:tasks.workunit.client.0.smithi019.stdout:5/186: read - d8/d11/f3b zero size 2024-02-22T15:50:27.122 INFO:tasks.workunit.client.0.smithi019.stdout:5/187: stat d8/d11/l1e 0 2024-02-22T15:50:27.122 INFO:tasks.workunit.client.0.smithi019.stdout:5/188: stat d8/d11/d21/d30 0 2024-02-22T15:50:27.132 INFO:tasks.workunit.client.0.smithi019.stdout:5/189: dwrite d8/f25 [4194304,4194304] 0 2024-02-22T15:50:27.134 INFO:tasks.workunit.client.0.smithi019.stdout:5/190: creat d8/d11/d33/f3c x:0 0 0 2024-02-22T15:50:27.136 INFO:tasks.workunit.client.0.smithi019.stdout:5/191: rename d8/d11/d21/d2c/c39 to d8/dd/c3d 0 2024-02-22T15:50:27.137 INFO:tasks.workunit.client.0.smithi019.stdout:5/192: write d8/fc [802114,105012] 0 2024-02-22T15:50:27.139 INFO:tasks.workunit.client.0.smithi019.stdout:5/193: creat d8/d11/f3e x:0 0 0 2024-02-22T15:50:27.142 INFO:tasks.workunit.client.0.smithi019.stdout:5/194: creat d8/dd/d19/d3a/f3f x:0 0 0 2024-02-22T15:50:27.143 INFO:tasks.workunit.client.0.smithi019.stdout:5/195: chown d8/dd/d19/d3a/f3f 1 1 2024-02-22T15:50:27.143 INFO:tasks.workunit.client.0.smithi019.stdout:5/196: chown d8/c32 2 1 2024-02-22T15:50:27.144 INFO:tasks.workunit.client.0.smithi019.stdout:5/197: stat d8/ff 0 2024-02-22T15:50:27.147 INFO:tasks.workunit.client.0.smithi019.stdout:5/198: read f5 [1081907,79494] 0 2024-02-22T15:50:27.148 INFO:tasks.workunit.client.0.smithi019.stdout:5/199: write d8/f2d [542264,67343] 0 2024-02-22T15:50:27.150 INFO:tasks.workunit.client.0.smithi019.stdout:5/200: mkdir d8/dd/d19/d40 0 2024-02-22T15:50:27.156 INFO:tasks.workunit.client.0.smithi019.stdout:5/201: dwrite d8/d11/f15 [0,4194304] 0 2024-02-22T15:50:27.165 INFO:tasks.workunit.client.0.smithi019.stdout:5/202: write d8/d11/d21/f22 [2446637,106706] 0 2024-02-22T15:50:27.171 INFO:tasks.workunit.client.0.smithi019.stdout:5/203: dwrite d8/fc [0,4194304] 0 2024-02-22T15:50:27.176 INFO:tasks.workunit.client.0.smithi019.stdout:5/204: rename d8/d11/d21/d2c/c37 to d8/dd/c41 0 2024-02-22T15:50:27.177 INFO:tasks.workunit.client.0.smithi019.stdout:5/205: stat d8/dd/d19/l26 0 2024-02-22T15:50:27.178 INFO:tasks.workunit.client.0.smithi019.stdout:5/206: chown d8/dd/d19/d38 374313 1 2024-02-22T15:50:27.186 INFO:tasks.workunit.client.0.smithi019.stdout:5/207: truncate d8/d11/d21/f36 3092744 0 2024-02-22T15:50:27.188 INFO:tasks.workunit.client.0.smithi019.stdout:5/208: symlink d8/d11/d21/d2c/l42 0 2024-02-22T15:50:27.189 INFO:tasks.workunit.client.0.smithi019.stdout:3/117: sync 2024-02-22T15:50:27.189 INFO:tasks.workunit.client.0.smithi019.stdout:1/140: sync 2024-02-22T15:50:27.190 INFO:tasks.workunit.client.0.smithi019.stdout:3/118: dread - d9/d14/d19/d1a/f20 zero size 2024-02-22T15:50:27.190 INFO:tasks.workunit.client.0.smithi019.stdout:5/209: creat d8/dd/d19/d3a/f43 x:0 0 0 2024-02-22T15:50:27.191 INFO:tasks.workunit.client.0.smithi019.stdout:1/141: creat d8/d15/f30 x:0 0 0 2024-02-22T15:50:27.192 INFO:tasks.workunit.client.0.smithi019.stdout:3/119: mknod d9/d14/c24 0 2024-02-22T15:50:27.193 INFO:tasks.workunit.client.0.smithi019.stdout:1/142: mknod d8/d2a/c31 0 2024-02-22T15:50:27.193 INFO:tasks.workunit.client.0.smithi019.stdout:1/143: stat c2 0 2024-02-22T15:50:27.194 INFO:tasks.workunit.client.0.smithi019.stdout:5/210: symlink d8/d11/d33/l44 0 2024-02-22T15:50:27.194 INFO:tasks.workunit.client.0.smithi019.stdout:3/120: mknod d9/d14/d19/d1a/c25 0 2024-02-22T15:50:27.198 INFO:tasks.workunit.client.0.smithi019.stdout:1/144: link d8/d15/f11 d8/d2a/f32 0 2024-02-22T15:50:27.199 INFO:tasks.workunit.client.0.smithi019.stdout:5/211: rename d8/dd/d19/d40 to d8/dd/d19/d38/d45 0 2024-02-22T15:50:27.202 INFO:tasks.workunit.client.0.smithi019.stdout:1/145: rmdir d8/d23 39 2024-02-22T15:50:27.203 INFO:tasks.workunit.client.0.smithi019.stdout:1/146: write d8/d15/f17 [551948,74755] 0 2024-02-22T15:50:27.207 INFO:tasks.workunit.client.0.smithi019.stdout:1/147: mkdir d8/d23/d33 0 2024-02-22T15:50:27.209 INFO:tasks.workunit.client.0.smithi019.stdout:1/148: mkdir d8/d15/d34 0 2024-02-22T15:50:27.211 INFO:tasks.workunit.client.0.smithi019.stdout:3/121: dwrite d9/d11/fd [0,4194304] 0 2024-02-22T15:50:27.211 INFO:tasks.workunit.client.0.smithi019.stdout:5/212: rename d8/dd/d19/d38/d45 to d8/dd/d19/d3a/d46 0 2024-02-22T15:50:27.214 INFO:tasks.workunit.client.0.smithi019.stdout:1/149: unlink d8/d15/f30 0 2024-02-22T15:50:27.216 INFO:tasks.workunit.client.0.smithi019.stdout:3/122: dread d9/d16/f1d [0,4194304] 0 2024-02-22T15:50:27.216 INFO:tasks.workunit.client.0.smithi019.stdout:5/213: mkdir d8/dd/d19/d38/d47 0 2024-02-22T15:50:27.218 INFO:tasks.workunit.client.0.smithi019.stdout:5/214: write d8/dd/d19/d3a/f3f [625239,60042] 0 2024-02-22T15:50:27.218 INFO:tasks.workunit.client.0.smithi019.stdout:1/150: creat d8/d23/f35 x:0 0 0 2024-02-22T15:50:27.219 INFO:tasks.workunit.client.0.smithi019.stdout:1/151: readlink d8/d15/l22 0 2024-02-22T15:50:27.221 INFO:tasks.workunit.client.0.smithi019.stdout:5/215: rename d8/d11/f15 to d8/f48 0 2024-02-22T15:50:27.223 INFO:tasks.workunit.client.0.smithi019.stdout:5/216: mknod d8/d11/d33/c49 0 2024-02-22T15:50:27.224 INFO:tasks.workunit.client.0.smithi019.stdout:5/217: truncate d8/dd/d19/d3a/f43 522020 0 2024-02-22T15:50:27.252 INFO:tasks.workunit.client.0.smithi019.stdout:3/123: truncate d9/d11/fd 1449678 0 2024-02-22T15:50:27.252 INFO:tasks.workunit.client.0.smithi019.stdout:3/124: readlink d9/d11/l10 0 2024-02-22T15:50:27.267 INFO:tasks.workunit.client.0.smithi019.stdout:3/125: dread f3 [0,4194304] 0 2024-02-22T15:50:27.269 INFO:tasks.workunit.client.0.smithi019.stdout:3/126: mkdir d9/d26 0 2024-02-22T15:50:27.273 INFO:tasks.workunit.client.0.smithi019.stdout:3/127: dread d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:27.274 INFO:tasks.workunit.client.0.smithi019.stdout:3/128: mkdir d9/d22/d27 0 2024-02-22T15:50:27.330 INFO:tasks.workunit.client.0.smithi019.stdout:7/103: sync 2024-02-22T15:50:27.332 INFO:tasks.workunit.client.0.smithi019.stdout:3/129: fdatasync d9/d11/f17 0 2024-02-22T15:50:27.333 INFO:tasks.workunit.client.0.smithi019.stdout:7/104: creat d9/f22 x:0 0 0 2024-02-22T15:50:27.336 INFO:tasks.workunit.client.0.smithi019.stdout:3/130: symlink d9/d26/l28 0 2024-02-22T15:50:27.338 INFO:tasks.workunit.client.0.smithi019.stdout:3/131: creat d9/d16/f29 x:0 0 0 2024-02-22T15:50:27.339 INFO:tasks.workunit.client.0.smithi019.stdout:3/132: readlink d9/d14/d19/l23 0 2024-02-22T15:50:27.340 INFO:tasks.workunit.client.0.smithi019.stdout:3/133: dread - d9/d11/f1e zero size 2024-02-22T15:50:27.342 INFO:tasks.workunit.client.0.smithi019.stdout:7/105: getdents d9 0 2024-02-22T15:50:27.344 INFO:tasks.workunit.client.0.smithi019.stdout:7/106: write d9/f22 [438546,29067] 0 2024-02-22T15:50:27.347 INFO:tasks.workunit.client.0.smithi019.stdout:7/107: unlink d9/f11 0 2024-02-22T15:50:27.349 INFO:tasks.workunit.client.0.smithi019.stdout:7/108: readlink d9/l18 0 2024-02-22T15:50:27.350 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:27 smithi019 ceph-mon[67418]: pgmap v29: 65 pgs: 65 active+clean; 427 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 2.8 MiB/s rd, 31 MiB/s wr, 396 op/s 2024-02-22T15:50:27.351 INFO:tasks.workunit.client.0.smithi019.stdout:3/134: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:27.352 INFO:tasks.workunit.client.0.smithi019.stdout:7/109: creat d9/f23 x:0 0 0 2024-02-22T15:50:27.353 INFO:tasks.workunit.client.0.smithi019.stdout:7/110: write d9/f22 [79317,75171] 0 2024-02-22T15:50:27.356 INFO:tasks.workunit.client.0.smithi019.stdout:7/111: creat d9/f24 x:0 0 0 2024-02-22T15:50:27.356 INFO:tasks.workunit.client.0.smithi019.stdout:3/135: dread d9/d11/fd [0,4194304] 0 2024-02-22T15:50:27.357 INFO:tasks.workunit.client.0.smithi019.stdout:7/112: creat d9/f25 x:0 0 0 2024-02-22T15:50:27.358 INFO:tasks.workunit.client.0.smithi019.stdout:3/136: mknod d9/d11/c2a 0 2024-02-22T15:50:27.360 INFO:tasks.workunit.client.0.smithi019.stdout:7/113: mkdir d9/d26 0 2024-02-22T15:50:27.361 INFO:tasks.workunit.client.0.smithi019.stdout:3/137: write d9/d11/fd [3204664,114341] 0 2024-02-22T15:50:27.365 INFO:tasks.workunit.client.0.smithi019.stdout:3/138: write d9/d16/f15 [3766358,9786] 0 2024-02-22T15:50:27.368 INFO:tasks.workunit.client.0.smithi019.stdout:3/139: creat d9/d16/f2b x:0 0 0 2024-02-22T15:50:27.371 INFO:tasks.workunit.client.0.smithi019.stdout:7/114: dwrite d9/f21 [0,4194304] 0 2024-02-22T15:50:27.373 INFO:tasks.workunit.client.0.smithi019.stdout:3/140: creat d9/d14/d19/d1a/f2c x:0 0 0 2024-02-22T15:50:27.379 INFO:tasks.workunit.client.0.smithi019.stdout:3/141: chown d9/d16/f15 1769 1 2024-02-22T15:50:27.381 INFO:tasks.workunit.client.0.smithi019.stdout:3/142: mkdir d9/d26/d2d 0 2024-02-22T15:50:27.382 INFO:tasks.workunit.client.0.smithi019.stdout:7/115: dread d9/f21 [0,4194304] 0 2024-02-22T15:50:27.382 INFO:tasks.workunit.client.0.smithi019.stdout:3/143: fdatasync d9/d11/fd 0 2024-02-22T15:50:27.384 INFO:tasks.workunit.client.0.smithi019.stdout:3/144: chown d9/d14/d19/d1a/c25 5611 1 2024-02-22T15:50:27.384 INFO:tasks.workunit.client.0.smithi019.stdout:7/116: write d9/f16 [2757527,84188] 0 2024-02-22T15:50:27.386 INFO:tasks.workunit.client.0.smithi019.stdout:7/117: write f8 [1908644,40663] 0 2024-02-22T15:50:27.387 INFO:tasks.workunit.client.0.smithi019.stdout:3/145: mknod d9/d14/d19/d1a/c2e 0 2024-02-22T15:50:27.390 INFO:tasks.workunit.client.0.smithi019.stdout:7/118: rename d9/c13 to d9/d26/c27 0 2024-02-22T15:50:27.390 INFO:tasks.workunit.client.0.smithi019.stdout:7/119: fdatasync d9/fa 0 2024-02-22T15:50:27.393 INFO:tasks.workunit.client.0.smithi019.stdout:3/146: creat d9/f2f x:0 0 0 2024-02-22T15:50:27.393 INFO:tasks.workunit.client.0.smithi019.stdout:3/147: fdatasync d9/d14/f1b 0 2024-02-22T15:50:27.397 INFO:tasks.workunit.client.0.smithi019.stdout:3/148: rename d9/d14/d19/d1a/c2e to d9/d16/c30 0 2024-02-22T15:50:27.399 INFO:tasks.workunit.client.0.smithi019.stdout:3/149: unlink d9/d14/d19/d1a/f20 0 2024-02-22T15:50:27.399 INFO:tasks.workunit.client.0.smithi019.stdout:3/150: chown d9/d14/d19 1030677 1 2024-02-22T15:50:27.401 INFO:tasks.workunit.client.0.smithi019.stdout:7/120: rmdir d9 39 2024-02-22T15:50:27.402 INFO:tasks.workunit.client.0.smithi019.stdout:3/151: creat d9/d14/f31 x:0 0 0 2024-02-22T15:50:27.405 INFO:tasks.workunit.client.0.smithi019.stdout:7/121: rename d9/d26/c27 to d9/c28 0 2024-02-22T15:50:27.406 INFO:tasks.workunit.client.0.smithi019.stdout:7/122: dread - d9/f25 zero size 2024-02-22T15:50:27.413 INFO:tasks.workunit.client.0.smithi019.stdout:7/123: dwrite d9/f17 [0,4194304] 0 2024-02-22T15:50:27.415 INFO:tasks.workunit.client.0.smithi019.stdout:7/124: write d9/f21 [927574,47326] 0 2024-02-22T15:50:27.416 INFO:tasks.workunit.client.0.smithi019.stdout:7/125: read - d9/f19 zero size 2024-02-22T15:50:27.419 INFO:tasks.workunit.client.0.smithi019.stdout:7/126: rename d9/f22 to d9/d26/f29 0 2024-02-22T15:50:27.420 INFO:tasks.workunit.client.0.smithi019.stdout:3/152: dread f3 [8388608,4194304] 0 2024-02-22T15:50:27.421 INFO:tasks.workunit.client.0.smithi019.stdout:3/153: chown d9/d11/l10 60751 1 2024-02-22T15:50:27.423 INFO:tasks.workunit.client.0.smithi019.stdout:7/127: link d9/ce d9/c2a 0 2024-02-22T15:50:27.434 INFO:tasks.workunit.client.0.smithi019.stdout:7/128: getdents d9 0 2024-02-22T15:50:27.435 INFO:tasks.workunit.client.0.smithi019.stdout:7/129: write d9/f24 [1028268,3011] 0 2024-02-22T15:50:27.438 INFO:tasks.workunit.client.0.smithi019.stdout:7/130: rename d9/f15 to d9/d26/f2b 0 2024-02-22T15:50:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:7/131: creat d9/f2c x:0 0 0 2024-02-22T15:50:27.440 INFO:tasks.workunit.client.0.smithi019.stdout:7/132: chown d9/c1f 3947 1 2024-02-22T15:50:27.441 INFO:tasks.workunit.client.0.smithi019.stdout:7/133: creat d9/d26/f2d x:0 0 0 2024-02-22T15:50:27.443 INFO:tasks.workunit.client.0.smithi019.stdout:7/134: creat d9/d26/f2e x:0 0 0 2024-02-22T15:50:27.443 INFO:tasks.workunit.client.0.smithi019.stdout:0/100: sync 2024-02-22T15:50:27.443 INFO:tasks.workunit.client.0.smithi019.stdout:6/86: sync 2024-02-22T15:50:27.443 INFO:tasks.workunit.client.0.smithi019.stdout:8/139: sync 2024-02-22T15:50:27.443 INFO:tasks.workunit.client.0.smithi019.stdout:4/87: sync 2024-02-22T15:50:27.444 INFO:tasks.workunit.client.0.smithi019.stdout:9/201: sync 2024-02-22T15:50:27.444 INFO:tasks.workunit.client.0.smithi019.stdout:2/131: sync 2024-02-22T15:50:27.444 INFO:tasks.workunit.client.0.smithi019.stdout:6/87: chown d6/l12 474 1 2024-02-22T15:50:27.445 INFO:tasks.workunit.client.0.smithi019.stdout:2/132: write d1/de/f16 [2451764,83541] 0 2024-02-22T15:50:27.445 INFO:tasks.workunit.client.0.smithi019.stdout:8/140: unlink c4 0 2024-02-22T15:50:27.446 INFO:tasks.workunit.client.0.smithi019.stdout:8/141: chown f6 38 1 2024-02-22T15:50:27.447 INFO:tasks.workunit.client.0.smithi019.stdout:0/101: mkdir d4/d22 0 2024-02-22T15:50:27.448 INFO:tasks.workunit.client.0.smithi019.stdout:8/142: write f1 [6137823,15430] 0 2024-02-22T15:50:27.448 INFO:tasks.workunit.client.0.smithi019.stdout:7/135: symlink d9/l2f 0 2024-02-22T15:50:27.448 INFO:tasks.workunit.client.0.smithi019.stdout:0/102: stat d4/d22 0 2024-02-22T15:50:27.448 INFO:tasks.workunit.client.0.smithi019.stdout:7/136: fdatasync f8 0 2024-02-22T15:50:27.449 INFO:tasks.workunit.client.0.smithi019.stdout:9/202: creat d2/d7/dc/f49 x:0 0 0 2024-02-22T15:50:27.450 INFO:tasks.workunit.client.0.smithi019.stdout:8/143: fdatasync d9/f29 0 2024-02-22T15:50:27.450 INFO:tasks.workunit.client.0.smithi019.stdout:0/103: dread - d4/f21 zero size 2024-02-22T15:50:27.451 INFO:tasks.workunit.client.0.smithi019.stdout:7/137: write d9/f1b [809692,49368] 0 2024-02-22T15:50:27.452 INFO:tasks.workunit.client.0.smithi019.stdout:6/88: symlink d6/l1b 0 2024-02-22T15:50:27.453 INFO:tasks.workunit.client.0.smithi019.stdout:0/104: mkdir d4/d23 0 2024-02-22T15:50:27.456 INFO:tasks.workunit.client.0.smithi019.stdout:4/88: dwrite d2/fa [0,4194304] 0 2024-02-22T15:50:27.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:27 smithi062 ceph-mon[48130]: pgmap v29: 65 pgs: 65 active+clean; 427 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 2.8 MiB/s rd, 31 MiB/s wr, 396 op/s 2024-02-22T15:50:27.458 INFO:tasks.workunit.client.0.smithi019.stdout:7/138: mknod d9/d26/c30 0 2024-02-22T15:50:27.458 INFO:tasks.workunit.client.0.smithi019.stdout:4/89: fsync d2/db/f17 0 2024-02-22T15:50:27.460 INFO:tasks.workunit.client.0.smithi019.stdout:8/144: creat d9/d14/d1f/d20/f2a x:0 0 0 2024-02-22T15:50:27.461 INFO:tasks.workunit.client.0.smithi019.stdout:8/145: chown d9/f29 0 1 2024-02-22T15:50:27.464 INFO:tasks.workunit.client.0.smithi019.stdout:6/89: creat d6/f1c x:0 0 0 2024-02-22T15:50:27.465 INFO:tasks.workunit.client.0.smithi019.stdout:0/105: mkdir d4/db/d24 0 2024-02-22T15:50:27.468 INFO:tasks.workunit.client.0.smithi019.stdout:4/90: dwrite d2/db/d14/f19 [0,4194304] 0 2024-02-22T15:50:27.471 INFO:tasks.workunit.client.0.smithi019.stdout:7/139: creat d9/d26/f31 x:0 0 0 2024-02-22T15:50:27.472 INFO:tasks.workunit.client.0.smithi019.stdout:2/133: getdents d1/d17/d19 0 2024-02-22T15:50:27.473 INFO:tasks.workunit.client.0.smithi019.stdout:2/134: stat d1/l7 0 2024-02-22T15:50:27.473 INFO:tasks.workunit.client.0.smithi019.stdout:6/90: dread d6/fb [0,4194304] 0 2024-02-22T15:50:27.474 INFO:tasks.workunit.client.0.smithi019.stdout:8/146: rename d9/d14/d17/l18 to d9/d14/d17/d1c/l2b 0 2024-02-22T15:50:27.476 INFO:tasks.workunit.client.0.smithi019.stdout:2/135: read d1/de/f1e [640903,58013] 0 2024-02-22T15:50:27.476 INFO:tasks.workunit.client.0.smithi019.stdout:6/91: truncate d6/f13 4611030 0 2024-02-22T15:50:27.482 INFO:tasks.workunit.client.0.smithi019.stdout:8/147: symlink d9/d14/d1f/d20/l2c 0 2024-02-22T15:50:27.485 INFO:tasks.workunit.client.0.smithi019.stdout:0/106: rmdir d4/db/d1d 0 2024-02-22T15:50:27.485 INFO:tasks.workunit.client.0.smithi019.stdout:4/91: dwrite d2/db/f17 [0,4194304] 0 2024-02-22T15:50:27.488 INFO:tasks.workunit.client.0.smithi019.stdout:7/140: dwrite d9/f1b [0,4194304] 0 2024-02-22T15:50:27.489 INFO:tasks.workunit.client.0.smithi019.stdout:8/148: mknod d9/d14/d1f/d20/c2d 0 2024-02-22T15:50:27.491 INFO:tasks.workunit.client.0.smithi019.stdout:2/136: dread d1/f14 [0,4194304] 0 2024-02-22T15:50:27.492 INFO:tasks.workunit.client.0.smithi019.stdout:4/92: fsync d2/db/d14/f18 0 2024-02-22T15:50:27.494 INFO:tasks.workunit.client.0.smithi019.stdout:7/141: dread d9/f24 [0,4194304] 0 2024-02-22T15:50:27.496 INFO:tasks.workunit.client.0.smithi019.stdout:2/137: rmdir d1 39 2024-02-22T15:50:27.501 INFO:tasks.workunit.client.0.smithi019.stdout:6/92: write d6/f10 [2200419,36485] 0 2024-02-22T15:50:27.501 INFO:tasks.workunit.client.0.smithi019.stdout:7/142: creat d9/d26/f32 x:0 0 0 2024-02-22T15:50:27.501 INFO:tasks.workunit.client.0.smithi019.stdout:0/107: link d4/db/c16 d4/d5/d7/c25 0 2024-02-22T15:50:27.504 INFO:tasks.workunit.client.0.smithi019.stdout:2/138: chown d1/de 211 1 2024-02-22T15:50:27.504 INFO:tasks.workunit.client.0.smithi019.stdout:6/93: fsync d6/f14 0 2024-02-22T15:50:27.506 INFO:tasks.workunit.client.0.smithi019.stdout:7/143: write d9/d26/f2e [1029715,78001] 0 2024-02-22T15:50:27.509 INFO:tasks.workunit.client.0.smithi019.stdout:8/149: dwrite d9/d14/d17/d1c/f24 [0,4194304] 0 2024-02-22T15:50:27.509 INFO:tasks.workunit.client.0.smithi019.stdout:4/93: dwrite d2/db/d14/f18 [0,4194304] 0 2024-02-22T15:50:27.510 INFO:tasks.workunit.client.0.smithi019.stdout:7/144: dread - d9/f19 zero size 2024-02-22T15:50:27.512 INFO:tasks.workunit.client.0.smithi019.stdout:4/94: chown d2/c3 14280005 1 2024-02-22T15:50:27.513 INFO:tasks.workunit.client.0.smithi019.stdout:7/145: write d9/f23 [658967,14935] 0 2024-02-22T15:50:27.515 INFO:tasks.workunit.client.0.smithi019.stdout:2/139: rename d1/de/d15/l28 to d1/d17/d19/l2c 0 2024-02-22T15:50:27.516 INFO:tasks.workunit.client.0.smithi019.stdout:6/94: dread d6/fb [0,4194304] 0 2024-02-22T15:50:27.516 INFO:tasks.workunit.client.0.smithi019.stdout:8/150: symlink d9/d14/d17/l2e 0 2024-02-22T15:50:27.517 INFO:tasks.workunit.client.0.smithi019.stdout:6/95: stat d6/f9 0 2024-02-22T15:50:27.519 INFO:tasks.workunit.client.0.smithi019.stdout:6/96: dread d6/fe [0,4194304] 0 2024-02-22T15:50:27.520 INFO:tasks.workunit.client.0.smithi019.stdout:8/151: truncate d9/d14/d17/f25 828636 0 2024-02-22T15:50:27.520 INFO:tasks.workunit.client.0.smithi019.stdout:7/146: rename d9/d26/f2d to d9/d26/f33 0 2024-02-22T15:50:27.521 INFO:tasks.workunit.client.0.smithi019.stdout:6/97: readlink d6/l19 0 2024-02-22T15:50:27.523 INFO:tasks.workunit.client.0.smithi019.stdout:7/147: write d9/f19 [955828,94599] 0 2024-02-22T15:50:27.526 INFO:tasks.workunit.client.0.smithi019.stdout:6/98: unlink d6/f13 0 2024-02-22T15:50:27.526 INFO:tasks.workunit.client.0.smithi019.stdout:2/140: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:27.526 INFO:tasks.workunit.client.0.smithi019.stdout:4/95: link d2/l7 d2/db/d14/d1b/l1c 0 2024-02-22T15:50:27.527 INFO:tasks.workunit.client.0.smithi019.stdout:8/152: mknod d9/d14/d1f/d20/c2f 0 2024-02-22T15:50:27.527 INFO:tasks.workunit.client.0.smithi019.stdout:6/99: stat d6/c15 0 2024-02-22T15:50:27.530 INFO:tasks.workunit.client.0.smithi019.stdout:4/96: mknod d2/db/c1d 0 2024-02-22T15:50:27.531 INFO:tasks.workunit.client.0.smithi019.stdout:6/100: creat d6/f1d x:0 0 0 2024-02-22T15:50:27.532 INFO:tasks.workunit.client.0.smithi019.stdout:8/153: mkdir d9/d14/d1f/d20/d30 0 2024-02-22T15:50:27.534 INFO:tasks.workunit.client.0.smithi019.stdout:2/141: creat d1/de/f2d x:0 0 0 2024-02-22T15:50:27.535 INFO:tasks.workunit.client.0.smithi019.stdout:4/97: mknod d2/c1e 0 2024-02-22T15:50:27.536 INFO:tasks.workunit.client.0.smithi019.stdout:7/148: dread d9/f19 [0,4194304] 0 2024-02-22T15:50:27.537 INFO:tasks.workunit.client.0.smithi019.stdout:8/154: mknod d9/d14/d1f/d20/c31 0 2024-02-22T15:50:27.539 INFO:tasks.workunit.client.0.smithi019.stdout:8/155: read f6 [117658,10726] 0 2024-02-22T15:50:27.542 INFO:tasks.workunit.client.0.smithi019.stdout:7/149: creat d9/f34 x:0 0 0 2024-02-22T15:50:27.543 INFO:tasks.workunit.client.0.smithi019.stdout:2/142: dread d1/de/f16 [0,4194304] 0 2024-02-22T15:50:27.544 INFO:tasks.workunit.client.0.smithi019.stdout:7/150: dread - d9/f34 zero size 2024-02-22T15:50:27.545 INFO:tasks.workunit.client.0.smithi019.stdout:7/151: readlink d9/l20 0 2024-02-22T15:50:27.548 INFO:tasks.workunit.client.0.smithi019.stdout:8/156: dwrite d9/ff [0,4194304] 0 2024-02-22T15:50:27.554 INFO:tasks.workunit.client.0.smithi019.stdout:7/152: unlink d9/f17 0 2024-02-22T15:50:27.555 INFO:tasks.workunit.client.0.smithi019.stdout:6/101: getdents d6 0 2024-02-22T15:50:27.555 INFO:tasks.workunit.client.0.smithi019.stdout:6/102: stat d6/fb 0 2024-02-22T15:50:27.556 INFO:tasks.workunit.client.0.smithi019.stdout:2/143: fsync d1/de/f2d 0 2024-02-22T15:50:27.557 INFO:tasks.workunit.client.0.smithi019.stdout:4/98: creat d2/f1f x:0 0 0 2024-02-22T15:50:27.559 INFO:tasks.workunit.client.0.smithi019.stdout:7/153: dread d9/f1b [0,4194304] 0 2024-02-22T15:50:27.560 INFO:tasks.workunit.client.0.smithi019.stdout:6/103: symlink d6/l1e 0 2024-02-22T15:50:27.562 INFO:tasks.workunit.client.0.smithi019.stdout:6/104: truncate d6/ff 1214076 0 2024-02-22T15:50:27.563 INFO:tasks.workunit.client.0.smithi019.stdout:4/99: creat d2/db/f20 x:0 0 0 2024-02-22T15:50:27.564 INFO:tasks.workunit.client.0.smithi019.stdout:7/154: mknod d9/d26/c35 0 2024-02-22T15:50:27.565 INFO:tasks.workunit.client.0.smithi019.stdout:4/100: truncate d2/db/f17 4557456 0 2024-02-22T15:50:27.566 INFO:tasks.workunit.client.0.smithi019.stdout:6/105: rmdir d6 39 2024-02-22T15:50:27.568 INFO:tasks.workunit.client.0.smithi019.stdout:8/157: rmdir d9/d14/d1f/d20/d30 0 2024-02-22T15:50:27.570 INFO:tasks.workunit.client.0.smithi019.stdout:2/144: creat d1/f2e x:0 0 0 2024-02-22T15:50:27.570 INFO:tasks.workunit.client.0.smithi019.stdout:8/158: dread - d9/f29 zero size 2024-02-22T15:50:27.571 INFO:tasks.workunit.client.0.smithi019.stdout:2/145: fsync d1/de/f2d 0 2024-02-22T15:50:27.572 INFO:tasks.workunit.client.0.smithi019.stdout:4/101: chown d2/db/d14/d1b/l1c 7510603 1 2024-02-22T15:50:27.572 INFO:tasks.workunit.client.0.smithi019.stdout:6/106: chown d6/l1e 16522822 1 2024-02-22T15:50:27.572 INFO:tasks.workunit.client.0.smithi019.stdout:7/155: dwrite d9/f25 [0,4194304] 0 2024-02-22T15:50:27.573 INFO:tasks.workunit.client.0.smithi019.stdout:4/102: fdatasync d2/f6 0 2024-02-22T15:50:27.575 INFO:tasks.workunit.client.0.smithi019.stdout:6/107: write d6/f18 [1371195,35129] 0 2024-02-22T15:50:27.580 INFO:tasks.workunit.client.0.smithi019.stdout:8/159: creat d9/f32 x:0 0 0 2024-02-22T15:50:27.583 INFO:tasks.workunit.client.0.smithi019.stdout:2/146: dwrite d1/f1f [4194304,4194304] 0 2024-02-22T15:50:27.583 INFO:tasks.workunit.client.0.smithi019.stdout:4/103: mkdir d2/db/d14/d1b/d21 0 2024-02-22T15:50:27.584 INFO:tasks.workunit.client.0.smithi019.stdout:2/147: dread - d1/f26 zero size 2024-02-22T15:50:27.586 INFO:tasks.workunit.client.0.smithi019.stdout:8/160: creat d9/d14/d1f/f33 x:0 0 0 2024-02-22T15:50:27.587 INFO:tasks.workunit.client.0.smithi019.stdout:2/148: write d1/f1f [4805770,19359] 0 2024-02-22T15:50:27.589 INFO:tasks.workunit.client.0.smithi019.stdout:2/149: chown d1/f26 87808906 1 2024-02-22T15:50:27.589 INFO:tasks.workunit.client.0.smithi019.stdout:7/156: dread d9/f1b [0,4194304] 0 2024-02-22T15:50:27.590 INFO:tasks.workunit.client.0.smithi019.stdout:8/161: creat d9/d14/f34 x:0 0 0 2024-02-22T15:50:27.591 INFO:tasks.workunit.client.0.smithi019.stdout:2/150: dread - d1/f2e zero size 2024-02-22T15:50:27.594 INFO:tasks.workunit.client.0.smithi019.stdout:7/157: mknod d9/c36 0 2024-02-22T15:50:27.594 INFO:tasks.workunit.client.0.smithi019.stdout:4/104: dwrite d2/db/f20 [0,4194304] 0 2024-02-22T15:50:27.595 INFO:tasks.workunit.client.0.smithi019.stdout:8/162: write d9/d14/d17/d1c/f1e [544253,121376] 0 2024-02-22T15:50:27.595 INFO:tasks.workunit.client.0.smithi019.stdout:2/151: readlink d1/d17/d19/l2c 0 2024-02-22T15:50:27.597 INFO:tasks.workunit.client.0.smithi019.stdout:4/105: chown d2/db/c1d 168315 1 2024-02-22T15:50:27.598 INFO:tasks.workunit.client.0.smithi019.stdout:7/158: truncate d9/d26/f32 952868 0 2024-02-22T15:50:27.603 INFO:tasks.workunit.client.0.smithi019.stdout:2/152: creat d1/d17/d19/f2f x:0 0 0 2024-02-22T15:50:27.603 INFO:tasks.workunit.client.0.smithi019.stdout:7/159: creat d9/d26/f37 x:0 0 0 2024-02-22T15:50:27.605 INFO:tasks.workunit.client.0.smithi019.stdout:4/106: symlink d2/db/d14/d1b/d21/l22 0 2024-02-22T15:50:27.606 INFO:tasks.workunit.client.0.smithi019.stdout:8/163: dread d9/d14/d17/d1c/f1e [0,4194304] 0 2024-02-22T15:50:27.609 INFO:tasks.workunit.client.0.smithi019.stdout:7/160: dread d9/f21 [0,4194304] 0 2024-02-22T15:50:27.611 INFO:tasks.workunit.client.0.smithi019.stdout:7/161: chown d9/f2c 28 1 2024-02-22T15:50:27.612 INFO:tasks.workunit.client.0.smithi019.stdout:8/164: dread d9/d14/f19 [0,4194304] 0 2024-02-22T15:50:27.613 INFO:tasks.workunit.client.0.smithi019.stdout:4/107: rename d2/db/d14/d1b to d2/db/d14/d23 0 2024-02-22T15:50:27.614 INFO:tasks.workunit.client.0.smithi019.stdout:7/162: chown d9/c12 6418 1 2024-02-22T15:50:27.614 INFO:tasks.workunit.client.0.smithi019.stdout:8/165: mknod d9/d14/d1f/c35 0 2024-02-22T15:50:27.615 INFO:tasks.workunit.client.0.smithi019.stdout:8/166: readlink d9/d14/d17/l1b 0 2024-02-22T15:50:27.616 INFO:tasks.workunit.client.0.smithi019.stdout:4/108: symlink d2/db/d14/l24 0 2024-02-22T15:50:27.616 INFO:tasks.workunit.client.0.smithi019.stdout:8/167: stat d9/d14/d1f/f21 0 2024-02-22T15:50:27.616 INFO:tasks.workunit.client.0.smithi019.stdout:1/152: sync 2024-02-22T15:50:27.616 INFO:tasks.workunit.client.0.smithi019.stdout:5/218: sync 2024-02-22T15:50:27.618 INFO:tasks.workunit.client.0.smithi019.stdout:5/219: fdatasync f5 0 2024-02-22T15:50:27.620 INFO:tasks.workunit.client.0.smithi019.stdout:4/109: write d2/db/d14/f18 [4904431,107366] 0 2024-02-22T15:50:27.620 INFO:tasks.workunit.client.0.smithi019.stdout:7/163: getdents d9 0 2024-02-22T15:50:27.623 INFO:tasks.workunit.client.0.smithi019.stdout:1/153: rename d8/d2a/c31 to d8/d2a/c36 0 2024-02-22T15:50:27.624 INFO:tasks.workunit.client.0.smithi019.stdout:4/110: dread d2/db/f20 [0,4194304] 0 2024-02-22T15:50:27.627 INFO:tasks.workunit.client.0.smithi019.stdout:8/168: link d9/d14/f19 d9/d14/f36 0 2024-02-22T15:50:27.628 INFO:tasks.workunit.client.0.smithi019.stdout:5/220: rename d8/d11/c12 to d8/dd/d19/c4a 0 2024-02-22T15:50:27.632 INFO:tasks.workunit.client.0.smithi019.stdout:5/221: write d8/f48 [4519053,30764] 0 2024-02-22T15:50:27.635 INFO:tasks.workunit.client.0.smithi019.stdout:4/111: getdents d2 0 2024-02-22T15:50:27.637 INFO:tasks.workunit.client.0.smithi019.stdout:7/164: truncate d9/f19 295772 0 2024-02-22T15:50:27.637 INFO:tasks.workunit.client.0.smithi019.stdout:8/169: dread f6 [0,4194304] 0 2024-02-22T15:50:27.637 INFO:tasks.workunit.client.0.smithi019.stdout:5/222: symlink d8/dd/d19/d38/d47/l4b 0 2024-02-22T15:50:27.639 INFO:tasks.workunit.client.0.smithi019.stdout:7/165: mkdir d9/d38 0 2024-02-22T15:50:27.639 INFO:tasks.workunit.client.0.smithi019.stdout:8/170: mknod d9/d14/d1f/d20/c37 0 2024-02-22T15:50:27.640 INFO:tasks.workunit.client.0.smithi019.stdout:7/166: truncate f8 2167046 0 2024-02-22T15:50:27.641 INFO:tasks.workunit.client.0.smithi019.stdout:7/167: dread - d9/d26/f33 zero size 2024-02-22T15:50:27.641 INFO:tasks.workunit.client.0.smithi019.stdout:8/171: creat d9/d14/f38 x:0 0 0 2024-02-22T15:50:27.642 INFO:tasks.workunit.client.0.smithi019.stdout:7/168: write d9/f34 [741940,70294] 0 2024-02-22T15:50:27.643 INFO:tasks.workunit.client.0.smithi019.stdout:8/172: read - d9/f29 zero size 2024-02-22T15:50:27.645 INFO:tasks.workunit.client.0.smithi019.stdout:8/173: chown d9/d14/d17/d1c 1 1 2024-02-22T15:50:27.651 INFO:tasks.workunit.client.0.smithi019.stdout:7/169: dread f8 [0,4194304] 0 2024-02-22T15:50:27.657 INFO:tasks.workunit.client.0.smithi019.stdout:7/170: link d9/c2a d9/c39 0 2024-02-22T15:50:27.657 INFO:tasks.workunit.client.0.smithi019.stdout:5/223: dread f5 [0,4194304] 0 2024-02-22T15:50:27.660 INFO:tasks.workunit.client.0.smithi019.stdout:8/174: dwrite d9/d14/f36 [0,4194304] 0 2024-02-22T15:50:27.660 INFO:tasks.workunit.client.0.smithi019.stdout:4/112: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:27.660 INFO:tasks.workunit.client.0.smithi019.stdout:5/224: chown d8/d11/c1d 441001 1 2024-02-22T15:50:27.660 INFO:tasks.workunit.client.0.smithi019.stdout:7/171: mkdir d9/d3a 0 2024-02-22T15:50:27.662 INFO:tasks.workunit.client.0.smithi019.stdout:5/225: chown d8/dd/d19/d3a/d46 10116672 1 2024-02-22T15:50:27.663 INFO:tasks.workunit.client.0.smithi019.stdout:4/113: rmdir d2/db/d14 39 2024-02-22T15:50:27.665 INFO:tasks.workunit.client.0.smithi019.stdout:8/175: mknod d9/d14/d1f/d20/c39 0 2024-02-22T15:50:27.665 INFO:tasks.workunit.client.0.smithi019.stdout:8/176: write d9/f12 [1995627,7810] 0 2024-02-22T15:50:27.666 INFO:tasks.workunit.client.0.smithi019.stdout:5/226: mknod d8/dd/d19/c4c 0 2024-02-22T15:50:27.669 INFO:tasks.workunit.client.0.smithi019.stdout:8/177: symlink d9/d14/d17/l3a 0 2024-02-22T15:50:27.672 INFO:tasks.workunit.client.0.smithi019.stdout:4/114: creat d2/db/f25 x:0 0 0 2024-02-22T15:50:27.673 INFO:tasks.workunit.client.0.smithi019.stdout:7/172: dwrite d9/f1b [0,4194304] 0 2024-02-22T15:50:27.673 INFO:tasks.workunit.client.0.smithi019.stdout:8/178: truncate d9/d14/d1f/f33 741998 0 2024-02-22T15:50:27.674 INFO:tasks.workunit.client.0.smithi019.stdout:5/227: link d8/l10 d8/d11/d21/d30/l4d 0 2024-02-22T15:50:27.674 INFO:tasks.workunit.client.0.smithi019.stdout:7/173: readlink d9/l18 0 2024-02-22T15:50:27.674 INFO:tasks.workunit.client.0.smithi019.stdout:5/228: read - d8/d11/f3e zero size 2024-02-22T15:50:27.675 INFO:tasks.workunit.client.0.smithi019.stdout:7/174: dread - d9/d26/f33 zero size 2024-02-22T15:50:27.678 INFO:tasks.workunit.client.0.smithi019.stdout:4/115: creat d2/db/f26 x:0 0 0 2024-02-22T15:50:27.679 INFO:tasks.workunit.client.0.smithi019.stdout:8/179: creat d9/f3b x:0 0 0 2024-02-22T15:50:27.679 INFO:tasks.workunit.client.0.smithi019.stdout:4/116: dread - d2/db/f25 zero size 2024-02-22T15:50:27.680 INFO:tasks.workunit.client.0.smithi019.stdout:7/175: rename d9/f2c to d9/d3a/f3b 0 2024-02-22T15:50:27.683 INFO:tasks.workunit.client.0.smithi019.stdout:4/117: write d2/db/d14/f19 [30600,70790] 0 2024-02-22T15:50:27.685 INFO:tasks.workunit.client.0.smithi019.stdout:8/180: mknod d9/c3c 0 2024-02-22T15:50:27.686 INFO:tasks.workunit.client.0.smithi019.stdout:7/176: creat d9/d38/f3c x:0 0 0 2024-02-22T15:50:27.687 INFO:tasks.workunit.client.0.smithi019.stdout:4/118: creat d2/db/d14/f27 x:0 0 0 2024-02-22T15:50:27.688 INFO:tasks.workunit.client.0.smithi019.stdout:8/181: mkdir d9/d14/d17/d3d 0 2024-02-22T15:50:27.690 INFO:tasks.workunit.client.0.smithi019.stdout:4/119: write d2/db/f20 [4116670,24688] 0 2024-02-22T15:50:27.692 INFO:tasks.workunit.client.0.smithi019.stdout:7/177: fsync d9/f1b 0 2024-02-22T15:50:27.692 INFO:tasks.workunit.client.0.smithi019.stdout:4/120: write d2/f1f [214919,69751] 0 2024-02-22T15:50:27.693 INFO:tasks.workunit.client.0.smithi019.stdout:8/182: getdents d9/d14/d17/d3d 0 2024-02-22T15:50:27.693 INFO:tasks.workunit.client.0.smithi019.stdout:7/178: chown d9/d3a 845985475 1 2024-02-22T15:50:27.694 INFO:tasks.workunit.client.0.smithi019.stdout:4/121: write d2/db/f25 [157178,1188] 0 2024-02-22T15:50:27.694 INFO:tasks.workunit.client.0.smithi019.stdout:5/229: truncate d8/dd/d19/d3a/f43 144271 0 2024-02-22T15:50:27.695 INFO:tasks.workunit.client.0.smithi019.stdout:8/183: read f5 [5709360,104644] 0 2024-02-22T15:50:27.696 INFO:tasks.workunit.client.0.smithi019.stdout:7/179: unlink l7 0 2024-02-22T15:50:27.698 INFO:tasks.workunit.client.0.smithi019.stdout:7/180: chown d9/fa 1 1 2024-02-22T15:50:27.698 INFO:tasks.workunit.client.0.smithi019.stdout:5/230: creat d8/f4e x:0 0 0 2024-02-22T15:50:27.699 INFO:tasks.workunit.client.0.smithi019.stdout:8/184: creat d9/d14/f3e x:0 0 0 2024-02-22T15:50:27.700 INFO:tasks.workunit.client.0.smithi019.stdout:5/231: fdatasync d8/dd/d19/d3a/f3f 0 2024-02-22T15:50:27.703 INFO:tasks.workunit.client.0.smithi019.stdout:7/181: dread d9/d26/f32 [0,4194304] 0 2024-02-22T15:50:27.703 INFO:tasks.workunit.client.0.smithi019.stdout:8/185: mkdir d9/d14/d17/d3d/d3f 0 2024-02-22T15:50:27.704 INFO:tasks.workunit.client.0.smithi019.stdout:4/122: dwrite d2/f1f [0,4194304] 0 2024-02-22T15:50:27.705 INFO:tasks.workunit.client.0.smithi019.stdout:5/232: symlink d8/l4f 0 2024-02-22T15:50:27.705 INFO:tasks.workunit.client.0.smithi019.stdout:7/182: symlink d9/d26/l3d 0 2024-02-22T15:50:27.706 INFO:tasks.workunit.client.0.smithi019.stdout:5/233: stat d8/dd/d19/f29 0 2024-02-22T15:50:27.710 INFO:tasks.workunit.client.0.smithi019.stdout:8/186: link d9/d14/d1f/f33 d9/d14/d17/f40 0 2024-02-22T15:50:27.713 INFO:tasks.workunit.client.0.smithi019.stdout:4/123: dwrite d2/fa [0,4194304] 0 2024-02-22T15:50:27.714 INFO:tasks.workunit.client.0.smithi019.stdout:8/187: rmdir d9/d14/d1f/d20 39 2024-02-22T15:50:27.715 INFO:tasks.workunit.client.0.smithi019.stdout:5/234: dread d8/fc [0,4194304] 0 2024-02-22T15:50:27.717 INFO:tasks.workunit.client.0.smithi019.stdout:8/188: write d9/d14/d17/d1c/f24 [4087681,65674] 0 2024-02-22T15:50:27.718 INFO:tasks.workunit.client.0.smithi019.stdout:5/235: write d8/d11/d33/f3c [774931,112104] 0 2024-02-22T15:50:27.720 INFO:tasks.workunit.client.0.smithi019.stdout:4/124: dread d2/fa [0,4194304] 0 2024-02-22T15:50:27.722 INFO:tasks.workunit.client.0.smithi019.stdout:5/236: symlink d8/dd/d19/d3a/l50 0 2024-02-22T15:50:27.722 INFO:tasks.workunit.client.0.smithi019.stdout:4/125: write d2/db/f25 [1006984,87086] 0 2024-02-22T15:50:27.725 INFO:tasks.workunit.client.0.smithi019.stdout:5/237: write d8/f23 [226464,27172] 0 2024-02-22T15:50:27.727 INFO:tasks.workunit.client.0.smithi019.stdout:5/238: fsync d8/f2d 0 2024-02-22T15:50:27.729 INFO:tasks.workunit.client.0.smithi019.stdout:4/126: creat d2/db/d14/d23/f28 x:0 0 0 2024-02-22T15:50:27.730 INFO:tasks.workunit.client.0.smithi019.stdout:4/127: readlink d2/db/d14/d23/l1c 0 2024-02-22T15:50:27.732 INFO:tasks.workunit.client.0.smithi019.stdout:7/183: getdents d9/d26 0 2024-02-22T15:50:27.735 INFO:tasks.workunit.client.0.smithi019.stdout:8/189: link d9/d14/d1f/d20/f2a d9/d14/d1f/f41 0 2024-02-22T15:50:27.736 INFO:tasks.workunit.client.0.smithi019.stdout:4/128: creat d2/db/d14/d23/d21/f29 x:0 0 0 2024-02-22T15:50:27.740 INFO:tasks.workunit.client.0.smithi019.stdout:4/129: symlink d2/db/d14/d23/l2a 0 2024-02-22T15:50:27.741 INFO:tasks.workunit.client.0.smithi019.stdout:4/130: chown d2/db 5482 1 2024-02-22T15:50:27.745 INFO:tasks.workunit.client.0.smithi019.stdout:4/131: dread d2/db/f20 [0,4194304] 0 2024-02-22T15:50:27.749 INFO:tasks.workunit.client.0.smithi019.stdout:5/239: write d8/d11/d21/f36 [2431442,51157] 0 2024-02-22T15:50:27.752 INFO:tasks.workunit.client.0.smithi019.stdout:7/184: fdatasync d9/f19 0 2024-02-22T15:50:27.753 INFO:tasks.workunit.client.0.smithi019.stdout:8/190: getdents d9/d14/d17/d3d 0 2024-02-22T15:50:27.753 INFO:tasks.workunit.client.0.smithi019.stdout:4/132: dwrite d2/db/d14/f19 [0,4194304] 0 2024-02-22T15:50:27.758 INFO:tasks.workunit.client.0.smithi019.stdout:5/240: dwrite d8/f25 [0,4194304] 0 2024-02-22T15:50:27.761 INFO:tasks.workunit.client.0.smithi019.stdout:4/133: creat d2/db/d14/f2b x:0 0 0 2024-02-22T15:50:27.765 INFO:tasks.workunit.client.0.smithi019.stdout:4/134: chown d2/c3 747 1 2024-02-22T15:50:27.765 INFO:tasks.workunit.client.0.smithi019.stdout:7/185: truncate d9/f24 1651807 0 2024-02-22T15:50:27.769 INFO:tasks.workunit.client.0.smithi019.stdout:4/135: creat d2/db/d14/d23/d21/f2c x:0 0 0 2024-02-22T15:50:27.770 INFO:tasks.workunit.client.0.smithi019.stdout:7/186: creat d9/d38/f3e x:0 0 0 2024-02-22T15:50:27.772 INFO:tasks.workunit.client.0.smithi019.stdout:8/191: dwrite d9/d14/d1f/f41 [0,4194304] 0 2024-02-22T15:50:27.773 INFO:tasks.workunit.client.0.smithi019.stdout:5/241: dwrite d8/ff [0,4194304] 0 2024-02-22T15:50:27.773 INFO:tasks.workunit.client.0.smithi019.stdout:7/187: chown d9/d38/f3c 232 1 2024-02-22T15:50:27.778 INFO:tasks.workunit.client.0.smithi019.stdout:5/242: symlink d8/dd/d19/d3a/d46/l51 0 2024-02-22T15:50:27.782 INFO:tasks.workunit.client.0.smithi019.stdout:8/192: write d9/d14/d17/d1c/f1e [1665626,20687] 0 2024-02-22T15:50:27.782 INFO:tasks.workunit.client.0.smithi019.stdout:4/136: write d2/db/d14/f2b [540505,77125] 0 2024-02-22T15:50:27.786 INFO:tasks.workunit.client.0.smithi019.stdout:5/243: rename d8/d11/d21/d2c/c34 to d8/dd/d19/d3a/d46/c52 0 2024-02-22T15:50:27.787 INFO:tasks.workunit.client.0.smithi019.stdout:7/188: dwrite d9/d38/f3c [0,4194304] 0 2024-02-22T15:50:27.791 INFO:tasks.workunit.client.0.smithi019.stdout:5/244: creat d8/dd/d19/d3a/d46/f53 x:0 0 0 2024-02-22T15:50:27.792 INFO:tasks.workunit.client.0.smithi019.stdout:5/245: read d8/dd/d19/f29 [13482,14501] 0 2024-02-22T15:50:27.792 INFO:tasks.workunit.client.0.smithi019.stdout:7/189: mknod d9/d3a/c3f 0 2024-02-22T15:50:27.793 INFO:tasks.workunit.client.0.smithi019.stdout:5/246: fdatasync d8/d11/f1c 0 2024-02-22T15:50:27.794 INFO:tasks.workunit.client.0.smithi019.stdout:7/190: read d9/f21 [1952397,116852] 0 2024-02-22T15:50:27.796 INFO:tasks.workunit.client.0.smithi019.stdout:5/247: mkdir d8/dd/d19/d38/d54 0 2024-02-22T15:50:27.797 INFO:tasks.workunit.client.0.smithi019.stdout:7/191: chown d9/d26/f31 54 1 2024-02-22T15:50:27.798 INFO:tasks.workunit.client.0.smithi019.stdout:5/248: dread d8/f23 [0,4194304] 0 2024-02-22T15:50:27.799 INFO:tasks.workunit.client.0.smithi019.stdout:7/192: write d9/f1d [1559208,37753] 0 2024-02-22T15:50:27.801 INFO:tasks.workunit.client.0.smithi019.stdout:7/193: mknod d9/d38/c40 0 2024-02-22T15:50:27.804 INFO:tasks.workunit.client.0.smithi019.stdout:5/249: dread d8/d11/d21/f22 [0,4194304] 0 2024-02-22T15:50:27.805 INFO:tasks.workunit.client.0.smithi019.stdout:5/250: fdatasync d8/dd/d19/d3a/f3f 0 2024-02-22T15:50:27.808 INFO:tasks.workunit.client.0.smithi019.stdout:5/251: mknod d8/dd/d19/c55 0 2024-02-22T15:50:27.811 INFO:tasks.workunit.client.0.smithi019.stdout:7/194: dwrite d9/f1b [4194304,4194304] 0 2024-02-22T15:50:27.814 INFO:tasks.workunit.client.0.smithi019.stdout:5/252: symlink d8/dd/d19/d38/d47/l56 0 2024-02-22T15:50:27.816 INFO:tasks.workunit.client.0.smithi019.stdout:5/253: fsync d8/d11/f3b 0 2024-02-22T15:50:27.816 INFO:tasks.workunit.client.0.smithi019.stdout:7/195: mkdir d9/d38/d41 0 2024-02-22T15:50:27.819 INFO:tasks.workunit.client.0.smithi019.stdout:7/196: readlink d9/l1c 0 2024-02-22T15:50:27.822 INFO:tasks.workunit.client.0.smithi019.stdout:8/193: dwrite d9/d14/d17/f40 [0,4194304] 0 2024-02-22T15:50:27.823 INFO:tasks.workunit.client.0.smithi019.stdout:5/254: dread d8/d11/d21/f36 [0,4194304] 0 2024-02-22T15:50:27.826 INFO:tasks.workunit.client.0.smithi019.stdout:5/255: write d8/fe [3197428,68882] 0 2024-02-22T15:50:27.827 INFO:tasks.workunit.client.0.smithi019.stdout:3/154: sync 2024-02-22T15:50:27.834 INFO:tasks.workunit.client.0.smithi019.stdout:5/256: mknod d8/d11/c57 0 2024-02-22T15:50:27.834 INFO:tasks.workunit.client.0.smithi019.stdout:7/197: dwrite d9/d3a/f3b [0,4194304] 0 2024-02-22T15:50:27.838 INFO:tasks.workunit.client.0.smithi019.stdout:3/155: rename d9/d14/d19/d1a to d9/d26/d32 0 2024-02-22T15:50:27.839 INFO:tasks.workunit.client.0.smithi019.stdout:3/156: dread - d9/f2f zero size 2024-02-22T15:50:27.841 INFO:tasks.workunit.client.0.smithi019.stdout:5/257: write d8/d11/d21/f36 [1904600,114362] 0 2024-02-22T15:50:27.842 INFO:tasks.workunit.client.0.smithi019.stdout:8/194: link d9/d14/d1f/d20/c2d d9/d14/d17/d1c/c42 0 2024-02-22T15:50:27.843 INFO:tasks.workunit.client.0.smithi019.stdout:7/198: mkdir d9/d38/d41/d42 0 2024-02-22T15:50:27.844 INFO:tasks.workunit.client.0.smithi019.stdout:7/199: write d9/d38/f3c [2126591,65959] 0 2024-02-22T15:50:27.846 INFO:tasks.workunit.client.0.smithi019.stdout:3/157: rename d9/d16/f15 to d9/d26/d32/f33 0 2024-02-22T15:50:27.853 INFO:tasks.workunit.client.0.smithi019.stdout:8/195: getdents d9/d14/d17/d3d 0 2024-02-22T15:50:27.854 INFO:tasks.workunit.client.0.smithi019.stdout:3/158: getdents d9/d26/d2d 0 2024-02-22T15:50:27.855 INFO:tasks.workunit.client.0.smithi019.stdout:8/196: write d9/d14/d1f/d20/f2a [1656642,72397] 0 2024-02-22T15:50:27.857 INFO:tasks.workunit.client.0.smithi019.stdout:7/200: truncate d9/f1d 1362407 0 2024-02-22T15:50:27.858 INFO:tasks.workunit.client.0.smithi019.stdout:5/258: link d8/l10 d8/d11/d21/l58 0 2024-02-22T15:50:27.861 INFO:tasks.workunit.client.0.smithi019.stdout:7/201: stat d9/c39 0 2024-02-22T15:50:27.861 INFO:tasks.workunit.client.0.smithi019.stdout:3/159: symlink d9/d22/d27/l34 0 2024-02-22T15:50:27.865 INFO:tasks.workunit.client.0.smithi019.stdout:7/202: unlink d9/d38/c40 0 2024-02-22T15:50:27.865 INFO:tasks.workunit.client.0.smithi019.stdout:5/259: dwrite d8/f2d [0,4194304] 0 2024-02-22T15:50:27.867 INFO:tasks.workunit.client.0.smithi019.stdout:7/203: write d9/f1b [3552759,113914] 0 2024-02-22T15:50:27.868 INFO:tasks.workunit.client.0.smithi019.stdout:5/260: read d8/dd/d19/f29 [94980,19668] 0 2024-02-22T15:50:27.870 INFO:tasks.workunit.client.0.smithi019.stdout:8/197: truncate d9/d14/f36 2843172 0 2024-02-22T15:50:27.871 INFO:tasks.workunit.client.0.smithi019.stdout:7/204: chown d9/f34 42625 1 2024-02-22T15:50:27.872 INFO:tasks.workunit.client.0.smithi019.stdout:5/261: symlink d8/d11/d21/l59 0 2024-02-22T15:50:27.874 INFO:tasks.workunit.client.0.smithi019.stdout:5/262: write d8/f48 [2635514,20897] 0 2024-02-22T15:50:27.875 INFO:tasks.workunit.client.0.smithi019.stdout:8/198: write f5 [4082598,124428] 0 2024-02-22T15:50:27.877 INFO:tasks.workunit.client.0.smithi019.stdout:5/263: write d8/dd/d19/d3a/d46/f53 [530381,93553] 0 2024-02-22T15:50:27.879 INFO:tasks.workunit.client.0.smithi019.stdout:7/205: rename d9/l18 to d9/d26/l43 0 2024-02-22T15:50:27.881 INFO:tasks.workunit.client.0.smithi019.stdout:5/264: creat d8/dd/d19/d38/f5a x:0 0 0 2024-02-22T15:50:27.882 INFO:tasks.workunit.client.0.smithi019.stdout:7/206: mknod d9/c44 0 2024-02-22T15:50:27.885 INFO:tasks.workunit.client.0.smithi019.stdout:5/265: creat d8/dd/f5b x:0 0 0 2024-02-22T15:50:27.885 INFO:tasks.workunit.client.0.smithi019.stdout:7/207: mkdir d9/d38/d41/d45 0 2024-02-22T15:50:27.886 INFO:tasks.workunit.client.0.smithi019.stdout:7/208: fsync d9/d3a/f3b 0 2024-02-22T15:50:27.890 INFO:tasks.workunit.client.0.smithi019.stdout:3/160: dread d9/fb [0,4194304] 0 2024-02-22T15:50:27.893 INFO:tasks.workunit.client.0.smithi019.stdout:5/266: dwrite d8/d11/f3e [0,4194304] 0 2024-02-22T15:50:27.949 INFO:tasks.workunit.client.0.smithi019.stdout:7/209: rmdir d9/d38/d41/d42 0 2024-02-22T15:50:27.950 INFO:tasks.workunit.client.0.smithi019.stdout:3/161: creat d9/d26/f35 x:0 0 0 2024-02-22T15:50:27.951 INFO:tasks.workunit.client.0.smithi019.stdout:5/267: unlink d8/c32 0 2024-02-22T15:50:27.957 INFO:tasks.workunit.client.0.smithi019.stdout:5/268: dwrite d8/d11/f1c [0,4194304] 0 2024-02-22T15:50:28.031 INFO:tasks.workunit.client.0.smithi019.stdout:7/210: mknod d9/d38/c46 0 2024-02-22T15:50:28.032 INFO:tasks.workunit.client.0.smithi019.stdout:3/162: fdatasync d9/d16/f1d 0 2024-02-22T15:50:28.034 INFO:tasks.workunit.client.0.smithi019.stdout:5/269: mknod d8/d11/d21/c5c 0 2024-02-22T15:50:28.035 INFO:tasks.workunit.client.0.smithi019.stdout:7/211: chown d9/fa 16448362 1 2024-02-22T15:50:28.037 INFO:tasks.workunit.client.0.smithi019.stdout:5/270: write d8/d11/d21/d2c/f31 [313445,18608] 0 2024-02-22T15:50:28.038 INFO:tasks.workunit.client.0.smithi019.stdout:8/199: dwrite d9/d14/f36 [0,4194304] 0 2024-02-22T15:50:28.038 INFO:tasks.workunit.client.0.smithi019.stdout:3/163: unlink d9/fa 0 2024-02-22T15:50:28.040 INFO:tasks.workunit.client.0.smithi019.stdout:8/200: chown d9/d14/d17/l1b 212588 1 2024-02-22T15:50:28.040 INFO:tasks.workunit.client.0.smithi019.stdout:8/201: chown f1 0 1 2024-02-22T15:50:28.042 INFO:tasks.workunit.client.0.smithi019.stdout:5/271: creat d8/d11/d21/d2c/f5d x:0 0 0 2024-02-22T15:50:28.042 INFO:tasks.workunit.client.0.smithi019.stdout:8/202: truncate d9/d14/f3e 598822 0 2024-02-22T15:50:28.043 INFO:tasks.workunit.client.0.smithi019.stdout:3/164: mknod d9/d14/c36 0 2024-02-22T15:50:28.044 INFO:tasks.workunit.client.0.smithi019.stdout:8/203: readlink d9/d14/d1f/d20/l2c 0 2024-02-22T15:50:28.047 INFO:tasks.workunit.client.0.smithi019.stdout:7/212: write d9/f19 [360213,101856] 0 2024-02-22T15:50:28.048 INFO:tasks.workunit.client.0.smithi019.stdout:8/204: rmdir d9/d14/d17/d3d 39 2024-02-22T15:50:28.053 INFO:tasks.workunit.client.0.smithi019.stdout:3/165: dwrite d9/d16/f1d [0,4194304] 0 2024-02-22T15:50:28.054 INFO:tasks.workunit.client.0.smithi019.stdout:7/213: dread d9/d26/f29 [0,4194304] 0 2024-02-22T15:50:28.055 INFO:tasks.workunit.client.0.smithi019.stdout:8/205: mknod d9/d14/d1f/c43 0 2024-02-22T15:50:28.056 INFO:tasks.workunit.client.0.smithi019.stdout:3/166: symlink d9/d26/l37 0 2024-02-22T15:50:28.057 INFO:tasks.workunit.client.0.smithi019.stdout:5/272: truncate d8/d11/f3e 1964289 0 2024-02-22T15:50:28.062 INFO:tasks.workunit.client.0.smithi019.stdout:7/214: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:28.064 INFO:tasks.workunit.client.0.smithi019.stdout:3/167: rename d9/d14/f31 to d9/d26/d32/f38 0 2024-02-22T15:50:28.068 INFO:tasks.workunit.client.0.smithi019.stdout:7/215: write d9/d26/f29 [748650,103084] 0 2024-02-22T15:50:28.069 INFO:tasks.workunit.client.0.smithi019.stdout:8/206: link d9/ce d9/c44 0 2024-02-22T15:50:28.071 INFO:tasks.workunit.client.0.smithi019.stdout:5/273: dwrite d8/dd/d19/f29 [0,4194304] 0 2024-02-22T15:50:28.074 INFO:tasks.workunit.client.0.smithi019.stdout:3/168: getdents d9/d26 0 2024-02-22T15:50:28.076 INFO:tasks.workunit.client.0.smithi019.stdout:5/274: creat d8/dd/d19/d3a/f5e x:0 0 0 2024-02-22T15:50:28.078 INFO:tasks.workunit.client.0.smithi019.stdout:0/108: sync 2024-02-22T15:50:28.078 INFO:tasks.workunit.client.0.smithi019.stdout:6/108: sync 2024-02-22T15:50:28.078 INFO:tasks.workunit.client.0.smithi019.stdout:9/203: sync 2024-02-22T15:50:28.079 INFO:tasks.workunit.client.0.smithi019.stdout:8/207: creat d9/d14/d17/f45 x:0 0 0 2024-02-22T15:50:28.081 INFO:tasks.workunit.client.0.smithi019.stdout:7/216: write f8 [3008823,60759] 0 2024-02-22T15:50:28.081 INFO:tasks.workunit.client.0.smithi019.stdout:5/275: symlink d8/dd/d19/d38/d47/l5f 0 2024-02-22T15:50:28.083 INFO:tasks.workunit.client.0.smithi019.stdout:6/109: mknod d6/c1f 0 2024-02-22T15:50:28.085 INFO:tasks.workunit.client.0.smithi019.stdout:9/204: mkdir d2/d1f/d2e/d48/d36/d4a 0 2024-02-22T15:50:28.087 INFO:tasks.workunit.client.0.smithi019.stdout:9/205: chown d2/c5 300843 1 2024-02-22T15:50:28.088 INFO:tasks.workunit.client.0.smithi019.stdout:7/217: rmdir d9 39 2024-02-22T15:50:28.089 INFO:tasks.workunit.client.0.smithi019.stdout:0/109: dread d4/d9/f11 [0,4194304] 0 2024-02-22T15:50:28.090 INFO:tasks.workunit.client.0.smithi019.stdout:8/208: dwrite d9/d14/d17/f40 [0,4194304] 0 2024-02-22T15:50:28.095 INFO:tasks.workunit.client.0.smithi019.stdout:9/206: rename d2/d7/f2a to d2/d1f/f4b 0 2024-02-22T15:50:28.097 INFO:tasks.workunit.client.0.smithi019.stdout:7/218: dread d9/f21 [0,4194304] 0 2024-02-22T15:50:28.101 INFO:tasks.workunit.client.0.smithi019.stdout:0/110: creat d4/d9/f26 x:0 0 0 2024-02-22T15:50:28.102 INFO:tasks.workunit.client.0.smithi019.stdout:8/209: symlink d9/d14/l46 0 2024-02-22T15:50:28.102 INFO:tasks.workunit.client.0.smithi019.stdout:0/111: write d4/d9/f13 [189092,110330] 0 2024-02-22T15:50:28.105 INFO:tasks.workunit.client.0.smithi019.stdout:9/207: chown d2/d1f/d2e/d48/c34 1991 1 2024-02-22T15:50:28.107 INFO:tasks.workunit.client.0.smithi019.stdout:5/276: link d8/dd/d19/d3a/d46/c52 d8/d11/c60 0 2024-02-22T15:50:28.110 INFO:tasks.workunit.client.0.smithi019.stdout:0/112: dread d4/d9/f17 [0,4194304] 0 2024-02-22T15:50:28.112 INFO:tasks.workunit.client.0.smithi019.stdout:0/113: chown f2 34379 1 2024-02-22T15:50:28.114 INFO:tasks.workunit.client.0.smithi019.stdout:6/110: dread d6/f10 [0,4194304] 0 2024-02-22T15:50:28.115 INFO:tasks.workunit.client.0.smithi019.stdout:7/219: dwrite d9/f23 [0,4194304] 0 2024-02-22T15:50:28.116 INFO:tasks.workunit.client.0.smithi019.stdout:0/114: creat d4/f27 x:0 0 0 2024-02-22T15:50:28.117 INFO:tasks.workunit.client.0.smithi019.stdout:6/111: rmdir d6 39 2024-02-22T15:50:28.118 INFO:tasks.workunit.client.0.smithi019.stdout:9/208: dwrite d2/f30 [0,4194304] 0 2024-02-22T15:50:28.118 INFO:tasks.workunit.client.0.smithi019.stdout:5/277: truncate d8/f2d 2475528 0 2024-02-22T15:50:28.119 INFO:tasks.workunit.client.0.smithi019.stdout:7/220: mknod d9/d3a/c47 0 2024-02-22T15:50:28.120 INFO:tasks.workunit.client.0.smithi019.stdout:0/115: fsync d4/d9/f11 0 2024-02-22T15:50:28.122 INFO:tasks.workunit.client.0.smithi019.stdout:9/209: mknod d2/d7/dc/d27/c4c 0 2024-02-22T15:50:28.124 INFO:tasks.workunit.client.0.smithi019.stdout:7/221: creat d9/d38/d41/f48 x:0 0 0 2024-02-22T15:50:28.125 INFO:tasks.workunit.client.0.smithi019.stdout:0/116: creat d4/d23/f28 x:0 0 0 2024-02-22T15:50:28.127 INFO:tasks.workunit.client.0.smithi019.stdout:5/278: dwrite d8/f25 [4194304,4194304] 0 2024-02-22T15:50:28.129 INFO:tasks.workunit.client.0.smithi019.stdout:5/279: fsync d8/dd/d19/f29 0 2024-02-22T15:50:28.132 INFO:tasks.workunit.client.0.smithi019.stdout:9/210: link d2/d1f/d2e/d48/d36/f43 d2/d1f/d2e/d48/f4d 0 2024-02-22T15:50:28.134 INFO:tasks.workunit.client.0.smithi019.stdout:6/112: write d6/fe [1015319,46837] 0 2024-02-22T15:50:28.135 INFO:tasks.workunit.client.0.smithi019.stdout:5/280: rmdir d8/dd/d19/d3a/d46 39 2024-02-22T15:50:28.138 INFO:tasks.workunit.client.0.smithi019.stdout:9/211: creat d2/d1f/d2e/d48/f4e x:0 0 0 2024-02-22T15:50:28.138 INFO:tasks.workunit.client.0.smithi019.stdout:6/113: fsync d6/fb 0 2024-02-22T15:50:28.139 INFO:tasks.workunit.client.0.smithi019.stdout:6/114: dread - d6/f1c zero size 2024-02-22T15:50:28.143 INFO:tasks.workunit.client.0.smithi019.stdout:6/115: creat d6/f20 x:0 0 0 2024-02-22T15:50:28.144 INFO:tasks.workunit.client.0.smithi019.stdout:9/212: fsync d2/d7/f1b 0 2024-02-22T15:50:28.145 INFO:tasks.workunit.client.0.smithi019.stdout:6/116: symlink d6/l21 0 2024-02-22T15:50:28.146 INFO:tasks.workunit.client.0.smithi019.stdout:9/213: creat d2/f4f x:0 0 0 2024-02-22T15:50:28.149 INFO:tasks.workunit.client.0.smithi019.stdout:9/214: rename d2/d7/d14/d2c to d2/d1f/d2e/d48/d50 0 2024-02-22T15:50:28.150 INFO:tasks.workunit.client.0.smithi019.stdout:9/215: write d2/d7/dc/f49 [160104,50101] 0 2024-02-22T15:50:28.163 INFO:tasks.workunit.client.0.smithi019.stdout:5/281: chown d8/f2d 489 1 2024-02-22T15:50:28.164 INFO:tasks.workunit.client.0.smithi019.stdout:7/222: dread d9/f16 [0,4194304] 0 2024-02-22T15:50:28.171 INFO:tasks.workunit.client.0.smithi019.stdout:7/223: dwrite d9/d26/f33 [0,4194304] 0 2024-02-22T15:50:28.172 INFO:tasks.workunit.client.0.smithi019.stdout:7/224: stat d9/d26/f32 0 2024-02-22T15:50:28.172 INFO:tasks.workunit.client.0.smithi019.stdout:6/117: dread d6/fc [0,4194304] 0 2024-02-22T15:50:28.174 INFO:tasks.workunit.client.0.smithi019.stdout:7/225: rmdir d9/d38/d41 39 2024-02-22T15:50:28.175 INFO:tasks.workunit.client.0.smithi019.stdout:6/118: creat d6/f22 x:0 0 0 2024-02-22T15:50:28.176 INFO:tasks.workunit.client.0.smithi019.stdout:7/226: unlink d9/l1c 0 2024-02-22T15:50:28.178 INFO:tasks.workunit.client.0.smithi019.stdout:6/119: symlink d6/l23 0 2024-02-22T15:50:28.179 INFO:tasks.workunit.client.0.smithi019.stdout:7/227: dread d9/f19 [0,4194304] 0 2024-02-22T15:50:28.181 INFO:tasks.workunit.client.0.smithi019.stdout:7/228: mkdir d9/d3a/d49 0 2024-02-22T15:50:28.186 INFO:tasks.workunit.client.0.smithi019.stdout:6/120: rmdir d6 39 2024-02-22T15:50:28.188 INFO:tasks.workunit.client.0.smithi019.stdout:6/121: readlink d6/l1e 0 2024-02-22T15:50:28.189 INFO:tasks.workunit.client.0.smithi019.stdout:6/122: write d6/f9 [2174846,22268] 0 2024-02-22T15:50:28.190 INFO:tasks.workunit.client.0.smithi019.stdout:7/229: dwrite d9/f19 [0,4194304] 0 2024-02-22T15:50:28.190 INFO:tasks.workunit.client.0.smithi019.stdout:6/123: chown d6/f1c 228527599 1 2024-02-22T15:50:28.192 INFO:tasks.workunit.client.0.smithi019.stdout:7/230: chown d9/d38/f3c 10 1 2024-02-22T15:50:28.194 INFO:tasks.workunit.client.0.smithi019.stdout:7/231: chown d9/c12 27937824 1 2024-02-22T15:50:28.196 INFO:tasks.workunit.client.0.smithi019.stdout:6/124: dread d6/fe [0,4194304] 0 2024-02-22T15:50:28.197 INFO:tasks.workunit.client.0.smithi019.stdout:7/232: symlink d9/d38/d41/l4a 0 2024-02-22T15:50:28.202 INFO:tasks.workunit.client.0.smithi019.stdout:7/233: rename d9/c39 to d9/c4b 0 2024-02-22T15:50:28.203 INFO:tasks.workunit.client.0.smithi019.stdout:6/125: dwrite d6/f1c [0,4194304] 0 2024-02-22T15:50:28.212 INFO:tasks.workunit.client.0.smithi019.stdout:7/234: dwrite d9/d38/f3c [0,4194304] 0 2024-02-22T15:50:28.218 INFO:tasks.workunit.client.0.smithi019.stdout:7/235: unlink d9/d3a/f3b 0 2024-02-22T15:50:28.218 INFO:tasks.workunit.client.0.smithi019.stdout:7/236: stat d9/d26 0 2024-02-22T15:50:28.218 INFO:tasks.workunit.client.0.smithi019.stdout:7/237: truncate d9/d26/f2e 1485759 0 2024-02-22T15:50:28.219 INFO:tasks.workunit.client.0.smithi019.stdout:6/126: truncate d6/f10 2087569 0 2024-02-22T15:50:28.221 INFO:tasks.workunit.client.0.smithi019.stdout:6/127: symlink d6/l24 0 2024-02-22T15:50:28.222 INFO:tasks.workunit.client.0.smithi019.stdout:6/128: creat d6/f25 x:0 0 0 2024-02-22T15:50:28.223 INFO:tasks.workunit.client.0.smithi019.stdout:6/129: fsync d6/f22 0 2024-02-22T15:50:28.223 INFO:tasks.workunit.client.0.smithi019.stdout:7/238: dread d9/d26/f2e [0,4194304] 0 2024-02-22T15:50:28.225 INFO:tasks.workunit.client.0.smithi019.stdout:6/130: mknod d6/c26 0 2024-02-22T15:50:28.231 INFO:tasks.workunit.client.0.smithi019.stdout:7/239: dwrite d9/d26/f2b [0,4194304] 0 2024-02-22T15:50:28.240 INFO:tasks.workunit.client.0.smithi019.stdout:7/240: dwrite d9/f21 [0,4194304] 0 2024-02-22T15:50:28.243 INFO:tasks.workunit.client.0.smithi019.stdout:7/241: mkdir d9/d4c 0 2024-02-22T15:50:28.249 INFO:tasks.workunit.client.0.smithi019.stdout:2/153: sync 2024-02-22T15:50:28.249 INFO:tasks.workunit.client.0.smithi019.stdout:1/154: sync 2024-02-22T15:50:28.250 INFO:tasks.workunit.client.0.smithi019.stdout:7/242: creat d9/f4d x:0 0 0 2024-02-22T15:50:28.251 INFO:tasks.workunit.client.0.smithi019.stdout:7/243: write f8 [50437,60385] 0 2024-02-22T15:50:28.253 INFO:tasks.workunit.client.0.smithi019.stdout:2/154: creat d1/d17/f30 x:0 0 0 2024-02-22T15:50:28.255 INFO:tasks.workunit.client.0.smithi019.stdout:7/244: creat d9/d38/f4e x:0 0 0 2024-02-22T15:50:28.258 INFO:tasks.workunit.client.0.smithi019.stdout:7/245: dread d9/f24 [0,4194304] 0 2024-02-22T15:50:28.260 INFO:tasks.workunit.client.0.smithi019.stdout:2/155: link d1/f26 d1/de/f31 0 2024-02-22T15:50:28.262 INFO:tasks.workunit.client.0.smithi019.stdout:2/156: write d1/de/ff [4241754,30658] 0 2024-02-22T15:50:28.264 INFO:tasks.workunit.client.0.smithi019.stdout:2/157: dread - d1/f26 zero size 2024-02-22T15:50:28.267 INFO:tasks.workunit.client.0.smithi019.stdout:2/158: fdatasync d1/f1f 0 2024-02-22T15:50:28.269 INFO:tasks.workunit.client.0.smithi019.stdout:7/246: dwrite d9/d38/f4e [0,4194304] 0 2024-02-22T15:50:28.270 INFO:tasks.workunit.client.0.smithi019.stdout:1/155: dwrite d8/fd [4194304,4194304] 0 2024-02-22T15:50:28.270 INFO:tasks.workunit.client.0.smithi019.stdout:2/159: mknod d1/d17/c32 0 2024-02-22T15:50:28.280 INFO:tasks.workunit.client.0.smithi019.stdout:7/247: dwrite d9/d26/f32 [0,4194304] 0 2024-02-22T15:50:28.282 INFO:tasks.workunit.client.0.smithi019.stdout:2/160: mkdir d1/de/d33 0 2024-02-22T15:50:28.284 INFO:tasks.workunit.client.0.smithi019.stdout:1/156: creat d8/d23/d33/f37 x:0 0 0 2024-02-22T15:50:28.285 INFO:tasks.workunit.client.0.smithi019.stdout:1/157: chown d8 0 1 2024-02-22T15:50:28.285 INFO:tasks.workunit.client.0.smithi019.stdout:7/248: getdents d9/d38/d41/d45 0 2024-02-22T15:50:28.286 INFO:tasks.workunit.client.0.smithi019.stdout:2/161: creat d1/d17/d1c/f34 x:0 0 0 2024-02-22T15:50:28.288 INFO:tasks.workunit.client.0.smithi019.stdout:7/249: symlink d9/d3a/l4f 0 2024-02-22T15:50:28.289 INFO:tasks.workunit.client.0.smithi019.stdout:7/250: write f8 [3567084,126375] 0 2024-02-22T15:50:28.290 INFO:tasks.workunit.client.0.smithi019.stdout:2/162: unlink d1/d17/c32 0 2024-02-22T15:50:28.290 INFO:tasks.workunit.client.0.smithi019.stdout:2/163: chown d1/d17/d19 344 1 2024-02-22T15:50:28.294 INFO:tasks.workunit.client.0.smithi019.stdout:1/158: link d8/d2a/c36 d8/d18/c38 0 2024-02-22T15:50:28.294 INFO:tasks.workunit.client.0.smithi019.stdout:2/164: creat d1/de/d15/f35 x:0 0 0 2024-02-22T15:50:28.298 INFO:tasks.workunit.client.0.smithi019.stdout:7/251: dwrite d9/f4d [0,4194304] 0 2024-02-22T15:50:28.306 INFO:tasks.workunit.client.0.smithi019.stdout:1/159: rmdir d8/d23 39 2024-02-22T15:50:28.307 INFO:tasks.workunit.client.0.smithi019.stdout:2/165: dwrite d1/f2e [0,4194304] 0 2024-02-22T15:50:28.309 INFO:tasks.workunit.client.0.smithi019.stdout:1/160: rmdir d8 39 2024-02-22T15:50:28.312 INFO:tasks.workunit.client.0.smithi019.stdout:2/166: unlink d1/de/f2d 0 2024-02-22T15:50:28.313 INFO:tasks.workunit.client.0.smithi019.stdout:2/167: write d1/f12 [2602383,115489] 0 2024-02-22T15:50:28.314 INFO:tasks.workunit.client.0.smithi019.stdout:7/252: write d9/d26/f2e [2418827,30238] 0 2024-02-22T15:50:28.315 INFO:tasks.workunit.client.0.smithi019.stdout:7/253: fsync d9/d38/f4e 0 2024-02-22T15:50:28.317 INFO:tasks.workunit.client.0.smithi019.stdout:2/168: write d1/de/d15/f35 [97015,63250] 0 2024-02-22T15:50:28.320 INFO:tasks.workunit.client.0.smithi019.stdout:2/169: creat d1/d17/d21/f36 x:0 0 0 2024-02-22T15:50:28.322 INFO:tasks.workunit.client.0.smithi019.stdout:1/161: dread d8/f1a [0,4194304] 0 2024-02-22T15:50:28.322 INFO:tasks.workunit.client.0.smithi019.stdout:7/254: dwrite f6 [0,4194304] 0 2024-02-22T15:50:28.330 INFO:tasks.workunit.client.0.smithi019.stdout:2/170: getdents d1/d17/d1c 0 2024-02-22T15:50:28.333 INFO:tasks.workunit.client.0.smithi019.stdout:2/171: creat d1/d17/d19/f37 x:0 0 0 2024-02-22T15:50:28.335 INFO:tasks.workunit.client.0.smithi019.stdout:7/255: dread d9/d26/f2e [0,4194304] 0 2024-02-22T15:50:28.335 INFO:tasks.workunit.client.0.smithi019.stdout:1/162: dwrite d8/d23/d33/f37 [0,4194304] 0 2024-02-22T15:50:28.336 INFO:tasks.workunit.client.0.smithi019.stdout:7/256: write d9/f34 [1241255,61387] 0 2024-02-22T15:50:28.337 INFO:tasks.workunit.client.0.smithi019.stdout:2/172: mkdir d1/d17/d38 0 2024-02-22T15:50:28.337 INFO:tasks.workunit.client.0.smithi019.stdout:2/173: stat d1/f26 0 2024-02-22T15:50:28.340 INFO:tasks.workunit.client.0.smithi019.stdout:7/257: mknod d9/d38/d41/c50 0 2024-02-22T15:50:28.342 INFO:tasks.workunit.client.0.smithi019.stdout:2/174: rename d1/l7 to d1/de/d15/l39 0 2024-02-22T15:50:28.343 INFO:tasks.workunit.client.0.smithi019.stdout:7/258: mknod d9/d38/d41/c51 0 2024-02-22T15:50:28.344 INFO:tasks.workunit.client.0.smithi019.stdout:1/163: dwrite d8/d18/f21 [0,4194304] 0 2024-02-22T15:50:28.348 INFO:tasks.workunit.client.0.smithi019.stdout:1/164: creat d8/d15/d34/f39 x:0 0 0 2024-02-22T15:50:28.349 INFO:tasks.workunit.client.0.smithi019.stdout:7/259: creat d9/d38/f52 x:0 0 0 2024-02-22T15:50:28.351 INFO:tasks.workunit.client.0.smithi019.stdout:2/175: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:28.353 INFO:tasks.workunit.client.0.smithi019.stdout:1/165: rename d8/d23/f35 to d8/d2a/f3a 0 2024-02-22T15:50:28.359 INFO:tasks.workunit.client.0.smithi019.stdout:1/166: rename d8/fc to d8/d18/f3b 0 2024-02-22T15:50:28.360 INFO:tasks.workunit.client.0.smithi019.stdout:7/260: dwrite d9/f34 [0,4194304] 0 2024-02-22T15:50:28.363 INFO:tasks.workunit.client.0.smithi019.stdout:7/261: chown d9/d26/c30 123877780 1 2024-02-22T15:50:28.363 INFO:tasks.workunit.client.0.smithi019.stdout:2/176: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:28.366 INFO:tasks.workunit.client.0.smithi019.stdout:7/262: chown d9/f19 0 1 2024-02-22T15:50:28.367 INFO:tasks.workunit.client.0.smithi019.stdout:7/263: stat d9/f23 0 2024-02-22T15:50:28.368 INFO:tasks.workunit.client.0.smithi019.stdout:2/177: write d1/de/f16 [1566291,65620] 0 2024-02-22T15:50:28.370 INFO:tasks.workunit.client.0.smithi019.stdout:7/264: readlink d9/d26/l43 0 2024-02-22T15:50:28.370 INFO:tasks.workunit.client.0.smithi019.stdout:2/178: write d1/d17/d19/f37 [452725,118534] 0 2024-02-22T15:50:28.372 INFO:tasks.workunit.client.0.smithi019.stdout:2/179: truncate d1/de/ff 4659107 0 2024-02-22T15:50:28.374 INFO:tasks.workunit.client.0.smithi019.stdout:7/265: rename d9/c4b to d9/c53 0 2024-02-22T15:50:28.378 INFO:tasks.workunit.client.0.smithi019.stdout:1/167: rmdir d8/d15/d34 39 2024-02-22T15:50:28.381 INFO:tasks.workunit.client.0.smithi019.stdout:1/168: rmdir d8/d23/d33 39 2024-02-22T15:50:28.384 INFO:tasks.workunit.client.0.smithi019.stdout:7/266: dwrite d9/d26/f37 [0,4194304] 0 2024-02-22T15:50:28.384 INFO:tasks.workunit.client.0.smithi019.stdout:2/180: dread d1/d17/d19/f37 [0,4194304] 0 2024-02-22T15:50:28.388 INFO:tasks.workunit.client.0.smithi019.stdout:2/181: creat d1/de/d15/f3a x:0 0 0 2024-02-22T15:50:28.394 INFO:tasks.workunit.client.0.smithi019.stdout:1/169: dwrite d8/d15/f26 [0,4194304] 0 2024-02-22T15:50:28.395 INFO:tasks.workunit.client.0.smithi019.stdout:7/267: dwrite d9/f23 [0,4194304] 0 2024-02-22T15:50:28.398 INFO:tasks.workunit.client.0.smithi019.stdout:2/182: rename d1/d17/d21/l29 to d1/d17/d21/l3b 0 2024-02-22T15:50:28.400 INFO:tasks.workunit.client.0.smithi019.stdout:1/170: rmdir d8/d15 39 2024-02-22T15:50:28.403 INFO:tasks.workunit.client.0.smithi019.stdout:2/183: symlink d1/de/d33/l3c 0 2024-02-22T15:50:28.405 INFO:tasks.workunit.client.0.smithi019.stdout:7/268: dwrite d9/f21 [0,4194304] 0 2024-02-22T15:50:28.405 INFO:tasks.workunit.client.0.smithi019.stdout:1/171: symlink d8/d23/d33/l3c 0 2024-02-22T15:50:28.407 INFO:tasks.workunit.client.0.smithi019.stdout:2/184: mkdir d1/de/d15/d3d 0 2024-02-22T15:50:28.408 INFO:tasks.workunit.client.0.smithi019.stdout:7/269: truncate d9/d26/f32 4225801 0 2024-02-22T15:50:28.412 INFO:tasks.workunit.client.0.smithi019.stdout:1/172: symlink d8/d15/l3d 0 2024-02-22T15:50:28.414 INFO:tasks.workunit.client.0.smithi019.stdout:2/185: creat d1/de/d15/d3d/f3e x:0 0 0 2024-02-22T15:50:28.415 INFO:tasks.workunit.client.0.smithi019.stdout:7/270: dread f8 [0,4194304] 0 2024-02-22T15:50:28.418 INFO:tasks.workunit.client.0.smithi019.stdout:1/173: creat d8/d15/d34/f3e x:0 0 0 2024-02-22T15:50:28.451 INFO:tasks.workunit.client.0.smithi019.stdout:4/137: sync 2024-02-22T15:50:28.452 INFO:tasks.workunit.client.0.smithi019.stdout:7/271: link d9/d26/c35 d9/d3a/c54 0 2024-02-22T15:50:28.452 INFO:tasks.workunit.client.0.smithi019.stdout:1/174: dread - d8/d2a/f2b zero size 2024-02-22T15:50:28.454 INFO:tasks.workunit.client.0.smithi019.stdout:4/138: dread - d2/db/d14/d23/f28 zero size 2024-02-22T15:50:28.455 INFO:tasks.workunit.client.0.smithi019.stdout:2/186: dwrite d1/de/d15/d3d/f3e [0,4194304] 0 2024-02-22T15:50:28.457 INFO:tasks.workunit.client.0.smithi019.stdout:7/272: symlink d9/d38/l55 0 2024-02-22T15:50:28.458 INFO:tasks.workunit.client.0.smithi019.stdout:2/187: symlink d1/d17/l3f 0 2024-02-22T15:50:28.459 INFO:tasks.workunit.client.0.smithi019.stdout:7/273: fdatasync d9/d38/d41/f48 0 2024-02-22T15:50:28.461 INFO:tasks.workunit.client.0.smithi019.stdout:7/274: fsync d9/f16 0 2024-02-22T15:50:28.462 INFO:tasks.workunit.client.0.smithi019.stdout:4/139: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:28.464 INFO:tasks.workunit.client.0.smithi019.stdout:4/140: write d2/db/d14/d23/d21/f2c [280300,109297] 0 2024-02-22T15:50:28.464 INFO:tasks.workunit.client.0.smithi019.stdout:7/275: mknod d9/d3a/c56 0 2024-02-22T15:50:28.466 INFO:tasks.workunit.client.0.smithi019.stdout:7/276: fsync d9/d38/f4e 0 2024-02-22T15:50:28.466 INFO:tasks.workunit.client.0.smithi019.stdout:4/141: dread - d2/db/d14/d23/d21/f29 zero size 2024-02-22T15:50:28.468 INFO:tasks.workunit.client.0.smithi019.stdout:7/277: creat d9/d38/d41/d45/f57 x:0 0 0 2024-02-22T15:50:28.469 INFO:tasks.workunit.client.0.smithi019.stdout:7/278: read - d9/d26/f31 zero size 2024-02-22T15:50:28.470 INFO:tasks.workunit.client.0.smithi019.stdout:4/142: rename d2/db/f20 to d2/db/d14/d23/d21/f2d 0 2024-02-22T15:50:28.471 INFO:tasks.workunit.client.0.smithi019.stdout:7/279: rmdir d9/d38/d41 39 2024-02-22T15:50:28.472 INFO:tasks.workunit.client.0.smithi019.stdout:7/280: stat d9/d38/d41/l4a 0 2024-02-22T15:50:28.473 INFO:tasks.workunit.client.0.smithi019.stdout:4/143: symlink d2/db/d14/d23/d21/l2e 0 2024-02-22T15:50:28.474 INFO:tasks.workunit.client.0.smithi019.stdout:7/281: chown d9/c28 37911950 1 2024-02-22T15:50:28.475 INFO:tasks.workunit.client.0.smithi019.stdout:7/282: chown d9/d4c 29 1 2024-02-22T15:50:28.478 INFO:tasks.workunit.client.0.smithi019.stdout:7/283: symlink d9/d38/d41/l58 0 2024-02-22T15:50:28.481 INFO:tasks.workunit.client.0.smithi019.stdout:7/284: creat d9/d38/f59 x:0 0 0 2024-02-22T15:50:28.482 INFO:tasks.workunit.client.0.smithi019.stdout:7/285: write d9/f1b [4954635,127252] 0 2024-02-22T15:50:28.487 INFO:tasks.workunit.client.0.smithi019.stdout:7/286: creat d9/d38/f5a x:0 0 0 2024-02-22T15:50:28.487 INFO:tasks.workunit.client.0.smithi019.stdout:4/144: truncate d2/db/d14/f18 4889477 0 2024-02-22T15:50:28.488 INFO:tasks.workunit.client.0.smithi019.stdout:4/145: stat d2/db/d14/d23/d21 0 2024-02-22T15:50:28.489 INFO:tasks.workunit.client.0.smithi019.stdout:7/287: mkdir d9/d4c/d5b 0 2024-02-22T15:50:28.496 INFO:tasks.workunit.client.0.smithi019.stdout:4/146: write d2/db/d14/d23/d21/f2d [4813909,87201] 0 2024-02-22T15:50:28.497 INFO:tasks.workunit.client.0.smithi019.stdout:7/288: truncate d9/d26/f29 457268 0 2024-02-22T15:50:28.499 INFO:tasks.workunit.client.0.smithi019.stdout:4/147: creat d2/db/d14/d23/f2f x:0 0 0 2024-02-22T15:50:28.500 INFO:tasks.workunit.client.0.smithi019.stdout:7/289: symlink d9/l5c 0 2024-02-22T15:50:28.501 INFO:tasks.workunit.client.0.smithi019.stdout:4/148: symlink d2/db/d14/d23/l30 0 2024-02-22T15:50:28.502 INFO:tasks.workunit.client.0.smithi019.stdout:7/290: mkdir d9/d3a/d49/d5d 0 2024-02-22T15:50:28.509 INFO:tasks.workunit.client.0.smithi019.stdout:4/149: dwrite d2/fa [0,4194304] 0 2024-02-22T15:50:28.512 INFO:tasks.workunit.client.0.smithi019.stdout:7/291: dwrite d9/d26/f32 [0,4194304] 0 2024-02-22T15:50:28.516 INFO:tasks.workunit.client.0.smithi019.stdout:4/150: dread d2/db/d14/f19 [0,4194304] 0 2024-02-22T15:50:28.520 INFO:tasks.workunit.client.0.smithi019.stdout:4/151: rmdir d2/db/d14/d23 39 2024-02-22T15:50:28.522 INFO:tasks.workunit.client.0.smithi019.stdout:4/152: chown d2/db/d14/d23/d21/f2d 14062 1 2024-02-22T15:50:28.525 INFO:tasks.workunit.client.0.smithi019.stdout:7/292: dread d9/f1d [0,4194304] 0 2024-02-22T15:50:28.525 INFO:tasks.workunit.client.0.smithi019.stdout:7/293: chown d9/d38 4 1 2024-02-22T15:50:28.526 INFO:tasks.workunit.client.0.smithi019.stdout:7/294: mknod d9/d26/c5e 0 2024-02-22T15:50:28.528 INFO:tasks.workunit.client.0.smithi019.stdout:7/295: creat d9/d38/d41/f5f x:0 0 0 2024-02-22T15:50:28.528 INFO:tasks.workunit.client.0.smithi019.stdout:7/296: read - d9/d26/f31 zero size 2024-02-22T15:50:28.531 INFO:tasks.workunit.client.0.smithi019.stdout:7/297: unlink d9/c36 0 2024-02-22T15:50:28.532 INFO:tasks.workunit.client.0.smithi019.stdout:4/153: truncate d2/fa 3928625 0 2024-02-22T15:50:28.533 INFO:tasks.workunit.client.0.smithi019.stdout:4/154: readlink d2/db/d14/d23/l1c 0 2024-02-22T15:50:28.534 INFO:tasks.workunit.client.0.smithi019.stdout:7/298: rename d9/d26/f37 to d9/d3a/f60 0 2024-02-22T15:50:28.535 INFO:tasks.workunit.client.0.smithi019.stdout:4/155: mkdir d2/d31 0 2024-02-22T15:50:28.543 INFO:tasks.workunit.client.0.smithi019.stdout:4/156: read - d2/db/d14/d23/f2f zero size 2024-02-22T15:50:28.544 INFO:tasks.workunit.client.0.smithi019.stdout:4/157: fsync d2/db/f26 0 2024-02-22T15:50:28.547 INFO:tasks.workunit.client.0.smithi019.stdout:4/158: getdents d2/db/d14/d23 0 2024-02-22T15:50:28.547 INFO:tasks.workunit.client.0.smithi019.stdout:4/159: chown d2/c5 61807 1 2024-02-22T15:50:28.548 INFO:tasks.workunit.client.0.smithi019.stdout:4/160: write d2/db/d14/f27 [370473,112456] 0 2024-02-22T15:50:28.550 INFO:tasks.workunit.client.0.smithi019.stdout:4/161: fdatasync d2/db/d14/f19 0 2024-02-22T15:50:28.553 INFO:tasks.workunit.client.0.smithi019.stdout:4/162: dread d2/db/d14/f19 [0,4194304] 0 2024-02-22T15:50:28.555 INFO:tasks.workunit.client.0.smithi019.stdout:4/163: creat d2/f32 x:0 0 0 2024-02-22T15:50:28.662 INFO:tasks.workunit.client.0.smithi019.stdout:8/210: sync 2024-02-22T15:50:28.662 INFO:tasks.workunit.client.0.smithi019.stdout:3/169: sync 2024-02-22T15:50:28.664 INFO:tasks.workunit.client.0.smithi019.stdout:3/170: mkdir d9/d26/d2d/d39 0 2024-02-22T15:50:28.664 INFO:tasks.workunit.client.0.smithi019.stdout:8/211: creat d9/d14/d17/f47 x:0 0 0 2024-02-22T15:50:28.667 INFO:tasks.workunit.client.0.smithi019.stdout:3/171: rename d9/d26/d32/f38 to d9/d26/d2d/f3a 0 2024-02-22T15:50:28.669 INFO:tasks.workunit.client.0.smithi019.stdout:8/212: link f5 d9/d14/d17/d1c/f48 0 2024-02-22T15:50:28.670 INFO:tasks.workunit.client.0.smithi019.stdout:3/172: rename d9/lf to d9/d26/d2d/d39/l3b 0 2024-02-22T15:50:28.673 INFO:tasks.workunit.client.0.smithi019.stdout:8/213: rename d9/d14/d1f/d20/c37 to d9/d14/d17/d3d/c49 0 2024-02-22T15:50:28.674 INFO:tasks.workunit.client.0.smithi019.stdout:3/173: truncate d9/d11/f1e 298417 0 2024-02-22T15:50:28.674 INFO:tasks.workunit.client.0.smithi019.stdout:8/214: chown d9/d14/d17/f25 14962 1 2024-02-22T15:50:28.676 INFO:tasks.workunit.client.0.smithi019.stdout:8/215: unlink f1 0 2024-02-22T15:50:28.679 INFO:tasks.workunit.client.0.smithi019.stdout:3/174: rename d9/d16/f1d to d9/d14/f3c 0 2024-02-22T15:50:28.685 INFO:tasks.workunit.client.0.smithi019.stdout:8/216: creat d9/d14/d17/f4a x:0 0 0 2024-02-22T15:50:28.694 INFO:tasks.workunit.client.0.smithi019.stdout:8/217: truncate d9/f12 3368458 0 2024-02-22T15:50:28.695 INFO:tasks.workunit.client.0.smithi019.stdout:8/218: readlink d9/d14/d17/l1b 0 2024-02-22T15:50:28.698 INFO:tasks.workunit.client.0.smithi019.stdout:8/219: mknod d9/d14/d1f/c4b 0 2024-02-22T15:50:28.698 INFO:tasks.workunit.client.0.smithi019.stdout:3/175: dwrite d9/d11/fd [4194304,4194304] 0 2024-02-22T15:50:28.702 INFO:tasks.workunit.client.0.smithi019.stdout:3/176: creat d9/d22/d27/f3d x:0 0 0 2024-02-22T15:50:28.703 INFO:tasks.workunit.client.0.smithi019.stdout:8/220: dread d9/d14/d1f/d20/f2a [0,4194304] 0 2024-02-22T15:50:28.705 INFO:tasks.workunit.client.0.smithi019.stdout:8/221: dread - d9/d14/d1f/f21 zero size 2024-02-22T15:50:28.705 INFO:tasks.workunit.client.0.smithi019.stdout:3/177: write d9/d14/f3c [2879185,24556] 0 2024-02-22T15:50:28.711 INFO:tasks.workunit.client.0.smithi019.stdout:8/222: dread d9/d14/f36 [0,4194304] 0 2024-02-22T15:50:28.712 INFO:tasks.workunit.client.0.smithi019.stdout:3/178: getdents d9/d22/d27 0 2024-02-22T15:50:28.714 INFO:tasks.workunit.client.0.smithi019.stdout:5/282: sync 2024-02-22T15:50:28.714 INFO:tasks.workunit.client.0.smithi019.stdout:0/117: sync 2024-02-22T15:50:28.714 INFO:tasks.workunit.client.0.smithi019.stdout:9/216: sync 2024-02-22T15:50:28.714 INFO:tasks.workunit.client.0.smithi019.stdout:3/179: write d9/d26/d32/f33 [1899390,124350] 0 2024-02-22T15:50:28.716 INFO:tasks.workunit.client.0.smithi019.stdout:0/118: fdatasync d4/d9/f11 0 2024-02-22T15:50:28.720 INFO:tasks.workunit.client.0.smithi019.stdout:5/283: creat d8/dd/d19/d3a/d46/f61 x:0 0 0 2024-02-22T15:50:28.721 INFO:tasks.workunit.client.0.smithi019.stdout:9/217: rename d2/d7/dc/d27/c4c to d2/d1f/d2e/c51 0 2024-02-22T15:50:28.724 INFO:tasks.workunit.client.0.smithi019.stdout:8/223: dread d9/d14/d17/f40 [0,4194304] 0 2024-02-22T15:50:28.729 INFO:tasks.workunit.client.0.smithi019.stdout:3/180: dwrite d9/d16/f2b [0,4194304] 0 2024-02-22T15:50:28.730 INFO:tasks.workunit.client.0.smithi019.stdout:0/119: dwrite f2 [4194304,4194304] 0 2024-02-22T15:50:28.732 INFO:tasks.workunit.client.0.smithi019.stdout:0/120: readlink d4/l12 0 2024-02-22T15:50:28.736 INFO:tasks.workunit.client.0.smithi019.stdout:8/224: write d9/d14/d17/f4a [224125,100635] 0 2024-02-22T15:50:28.736 INFO:tasks.workunit.client.0.smithi019.stdout:5/284: dwrite d8/d11/f3b [0,4194304] 0 2024-02-22T15:50:28.736 INFO:tasks.workunit.client.0.smithi019.stdout:9/218: truncate d2/d1f/d2e/f42 2332421 0 2024-02-22T15:50:28.740 INFO:tasks.workunit.client.0.smithi019.stdout:0/121: mknod d4/d5/c29 0 2024-02-22T15:50:28.742 INFO:tasks.workunit.client.0.smithi019.stdout:5/285: symlink d8/d11/l62 0 2024-02-22T15:50:28.743 INFO:tasks.workunit.client.0.smithi019.stdout:3/181: unlink d9/d11/c18 0 2024-02-22T15:50:28.744 INFO:tasks.workunit.client.0.smithi019.stdout:8/225: mkdir d9/d14/d1f/d4c 0 2024-02-22T15:50:28.745 INFO:tasks.workunit.client.0.smithi019.stdout:0/122: mknod d4/d5/dc/c2a 0 2024-02-22T15:50:28.751 INFO:tasks.workunit.client.0.smithi019.stdout:9/219: rename d2/d1f/d2e/d48/d50 to d2/d7/d52 0 2024-02-22T15:50:28.751 INFO:tasks.workunit.client.0.smithi019.stdout:0/123: symlink d4/d5/l2b 0 2024-02-22T15:50:28.752 INFO:tasks.workunit.client.0.smithi019.stdout:0/124: fdatasync d4/d9/f13 0 2024-02-22T15:50:28.753 INFO:tasks.workunit.client.0.smithi019.stdout:3/182: unlink d9/d26/d2d/d39/l3b 0 2024-02-22T15:50:28.753 INFO:tasks.workunit.client.0.smithi019.stdout:8/226: symlink d9/d14/d1f/d4c/l4d 0 2024-02-22T15:50:28.755 INFO:tasks.workunit.client.0.smithi019.stdout:3/183: creat d9/d22/d27/f3e x:0 0 0 2024-02-22T15:50:28.758 INFO:tasks.workunit.client.0.smithi019.stdout:3/184: mkdir d9/d26/d3f 0 2024-02-22T15:50:28.760 INFO:tasks.workunit.client.0.smithi019.stdout:8/227: creat d9/d14/d1f/f4e x:0 0 0 2024-02-22T15:50:28.760 INFO:tasks.workunit.client.0.smithi019.stdout:3/185: write d9/d22/d27/f3e [935641,91645] 0 2024-02-22T15:50:28.761 INFO:tasks.workunit.client.0.smithi019.stdout:3/186: write d9/d11/f1e [510268,72010] 0 2024-02-22T15:50:28.763 INFO:tasks.workunit.client.0.smithi019.stdout:3/187: write f3 [10483003,51029] 0 2024-02-22T15:50:28.764 INFO:tasks.workunit.client.0.smithi019.stdout:8/228: symlink d9/d14/d17/d3d/d3f/l4f 0 2024-02-22T15:50:28.772 INFO:tasks.workunit.client.0.smithi019.stdout:8/229: creat d9/d14/d17/f50 x:0 0 0 2024-02-22T15:50:28.772 INFO:tasks.workunit.client.0.smithi019.stdout:9/220: dread d2/d7/f1b [0,4194304] 0 2024-02-22T15:50:28.772 INFO:tasks.workunit.client.0.smithi019.stdout:8/230: chown d9/d14/d1f/d20/c31 0 1 2024-02-22T15:50:28.772 INFO:tasks.workunit.client.0.smithi019.stdout:9/221: chown d2/d7/d14/l3c 45 1 2024-02-22T15:50:28.774 INFO:tasks.workunit.client.0.smithi019.stdout:9/222: readlink d2/d7/d14/l3c 0 2024-02-22T15:50:28.775 INFO:tasks.workunit.client.0.smithi019.stdout:8/231: creat d9/d14/d17/d1c/f51 x:0 0 0 2024-02-22T15:50:28.777 INFO:tasks.workunit.client.0.smithi019.stdout:2/188: sync 2024-02-22T15:50:28.777 INFO:tasks.workunit.client.0.smithi019.stdout:6/131: sync 2024-02-22T15:50:28.777 INFO:tasks.workunit.client.0.smithi019.stdout:7/299: sync 2024-02-22T15:50:28.777 INFO:tasks.workunit.client.0.smithi019.stdout:1/175: sync 2024-02-22T15:50:28.777 INFO:tasks.workunit.client.0.smithi019.stdout:4/164: sync 2024-02-22T15:50:28.778 INFO:tasks.workunit.client.0.smithi019.stdout:6/132: readlink d6/la 0 2024-02-22T15:50:28.779 INFO:tasks.workunit.client.0.smithi019.stdout:0/125: truncate f2 3914044 0 2024-02-22T15:50:28.781 INFO:tasks.workunit.client.0.smithi019.stdout:4/165: chown d2/fd 11332804 1 2024-02-22T15:50:28.781 INFO:tasks.workunit.client.0.smithi019.stdout:5/286: dwrite d8/f2d [0,4194304] 0 2024-02-22T15:50:28.785 INFO:tasks.workunit.client.0.smithi019.stdout:6/133: truncate d6/f20 298191 0 2024-02-22T15:50:28.785 INFO:tasks.workunit.client.0.smithi019.stdout:7/300: unlink d9/d38/d41/l4a 0 2024-02-22T15:50:28.786 INFO:tasks.workunit.client.0.smithi019.stdout:1/176: rename d8/d15/d34/f3e to d8/d2a/f3f 0 2024-02-22T15:50:28.786 INFO:tasks.workunit.client.0.smithi019.stdout:6/134: fsync d6/f17 0 2024-02-22T15:50:28.788 INFO:tasks.workunit.client.0.smithi019.stdout:8/232: write d9/d14/d17/d1c/f48 [7504672,27732] 0 2024-02-22T15:50:28.790 INFO:tasks.workunit.client.0.smithi019.stdout:4/166: fdatasync d2/db/f26 0 2024-02-22T15:50:28.790 INFO:tasks.workunit.client.0.smithi019.stdout:1/177: readlink d8/d15/l22 0 2024-02-22T15:50:28.790 INFO:tasks.workunit.client.0.smithi019.stdout:0/126: mkdir d4/d22/d2c 0 2024-02-22T15:50:28.791 INFO:tasks.workunit.client.0.smithi019.stdout:5/287: creat d8/d11/d21/d2c/f63 x:0 0 0 2024-02-22T15:50:28.797 INFO:tasks.workunit.client.0.smithi019.stdout:7/301: mkdir d9/d38/d41/d45/d61 0 2024-02-22T15:50:28.797 INFO:tasks.workunit.client.0.smithi019.stdout:4/167: dread - d2/db/f26 zero size 2024-02-22T15:50:28.801 INFO:tasks.workunit.client.0.smithi019.stdout:8/233: write d9/d14/d1f/f21 [634521,50757] 0 2024-02-22T15:50:28.802 INFO:tasks.workunit.client.0.smithi019.stdout:1/178: rename d8/fb to d8/d2a/f40 0 2024-02-22T15:50:28.804 INFO:tasks.workunit.client.0.smithi019.stdout:7/302: stat d9/d26/f32 0 2024-02-22T15:50:28.804 INFO:tasks.workunit.client.0.smithi019.stdout:2/189: dwrite d1/d17/f30 [0,4194304] 0 2024-02-22T15:50:28.804 INFO:tasks.workunit.client.0.smithi019.stdout:3/188: dwrite d9/d11/f17 [0,4194304] 0 2024-02-22T15:50:28.805 INFO:tasks.workunit.client.0.smithi019.stdout:5/288: creat d8/dd/d19/d38/d47/f64 x:0 0 0 2024-02-22T15:50:28.807 INFO:tasks.workunit.client.0.smithi019.stdout:4/168: fsync d2/db/d14/d23/d21/f2c 0 2024-02-22T15:50:28.808 INFO:tasks.workunit.client.0.smithi019.stdout:8/234: creat d9/d14/f52 x:0 0 0 2024-02-22T15:50:28.811 INFO:tasks.workunit.client.0.smithi019.stdout:3/189: readlink d9/d26/l28 0 2024-02-22T15:50:28.813 INFO:tasks.workunit.client.0.smithi019.stdout:6/135: dwrite d6/f25 [0,4194304] 0 2024-02-22T15:50:28.815 INFO:tasks.workunit.client.0.smithi019.stdout:5/289: creat d8/d11/f65 x:0 0 0 2024-02-22T15:50:28.816 INFO:tasks.workunit.client.0.smithi019.stdout:6/136: write d6/f14 [4660887,11294] 0 2024-02-22T15:50:28.819 INFO:tasks.workunit.client.0.smithi019.stdout:7/303: dread d9/d26/f2b [0,4194304] 0 2024-02-22T15:50:28.819 INFO:tasks.workunit.client.0.smithi019.stdout:6/137: dread - d6/f1d zero size 2024-02-22T15:50:28.822 INFO:tasks.workunit.client.0.smithi019.stdout:0/127: truncate f2 61951 0 2024-02-22T15:50:28.822 INFO:tasks.workunit.client.0.smithi019.stdout:9/223: dread d2/d7/dc/f24 [4194304,4194304] 0 2024-02-22T15:50:28.823 INFO:tasks.workunit.client.0.smithi019.stdout:3/190: unlink d9/d16/f29 0 2024-02-22T15:50:28.826 INFO:tasks.workunit.client.0.smithi019.stdout:5/290: symlink d8/d11/l66 0 2024-02-22T15:50:28.828 INFO:tasks.workunit.client.0.smithi019.stdout:4/169: unlink d2/l7 0 2024-02-22T15:50:28.829 INFO:tasks.workunit.client.0.smithi019.stdout:5/291: chown d8/dd/d19 379 1 2024-02-22T15:50:28.831 INFO:tasks.workunit.client.0.smithi019.stdout:6/138: symlink d6/l27 0 2024-02-22T15:50:28.834 INFO:tasks.workunit.client.0.smithi019.stdout:0/128: mkdir d4/db/d2d 0 2024-02-22T15:50:28.836 INFO:tasks.workunit.client.0.smithi019.stdout:7/304: dwrite d9/d26/f33 [4194304,4194304] 0 2024-02-22T15:50:28.837 INFO:tasks.workunit.client.0.smithi019.stdout:5/292: chown d8/dd/d19/d38/f5a 0 1 2024-02-22T15:50:28.837 INFO:tasks.workunit.client.0.smithi019.stdout:6/139: unlink d6/c1a 0 2024-02-22T15:50:28.843 INFO:tasks.workunit.client.0.smithi019.stdout:7/305: write d9/d38/f59 [375536,49273] 0 2024-02-22T15:50:28.843 INFO:tasks.workunit.client.0.smithi019.stdout:0/129: symlink d4/d5/l2e 0 2024-02-22T15:50:28.845 INFO:tasks.workunit.client.0.smithi019.stdout:6/140: chown d6/f14 2201923 1 2024-02-22T15:50:28.847 INFO:tasks.workunit.client.0.smithi019.stdout:1/179: dwrite d8/d18/f3b [0,4194304] 0 2024-02-22T15:50:28.848 INFO:tasks.workunit.client.0.smithi019.stdout:9/224: dwrite d2/f30 [0,4194304] 0 2024-02-22T15:50:28.849 INFO:tasks.workunit.client.0.smithi019.stdout:7/306: mknod d9/d26/c62 0 2024-02-22T15:50:28.852 INFO:tasks.workunit.client.0.smithi019.stdout:0/130: creat d4/db/d1b/f2f x:0 0 0 2024-02-22T15:50:28.852 INFO:tasks.workunit.client.0.smithi019.stdout:2/190: dwrite d1/de/f1e [0,4194304] 0 2024-02-22T15:50:28.852 INFO:tasks.workunit.client.0.smithi019.stdout:6/141: creat d6/f28 x:0 0 0 2024-02-22T15:50:28.854 INFO:tasks.workunit.client.0.smithi019.stdout:1/180: mknod d8/d15/d34/c41 0 2024-02-22T15:50:28.863 INFO:tasks.workunit.client.0.smithi019.stdout:7/307: symlink d9/d4c/d5b/l63 0 2024-02-22T15:50:28.864 INFO:tasks.workunit.client.0.smithi019.stdout:9/225: dwrite d2/f6 [8388608,4194304] 0 2024-02-22T15:50:28.867 INFO:tasks.workunit.client.0.smithi019.stdout:7/308: fsync d9/f16 0 2024-02-22T15:50:28.868 INFO:tasks.workunit.client.0.smithi019.stdout:1/181: dwrite d8/d18/f21 [4194304,4194304] 0 2024-02-22T15:50:28.871 INFO:tasks.workunit.client.0.smithi019.stdout:7/309: creat d9/d4c/d5b/f64 x:0 0 0 2024-02-22T15:50:28.873 INFO:tasks.workunit.client.0.smithi019.stdout:9/226: dread d2/f30 [0,4194304] 0 2024-02-22T15:50:28.876 INFO:tasks.workunit.client.0.smithi019.stdout:9/227: symlink d2/l53 0 2024-02-22T15:50:28.879 INFO:tasks.workunit.client.0.smithi019.stdout:1/182: dwrite d8/d23/d33/f37 [0,4194304] 0 2024-02-22T15:50:28.880 INFO:tasks.workunit.client.0.smithi019.stdout:7/310: rename d9/d38/d41/d45/f57 to d9/d4c/d5b/f65 0 2024-02-22T15:50:28.881 INFO:tasks.workunit.client.0.smithi019.stdout:1/183: rename d8 to d8/d23/d33/d42 22 2024-02-22T15:50:28.883 INFO:tasks.workunit.client.0.smithi019.stdout:9/228: creat d2/d7/dc/d27/d37/f54 x:0 0 0 2024-02-22T15:50:28.884 INFO:tasks.workunit.client.0.smithi019.stdout:7/311: creat d9/d4c/f66 x:0 0 0 2024-02-22T15:50:28.887 INFO:tasks.workunit.client.0.smithi019.stdout:9/229: rename d2/d7/l2b to d2/d7/dc/l55 0 2024-02-22T15:50:28.888 INFO:tasks.workunit.client.0.smithi019.stdout:1/184: creat d8/d15/f43 x:0 0 0 2024-02-22T15:50:28.889 INFO:tasks.workunit.client.0.smithi019.stdout:9/230: mkdir d2/d1f/d2e/d56 0 2024-02-22T15:50:28.890 INFO:tasks.workunit.client.0.smithi019.stdout:1/185: unlink d8/d15/f11 0 2024-02-22T15:50:28.891 INFO:tasks.workunit.client.0.smithi019.stdout:1/186: fdatasync d8/d2a/f3f 0 2024-02-22T15:50:28.896 INFO:tasks.workunit.client.0.smithi019.stdout:1/187: unlink d8/d2a/f40 0 2024-02-22T15:50:28.899 INFO:tasks.workunit.client.0.smithi019.stdout:1/188: mkdir d8/d44 0 2024-02-22T15:50:28.899 INFO:tasks.workunit.client.0.smithi019.stdout:1/189: readlink d8/d15/l22 0 2024-02-22T15:50:28.900 INFO:tasks.workunit.client.0.smithi019.stdout:6/142: rmdir d6 39 2024-02-22T15:50:28.906 INFO:tasks.workunit.client.0.smithi019.stdout:6/143: dread d6/f20 [0,4194304] 0 2024-02-22T15:50:28.908 INFO:tasks.workunit.client.0.smithi019.stdout:1/190: dread d8/f1a [0,4194304] 0 2024-02-22T15:50:28.912 INFO:tasks.workunit.client.0.smithi019.stdout:0/131: truncate d4/d9/f11 696000 0 2024-02-22T15:50:28.913 INFO:tasks.workunit.client.0.smithi019.stdout:2/191: dwrite d1/f26 [0,4194304] 0 2024-02-22T15:50:28.914 INFO:tasks.workunit.client.0.smithi019.stdout:9/231: truncate d2/d7/f12 9375844 0 2024-02-22T15:50:28.919 INFO:tasks.workunit.client.0.smithi019.stdout:1/191: rename d8/f1a to d8/d44/f45 0 2024-02-22T15:50:28.920 INFO:tasks.workunit.client.0.smithi019.stdout:0/132: dread - d4/f27 zero size 2024-02-22T15:50:28.923 INFO:tasks.workunit.client.0.smithi019.stdout:1/192: chown d8/fd 1 1 2024-02-22T15:50:28.923 INFO:tasks.workunit.client.0.smithi019.stdout:2/192: readlink d1/d17/d19/l23 0 2024-02-22T15:50:28.923 INFO:tasks.workunit.client.0.smithi019.stdout:1/193: stat d8/d15/f43 0 2024-02-22T15:50:28.924 INFO:tasks.workunit.client.0.smithi019.stdout:2/193: write d1/d17/d21/f36 [429333,44860] 0 2024-02-22T15:50:28.925 INFO:tasks.workunit.client.0.smithi019.stdout:0/133: truncate d4/f27 195854 0 2024-02-22T15:50:28.926 INFO:tasks.workunit.client.0.smithi019.stdout:9/232: rmdir d2 39 2024-02-22T15:50:28.932 INFO:tasks.workunit.client.0.smithi019.stdout:2/194: unlink d1/de/d15/f3a 0 2024-02-22T15:50:28.939 INFO:tasks.workunit.client.0.smithi019.stdout:0/134: creat d4/d22/d2c/f30 x:0 0 0 2024-02-22T15:50:28.940 INFO:tasks.workunit.client.0.smithi019.stdout:6/144: dread d6/fb [0,4194304] 0 2024-02-22T15:50:28.941 INFO:tasks.workunit.client.0.smithi019.stdout:9/233: rmdir d2/d7/d52 39 2024-02-22T15:50:28.942 INFO:tasks.workunit.client.0.smithi019.stdout:0/135: rmdir d4/d5 39 2024-02-22T15:50:28.944 INFO:tasks.workunit.client.0.smithi019.stdout:0/136: fsync f1 0 2024-02-22T15:50:28.946 INFO:tasks.workunit.client.0.smithi019.stdout:1/194: getdents d8/d15 0 2024-02-22T15:50:28.949 INFO:tasks.workunit.client.0.smithi019.stdout:2/195: truncate d1/f18 1743619 0 2024-02-22T15:50:28.953 INFO:tasks.workunit.client.0.smithi019.stdout:6/145: dwrite d6/ff [0,4194304] 0 2024-02-22T15:50:28.954 INFO:tasks.workunit.client.0.smithi019.stdout:0/137: truncate d4/db/d1b/f1c 341318 0 2024-02-22T15:50:28.955 INFO:tasks.workunit.client.0.smithi019.stdout:9/234: write d2/d1f/d2e/f42 [448147,84565] 0 2024-02-22T15:50:28.957 INFO:tasks.workunit.client.0.smithi019.stdout:9/235: readlink d2/l53 0 2024-02-22T15:50:28.959 INFO:tasks.workunit.client.0.smithi019.stdout:6/146: creat d6/f29 x:0 0 0 2024-02-22T15:50:28.960 INFO:tasks.workunit.client.0.smithi019.stdout:9/236: chown d2/c25 4565 1 2024-02-22T15:50:28.964 INFO:tasks.workunit.client.0.smithi019.stdout:9/237: mkdir d2/d1f/d2e/d57 0 2024-02-22T15:50:28.966 INFO:tasks.workunit.client.0.smithi019.stdout:6/147: creat d6/f2a x:0 0 0 2024-02-22T15:50:28.967 INFO:tasks.workunit.client.0.smithi019.stdout:9/238: chown d2/d1f/d2e/d57 1425 1 2024-02-22T15:50:28.967 INFO:tasks.workunit.client.0.smithi019.stdout:1/195: dwrite d8/f14 [0,4194304] 0 2024-02-22T15:50:28.969 INFO:tasks.workunit.client.0.smithi019.stdout:0/138: dwrite d4/d9/f1a [0,4194304] 0 2024-02-22T15:50:28.970 INFO:tasks.workunit.client.0.smithi019.stdout:9/239: fdatasync d2/d1f/d2e/f42 0 2024-02-22T15:50:28.970 INFO:tasks.workunit.client.0.smithi019.stdout:6/148: creat d6/f2b x:0 0 0 2024-02-22T15:50:28.972 INFO:tasks.workunit.client.0.smithi019.stdout:6/149: fsync d6/f18 0 2024-02-22T15:50:28.976 INFO:tasks.workunit.client.0.smithi019.stdout:6/150: truncate d6/f29 617069 0 2024-02-22T15:50:28.976 INFO:tasks.workunit.client.0.smithi019.stdout:1/196: dread - d8/d2a/f2b zero size 2024-02-22T15:50:28.977 INFO:tasks.workunit.client.0.smithi019.stdout:9/240: symlink d2/d7/dc/d27/d37/l58 0 2024-02-22T15:50:28.978 INFO:tasks.workunit.client.0.smithi019.stdout:6/151: write d6/f14 [984495,124839] 0 2024-02-22T15:50:28.978 INFO:tasks.workunit.client.0.smithi019.stdout:9/241: stat d2/fb 0 2024-02-22T15:50:28.981 INFO:tasks.workunit.client.0.smithi019.stdout:1/197: mknod d8/d2a/c46 0 2024-02-22T15:50:28.981 INFO:tasks.workunit.client.0.smithi019.stdout:9/242: chown d2/d1f/d2e/d48/d45 31 1 2024-02-22T15:50:28.982 INFO:tasks.workunit.client.0.smithi019.stdout:0/139: dwrite d4/f19 [0,4194304] 0 2024-02-22T15:50:28.982 INFO:tasks.workunit.client.0.smithi019.stdout:6/152: creat d6/f2c x:0 0 0 2024-02-22T15:50:28.984 INFO:tasks.workunit.client.0.smithi019.stdout:6/153: creat d6/f2d x:0 0 0 2024-02-22T15:50:28.985 INFO:tasks.workunit.client.0.smithi019.stdout:6/154: readlink d6/l24 0 2024-02-22T15:50:28.986 INFO:tasks.workunit.client.0.smithi019.stdout:0/140: unlink d4/l20 0 2024-02-22T15:50:28.988 INFO:tasks.workunit.client.0.smithi019.stdout:0/141: chown d4/d5 857563 1 2024-02-22T15:50:28.994 INFO:tasks.workunit.client.0.smithi019.stdout:1/198: dwrite d8/d2a/f2b [0,4194304] 0 2024-02-22T15:50:28.995 INFO:tasks.workunit.client.0.smithi019.stdout:1/199: fdatasync d8/d15/d34/f39 0 2024-02-22T15:50:28.998 INFO:tasks.workunit.client.0.smithi019.stdout:0/142: fsync d4/d9/f1a 0 2024-02-22T15:50:28.998 INFO:tasks.workunit.client.0.smithi019.stdout:1/200: creat d8/d18/f47 x:0 0 0 2024-02-22T15:50:28.998 INFO:tasks.workunit.client.0.smithi019.stdout:1/201: fdatasync d8/d15/f19 0 2024-02-22T15:50:29.000 INFO:tasks.workunit.client.0.smithi019.stdout:0/143: truncate d4/db/d1b/f2f 19262 0 2024-02-22T15:50:29.002 INFO:tasks.workunit.client.0.smithi019.stdout:0/144: mknod d4/d5/dc/c31 0 2024-02-22T15:50:29.003 INFO:tasks.workunit.client.0.smithi019.stdout:0/145: readlink d4/d5/l2e 0 2024-02-22T15:50:29.009 INFO:tasks.workunit.client.0.smithi019.stdout:1/202: dwrite d8/d23/d33/f37 [4194304,4194304] 0 2024-02-22T15:50:29.011 INFO:tasks.workunit.client.0.smithi019.stdout:0/146: dread d4/d9/f13 [0,4194304] 0 2024-02-22T15:50:29.014 INFO:tasks.workunit.client.0.smithi019.stdout:6/155: dwrite d6/f20 [0,4194304] 0 2024-02-22T15:50:29.015 INFO:tasks.workunit.client.0.smithi019.stdout:1/203: write d8/d2a/f2c [2754400,25249] 0 2024-02-22T15:50:29.027 INFO:tasks.workunit.client.0.smithi019.stdout:0/147: dwrite d4/f19 [0,4194304] 0 2024-02-22T15:50:29.029 INFO:tasks.workunit.client.0.smithi019.stdout:6/156: dread d6/f1c [0,4194304] 0 2024-02-22T15:50:29.029 INFO:tasks.workunit.client.0.smithi019.stdout:1/204: dwrite d8/d15/f1b [4194304,4194304] 0 2024-02-22T15:50:29.033 INFO:tasks.workunit.client.0.smithi019.stdout:0/148: truncate d4/d9/f11 319858 0 2024-02-22T15:50:29.040 INFO:tasks.workunit.client.0.smithi019.stdout:1/205: mknod d8/d2a/c48 0 2024-02-22T15:50:29.042 INFO:tasks.workunit.client.0.smithi019.stdout:1/206: chown d8/d15/f19 20320 1 2024-02-22T15:50:29.043 INFO:tasks.workunit.client.0.smithi019.stdout:6/157: dwrite d6/f17 [0,4194304] 0 2024-02-22T15:50:29.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 -- 172.21.15.19:0/1879195015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 msgr2=0x7f82200fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 --2- 172.21.15.19:0/1879195015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f82200fd420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8214004940 tx=0x7f8214009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 -- 172.21.15.19:0/1879195015 shutdown_connections 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 --2- 172.21.15.19:0/1879195015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f82200fd420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 --2- 172.21.15.19:0/1879195015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 0x7f82200fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 -- 172.21.15.19:0/1879195015 >> 172.21.15.19:0/1879195015 conn(0x7f82200f74e0 msgr2=0x7f82200f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.060+0000 7f8225b5c700 1 -- 172.21.15.19:0/1879195015 shutdown_connections 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.061+0000 7f8225b5c700 1 -- 172.21.15.19:0/1879195015 wait complete. 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.061+0000 7f8225b5c700 1 Processor -- start 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f8225b5c700 1 -- start start 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f8225b5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 0x7f822010b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f8225b5c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f822010bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f8225b5c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010c150 con 0x7f82200fcfb0 2024-02-22T15:50:29.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f8225b5c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010ca40 con 0x7f82200fbe40 2024-02-22T15:50:29.045 INFO:tasks.workunit.client.0.smithi019.stdout:1/207: fsync d8/f14 0 2024-02-22T15:50:29.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f821effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f822010bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f821effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f822010bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53768/0 (socket says 172.21.15.19:53768) 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f821effd700 1 -- 172.21.15.19:0/3669537035 learned_addr learned my addr 172.21.15.19:0/3669537035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.062+0000 7f821f7fe700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 0x7f822010b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.063+0000 7f821f7fe700 1 -- 172.21.15.19:0/3669537035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 msgr2=0x7f822010bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.063+0000 7f821f7fe700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f822010bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.063+0000 7f821f7fe700 1 -- 172.21.15.19:0/3669537035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214005040 con 0x7f82200fbe40 2024-02-22T15:50:29.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.064+0000 7f821f7fe700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 0x7f822010b610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8210002820 tx=0x7f8210004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.064+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8210007cc0 con 0x7f82200fbe40 2024-02-22T15:50:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.064+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f821000f040 con 0x7f82200fbe40 2024-02-22T15:50:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.064+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82100137d0 con 0x7f82200fbe40 2024-02-22T15:50:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.064+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822010cd20 con 0x7f82200fbe40 2024-02-22T15:50:29.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.064+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822010d260 con 0x7f82200fbe40 2024-02-22T15:50:29.048 INFO:tasks.workunit.client.0.smithi019.stdout:0/149: dwrite d4/f27 [0,4194304] 0 2024-02-22T15:50:29.048 INFO:tasks.workunit.client.0.smithi019.stdout:6/158: mknod d6/c2e 0 2024-02-22T15:50:29.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.065+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822004bed0 con 0x7f82200fbe40 2024-02-22T15:50:29.052 INFO:tasks.workunit.client.0.smithi019.stdout:1/208: mknod d8/d15/d34/c49 0 2024-02-22T15:50:29.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.067+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f821001c400 con 0x7f82200fbe40 2024-02-22T15:50:29.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.068+0000 7f8207fff700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f820806af90 0x7f820806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:29.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.068+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8210089570 con 0x7f82200fbe40 2024-02-22T15:50:29.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.069+0000 7f821effd700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f820806af90 0x7f820806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:29.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.070+0000 7f821effd700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f820806af90 0x7f820806d440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f821400b3b0 tx=0x7f8214009e30 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:29.054 INFO:tasks.workunit.client.0.smithi019.stdout:6/159: creat d6/f2f x:0 0 0 2024-02-22T15:50:29.056 INFO:tasks.workunit.client.0.smithi019.stdout:6/160: mknod d6/c30 0 2024-02-22T15:50:29.062 INFO:tasks.workunit.client.0.smithi019.stdout:1/209: dwrite d8/d18/f47 [0,4194304] 0 2024-02-22T15:50:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.084+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f8210053940 con 0x7f82200fbe40 2024-02-22T15:50:29.066 INFO:tasks.workunit.client.0.smithi019.stdout:1/210: creat d8/d44/f4a x:0 0 0 2024-02-22T15:50:29.067 INFO:tasks.workunit.client.0.smithi019.stdout:1/211: dread - d8/d15/f43 zero size 2024-02-22T15:50:29.075 INFO:tasks.workunit.client.0.smithi019.stdout:1/212: dread d8/d15/f26 [0,4194304] 0 2024-02-22T15:50:29.083 INFO:tasks.workunit.client.0.smithi019.stdout:1/213: dwrite d8/d44/f4a [0,4194304] 0 2024-02-22T15:50:29.084 INFO:tasks.workunit.client.0.smithi019.stdout:6/161: dread d6/f14 [0,4194304] 0 2024-02-22T15:50:29.089 INFO:tasks.workunit.client.0.smithi019.stdout:6/162: fdatasync d6/f10 0 2024-02-22T15:50:29.090 INFO:tasks.workunit.client.0.smithi019.stdout:1/214: dwrite d8/fd [4194304,4194304] 0 2024-02-22T15:50:29.091 INFO:tasks.workunit.client.0.smithi019.stdout:6/163: truncate d6/f2c 691540 0 2024-02-22T15:50:29.093 INFO:tasks.workunit.client.0.smithi019.stdout:1/215: mkdir d8/d18/d4b 0 2024-02-22T15:50:29.094 INFO:tasks.workunit.client.0.smithi019.stdout:6/164: mkdir d6/d31 0 2024-02-22T15:50:29.095 INFO:tasks.workunit.client.0.smithi019.stdout:1/216: fdatasync d8/d15/f26 0 2024-02-22T15:50:29.099 INFO:tasks.workunit.client.0.smithi019.stdout:1/217: rename d8/d23/c25 to d8/d15/c4c 0 2024-02-22T15:50:29.102 INFO:tasks.workunit.client.0.smithi019.stdout:6/165: dwrite d6/fb [4194304,4194304] 0 2024-02-22T15:50:29.103 INFO:tasks.workunit.client.0.smithi019.stdout:6/166: readlink d6/l21 0 2024-02-22T15:50:29.103 INFO:tasks.workunit.client.0.smithi019.stdout:6/167: fdatasync d6/f29 0 2024-02-22T15:50:29.108 INFO:tasks.workunit.client.0.smithi019.stdout:6/168: dread d6/f1c [0,4194304] 0 2024-02-22T15:50:29.108 INFO:tasks.workunit.client.0.smithi019.stdout:6/169: chown d6/f2a 0 1 2024-02-22T15:50:29.109 INFO:tasks.workunit.client.0.smithi019.stdout:6/170: write d6/f1d [861561,25149] 0 2024-02-22T15:50:29.112 INFO:tasks.workunit.client.0.smithi019.stdout:6/171: unlink d6/f2f 0 2024-02-22T15:50:29.118 INFO:tasks.workunit.client.0.smithi019.stdout:1/218: rename d8/d15/f12 to d8/d15/d34/f4d 0 2024-02-22T15:50:29.120 INFO:tasks.workunit.client.0.smithi019.stdout:1/219: mknod d8/d44/c4e 0 2024-02-22T15:50:29.120 INFO:tasks.workunit.client.0.smithi019.stdout:1/220: fsync d8/d18/f3b 0 2024-02-22T15:50:29.121 INFO:tasks.workunit.client.0.smithi019.stdout:6/172: rmdir d6 39 2024-02-22T15:50:29.123 INFO:tasks.workunit.client.0.smithi019.stdout:6/173: stat d6/f14 0 2024-02-22T15:50:29.127 INFO:tasks.workunit.client.0.smithi019.stdout:6/174: creat d6/d31/f32 x:0 0 0 2024-02-22T15:50:29.128 INFO:tasks.workunit.client.0.smithi019.stdout:1/221: link d8/d18/l1e d8/d18/l4f 0 2024-02-22T15:50:29.131 INFO:tasks.workunit.client.0.smithi019.stdout:1/222: mkdir d8/d23/d50 0 2024-02-22T15:50:29.134 INFO:tasks.workunit.client.0.smithi019.stdout:1/223: readlink d8/d18/l4f 0 2024-02-22T15:50:29.135 INFO:tasks.workunit.client.0.smithi019.stdout:1/224: rmdir d8 39 2024-02-22T15:50:29.141 INFO:tasks.workunit.client.0.smithi019.stdout:6/175: dwrite d6/f22 [0,4194304] 0 2024-02-22T15:50:29.143 INFO:tasks.workunit.client.0.smithi019.stdout:6/176: write d6/f1d [1242647,50436] 0 2024-02-22T15:50:29.144 INFO:tasks.workunit.client.0.smithi019.stdout:1/225: getdents d8/d23/d50 0 2024-02-22T15:50:29.145 INFO:tasks.workunit.client.0.smithi019.stdout:1/226: write d8/fd [873230,7919] 0 2024-02-22T15:50:29.153 INFO:tasks.workunit.client.0.smithi019.stdout:6/177: dwrite d6/f2d [0,4194304] 0 2024-02-22T15:50:29.155 INFO:tasks.workunit.client.0.smithi019.stdout:1/227: rename d8/d15/c4c to d8/d15/d34/c51 0 2024-02-22T15:50:29.160 INFO:tasks.workunit.client.0.smithi019.stdout:8/235: sync 2024-02-22T15:50:29.161 INFO:tasks.workunit.client.0.smithi019.stdout:6/178: rmdir d6 39 2024-02-22T15:50:29.163 INFO:tasks.workunit.client.0.smithi019.stdout:6/179: stat d6/l27 0 2024-02-22T15:50:29.164 INFO:tasks.workunit.client.0.smithi019.stdout:1/228: symlink d8/d23/d50/l52 0 2024-02-22T15:50:29.165 INFO:tasks.workunit.client.0.smithi019.stdout:6/180: mknod d6/c33 0 2024-02-22T15:50:29.166 INFO:tasks.workunit.client.0.smithi019.stdout:6/181: stat d6/l24 0 2024-02-22T15:50:29.172 INFO:tasks.workunit.client.0.smithi019.stdout:1/229: fsync d8/fd 0 2024-02-22T15:50:29.175 INFO:tasks.workunit.client.0.smithi019.stdout:1/230: creat d8/f53 x:0 0 0 2024-02-22T15:50:29.177 INFO:tasks.workunit.client.0.smithi019.stdout:1/231: dread - d8/d15/d34/f39 zero size 2024-02-22T15:50:29.180 INFO:tasks.workunit.client.0.smithi019.stdout:1/232: write d8/d15/f20 [210412,94194] 0 2024-02-22T15:50:29.180 INFO:tasks.workunit.client.0.smithi019.stdout:8/236: dwrite f6 [0,4194304] 0 2024-02-22T15:50:29.181 INFO:tasks.workunit.client.0.smithi019.stdout:1/233: chown d8/d23/c29 6889 1 2024-02-22T15:50:29.184 INFO:tasks.workunit.client.0.smithi019.stdout:8/237: dread - d9/d14/f34 zero size 2024-02-22T15:50:29.186 INFO:tasks.workunit.client.0.smithi019.stdout:8/238: dread - d9/d14/d17/f28 zero size 2024-02-22T15:50:29.188 INFO:tasks.workunit.client.0.smithi019.stdout:1/234: dread d8/d18/f21 [4194304,4194304] 0 2024-02-22T15:50:29.189 INFO:tasks.workunit.client.0.smithi019.stdout:8/239: creat d9/d14/f53 x:0 0 0 2024-02-22T15:50:29.189 INFO:tasks.workunit.client.0.smithi019.stdout:1/235: chown d8/d18/f47 19042374 1 2024-02-22T15:50:29.197 INFO:tasks.workunit.client.0.smithi019.stdout:8/240: rmdir d9 39 2024-02-22T15:50:29.198 INFO:tasks.workunit.client.0.smithi019.stdout:1/236: write d8/d15/f24 [1399313,99439] 0 2024-02-22T15:50:29.199 INFO:tasks.workunit.client.0.smithi019.stdout:1/237: readlink d8/d23/d50/l52 0 2024-02-22T15:50:29.201 INFO:tasks.workunit.client.0.smithi019.stdout:1/238: write d8/d23/d33/f37 [9372689,54790] 0 2024-02-22T15:50:29.203 INFO:tasks.workunit.client.0.smithi019.stdout:1/239: mkdir d8/d54 0 2024-02-22T15:50:29.204 INFO:tasks.workunit.client.0.smithi019.stdout:8/241: getdents d9/d14/d17/d3d/d3f 0 2024-02-22T15:50:29.205 INFO:tasks.workunit.client.0.smithi019.stdout:8/242: truncate d9/d14/d17/d1c/f51 758903 0 2024-02-22T15:50:29.206 INFO:tasks.workunit.client.0.smithi019.stdout:8/243: stat d9/d14/d1f/d20/c39 0 2024-02-22T15:50:29.208 INFO:tasks.workunit.client.0.smithi019.stdout:1/240: write d8/d44/f45 [2092254,75179] 0 2024-02-22T15:50:29.209 INFO:tasks.workunit.client.0.smithi019.stdout:1/241: chown d8/d18 17235639 1 2024-02-22T15:50:29.211 INFO:tasks.workunit.client.0.smithi019.stdout:1/242: fdatasync d8/d44/f4a 0 2024-02-22T15:50:29.215 INFO:tasks.workunit.client.0.smithi019.stdout:8/244: dwrite d9/d14/d17/d1c/f1e [0,4194304] 0 2024-02-22T15:50:29.216 INFO:tasks.workunit.client.0.smithi019.stdout:1/243: unlink d8/c16 0 2024-02-22T15:50:29.217 INFO:tasks.workunit.client.0.smithi019.stdout:8/245: dread - d9/d14/f38 zero size 2024-02-22T15:50:29.225 INFO:tasks.workunit.client.0.smithi019.stdout:8/246: mknod d9/d14/d17/c54 0 2024-02-22T15:50:29.226 INFO:tasks.workunit.client.0.smithi019.stdout:1/244: dwrite d8/d15/f1b [4194304,4194304] 0 2024-02-22T15:50:29.230 INFO:tasks.workunit.client.0.smithi019.stdout:8/247: getdents d9/d14/d17/d3d/d3f 0 2024-02-22T15:50:29.234 INFO:tasks.workunit.client.0.smithi019.stdout:8/248: symlink d9/d14/d17/l55 0 2024-02-22T15:50:29.236 INFO:tasks.workunit.client.0.smithi019.stdout:1/245: dwrite d8/d18/f47 [4194304,4194304] 0 2024-02-22T15:50:29.236 INFO:tasks.workunit.client.0.smithi019.stdout:8/249: write d9/f3b [187068,127451] 0 2024-02-22T15:50:29.239 INFO:tasks.workunit.client.0.smithi019.stdout:8/250: creat d9/d14/d17/d3d/d3f/f56 x:0 0 0 2024-02-22T15:50:29.246 INFO:tasks.workunit.client.0.smithi019.stdout:8/251: dwrite d9/ff [0,4194304] 0 2024-02-22T15:50:29.256 INFO:tasks.workunit.client.0.smithi019.stdout:8/252: dwrite d9/d14/d17/d1c/f51 [0,4194304] 0 2024-02-22T15:50:29.256 INFO:tasks.workunit.client.0.smithi019.stdout:1/246: truncate d8/d23/d33/f37 8558265 0 2024-02-22T15:50:29.258 INFO:tasks.workunit.client.0.smithi019.stdout:8/253: truncate d9/d14/f53 1039581 0 2024-02-22T15:50:29.264 INFO:tasks.workunit.client.0.smithi019.stdout:8/254: dread - d9/d14/d17/d3d/d3f/f56 zero size 2024-02-22T15:50:29.265 INFO:tasks.workunit.client.0.smithi019.stdout:1/247: creat d8/d23/f55 x:0 0 0 2024-02-22T15:50:29.266 INFO:tasks.workunit.client.0.smithi019.stdout:1/248: fsync d8/d44/f4a 0 2024-02-22T15:50:29.267 INFO:tasks.workunit.client.0.smithi019.stdout:8/255: symlink d9/d14/d1f/d20/l57 0 2024-02-22T15:50:29.268 INFO:tasks.workunit.client.0.smithi019.stdout:8/256: write d9/d14/f38 [544155,109789] 0 2024-02-22T15:50:29.268 INFO:tasks.workunit.client.0.smithi019.stdout:1/249: stat d8/d2a/f32 0 2024-02-22T15:50:29.269 INFO:tasks.workunit.client.0.smithi019.stdout:8/257: fdatasync d9/d14/d17/d1c/f24 0 2024-02-22T15:50:29.270 INFO:tasks.workunit.client.0.smithi019.stdout:8/258: chown d9/d14/d1f/c35 974 1 2024-02-22T15:50:29.271 INFO:tasks.workunit.client.0.smithi019.stdout:1/250: unlink d8/d18/f2d 0 2024-02-22T15:50:29.273 INFO:tasks.workunit.client.0.smithi019.stdout:1/251: fdatasync d8/d18/f21 0 2024-02-22T15:50:29.274 INFO:tasks.workunit.client.0.smithi019.stdout:1/252: write d8/d15/f17 [775909,41170] 0 2024-02-22T15:50:29.345 INFO:tasks.workunit.client.0.smithi019.stdout:1/253: symlink d8/d23/d33/l56 0 2024-02-22T15:50:29.347 INFO:tasks.workunit.client.0.smithi019.stdout:1/254: stat d8/d18/d4b 0 2024-02-22T15:50:29.350 INFO:tasks.workunit.client.0.smithi019.stdout:1/255: link d8/d2a/c48 d8/c57 0 2024-02-22T15:50:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.393+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82200f9040 con 0x7f820806af90 2024-02-22T15:50:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.394+0000 7f8207fff700 1 -- 172.21.15.19:0/3669537035 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f82200f9040 con 0x7f820806af90 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f820806af90 msgr2=0x7f820806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f820806af90 0x7f820806d440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f821400b3b0 tx=0x7f8214009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 msgr2=0x7f822010b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 0x7f822010b610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8210002820 tx=0x7f8210004570 comp rx=0 tx=0).stop 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 shutdown_connections 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82200fbe40 0x7f822010b610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f820806af90 0x7f820806d440 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 --2- 172.21.15.19:0/3669537035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82200fcfb0 0x7f822010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 >> 172.21.15.19:0/3669537035 conn(0x7f82200f74e0 msgr2=0x7f82200f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 shutdown_connections 2024-02-22T15:50:29.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.397+0000 7f8225b5c700 1 -- 172.21.15.19:0/3669537035 wait complete. 2024-02-22T15:50:29.381 INFO:tasks.workunit.client.0.smithi019.stdout:4/170: sync 2024-02-22T15:50:29.381 INFO:tasks.workunit.client.0.smithi019.stdout:3/191: sync 2024-02-22T15:50:29.381 INFO:tasks.workunit.client.0.smithi019.stdout:5/293: sync 2024-02-22T15:50:29.383 INFO:tasks.workunit.client.0.smithi019.stdout:1/256: dwrite d8/d2a/f2c [0,4194304] 0 2024-02-22T15:50:29.383 INFO:tasks.workunit.client.0.smithi019.stdout:3/192: readlink d9/d26/l37 0 2024-02-22T15:50:29.385 INFO:tasks.workunit.client.0.smithi019.stdout:4/171: read d2/fa [2561787,58522] 0 2024-02-22T15:50:29.388 INFO:tasks.workunit.client.0.smithi019.stdout:5/294: link d8/d11/d33/f3c d8/dd/f67 0 2024-02-22T15:50:29.389 INFO:tasks.workunit.client.0.smithi019.stdout:1/257: mknod d8/d18/c58 0 2024-02-22T15:50:29.394 INFO:tasks.workunit.client.0.smithi019.stdout:1/258: symlink d8/l59 0 2024-02-22T15:50:29.396 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:50:29.398 INFO:tasks.workunit.client.0.smithi019.stdout:4/172: dwrite d2/f32 [0,4194304] 0 2024-02-22T15:50:29.401 INFO:tasks.workunit.client.0.smithi019.stdout:1/259: creat d8/f5a x:0 0 0 2024-02-22T15:50:29.403 INFO:tasks.workunit.client.0.smithi019.stdout:5/295: dwrite d8/d11/f1c [0,4194304] 0 2024-02-22T15:50:29.405 INFO:tasks.workunit.client.0.smithi019.stdout:4/173: chown d2/db/d14/d23/d21/f2c 41824360 1 2024-02-22T15:50:29.406 INFO:tasks.workunit.client.0.smithi019.stdout:1/260: dread - d8/d2a/f3a zero size 2024-02-22T15:50:29.410 INFO:tasks.workunit.client.0.smithi019.stdout:5/296: rename d8/d11/d21/f36 to d8/dd/d19/d38/d47/f68 0 2024-02-22T15:50:29.411 INFO:tasks.workunit.client.0.smithi019.stdout:4/174: stat d2/db/d14/d23/l30 0 2024-02-22T15:50:29.411 INFO:tasks.workunit.client.0.smithi019.stdout:5/297: write f5 [2300314,28661] 0 2024-02-22T15:50:29.415 INFO:tasks.workunit.client.0.smithi019.stdout:1/261: mkdir d8/d15/d5b 0 2024-02-22T15:50:29.416 INFO:tasks.workunit.client.0.smithi019.stdout:3/193: truncate d9/d11/f17 3323160 0 2024-02-22T15:50:29.419 INFO:tasks.workunit.client.0.smithi019.stdout:4/175: mkdir d2/db/d14/d23/d33 0 2024-02-22T15:50:29.421 INFO:tasks.workunit.client.0.smithi019.stdout:4/176: write d2/f32 [4329677,12062] 0 2024-02-22T15:50:29.424 INFO:tasks.workunit.client.0.smithi019.stdout:5/298: dread d8/dd/d19/d3a/f3f [0,4194304] 0 2024-02-22T15:50:29.426 INFO:tasks.workunit.client.0.smithi019.stdout:5/299: readlink d8/dd/d19/l26 0 2024-02-22T15:50:29.427 INFO:tasks.workunit.client.0.smithi019.stdout:1/262: creat d8/d15/f5c x:0 0 0 2024-02-22T15:50:29.430 INFO:tasks.workunit.client.0.smithi019.stdout:3/194: dwrite d9/d14/f3c [0,4194304] 0 2024-02-22T15:50:29.433 INFO:tasks.workunit.client.0.smithi019.stdout:4/177: rename d2/db/d14/d23/l2a to d2/db/d14/d23/d33/l34 0 2024-02-22T15:50:29.434 INFO:tasks.workunit.client.0.smithi019.stdout:3/195: truncate d9/d26/d32/f2c 237648 0 2024-02-22T15:50:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:4/178: chown d2/f32 155560 1 2024-02-22T15:50:29.435 INFO:tasks.workunit.client.0.smithi019.stdout:5/300: mknod d8/d11/d21/d30/c69 0 2024-02-22T15:50:29.438 INFO:tasks.workunit.client.0.smithi019.stdout:1/263: dwrite d8/f5a [0,4194304] 0 2024-02-22T15:50:29.440 INFO:tasks.workunit.client.0.smithi019.stdout:1/264: read - d8/f53 zero size 2024-02-22T15:50:29.445 INFO:tasks.workunit.client.0.smithi019.stdout:4/179: mkdir d2/db/d14/d23/d21/d35 0 2024-02-22T15:50:29.451 INFO:tasks.workunit.client.0.smithi019.stdout:3/196: dwrite d9/d26/f35 [0,4194304] 0 2024-02-22T15:50:29.456 INFO:tasks.workunit.client.0.smithi019.stdout:1/265: dread d8/d15/f17 [0,4194304] 0 2024-02-22T15:50:29.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:29 smithi062 ceph-mon[48130]: pgmap v30: 65 pgs: 65 active+clean; 427 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 219 op/s 2024-02-22T15:50:29.462 INFO:tasks.workunit.client.0.smithi019.stdout:4/180: dwrite d2/db/d14/f27 [0,4194304] 0 2024-02-22T15:50:29.463 INFO:tasks.workunit.client.0.smithi019.stdout:1/266: rename d8/d23/d50 to d8/d44/d5d 0 2024-02-22T15:50:29.465 INFO:tasks.workunit.client.0.smithi019.stdout:3/197: dwrite d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:29.466 INFO:tasks.workunit.client.0.smithi019.stdout:1/267: fsync d8/d2a/f3a 0 2024-02-22T15:50:29.467 INFO:tasks.workunit.client.0.smithi019.stdout:4/181: chown d2/lc 9 1 2024-02-22T15:50:29.467 INFO:tasks.workunit.client.0.smithi019.stdout:3/198: write d9/d22/d27/f3d [21034,53919] 0 2024-02-22T15:50:29.470 INFO:tasks.workunit.client.0.smithi019.stdout:3/199: symlink d9/d26/d2d/l40 0 2024-02-22T15:50:29.471 INFO:tasks.workunit.client.0.smithi019.stdout:3/200: readlink d9/d26/l37 0 2024-02-22T15:50:29.472 INFO:tasks.workunit.client.0.smithi019.stdout:5/301: dread d8/fc [0,4194304] 0 2024-02-22T15:50:29.473 INFO:tasks.workunit.client.0.smithi019.stdout:4/182: rename d2/db/d14/f2b to d2/db/d14/d23/f36 0 2024-02-22T15:50:29.475 INFO:tasks.workunit.client.0.smithi019.stdout:1/268: dread - d8/d15/f43 zero size 2024-02-22T15:50:29.478 INFO:tasks.workunit.client.0.smithi019.stdout:1/269: chown d8/d2a/f2c 37 1 2024-02-22T15:50:29.480 INFO:tasks.workunit.client.0.smithi019.stdout:4/183: truncate d2/db/d14/d23/d21/f29 186009 0 2024-02-22T15:50:29.483 INFO:tasks.workunit.client.0.smithi019.stdout:3/201: dwrite f6 [0,4194304] 0 2024-02-22T15:50:29.483 INFO:tasks.workunit.client.0.smithi019.stdout:1/270: creat d8/d2a/f5e x:0 0 0 2024-02-22T15:50:29.485 INFO:tasks.workunit.client.0.smithi019.stdout:3/202: write d9/d14/f1b [3071464,118614] 0 2024-02-22T15:50:29.486 INFO:tasks.workunit.client.0.smithi019.stdout:3/203: chown d9/d26/f35 21618115 1 2024-02-22T15:50:29.488 INFO:tasks.workunit.client.0.smithi019.stdout:4/184: rename d2/db/d14/l16 to d2/db/d14/d23/d21/d35/l37 0 2024-02-22T15:50:29.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:29 smithi019 ceph-mon[67418]: pgmap v30: 65 pgs: 65 active+clean; 427 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 219 op/s 2024-02-22T15:50:29.498 INFO:tasks.workunit.client.0.smithi019.stdout:4/185: creat d2/db/d14/d23/d21/d35/f38 x:0 0 0 2024-02-22T15:50:29.498 INFO:tasks.workunit.client.0.smithi019.stdout:5/302: truncate d8/f2d 1689258 0 2024-02-22T15:50:29.499 INFO:tasks.workunit.client.0.smithi019.stdout:1/271: rmdir d8/d54 0 2024-02-22T15:50:29.499 INFO:tasks.workunit.client.0.smithi019.stdout:2/196: sync 2024-02-22T15:50:29.499 INFO:tasks.workunit.client.0.smithi019.stdout:7/312: sync 2024-02-22T15:50:29.500 INFO:tasks.workunit.client.0.smithi019.stdout:3/204: creat d9/d11/f41 x:0 0 0 2024-02-22T15:50:29.501 INFO:tasks.workunit.client.0.smithi019.stdout:7/313: write d9/d38/f3e [11969,116704] 0 2024-02-22T15:50:29.501 INFO:tasks.workunit.client.0.smithi019.stdout:4/186: mknod d2/db/d14/d23/d21/c39 0 2024-02-22T15:50:29.504 INFO:tasks.workunit.client.0.smithi019.stdout:7/314: mknod d9/d3a/c67 0 2024-02-22T15:50:29.504 INFO:tasks.workunit.client.0.smithi019.stdout:1/272: stat d8/d18/l4f 0 2024-02-22T15:50:29.506 INFO:tasks.workunit.client.0.smithi019.stdout:3/205: fdatasync d9/d11/f1f 0 2024-02-22T15:50:29.507 INFO:tasks.workunit.client.0.smithi019.stdout:5/303: truncate d8/d11/f3e 585847 0 2024-02-22T15:50:29.508 INFO:tasks.workunit.client.0.smithi019.stdout:7/315: chown d9/ce 202 1 2024-02-22T15:50:29.510 INFO:tasks.workunit.client.0.smithi019.stdout:3/206: creat d9/d16/f42 x:0 0 0 2024-02-22T15:50:29.511 INFO:tasks.workunit.client.0.smithi019.stdout:1/273: read d8/d23/d33/f37 [3137125,30651] 0 2024-02-22T15:50:29.514 INFO:tasks.workunit.client.0.smithi019.stdout:7/316: creat d9/d4c/f68 x:0 0 0 2024-02-22T15:50:29.518 INFO:tasks.workunit.client.0.smithi019.stdout:3/207: creat d9/d26/d32/f43 x:0 0 0 2024-02-22T15:50:29.519 INFO:tasks.workunit.client.0.smithi019.stdout:2/197: dwrite d1/d17/d1c/f34 [0,4194304] 0 2024-02-22T15:50:29.527 INFO:tasks.workunit.client.0.smithi019.stdout:4/187: dwrite d2/db/d14/d23/d21/f2d [4194304,4194304] 0 2024-02-22T15:50:29.534 INFO:tasks.workunit.client.0.smithi019.stdout:2/198: symlink d1/de/d15/d3d/l40 0 2024-02-22T15:50:29.534 INFO:tasks.workunit.client.0.smithi019.stdout:0/150: sync 2024-02-22T15:50:29.534 INFO:tasks.workunit.client.0.smithi019.stdout:6/182: sync 2024-02-22T15:50:29.535 INFO:tasks.workunit.client.0.smithi019.stdout:8/259: sync 2024-02-22T15:50:29.535 INFO:tasks.workunit.client.0.smithi019.stdout:9/243: sync 2024-02-22T15:50:29.535 INFO:tasks.workunit.client.0.smithi019.stdout:7/317: getdents d9/d38/d41/d45/d61 0 2024-02-22T15:50:29.535 INFO:tasks.workunit.client.0.smithi019.stdout:5/304: dwrite d8/d11/d21/d2c/f63 [0,4194304] 0 2024-02-22T15:50:29.536 INFO:tasks.workunit.client.0.smithi019.stdout:9/244: write d2/d1f/f47 [505148,22622] 0 2024-02-22T15:50:29.537 INFO:tasks.workunit.client.0.smithi019.stdout:4/188: creat d2/db/d14/d23/d21/f3a x:0 0 0 2024-02-22T15:50:29.543 INFO:tasks.workunit.client.0.smithi019.stdout:8/260: chown d9/d14/d1f/c35 86 1 2024-02-22T15:50:29.544 INFO:tasks.workunit.client.0.smithi019.stdout:1/274: dread d8/d15/f24 [0,4194304] 0 2024-02-22T15:50:29.548 INFO:tasks.workunit.client.0.smithi019.stdout:2/199: creat d1/d17/d21/f41 x:0 0 0 2024-02-22T15:50:29.551 INFO:tasks.workunit.client.0.smithi019.stdout:5/305: mknod d8/d11/c6a 0 2024-02-22T15:50:29.552 INFO:tasks.workunit.client.0.smithi019.stdout:7/318: creat d9/d38/f69 x:0 0 0 2024-02-22T15:50:29.552 INFO:tasks.workunit.client.0.smithi019.stdout:4/189: fsync d2/db/d14/d23/f2f 0 2024-02-22T15:50:29.552 INFO:tasks.workunit.client.0.smithi019.stdout:9/245: symlink d2/d1f/l59 0 2024-02-22T15:50:29.552 INFO:tasks.workunit.client.0.smithi019.stdout:1/275: chown d8/d44/f4a 0 1 2024-02-22T15:50:29.552 INFO:tasks.workunit.client.0.smithi019.stdout:8/261: unlink d9/d14/d1f/d20/f2a 0 2024-02-22T15:50:29.553 INFO:tasks.workunit.client.0.smithi019.stdout:3/208: truncate d9/d11/f1e 535669 0 2024-02-22T15:50:29.556 INFO:tasks.workunit.client.0.smithi019.stdout:5/306: fdatasync d8/dd/d19/d38/d47/f68 0 2024-02-22T15:50:29.557 INFO:tasks.workunit.client.0.smithi019.stdout:0/151: dwrite d4/d5/fe [4194304,4194304] 0 2024-02-22T15:50:29.559 INFO:tasks.workunit.client.0.smithi019.stdout:2/200: unlink d1/d17/d21/l3b 0 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 -- 172.21.15.19:0/22993041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a41035a0 msgr2=0x7f13a41039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 --2- 172.21.15.19:0/22993041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a41035a0 0x7f13a41039b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1394004940 tx=0x7f1394009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 -- 172.21.15.19:0/22993041 shutdown_connections 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 --2- 172.21.15.19:0/22993041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a4104760 0x7f13a4104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 --2- 172.21.15.19:0/22993041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a41035a0 0x7f13a41039b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 -- 172.21.15.19:0/22993041 >> 172.21.15.19:0/22993041 conn(0x7f13a4066d20 msgr2=0x7f13a4067140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 -- 172.21.15.19:0/22993041 shutdown_connections 2024-02-22T15:50:29.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.578+0000 7f13ac19b700 1 -- 172.21.15.19:0/22993041 wait complete. 2024-02-22T15:50:29.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.579+0000 7f13ac19b700 1 Processor -- start 2024-02-22T15:50:29.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.579+0000 7f13ac19b700 1 -- start start 2024-02-22T15:50:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.579+0000 7f13ac19b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a41035a0 0x7f13a407de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.579+0000 7f13ac19b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 0x7f13a407e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.579+0000 7f13ac19b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a407e970 con 0x7f13a4104760 2024-02-22T15:50:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.579+0000 7f13ac19b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a411da00 con 0x7f13a41035a0 2024-02-22T15:50:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.580+0000 7f13a9736700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 0x7f13a407e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:29.562 INFO:tasks.workunit.client.0.smithi019.stdout:7/319: mkdir d9/d38/d41/d45/d61/d6a 0 2024-02-22T15:50:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.580+0000 7f13a9736700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 0x7f13a407e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53788/0 (socket says 172.21.15.19:53788) 2024-02-22T15:50:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.580+0000 7f13a9736700 1 -- 172.21.15.19:0/757204272 learned_addr learned my addr 172.21.15.19:0/757204272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:50:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.580+0000 7f13a9736700 1 -- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a41035a0 msgr2=0x7f13a407de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.580+0000 7f13a9736700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a41035a0 0x7f13a407de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.580+0000 7f13a9736700 1 -- 172.21.15.19:0/757204272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394005040 con 0x7f13a4104760 2024-02-22T15:50:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.581+0000 7f13a9736700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 0x7f13a407e370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13a000eb40 tx=0x7f13a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.581+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a0007cc0 con 0x7f13a4104760 2024-02-22T15:50:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.581+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13a000f040 con 0x7f13a4104760 2024-02-22T15:50:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.581+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a0009570 con 0x7f13a4104760 2024-02-22T15:50:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.581+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a411dc80 con 0x7f13a4104760 2024-02-22T15:50:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.581+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a411e1c0 con 0x7f13a4104760 2024-02-22T15:50:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.583+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a404bed0 con 0x7f13a4104760 2024-02-22T15:50:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.584+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f13a00096d0 con 0x7f13a4104760 2024-02-22T15:50:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.585+0000 7f139a7fc700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f139006af90 0x7f139006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.585+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f13a0089530 con 0x7f13a4104760 2024-02-22T15:50:29.570 INFO:tasks.workunit.client.0.smithi019.stdout:5/307: rename d8/fc to d8/dd/d19/d38/d54/f6b 0 2024-02-22T15:50:29.570 INFO:tasks.workunit.client.0.smithi019.stdout:6/183: truncate d6/f14 733501 0 2024-02-22T15:50:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.585+0000 7f13a9f37700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f139006af90 0x7f139006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.586+0000 7f13a9f37700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f139006af90 0x7f139006d440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f139400b0d0 tx=0x7f1394009e30 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:29.571 INFO:tasks.workunit.client.0.smithi019.stdout:5/308: chown f1 150 1 2024-02-22T15:50:29.573 INFO:tasks.workunit.client.0.smithi019.stdout:3/209: mknod d9/c44 0 2024-02-22T15:50:29.573 INFO:tasks.workunit.client.0.smithi019.stdout:1/276: creat d8/d2a/f5f x:0 0 0 2024-02-22T15:50:29.574 INFO:tasks.workunit.client.0.smithi019.stdout:9/246: mkdir d2/d1f/d2e/d48/d45/d5a 0 2024-02-22T15:50:29.575 INFO:tasks.workunit.client.0.smithi019.stdout:8/262: dread d9/fa [0,4194304] 0 2024-02-22T15:50:29.576 INFO:tasks.workunit.client.0.smithi019.stdout:0/152: mkdir d4/d32 0 2024-02-22T15:50:29.576 INFO:tasks.workunit.client.0.smithi019.stdout:7/320: creat d9/d3a/f6b x:0 0 0 2024-02-22T15:50:29.577 INFO:tasks.workunit.client.0.smithi019.stdout:6/184: mknod d6/d31/c34 0 2024-02-22T15:50:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.596+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f13a004f4d0 con 0x7f13a4104760 2024-02-22T15:50:29.578 INFO:tasks.workunit.client.0.smithi019.stdout:2/201: mknod d1/c42 0 2024-02-22T15:50:29.578 INFO:tasks.workunit.client.0.smithi019.stdout:5/309: chown d8/d11/d21/l58 9326 1 2024-02-22T15:50:29.579 INFO:tasks.workunit.client.0.smithi019.stdout:7/321: readlink d9/d3a/l4f 0 2024-02-22T15:50:29.580 INFO:tasks.workunit.client.0.smithi019.stdout:1/277: truncate d8/d15/f20 1071868 0 2024-02-22T15:50:29.581 INFO:tasks.workunit.client.0.smithi019.stdout:2/202: chown d1/c42 371610923 1 2024-02-22T15:50:29.582 INFO:tasks.workunit.client.0.smithi019.stdout:2/203: chown d1/de/ff 146 1 2024-02-22T15:50:29.583 INFO:tasks.workunit.client.0.smithi019.stdout:3/210: dwrite d9/d16/f2b [0,4194304] 0 2024-02-22T15:50:29.583 INFO:tasks.workunit.client.0.smithi019.stdout:1/278: chown d8/d15/d34/f39 25113174 1 2024-02-22T15:50:29.604 INFO:tasks.workunit.client.0.smithi019.stdout:3/211: dread d9/fb [0,4194304] 0 2024-02-22T15:50:29.605 INFO:tasks.workunit.client.0.smithi019.stdout:3/212: write d9/d26/d32/f33 [3071127,104151] 0 2024-02-22T15:50:29.675 INFO:tasks.workunit.client.0.smithi019.stdout:6/185: creat d6/d31/f35 x:0 0 0 2024-02-22T15:50:29.679 INFO:tasks.workunit.client.0.smithi019.stdout:7/322: mknod d9/d38/d41/d45/d61/c6c 0 2024-02-22T15:50:29.680 INFO:tasks.workunit.client.0.smithi019.stdout:4/190: write d2/db/d14/f18 [4694093,79486] 0 2024-02-22T15:50:29.682 INFO:tasks.workunit.client.0.smithi019.stdout:2/204: symlink d1/de/d33/l43 0 2024-02-22T15:50:29.684 INFO:tasks.workunit.client.0.smithi019.stdout:6/186: dwrite d6/f2b [0,4194304] 0 2024-02-22T15:50:29.684 INFO:tasks.workunit.client.0.smithi019.stdout:3/213: mknod d9/d26/c45 0 2024-02-22T15:50:29.686 INFO:tasks.workunit.client.0.smithi019.stdout:4/191: truncate d2/db/d14/d23/d21/f29 458867 0 2024-02-22T15:50:29.688 INFO:tasks.workunit.client.0.smithi019.stdout:4/192: write d2/fd [1008894,131031] 0 2024-02-22T15:50:29.693 INFO:tasks.workunit.client.0.smithi019.stdout:4/193: truncate d2/db/f25 1294978 0 2024-02-22T15:50:29.699 INFO:tasks.workunit.client.0.smithi019.stdout:3/214: dwrite d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:29.701 INFO:tasks.workunit.client.0.smithi019.stdout:3/215: chown d9/d11 833 1 2024-02-22T15:50:29.702 INFO:tasks.workunit.client.0.smithi019.stdout:6/187: unlink d6/c2e 0 2024-02-22T15:50:29.702 INFO:tasks.workunit.client.0.smithi019.stdout:8/263: creat d9/f58 x:0 0 0 2024-02-22T15:50:29.703 INFO:tasks.workunit.client.0.smithi019.stdout:6/188: stat d6/c1f 0 2024-02-22T15:50:29.704 INFO:tasks.workunit.client.0.smithi019.stdout:8/264: chown d9/d14/f34 312647 1 2024-02-22T15:50:29.705 INFO:tasks.workunit.client.0.smithi019.stdout:4/194: dwrite d2/db/f25 [0,4194304] 0 2024-02-22T15:50:29.707 INFO:tasks.workunit.client.0.smithi019.stdout:1/279: rmdir d8/d18/d4b 0 2024-02-22T15:50:29.716 INFO:tasks.workunit.client.0.smithi019.stdout:7/323: write d9/f1d [2217002,52400] 0 2024-02-22T15:50:29.720 INFO:tasks.workunit.client.0.smithi019.stdout:6/189: unlink d6/f2b 0 2024-02-22T15:50:29.724 INFO:tasks.workunit.client.0.smithi019.stdout:0/153: dwrite f2 [0,4194304] 0 2024-02-22T15:50:29.727 INFO:tasks.workunit.client.0.smithi019.stdout:5/310: dwrite d8/d11/f3e [0,4194304] 0 2024-02-22T15:50:29.731 INFO:tasks.workunit.client.0.smithi019.stdout:3/216: getdents d9/d22/d27 0 2024-02-22T15:50:29.731 INFO:tasks.workunit.client.0.smithi019.stdout:7/324: chown d9/f25 9320 1 2024-02-22T15:50:29.733 INFO:tasks.workunit.client.0.smithi019.stdout:4/195: getdents d2/db/d14/d23/d33 0 2024-02-22T15:50:29.736 INFO:tasks.workunit.client.0.smithi019.stdout:8/265: dwrite f6 [4194304,4194304] 0 2024-02-22T15:50:29.742 INFO:tasks.workunit.client.0.smithi019.stdout:0/154: symlink d4/db/d1b/l33 0 2024-02-22T15:50:29.742 INFO:tasks.workunit.client.0.smithi019.stdout:1/280: dwrite d8/d2a/f2b [0,4194304] 0 2024-02-22T15:50:29.744 INFO:tasks.workunit.client.0.smithi019.stdout:4/196: rmdir d2 39 2024-02-22T15:50:29.747 INFO:tasks.workunit.client.0.smithi019.stdout:5/311: rename d8/d11/l66 to d8/dd/d19/d38/d54/l6c 0 2024-02-22T15:50:29.751 INFO:tasks.workunit.client.0.smithi019.stdout:6/190: dread d6/ff [0,4194304] 0 2024-02-22T15:50:29.751 INFO:tasks.workunit.client.0.smithi019.stdout:0/155: symlink d4/d5/dc/l34 0 2024-02-22T15:50:29.754 INFO:tasks.workunit.client.0.smithi019.stdout:7/325: creat d9/d38/d41/d45/d61/d6a/f6d x:0 0 0 2024-02-22T15:50:29.754 INFO:tasks.workunit.client.0.smithi019.stdout:2/205: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:29.755 INFO:tasks.workunit.client.0.smithi019.stdout:3/217: rename d9/d16/f2b to d9/d22/d27/f46 0 2024-02-22T15:50:29.756 INFO:tasks.workunit.client.0.smithi019.stdout:3/218: chown d9/d11/f1f 892823673 1 2024-02-22T15:50:29.757 INFO:tasks.workunit.client.0.smithi019.stdout:3/219: fdatasync f3 0 2024-02-22T15:50:29.757 INFO:tasks.workunit.client.0.smithi019.stdout:6/191: stat d6/la 0 2024-02-22T15:50:29.759 INFO:tasks.workunit.client.0.smithi019.stdout:5/312: symlink d8/dd/d19/d3a/d46/l6d 0 2024-02-22T15:50:29.760 INFO:tasks.workunit.client.0.smithi019.stdout:7/326: readlink d9/l20 0 2024-02-22T15:50:29.760 INFO:tasks.workunit.client.0.smithi019.stdout:5/313: chown d8/ff 40 1 2024-02-22T15:50:29.762 INFO:tasks.workunit.client.0.smithi019.stdout:2/206: unlink d1/de/d33/l3c 0 2024-02-22T15:50:29.762 INFO:tasks.workunit.client.0.smithi019.stdout:1/281: write d8/d23/d33/f37 [9225174,55322] 0 2024-02-22T15:50:29.762 INFO:tasks.workunit.client.0.smithi019.stdout:5/314: chown d8/dd/d19/d38/d54 267345693 1 2024-02-22T15:50:29.765 INFO:tasks.workunit.client.0.smithi019.stdout:8/266: dwrite d9/d14/f36 [0,4194304] 0 2024-02-22T15:50:29.766 INFO:tasks.workunit.client.0.smithi019.stdout:7/327: mknod d9/d4c/d5b/c6e 0 2024-02-22T15:50:29.767 INFO:tasks.workunit.client.0.smithi019.stdout:0/156: chown d4/d5/d7/c25 6951 1 2024-02-22T15:50:29.768 INFO:tasks.workunit.client.0.smithi019.stdout:3/220: write d9/d26/d2d/f3a [492776,119068] 0 2024-02-22T15:50:29.770 INFO:tasks.workunit.client.0.smithi019.stdout:6/192: link d6/f29 d6/d31/f36 0 2024-02-22T15:50:29.771 INFO:tasks.workunit.client.0.smithi019.stdout:7/328: read - d9/d3a/f6b zero size 2024-02-22T15:50:29.774 INFO:tasks.workunit.client.0.smithi019.stdout:0/157: mkdir d4/db/d1b/d35 0 2024-02-22T15:50:29.777 INFO:tasks.workunit.client.0.smithi019.stdout:6/193: write d6/f29 [1063808,89439] 0 2024-02-22T15:50:29.779 INFO:tasks.workunit.client.0.smithi019.stdout:1/282: symlink d8/d15/d5b/l60 0 2024-02-22T15:50:29.780 INFO:tasks.workunit.client.0.smithi019.stdout:4/197: dread d2/fd [0,4194304] 0 2024-02-22T15:50:29.782 INFO:tasks.workunit.client.0.smithi019.stdout:0/158: creat d4/d22/f36 x:0 0 0 2024-02-22T15:50:29.782 INFO:tasks.workunit.client.0.smithi019.stdout:2/207: dread d1/f18 [0,4194304] 0 2024-02-22T15:50:29.784 INFO:tasks.workunit.client.0.smithi019.stdout:6/194: mkdir d6/d31/d37 0 2024-02-22T15:50:29.787 INFO:tasks.workunit.client.0.smithi019.stdout:1/283: chown d8/d15/d34/f39 90439990 1 2024-02-22T15:50:29.789 INFO:tasks.workunit.client.0.smithi019.stdout:6/195: fdatasync d6/f22 0 2024-02-22T15:50:29.790 INFO:tasks.workunit.client.0.smithi019.stdout:8/267: dwrite d9/d14/f3e [0,4194304] 0 2024-02-22T15:50:29.790 INFO:tasks.workunit.client.0.smithi019.stdout:2/208: fdatasync d1/d17/d1c/f22 0 2024-02-22T15:50:29.793 INFO:tasks.workunit.client.0.smithi019.stdout:0/159: write d4/db/d1b/f2f [294615,2442] 0 2024-02-22T15:50:29.796 INFO:tasks.workunit.client.0.smithi019.stdout:6/196: write d6/f2c [241057,81934] 0 2024-02-22T15:50:29.799 INFO:tasks.workunit.client.0.smithi019.stdout:7/329: dwrite d9/d38/f3e [0,4194304] 0 2024-02-22T15:50:29.802 INFO:tasks.workunit.client.0.smithi019.stdout:3/221: dwrite d9/d22/d27/f46 [0,4194304] 0 2024-02-22T15:50:29.809 INFO:tasks.workunit.client.0.smithi019.stdout:7/330: write d9/f19 [1042199,20830] 0 2024-02-22T15:50:29.812 INFO:tasks.workunit.client.0.smithi019.stdout:1/284: truncate d8/d15/d34/f39 235190 0 2024-02-22T15:50:29.819 INFO:tasks.workunit.client.0.smithi019.stdout:1/285: write d8/d18/f21 [7794350,110683] 0 2024-02-22T15:50:29.819 INFO:tasks.workunit.client.0.smithi019.stdout:8/268: creat d9/d14/d17/d3d/f59 x:0 0 0 2024-02-22T15:50:29.820 INFO:tasks.workunit.client.0.smithi019.stdout:0/160: rename d4/db/d1b/d35 to d4/db/d24/d37 0 2024-02-22T15:50:29.821 INFO:tasks.workunit.client.0.smithi019.stdout:4/198: dwrite d2/db/d14/d23/d21/f2d [8388608,4194304] 0 2024-02-22T15:50:29.821 INFO:tasks.workunit.client.0.smithi019.stdout:3/222: creat d9/d22/d27/f47 x:0 0 0 2024-02-22T15:50:29.821 INFO:tasks.workunit.client.0.smithi019.stdout:7/331: symlink d9/d38/d41/d45/d61/d6a/l6f 0 2024-02-22T15:50:29.822 INFO:tasks.workunit.client.0.smithi019.stdout:6/197: dread d6/f14 [0,4194304] 0 2024-02-22T15:50:29.827 INFO:tasks.workunit.client.0.smithi019.stdout:0/161: write d4/d9/f13 [1014216,82162] 0 2024-02-22T15:50:29.827 INFO:tasks.workunit.client.0.smithi019.stdout:7/332: stat d9/d38/d41/d45/d61/d6a/f6d 0 2024-02-22T15:50:29.828 INFO:tasks.workunit.client.0.smithi019.stdout:6/198: creat d6/d31/f38 x:0 0 0 2024-02-22T15:50:29.829 INFO:tasks.workunit.client.0.smithi019.stdout:3/223: write d9/d22/d27/f3e [1244265,23511] 0 2024-02-22T15:50:29.829 INFO:tasks.workunit.client.0.smithi019.stdout:6/199: fdatasync d6/f20 0 2024-02-22T15:50:29.830 INFO:tasks.workunit.client.0.smithi019.stdout:2/209: truncate d1/de/ff 939716 0 2024-02-22T15:50:29.831 INFO:tasks.workunit.client.0.smithi019.stdout:6/200: stat d6/d31/f32 0 2024-02-22T15:50:29.832 INFO:tasks.workunit.client.0.smithi019.stdout:1/286: rename d8/l59 to d8/d23/d33/l61 0 2024-02-22T15:50:29.832 INFO:tasks.workunit.client.0.smithi019.stdout:0/162: truncate d4/d22/d2c/f30 1022039 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:8/269: symlink d9/d14/d1f/d4c/l5a 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:4/199: symlink d2/db/d14/l3b 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:7/333: creat d9/d38/d41/d45/f70 x:0 0 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:8/270: stat d9/d14/d17 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:4/200: fdatasync d2/f32 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:8/271: stat d9/d14/f38 0 2024-02-22T15:50:29.838 INFO:tasks.workunit.client.0.smithi019.stdout:7/334: write d9/d4c/f66 [574775,71275] 0 2024-02-22T15:50:29.840 INFO:tasks.workunit.client.0.smithi019.stdout:7/335: write d9/d26/f33 [9320775,9754] 0 2024-02-22T15:50:29.840 INFO:tasks.workunit.client.0.smithi019.stdout:6/201: creat d6/f39 x:0 0 0 2024-02-22T15:50:29.842 INFO:tasks.workunit.client.0.smithi019.stdout:0/163: mknod d4/d5/dc/c38 0 2024-02-22T15:50:29.842 INFO:tasks.workunit.client.0.smithi019.stdout:1/287: chown d8/d18/c38 10860 1 2024-02-22T15:50:29.843 INFO:tasks.workunit.client.0.smithi019.stdout:0/164: dread - d4/d9/f26 zero size 2024-02-22T15:50:29.849 INFO:tasks.workunit.client.0.smithi019.stdout:4/201: rename d2/d31 to d2/db/d14/d23/d21/d35/d3c 0 2024-02-22T15:50:29.851 INFO:tasks.workunit.client.0.smithi019.stdout:7/336: chown d9/cb 4415 1 2024-02-22T15:50:29.853 INFO:tasks.workunit.client.0.smithi019.stdout:2/210: mknod d1/d17/d38/c44 0 2024-02-22T15:50:29.853 INFO:tasks.workunit.client.0.smithi019.stdout:7/337: truncate d9/f1d 3252214 0 2024-02-22T15:50:29.854 INFO:tasks.workunit.client.0.smithi019.stdout:6/202: fsync d6/ff 0 2024-02-22T15:50:29.854 INFO:tasks.workunit.client.0.smithi019.stdout:1/288: symlink d8/d23/d33/l62 0 2024-02-22T15:50:29.856 INFO:tasks.workunit.client.0.smithi019.stdout:6/203: readlink d6/l1b 0 2024-02-22T15:50:29.857 INFO:tasks.workunit.client.0.smithi019.stdout:0/165: creat d4/d5/f39 x:0 0 0 2024-02-22T15:50:29.857 INFO:tasks.workunit.client.0.smithi019.stdout:8/272: dread f3 [0,4194304] 0 2024-02-22T15:50:29.859 INFO:tasks.workunit.client.0.smithi019.stdout:4/202: mkdir d2/db/d14/d23/d21/d35/d3d 0 2024-02-22T15:50:29.861 INFO:tasks.workunit.client.0.smithi019.stdout:2/211: symlink d1/d17/d21/l45 0 2024-02-22T15:50:29.864 INFO:tasks.workunit.client.0.smithi019.stdout:8/273: symlink d9/d14/d1f/d4c/l5b 0 2024-02-22T15:50:29.864 INFO:tasks.workunit.client.0.smithi019.stdout:3/224: dread d9/d11/f17 [0,4194304] 0 2024-02-22T15:50:29.867 INFO:tasks.workunit.client.0.smithi019.stdout:3/225: stat f3 0 2024-02-22T15:50:29.868 INFO:tasks.workunit.client.0.smithi019.stdout:2/212: mkdir d1/d46 0 2024-02-22T15:50:29.872 INFO:tasks.workunit.client.0.smithi019.stdout:1/289: rename d8/d15/f19 to d8/d15/f63 0 2024-02-22T15:50:29.873 INFO:tasks.workunit.client.0.smithi019.stdout:7/338: getdents d9/d38/d41 0 2024-02-22T15:50:29.875 INFO:tasks.workunit.client.0.smithi019.stdout:2/213: truncate d1/f1f 8696500 0 2024-02-22T15:50:29.876 INFO:tasks.workunit.client.0.smithi019.stdout:8/274: mkdir d9/d14/d1f/d5c 0 2024-02-22T15:50:29.881 INFO:tasks.workunit.client.0.smithi019.stdout:1/290: mkdir d8/d18/d64 0 2024-02-22T15:50:29.882 INFO:tasks.workunit.client.0.smithi019.stdout:8/275: stat d9/d14/d17/l55 0 2024-02-22T15:50:29.882 INFO:tasks.workunit.client.0.smithi019.stdout:6/204: dwrite d6/d31/f35 [0,4194304] 0 2024-02-22T15:50:29.882 INFO:tasks.workunit.client.0.smithi019.stdout:0/166: dwrite d4/d22/d2c/f30 [0,4194304] 0 2024-02-22T15:50:29.885 INFO:tasks.workunit.client.0.smithi019.stdout:1/291: chown d8/d18/d64 16 1 2024-02-22T15:50:29.888 INFO:tasks.workunit.client.0.smithi019.stdout:4/203: dwrite d2/db/d14/d23/f2f [0,4194304] 0 2024-02-22T15:50:29.892 INFO:tasks.workunit.client.0.smithi019.stdout:3/226: link d9/d26/d32/l1c d9/d26/d32/l48 0 2024-02-22T15:50:29.892 INFO:tasks.workunit.client.0.smithi019.stdout:0/167: mknod d4/d5/d7/c3a 0 2024-02-22T15:50:29.895 INFO:tasks.workunit.client.0.smithi019.stdout:1/292: creat d8/d15/d34/f65 x:0 0 0 2024-02-22T15:50:29.899 INFO:tasks.workunit.client.0.smithi019.stdout:3/227: mkdir d9/d22/d49 0 2024-02-22T15:50:29.899 INFO:tasks.workunit.client.0.smithi019.stdout:0/168: dread - d4/d9/f26 zero size 2024-02-22T15:50:29.901 INFO:tasks.workunit.client.0.smithi019.stdout:7/339: truncate d9/f21 2927151 0 2024-02-22T15:50:29.907 INFO:tasks.workunit.client.0.smithi019.stdout:3/228: fdatasync d9/d26/d32/f33 0 2024-02-22T15:50:29.908 INFO:tasks.workunit.client.0.smithi019.stdout:1/293: creat d8/d18/d64/f66 x:0 0 0 2024-02-22T15:50:29.910 INFO:tasks.workunit.client.0.smithi019.stdout:7/340: chown d9/f1b 7267 1 2024-02-22T15:50:29.912 INFO:tasks.workunit.client.0.smithi019.stdout:6/205: dwrite d6/f28 [0,4194304] 0 2024-02-22T15:50:29.913 INFO:tasks.workunit.client.0.smithi019.stdout:2/214: dwrite d1/de/d15/d3d/f3e [0,4194304] 0 2024-02-22T15:50:29.914 INFO:tasks.workunit.client.0.smithi019.stdout:0/169: write d4/d23/f28 [414927,9726] 0 2024-02-22T15:50:29.917 INFO:tasks.workunit.client.0.smithi019.stdout:3/229: dread - d9/f2f zero size 2024-02-22T15:50:29.917 INFO:tasks.workunit.client.0.smithi019.stdout:8/276: dwrite d9/d14/d17/d3d/f59 [0,4194304] 0 2024-02-22T15:50:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.948+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13a411e6a0 con 0x7f139006af90 2024-02-22T15:50:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.949+0000 7f139a7fc700 1 -- 172.21.15.19:0/757204272 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f13a411e6a0 con 0x7f139006af90 2024-02-22T15:50:29.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f139006af90 msgr2=0x7f139006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f139006af90 0x7f139006d440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f139400b0d0 tx=0x7f1394009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 msgr2=0x7f13a407e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 0x7f13a407e370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13a000eb40 tx=0x7f13a000b040 comp rx=0 tx=0).stop 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 shutdown_connections 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13a41035a0 0x7f13a407de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f139006af90 0x7f139006d440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 --2- 172.21.15.19:0/757204272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13a4104760 0x7f13a407e370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.951+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 >> 172.21.15.19:0/757204272 conn(0x7f13a4066d20 msgr2=0x7f13a4107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.952+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 shutdown_connections 2024-02-22T15:50:29.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:29.952+0000 7f13ac19b700 1 -- 172.21.15.19:0/757204272 wait complete. 2024-02-22T15:50:29.936 INFO:tasks.workunit.client.0.smithi019.stdout:4/204: dwrite d2/db/d14/d23/d21/f3a [0,4194304] 0 2024-02-22T15:50:29.937 INFO:tasks.workunit.client.0.smithi019.stdout:6/206: symlink d6/l3a 0 2024-02-22T15:50:29.940 INFO:tasks.workunit.client.0.smithi019.stdout:6/207: write d6/d31/f38 [747950,70592] 0 2024-02-22T15:50:29.941 INFO:tasks.workunit.client.0.smithi019.stdout:6/208: fdatasync d6/f2c 0 2024-02-22T15:50:29.941 INFO:tasks.workunit.client.0.smithi019.stdout:4/205: chown d2/db/d14/l24 0 1 2024-02-22T15:50:29.942 INFO:tasks.workunit.client.0.smithi019.stdout:6/209: chown c4 15113557 1 2024-02-22T15:50:29.944 INFO:tasks.workunit.client.0.smithi019.stdout:0/170: rename l0 to d4/db/d24/l3b 0 2024-02-22T15:50:29.944 INFO:tasks.workunit.client.0.smithi019.stdout:0/171: readlink d4/d5/dc/l34 0 2024-02-22T15:50:29.946 INFO:tasks.workunit.client.0.smithi019.stdout:7/341: link d9/d38/f3e d9/d3a/d49/f71 0 2024-02-22T15:50:29.948 INFO:tasks.workunit.client.0.smithi019.stdout:8/277: mkdir d9/d14/d17/d5d 0 2024-02-22T15:50:29.949 INFO:tasks.workunit.client.0.smithi019.stdout:6/210: dread d6/d31/f38 [0,4194304] 0 2024-02-22T15:50:29.950 INFO:tasks.workunit.client.0.smithi019.stdout:6/211: write d6/d31/f35 [1841337,128527] 0 2024-02-22T15:50:29.950 INFO:tasks.workunit.client.0.smithi019.stdout:8/278: write d9/d14/f19 [1362641,102929] 0 2024-02-22T15:50:29.951 INFO:tasks.workunit.client.0.smithi019.stdout:6/212: fdatasync d6/f2a 0 2024-02-22T15:50:29.952 INFO:tasks.workunit.client.0.smithi019.stdout:2/215: truncate d1/f14 132036 0 2024-02-22T15:50:29.952 INFO:tasks.workunit.client.0.smithi019.stdout:1/294: truncate d8/d15/f26 3236288 0 2024-02-22T15:50:29.954 INFO:tasks.workunit.client.0.smithi019.stdout:4/206: creat d2/db/d14/d23/d21/d35/d3c/f3e x:0 0 0 2024-02-22T15:50:29.955 INFO:tasks.workunit.client.0.smithi019.stdout:4/207: readlink d2/db/l15 0 2024-02-22T15:50:29.960 INFO:tasks.workunit.client.0.smithi019.stdout:0/172: symlink d4/d5/l3c 0 2024-02-22T15:50:29.960 INFO:tasks.workunit.client.0.smithi019.stdout:3/230: creat d9/d26/d3f/f4a x:0 0 0 2024-02-22T15:50:29.960 INFO:tasks.workunit.client.0.smithi019.stdout:1/295: write d8/d2a/f3a [51588,26290] 0 2024-02-22T15:50:29.965 INFO:tasks.workunit.client.0.smithi019.stdout:8/279: rmdir d9/d14/d1f 39 2024-02-22T15:50:29.965 INFO:tasks.workunit.client.0.smithi019.stdout:4/208: dread d2/fa [0,4194304] 0 2024-02-22T15:50:29.968 INFO:tasks.workunit.client.0.smithi019.stdout:6/213: truncate d6/f10 1166160 0 2024-02-22T15:50:29.971 INFO:tasks.workunit.client.0.smithi019.stdout:2/216: creat d1/d17/f47 x:0 0 0 2024-02-22T15:50:29.972 INFO:tasks.workunit.client.0.smithi019.stdout:7/342: creat d9/d3a/d49/d5d/f72 x:0 0 0 2024-02-22T15:50:29.975 INFO:tasks.workunit.client.0.smithi019.stdout:7/343: dread - d9/d38/f69 zero size 2024-02-22T15:50:29.975 INFO:tasks.workunit.client.0.smithi019.stdout:3/231: creat d9/d26/d2d/f4b x:0 0 0 2024-02-22T15:50:29.981 INFO:tasks.workunit.client.0.smithi019.stdout:6/214: dwrite d6/fb [0,4194304] 0 2024-02-22T15:50:29.983 INFO:tasks.workunit.client.0.smithi019.stdout:7/344: creat d9/d4c/d5b/f73 x:0 0 0 2024-02-22T15:50:29.984 INFO:tasks.workunit.client.0.smithi019.stdout:4/209: mkdir d2/d3f 0 2024-02-22T15:50:29.985 INFO:tasks.workunit.client.0.smithi019.stdout:1/296: dread d8/d18/f21 [0,4194304] 0 2024-02-22T15:50:29.990 INFO:tasks.workunit.client.0.smithi019.stdout:3/232: dwrite d9/d16/f42 [0,4194304] 0 2024-02-22T15:50:29.993 INFO:tasks.workunit.client.0.smithi019.stdout:8/280: mknod d9/d14/d1f/d5c/c5e 0 2024-02-22T15:50:29.995 INFO:tasks.workunit.client.0.smithi019.stdout:6/215: creat d6/d31/f3b x:0 0 0 2024-02-22T15:50:29.995 INFO:tasks.workunit.client.0.smithi019.stdout:4/210: dread - d2/db/d14/d23/d21/d35/d3c/f3e zero size 2024-02-22T15:50:29.995 INFO:tasks.workunit.client.0.smithi019.stdout:1/297: creat d8/d18/d64/f67 x:0 0 0 2024-02-22T15:50:29.996 INFO:tasks.workunit.client.0.smithi019.stdout:3/233: truncate d9/d26/d3f/f4a 23631 0 2024-02-22T15:50:29.999 INFO:tasks.workunit.client.0.smithi019.stdout:6/216: creat d6/f3c x:0 0 0 2024-02-22T15:50:30.004 INFO:tasks.workunit.client.0.smithi019.stdout:6/217: chown d6/l16 950 1 2024-02-22T15:50:30.007 INFO:tasks.workunit.client.0.smithi019.stdout:0/173: truncate d4/d9/f11 113229 0 2024-02-22T15:50:30.009 INFO:tasks.workunit.client.0.smithi019.stdout:8/281: dwrite d9/ff [0,4194304] 0 2024-02-22T15:50:30.010 INFO:tasks.workunit.client.0.smithi019.stdout:1/298: link d8/d18/f47 d8/d15/f68 0 2024-02-22T15:50:30.012 INFO:tasks.workunit.client.0.smithi019.stdout:6/218: symlink d6/d31/d37/l3d 0 2024-02-22T15:50:30.013 INFO:tasks.workunit.client.0.smithi019.stdout:4/211: dwrite d2/db/f17 [0,4194304] 0 2024-02-22T15:50:30.013 INFO:tasks.workunit.client.0.smithi019.stdout:0/174: mkdir d4/d22/d3d 0 2024-02-22T15:50:30.014 INFO:tasks.workunit.client.0.smithi019.stdout:1/299: rmdir d8/d2a 39 2024-02-22T15:50:30.019 INFO:tasks.workunit.client.0.smithi019.stdout:4/212: dread d2/db/f25 [0,4194304] 0 2024-02-22T15:50:30.023 INFO:tasks.workunit.client.0.smithi019.stdout:0/175: rename d4/d9/f1a to d4/f3e 0 2024-02-22T15:50:30.024 INFO:tasks.workunit.client.0.smithi019.stdout:1/300: creat d8/d15/f69 x:0 0 0 2024-02-22T15:50:30.029 INFO:tasks.workunit.client.0.smithi019.stdout:3/234: truncate f6 2496170 0 2024-02-22T15:50:30.031 INFO:tasks.workunit.client.0.smithi019.stdout:6/219: dwrite d6/f9 [0,4194304] 0 2024-02-22T15:50:30.032 INFO:tasks.workunit.client.0.smithi019.stdout:1/301: fsync d8/d15/d34/f4d 0 2024-02-22T15:50:30.034 INFO:tasks.workunit.client.0.smithi019.stdout:0/176: rmdir d4/db/d2d 0 2024-02-22T15:50:30.036 INFO:tasks.workunit.client.0.smithi019.stdout:6/220: mkdir d6/d3e 0 2024-02-22T15:50:30.036 INFO:tasks.workunit.client.0.smithi019.stdout:8/282: dread d9/fc [0,4194304] 0 2024-02-22T15:50:30.037 INFO:tasks.workunit.client.0.smithi019.stdout:6/221: stat d6/f1d 0 2024-02-22T15:50:30.039 INFO:tasks.workunit.client.0.smithi019.stdout:8/283: write d9/d14/f38 [175836,127875] 0 2024-02-22T15:50:30.040 INFO:tasks.workunit.client.0.smithi019.stdout:0/177: write d4/d9/f13 [347536,35898] 0 2024-02-22T15:50:30.041 INFO:tasks.workunit.client.0.smithi019.stdout:1/302: truncate d8/d15/f69 763658 0 2024-02-22T15:50:30.042 INFO:tasks.workunit.client.0.smithi019.stdout:3/235: rename d9/d14/c24 to d9/d22/d49/c4c 0 2024-02-22T15:50:30.043 INFO:tasks.workunit.client.0.smithi019.stdout:0/178: dread - d4/d5/f39 zero size 2024-02-22T15:50:30.047 INFO:tasks.workunit.client.0.smithi019.stdout:8/284: dread d9/d14/d17/d1c/f51 [0,4194304] 0 2024-02-22T15:50:30.048 INFO:tasks.workunit.client.0.smithi019.stdout:1/303: creat d8/d23/d33/f6a x:0 0 0 2024-02-22T15:50:30.051 INFO:tasks.workunit.client.0.smithi019.stdout:4/213: truncate d2/db/d14/f18 4458275 0 2024-02-22T15:50:30.053 INFO:tasks.workunit.client.0.smithi019.stdout:6/222: link d6/f3c d6/d3e/f3f 0 2024-02-22T15:50:30.054 INFO:tasks.workunit.client.0.smithi019.stdout:1/304: fsync d8/d18/f21 0 2024-02-22T15:50:30.058 INFO:tasks.workunit.client.0.smithi019.stdout:1/305: dread d8/f5a [0,4194304] 0 2024-02-22T15:50:30.059 INFO:tasks.workunit.client.0.smithi019.stdout:1/306: chown l7 25579 1 2024-02-22T15:50:30.059 INFO:tasks.workunit.client.0.smithi019.stdout:4/214: rename d2/db/d14/d23/d33/l34 to d2/db/d14/d23/d33/l40 0 2024-02-22T15:50:30.063 INFO:tasks.workunit.client.0.smithi019.stdout:3/236: creat d9/d11/f4d x:0 0 0 2024-02-22T15:50:30.064 INFO:tasks.workunit.client.0.smithi019.stdout:4/215: write d2/db/d14/f19 [1896811,28404] 0 2024-02-22T15:50:30.069 INFO:tasks.workunit.client.0.smithi019.stdout:3/237: symlink d9/d26/d32/l4e 0 2024-02-22T15:50:30.070 INFO:tasks.workunit.client.0.smithi019.stdout:3/238: chown d9/c21 6950 1 2024-02-22T15:50:30.071 INFO:tasks.workunit.client.0.smithi019.stdout:4/216: symlink d2/db/d14/d23/d21/d35/l41 0 2024-02-22T15:50:30.071 INFO:tasks.workunit.client.0.smithi019.stdout:3/239: chown d9/d22/d27/f47 13115337 1 2024-02-22T15:50:30.073 INFO:tasks.workunit.client.0.smithi019.stdout:1/307: symlink d8/l6b 0 2024-02-22T15:50:30.074 INFO:tasks.workunit.client.0.smithi019.stdout:6/223: rmdir d6 39 2024-02-22T15:50:30.076 INFO:tasks.workunit.client.0.smithi019.stdout:3/240: readlink d9/d26/d32/l1c 0 2024-02-22T15:50:30.076 INFO:tasks.workunit.client.0.smithi019.stdout:8/285: dread d9/f10 [0,4194304] 0 2024-02-22T15:50:30.077 INFO:tasks.workunit.client.0.smithi019.stdout:4/217: mkdir d2/db/d14/d23/d21/d42 0 2024-02-22T15:50:30.078 INFO:tasks.workunit.client.0.smithi019.stdout:1/308: read - d8/d2a/f5f zero size 2024-02-22T15:50:30.081 INFO:tasks.workunit.client.0.smithi019.stdout:3/241: creat d9/d22/f4f x:0 0 0 2024-02-22T15:50:30.083 INFO:tasks.workunit.client.0.smithi019.stdout:4/218: stat d2/c4 0 2024-02-22T15:50:30.085 INFO:tasks.workunit.client.0.smithi019.stdout:8/286: dwrite d9/d14/f38 [0,4194304] 0 2024-02-22T15:50:30.087 INFO:tasks.workunit.client.0.smithi019.stdout:6/224: unlink d6/d31/f38 0 2024-02-22T15:50:30.087 INFO:tasks.workunit.client.0.smithi019.stdout:1/309: creat d8/d15/d34/f6c x:0 0 0 2024-02-22T15:50:30.088 INFO:tasks.workunit.client.0.smithi019.stdout:3/242: creat d9/d22/f50 x:0 0 0 2024-02-22T15:50:30.090 INFO:tasks.workunit.client.0.smithi019.stdout:6/225: mknod d6/d31/c40 0 2024-02-22T15:50:30.092 INFO:tasks.workunit.client.0.smithi019.stdout:1/310: mknod d8/d23/d33/c6d 0 2024-02-22T15:50:30.093 INFO:tasks.workunit.client.0.smithi019.stdout:4/219: symlink d2/l43 0 2024-02-22T15:50:30.094 INFO:tasks.workunit.client.0.smithi019.stdout:8/287: mknod d9/d14/d17/c5f 0 2024-02-22T15:50:30.097 INFO:tasks.workunit.client.0.smithi019.stdout:6/226: dread d6/d31/f35 [0,4194304] 0 2024-02-22T15:50:30.098 INFO:tasks.workunit.client.0.smithi019.stdout:8/288: creat d9/d14/d1f/d5c/f60 x:0 0 0 2024-02-22T15:50:30.100 INFO:tasks.workunit.client.0.smithi019.stdout:3/243: dread d9/d22/d27/f3e [0,4194304] 0 2024-02-22T15:50:30.100 INFO:tasks.workunit.client.0.smithi019.stdout:8/289: readlink d9/d14/d1f/d4c/l5b 0 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- 172.21.15.19:0/1440678243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fc7f0 msgr2=0x7f2c9c0fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/1440678243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fc7f0 0x7f2c9c0fcc40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2c90004940 tx=0x7f2c90009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- 172.21.15.19:0/1440678243 shutdown_connections 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/1440678243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fc7f0 0x7f2c9c0fcc40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/1440678243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- 172.21.15.19:0/1440678243 >> 172.21.15.19:0/1440678243 conn(0x7f2c9c0f6ca0 msgr2=0x7f2c9c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- 172.21.15.19:0/1440678243 shutdown_connections 2024-02-22T15:50:30.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- 172.21.15.19:0/1440678243 wait complete. 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 Processor -- start 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- start start 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c10c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9c0fc7f0 0x7f2c9c10cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c10d350 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.125+0000 7f2ca0fec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c10dbf0 con 0x7f2c9c0fc7f0 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c10c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c10c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53802/0 (socket says 172.21.15.19:53802) 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 -- 172.21.15.19:0/137839463 learned_addr learned my addr 172.21.15.19:0/137839463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c99d9b700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9c0fc7f0 0x7f2c9c10cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:30.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 -- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9c0fc7f0 msgr2=0x7f2c9c10cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9c0fc7f0 0x7f2c9c10cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 -- 172.21.15.19:0/137839463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c90005040 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c9a59c700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c10c810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c8b0 tx=0x7f2c8c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c0106f0 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c8c010d00 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c10ded0 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c012dd0 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.126+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c10e410 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.127+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c04bed0 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.128+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f2c8c012600 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.128+0000 7f2c8affd700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2c8406aec0 0x7f2c8406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.128+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2c8c0896a0 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.128+0000 7f2c99d9b700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2c8406aec0 0x7f2c8406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:30.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.129+0000 7f2c99d9b700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2c8406aec0 0x7f2c8406d370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c90004f90 tx=0x7f2c90009e30 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:30.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.133+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f2c8c057690 con 0x7f2c9c0fb5e0 2024-02-22T15:50:30.153 INFO:tasks.workunit.client.0.smithi019.stdout:9/247: sync 2024-02-22T15:50:30.153 INFO:tasks.workunit.client.0.smithi019.stdout:4/220: dwrite d2/db/d14/f19 [0,4194304] 0 2024-02-22T15:50:30.155 INFO:tasks.workunit.client.0.smithi019.stdout:3/244: symlink d9/d26/d2d/d39/l51 0 2024-02-22T15:50:30.158 INFO:tasks.workunit.client.0.smithi019.stdout:6/227: dwrite d6/d31/f36 [0,4194304] 0 2024-02-22T15:50:30.160 INFO:tasks.workunit.client.0.smithi019.stdout:4/221: mkdir d2/db/d44 0 2024-02-22T15:50:30.161 INFO:tasks.workunit.client.0.smithi019.stdout:3/245: symlink d9/d26/d2d/d39/l52 0 2024-02-22T15:50:30.163 INFO:tasks.workunit.client.0.smithi019.stdout:3/246: chown d9/d26/d2d/d39 3 1 2024-02-22T15:50:30.166 INFO:tasks.workunit.client.0.smithi019.stdout:3/247: chown d9/d22/f4f 628032076 1 2024-02-22T15:50:30.167 INFO:tasks.workunit.client.0.smithi019.stdout:8/290: link d9/fc d9/d14/d1f/d20/f61 0 2024-02-22T15:50:30.168 INFO:tasks.workunit.client.0.smithi019.stdout:6/228: creat d6/f41 x:0 0 0 2024-02-22T15:50:30.169 INFO:tasks.workunit.client.0.smithi019.stdout:9/248: dwrite d2/d7/dc/f24 [0,4194304] 0 2024-02-22T15:50:30.170 INFO:tasks.workunit.client.0.smithi019.stdout:8/291: dread - d9/d14/f52 zero size 2024-02-22T15:50:30.172 INFO:tasks.workunit.client.0.smithi019.stdout:1/311: dread d8/d44/f4a [0,4194304] 0 2024-02-22T15:50:30.173 INFO:tasks.workunit.client.0.smithi019.stdout:4/222: mknod d2/db/d14/d23/c45 0 2024-02-22T15:50:30.174 INFO:tasks.workunit.client.0.smithi019.stdout:9/249: write d2/fb [3189399,75839] 0 2024-02-22T15:50:30.176 INFO:tasks.workunit.client.0.smithi019.stdout:8/292: symlink d9/d14/d17/d1c/l62 0 2024-02-22T15:50:30.179 INFO:tasks.workunit.client.0.smithi019.stdout:4/223: mkdir d2/db/d14/d23/d46 0 2024-02-22T15:50:30.183 INFO:tasks.workunit.client.0.smithi019.stdout:6/229: dwrite d6/fc [4194304,4194304] 0 2024-02-22T15:50:30.184 INFO:tasks.workunit.client.0.smithi019.stdout:6/230: write d6/f25 [887816,30382] 0 2024-02-22T15:50:30.187 INFO:tasks.workunit.client.0.smithi019.stdout:6/231: write d6/f25 [2722231,54785] 0 2024-02-22T15:50:30.188 INFO:tasks.workunit.client.0.smithi019.stdout:6/232: dread - d6/f2a zero size 2024-02-22T15:50:30.192 INFO:tasks.workunit.client.0.smithi019.stdout:4/224: symlink d2/d3f/l47 0 2024-02-22T15:50:30.194 INFO:tasks.workunit.client.0.smithi019.stdout:6/233: unlink d6/c26 0 2024-02-22T15:50:30.200 INFO:tasks.workunit.client.0.smithi019.stdout:9/250: dread d2/d1f/d2e/f42 [0,4194304] 0 2024-02-22T15:50:30.201 INFO:tasks.workunit.client.0.smithi019.stdout:9/251: write d2/f4f [160628,96479] 0 2024-02-22T15:50:30.204 INFO:tasks.workunit.client.0.smithi019.stdout:1/312: dread d8/d15/f1b [4194304,4194304] 0 2024-02-22T15:50:30.205 INFO:tasks.workunit.client.0.smithi019.stdout:3/248: write f6 [3124386,31435] 0 2024-02-22T15:50:30.205 INFO:tasks.workunit.client.0.smithi019.stdout:3/249: chown d9/d22 21708 1 2024-02-22T15:50:30.207 INFO:tasks.workunit.client.0.smithi019.stdout:3/250: read - d9/f2f zero size 2024-02-22T15:50:30.209 INFO:tasks.workunit.client.0.smithi019.stdout:1/313: dread d8/d15/f20 [0,4194304] 0 2024-02-22T15:50:30.209 INFO:tasks.workunit.client.0.smithi019.stdout:9/252: rename d2/d7/dc/l55 to d2/d1f/d2e/l5b 0 2024-02-22T15:50:30.212 INFO:tasks.workunit.client.0.smithi019.stdout:4/225: truncate d2/db/f25 416016 0 2024-02-22T15:50:30.214 INFO:tasks.workunit.client.0.smithi019.stdout:1/314: mknod d8/d23/c6e 0 2024-02-22T15:50:30.215 INFO:tasks.workunit.client.0.smithi019.stdout:9/253: symlink d2/d1f/d2e/d56/l5c 0 2024-02-22T15:50:30.217 INFO:tasks.workunit.client.0.smithi019.stdout:4/226: chown d2/db/d14/f18 415418372 1 2024-02-22T15:50:30.218 INFO:tasks.workunit.client.0.smithi019.stdout:3/251: dwrite d9/d26/d32/f43 [0,4194304] 0 2024-02-22T15:50:30.219 INFO:tasks.workunit.client.0.smithi019.stdout:9/254: chown d2/f19 35 1 2024-02-22T15:50:30.221 INFO:tasks.workunit.client.0.smithi019.stdout:3/252: fdatasync d9/d22/d27/f47 0 2024-02-22T15:50:30.223 INFO:tasks.workunit.client.0.smithi019.stdout:4/227: fsync d2/db/d14/d23/d21/f2c 0 2024-02-22T15:50:30.226 INFO:tasks.workunit.client.0.smithi019.stdout:3/253: mkdir d9/d26/d2d/d53 0 2024-02-22T15:50:30.227 INFO:tasks.workunit.client.0.smithi019.stdout:6/234: dwrite d6/f10 [0,4194304] 0 2024-02-22T15:50:30.228 INFO:tasks.workunit.client.0.smithi019.stdout:4/228: chown d2/db/d14/d23/d21/f2d 881715 1 2024-02-22T15:50:30.234 INFO:tasks.workunit.client.0.smithi019.stdout:1/315: link d8/d23/d33/l3c d8/d15/d5b/l6f 0 2024-02-22T15:50:30.236 INFO:tasks.workunit.client.0.smithi019.stdout:9/255: dwrite d2/d7/dc/d27/d37/f54 [0,4194304] 0 2024-02-22T15:50:30.239 INFO:tasks.workunit.client.0.smithi019.stdout:4/229: rename d2/db/d14/d23/d21/d35/d3c to d2/d3f/d48 0 2024-02-22T15:50:30.241 INFO:tasks.workunit.client.0.smithi019.stdout:9/256: mknod d2/d7/dc/d27/d37/c5d 0 2024-02-22T15:50:30.243 INFO:tasks.workunit.client.0.smithi019.stdout:9/257: write d2/d7/dc/f29 [28109,13607] 0 2024-02-22T15:50:30.244 INFO:tasks.workunit.client.0.smithi019.stdout:3/254: getdents d9/d26/d3f 0 2024-02-22T15:50:30.245 INFO:tasks.workunit.client.0.smithi019.stdout:6/235: getdents d6/d3e 0 2024-02-22T15:50:30.247 INFO:tasks.workunit.client.0.smithi019.stdout:6/236: write d6/f41 [767504,90542] 0 2024-02-22T15:50:30.247 INFO:tasks.workunit.client.0.smithi019.stdout:3/255: truncate d9/d11/f1f 5554257 0 2024-02-22T15:50:30.249 INFO:tasks.workunit.client.0.smithi019.stdout:6/237: write d6/f9 [4682857,93523] 0 2024-02-22T15:50:30.252 INFO:tasks.workunit.client.0.smithi019.stdout:4/230: rmdir d2/db/d14/d23/d46 0 2024-02-22T15:50:30.254 INFO:tasks.workunit.client.0.smithi019.stdout:3/256: symlink d9/d22/d27/l54 0 2024-02-22T15:50:30.256 INFO:tasks.workunit.client.0.smithi019.stdout:6/238: creat d6/d3e/f42 x:0 0 0 2024-02-22T15:50:30.256 INFO:tasks.workunit.client.0.smithi019.stdout:9/258: dwrite d2/d7/dc/d27/d37/f54 [0,4194304] 0 2024-02-22T15:50:30.259 INFO:tasks.workunit.client.0.smithi019.stdout:4/231: mknod d2/db/d14/d23/d21/d35/c49 0 2024-02-22T15:50:30.260 INFO:tasks.workunit.client.0.smithi019.stdout:6/239: mkdir d6/d31/d43 0 2024-02-22T15:50:30.262 INFO:tasks.workunit.client.0.smithi019.stdout:9/259: fdatasync d2/d1f/d2e/d48/d36/f43 0 2024-02-22T15:50:30.263 INFO:tasks.workunit.client.0.smithi019.stdout:1/316: dwrite d8/d18/f47 [4194304,4194304] 0 2024-02-22T15:50:30.264 INFO:tasks.workunit.client.0.smithi019.stdout:4/232: symlink d2/d3f/d48/l4a 0 2024-02-22T15:50:30.265 INFO:tasks.workunit.client.0.smithi019.stdout:9/260: stat d2/l35 0 2024-02-22T15:50:30.266 INFO:tasks.workunit.client.0.smithi019.stdout:6/240: creat d6/f44 x:0 0 0 2024-02-22T15:50:30.268 INFO:tasks.workunit.client.0.smithi019.stdout:1/317: stat d8/d18/f27 0 2024-02-22T15:50:30.269 INFO:tasks.workunit.client.0.smithi019.stdout:4/233: chown d2/c3 4 1 2024-02-22T15:50:30.270 INFO:tasks.workunit.client.0.smithi019.stdout:3/257: dwrite d9/d14/f3c [0,4194304] 0 2024-02-22T15:50:30.271 INFO:tasks.workunit.client.0.smithi019.stdout:1/318: read - d8/d15/d34/f65 zero size 2024-02-22T15:50:30.272 INFO:tasks.workunit.client.0.smithi019.stdout:6/241: rmdir d6/d31 39 2024-02-22T15:50:30.272 INFO:tasks.workunit.client.0.smithi019.stdout:9/261: read d2/f30 [3761844,54616] 0 2024-02-22T15:50:30.273 INFO:tasks.workunit.client.0.smithi019.stdout:3/258: readlink d9/d26/d2d/l40 0 2024-02-22T15:50:30.274 INFO:tasks.workunit.client.0.smithi019.stdout:4/234: stat d2/c5 0 2024-02-22T15:50:30.276 INFO:tasks.workunit.client.0.smithi019.stdout:9/262: chown d2/d1f/d2e/d48/d36/f3b 591499 1 2024-02-22T15:50:30.276 INFO:tasks.workunit.client.0.smithi019.stdout:6/242: write d6/f1c [1485529,129596] 0 2024-02-22T15:50:30.277 INFO:tasks.workunit.client.0.smithi019.stdout:1/319: symlink d8/d15/d34/l70 0 2024-02-22T15:50:30.278 INFO:tasks.workunit.client.0.smithi019.stdout:6/243: chown d6/f17 13606 1 2024-02-22T15:50:30.279 INFO:tasks.workunit.client.0.smithi019.stdout:3/259: unlink d9/d22/d27/f3e 0 2024-02-22T15:50:30.281 INFO:tasks.workunit.client.0.smithi019.stdout:4/235: mkdir d2/db/d14/d23/d4b 0 2024-02-22T15:50:30.283 INFO:tasks.workunit.client.0.smithi019.stdout:3/260: creat d9/d26/f55 x:0 0 0 2024-02-22T15:50:30.284 INFO:tasks.workunit.client.0.smithi019.stdout:3/261: fsync d9/d22/d27/f3d 0 2024-02-22T15:50:30.286 INFO:tasks.workunit.client.0.smithi019.stdout:9/263: rename d2/d1f/d2e/c51 to d2/d7/d14/c5e 0 2024-02-22T15:50:30.286 INFO:tasks.workunit.client.0.smithi019.stdout:4/236: write d2/db/d14/d23/f36 [622174,49250] 0 2024-02-22T15:50:30.287 INFO:tasks.workunit.client.0.smithi019.stdout:9/264: fsync d2/d1f/d2e/d48/d36/f3b 0 2024-02-22T15:50:30.288 INFO:tasks.workunit.client.0.smithi019.stdout:4/237: write d2/f32 [4215338,5575] 0 2024-02-22T15:50:30.289 INFO:tasks.workunit.client.0.smithi019.stdout:9/265: fdatasync d2/d1f/f47 0 2024-02-22T15:50:30.290 INFO:tasks.workunit.client.0.smithi019.stdout:3/262: creat d9/d26/d2d/d53/f56 x:0 0 0 2024-02-22T15:50:30.291 INFO:tasks.workunit.client.0.smithi019.stdout:4/238: fsync d2/db/d14/d23/d21/f29 0 2024-02-22T15:50:30.293 INFO:tasks.workunit.client.0.smithi019.stdout:3/263: write d9/d26/d2d/f3a [231766,102746] 0 2024-02-22T15:50:30.295 INFO:tasks.workunit.client.0.smithi019.stdout:9/266: symlink d2/d1f/d2e/l5f 0 2024-02-22T15:50:30.296 INFO:tasks.workunit.client.0.smithi019.stdout:1/320: truncate d8/d15/f69 514489 0 2024-02-22T15:50:30.299 INFO:tasks.workunit.client.0.smithi019.stdout:9/267: dread d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:30.299 INFO:tasks.workunit.client.0.smithi019.stdout:3/264: rmdir d9/d22/d27 39 2024-02-22T15:50:30.301 INFO:tasks.workunit.client.0.smithi019.stdout:9/268: stat d2/d7/dc/d27/d37/l38 0 2024-02-22T15:50:30.301 INFO:tasks.workunit.client.0.smithi019.stdout:3/265: write d9/d26/d3f/f4a [117607,20671] 0 2024-02-22T15:50:30.301 INFO:tasks.workunit.client.0.smithi019.stdout:4/239: mknod d2/db/d14/d23/c4c 0 2024-02-22T15:50:30.303 INFO:tasks.workunit.client.0.smithi019.stdout:1/321: creat d8/d15/d34/f71 x:0 0 0 2024-02-22T15:50:30.306 INFO:tasks.workunit.client.0.smithi019.stdout:4/240: fdatasync d2/fa 0 2024-02-22T15:50:30.307 INFO:tasks.workunit.client.0.smithi019.stdout:6/244: dread d6/f22 [0,4194304] 0 2024-02-22T15:50:30.308 INFO:tasks.workunit.client.0.smithi019.stdout:1/322: fsync d8/d15/f24 0 2024-02-22T15:50:30.308 INFO:tasks.workunit.client.0.smithi019.stdout:6/245: chown d6/f20 31792459 1 2024-02-22T15:50:30.311 INFO:tasks.workunit.client.0.smithi019.stdout:9/269: mkdir d2/d1f/d2e/d57/d60 0 2024-02-22T15:50:30.311 INFO:tasks.workunit.client.0.smithi019.stdout:3/266: dwrite f4 [0,4194304] 0 2024-02-22T15:50:30.316 INFO:tasks.workunit.client.0.smithi019.stdout:3/267: write d9/d26/d2d/d53/f56 [850954,44484] 0 2024-02-22T15:50:30.322 INFO:tasks.workunit.client.0.smithi019.stdout:6/246: dwrite d6/f25 [0,4194304] 0 2024-02-22T15:50:30.325 INFO:tasks.workunit.client.0.smithi019.stdout:6/247: readlink d6/l24 0 2024-02-22T15:50:30.326 INFO:tasks.workunit.client.0.smithi019.stdout:1/323: dwrite d8/d44/f4a [4194304,4194304] 0 2024-02-22T15:50:30.329 INFO:tasks.workunit.client.0.smithi019.stdout:6/248: dread - d6/d3e/f42 zero size 2024-02-22T15:50:30.330 INFO:tasks.workunit.client.0.smithi019.stdout:9/270: dwrite d2/f6 [4194304,4194304] 0 2024-02-22T15:50:30.331 INFO:tasks.workunit.client.0.smithi019.stdout:1/324: mkdir d8/d18/d72 0 2024-02-22T15:50:30.332 INFO:tasks.workunit.client.0.smithi019.stdout:9/271: chown d2 3925 1 2024-02-22T15:50:30.332 INFO:tasks.workunit.client.0.smithi019.stdout:6/249: stat d6/f41 0 2024-02-22T15:50:30.334 INFO:tasks.workunit.client.0.smithi019.stdout:1/325: chown d8/d18/f3b 356609 1 2024-02-22T15:50:30.337 INFO:tasks.workunit.client.0.smithi019.stdout:1/326: write d8/d23/f55 [818908,113531] 0 2024-02-22T15:50:30.339 INFO:tasks.workunit.client.0.smithi019.stdout:1/327: stat d8/f14 0 2024-02-22T15:50:30.339 INFO:tasks.workunit.client.0.smithi019.stdout:3/268: dwrite d9/d22/d27/f3d [0,4194304] 0 2024-02-22T15:50:30.342 INFO:tasks.workunit.client.0.smithi019.stdout:9/272: creat d2/d1f/d2e/d48/d45/f61 x:0 0 0 2024-02-22T15:50:30.346 INFO:tasks.workunit.client.0.smithi019.stdout:6/250: symlink d6/l45 0 2024-02-22T15:50:30.354 INFO:tasks.workunit.client.0.smithi019.stdout:1/328: link d8/l1f d8/d15/l73 0 2024-02-22T15:50:30.355 INFO:tasks.workunit.client.0.smithi019.stdout:9/273: link d2/d7/d14/c32 d2/d1f/d2e/d48/d45/d5a/c62 0 2024-02-22T15:50:30.358 INFO:tasks.workunit.client.0.smithi019.stdout:3/269: dwrite d9/d22/d27/f3d [0,4194304] 0 2024-02-22T15:50:30.364 INFO:tasks.workunit.client.0.smithi019.stdout:9/274: creat d2/d1f/d2e/d48/d45/d5a/f63 x:0 0 0 2024-02-22T15:50:30.367 INFO:tasks.workunit.client.0.smithi019.stdout:1/329: dwrite d8/d15/f63 [0,4194304] 0 2024-02-22T15:50:30.367 INFO:tasks.workunit.client.0.smithi019.stdout:3/270: dread d9/d26/d32/f43 [0,4194304] 0 2024-02-22T15:50:30.368 INFO:tasks.workunit.client.0.smithi019.stdout:1/330: fdatasync d8/d15/d34/f39 0 2024-02-22T15:50:30.370 INFO:tasks.workunit.client.0.smithi019.stdout:1/331: readlink d8/d44/d5d/l52 0 2024-02-22T15:50:30.371 INFO:tasks.workunit.client.0.smithi019.stdout:3/271: write d9/d14/f1b [2117128,87704] 0 2024-02-22T15:50:30.372 INFO:tasks.workunit.client.0.smithi019.stdout:9/275: getdents d2/d1f/d2e/d56 0 2024-02-22T15:50:30.373 INFO:tasks.workunit.client.0.smithi019.stdout:1/332: creat d8/d15/d34/f74 x:0 0 0 2024-02-22T15:50:30.374 INFO:tasks.workunit.client.0.smithi019.stdout:9/276: readlink d2/d1f/l59 0 2024-02-22T15:50:30.376 INFO:tasks.workunit.client.0.smithi019.stdout:1/333: write d8/d15/d34/f65 [682705,77135] 0 2024-02-22T15:50:30.376 INFO:tasks.workunit.client.0.smithi019.stdout:3/272: mknod d9/d26/d2d/d53/c57 0 2024-02-22T15:50:30.377 INFO:tasks.workunit.client.0.smithi019.stdout:1/334: read d8/d15/f20 [295632,4608] 0 2024-02-22T15:50:30.380 INFO:tasks.workunit.client.0.smithi019.stdout:3/273: mkdir d9/d22/d58 0 2024-02-22T15:50:30.380 INFO:tasks.workunit.client.0.smithi019.stdout:1/335: write d8/d44/f45 [1303649,3422] 0 2024-02-22T15:50:30.382 INFO:tasks.workunit.client.0.smithi019.stdout:3/274: chown d9/d22/d27/l34 1880903 1 2024-02-22T15:50:30.383 INFO:tasks.workunit.client.0.smithi019.stdout:1/336: mkdir d8/d2a/d75 0 2024-02-22T15:50:30.385 INFO:tasks.workunit.client.0.smithi019.stdout:9/277: dwrite d2/f30 [0,4194304] 0 2024-02-22T15:50:30.386 INFO:tasks.workunit.client.0.smithi019.stdout:3/275: truncate d9/d22/d27/f3d 4722195 0 2024-02-22T15:50:30.391 INFO:tasks.workunit.client.0.smithi019.stdout:9/278: dread d2/f30 [0,4194304] 0 2024-02-22T15:50:30.392 INFO:tasks.workunit.client.0.smithi019.stdout:9/279: read d2/d1f/d2e/f42 [1418999,122485] 0 2024-02-22T15:50:30.395 INFO:tasks.workunit.client.0.smithi019.stdout:1/337: mkdir d8/d2a/d75/d76 0 2024-02-22T15:50:30.395 INFO:tasks.workunit.client.0.smithi019.stdout:1/338: stat d8/fd 0 2024-02-22T15:50:30.397 INFO:tasks.workunit.client.0.smithi019.stdout:3/276: rmdir d9 39 2024-02-22T15:50:30.398 INFO:tasks.workunit.client.0.smithi019.stdout:1/339: symlink d8/d18/d64/l77 0 2024-02-22T15:50:30.401 INFO:tasks.workunit.client.0.smithi019.stdout:9/280: link d2/d1f/d2e/d48/d45/d5a/f63 d2/d1f/d2e/d56/f64 0 2024-02-22T15:50:30.402 INFO:tasks.workunit.client.0.smithi019.stdout:3/277: dread - d9/d22/f50 zero size 2024-02-22T15:50:30.403 INFO:tasks.workunit.client.0.smithi019.stdout:1/340: dread d8/d44/f4a [4194304,4194304] 0 2024-02-22T15:50:30.403 INFO:tasks.workunit.client.0.smithi019.stdout:3/278: chown d9/d11/f41 462 1 2024-02-22T15:50:30.406 INFO:tasks.workunit.client.0.smithi019.stdout:1/341: rmdir d8/d2a/d75/d76 0 2024-02-22T15:50:30.407 INFO:tasks.workunit.client.0.smithi019.stdout:1/342: chown d8/d44/c4e 457426856 1 2024-02-22T15:50:30.408 INFO:tasks.workunit.client.0.smithi019.stdout:3/279: getdents d9/d26 0 2024-02-22T15:50:30.408 INFO:tasks.workunit.client.0.smithi019.stdout:1/343: chown d8/d15/f20 30848456 1 2024-02-22T15:50:30.409 INFO:tasks.workunit.client.0.smithi019.stdout:3/280: truncate d9/d11/f4d 419119 0 2024-02-22T15:50:30.412 INFO:tasks.workunit.client.0.smithi019.stdout:1/344: creat d8/d2a/f78 x:0 0 0 2024-02-22T15:50:30.413 INFO:tasks.workunit.client.0.smithi019.stdout:1/345: chown d8/d15/f43 14962723 1 2024-02-22T15:50:30.414 INFO:tasks.workunit.client.0.smithi019.stdout:3/281: dread d9/d16/f42 [0,4194304] 0 2024-02-22T15:50:30.417 INFO:tasks.workunit.client.0.smithi019.stdout:3/282: mkdir d9/d26/d3f/d59 0 2024-02-22T15:50:30.419 INFO:tasks.workunit.client.0.smithi019.stdout:3/283: fsync d9/d26/d32/f33 0 2024-02-22T15:50:30.421 INFO:tasks.workunit.client.0.smithi019.stdout:1/346: dwrite d8/d18/f27 [0,4194304] 0 2024-02-22T15:50:30.425 INFO:tasks.workunit.client.0.smithi019.stdout:3/284: dread d9/d26/d2d/d53/f56 [0,4194304] 0 2024-02-22T15:50:30.425 INFO:tasks.workunit.client.0.smithi019.stdout:1/347: creat d8/f79 x:0 0 0 2024-02-22T15:50:30.427 INFO:tasks.workunit.client.0.smithi019.stdout:3/285: truncate d9/d26/d2d/f3a 1615003 0 2024-02-22T15:50:30.428 INFO:tasks.workunit.client.0.smithi019.stdout:3/286: write d9/d26/f35 [253156,108486] 0 2024-02-22T15:50:30.428 INFO:tasks.workunit.client.0.smithi019.stdout:5/315: sync 2024-02-22T15:50:30.430 INFO:tasks.workunit.client.0.smithi019.stdout:3/287: readlink d9/d26/d2d/d39/l51 0 2024-02-22T15:50:30.431 INFO:tasks.workunit.client.0.smithi019.stdout:5/316: rmdir d8/d11/d21 39 2024-02-22T15:50:30.433 INFO:tasks.workunit.client.0.smithi019.stdout:3/288: chown d9/d22/d58 1 1 2024-02-22T15:50:30.434 INFO:tasks.workunit.client.0.smithi019.stdout:1/348: getdents d8 0 2024-02-22T15:50:30.435 INFO:tasks.workunit.client.0.smithi019.stdout:1/349: dread - d8/d2a/f5f zero size 2024-02-22T15:50:30.437 INFO:tasks.workunit.client.0.smithi019.stdout:1/350: mknod d8/d15/d34/c7a 0 2024-02-22T15:50:30.438 INFO:tasks.workunit.client.0.smithi019.stdout:1/351: chown d8/d18 340 1 2024-02-22T15:50:30.440 INFO:tasks.workunit.client.0.smithi019.stdout:1/352: symlink d8/l7b 0 2024-02-22T15:50:30.441 INFO:tasks.workunit.client.0.smithi019.stdout:3/289: dwrite d9/d22/d27/f46 [4194304,4194304] 0 2024-02-22T15:50:30.444 INFO:tasks.workunit.client.0.smithi019.stdout:3/290: mknod d9/d22/d58/c5a 0 2024-02-22T15:50:30.446 INFO:tasks.workunit.client.0.smithi019.stdout:1/353: write d8/d15/f1b [8539447,83748] 0 2024-02-22T15:50:30.446 INFO:tasks.workunit.client.0.smithi019.stdout:3/291: mknod d9/d11/c5b 0 2024-02-22T15:50:30.453 INFO:tasks.workunit.client.0.smithi019.stdout:3/292: dwrite f4 [4194304,4194304] 0 2024-02-22T15:50:30.456 INFO:tasks.workunit.client.0.smithi019.stdout:3/293: mkdir d9/d26/d3f/d5c 0 2024-02-22T15:50:30.457 INFO:tasks.workunit.client.0.smithi019.stdout:1/354: dread d8/d15/f17 [0,4194304] 0 2024-02-22T15:50:30.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:30 smithi062 ceph-mon[48130]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:30.458 INFO:tasks.workunit.client.0.smithi019.stdout:3/294: mkdir d9/d22/d58/d5d 0 2024-02-22T15:50:30.464 INFO:tasks.workunit.client.0.smithi019.stdout:1/355: rmdir d8/d15/d34 39 2024-02-22T15:50:30.470 INFO:tasks.workunit.client.0.smithi019.stdout:1/356: truncate d8/f14 2877742 0 2024-02-22T15:50:30.470 INFO:tasks.workunit.client.0.smithi019.stdout:3/295: truncate d9/d26/f35 4178591 0 2024-02-22T15:50:30.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.490+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c9c10e910 con 0x7f2c8406aec0 2024-02-22T15:50:30.478 INFO:tasks.workunit.client.0.smithi019.stdout:3/296: dwrite d9/fb [0,4194304] 0 2024-02-22T15:50:30.483 INFO:tasks.workunit.client.0.smithi019.stdout:3/297: getdents d9 0 2024-02-22T15:50:30.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.502+0000 7f2c8affd700 1 -- 172.21.15.19:0/137839463 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f2c9c10e910 con 0x7f2c8406aec0 2024-02-22T15:50:30.484 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:50:30.484 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 starting - - - - 2024-02-22T15:50:30.484 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (9m) 42s ago 9m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:50:30.484 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (7m) 51s ago 7m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:50:30.484 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 starting - - - - 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (2m) 42s ago 2m 62.7M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (3m) 42s ago 3m 36.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (2m) 51s ago 2m 20.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (3m) 51s ago 3m 20.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:9283 running (10m) 42s ago 10m 169M - 17.2.7-527-g6ea2a55e e2e4ad29904c 1e57e04e2dad 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 running (67s) 51s ago 7m 475M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 60b7b18ac805 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (10m) 42s ago 10m 49.5M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (7m) 51s ago 7m 41.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 starting - - - - 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (7m) 51s ago 7m 16.4M - 1.5.0 0da6a335fe13 379a69e65053 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (6m) 42s ago 6m 69.5M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (6m) 42s ago 6m 71.3M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (5m) 42s ago 5m 62.8M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (5m) 51s ago 5m 61.7M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:50:30.485 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (5m) 51s ago 5m 63.4M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:50:30.486 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (4m) 51s ago 4m 61.9M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:50:30.486 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (7m) 42s ago 8m 57.9M - 2.43.0 a07b618ecd1d 05fe6b300615 2024-02-22T15:50:30.486 INFO:tasks.workunit.client.0.smithi019.stdout:1/357: dwrite d8/f14 [0,4194304] 0 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2c8406aec0 msgr2=0x7f2c8406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2c8406aec0 0x7f2c8406d370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2c90004f90 tx=0x7f2c90009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 msgr2=0x7f2c9c10c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c10c810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c8b0 tx=0x7f2c8c004470 comp rx=0 tx=0).stop 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 shutdown_connections 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2c8406aec0 0x7f2c8406d370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c9c0fb5e0 0x7f2c9c10c810 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 --2- 172.21.15.19:0/137839463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9c0fc7f0 0x7f2c9c10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 >> 172.21.15.19:0/137839463 conn(0x7f2c9c0f6ca0 msgr2=0x7f2c9c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 shutdown_connections 2024-02-22T15:50:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.506+0000 7f2ca0fec700 1 -- 172.21.15.19:0/137839463 wait complete. 2024-02-22T15:50:30.491 INFO:tasks.workunit.client.0.smithi019.stdout:1/358: symlink d8/d18/d64/l7c 0 2024-02-22T15:50:30.493 INFO:tasks.workunit.client.0.smithi019.stdout:1/359: mknod d8/d2a/d75/c7d 0 2024-02-22T15:50:30.495 INFO:tasks.workunit.client.0.smithi019.stdout:1/360: write d8/d2a/f78 [615969,124216] 0 2024-02-22T15:50:30.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:30 smithi019 ceph-mon[67418]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:30.499 INFO:tasks.workunit.client.0.smithi019.stdout:1/361: creat d8/d15/d34/f7e x:0 0 0 2024-02-22T15:50:30.501 INFO:tasks.workunit.client.0.smithi019.stdout:1/362: mkdir d8/d18/d64/d7f 0 2024-02-22T15:50:30.502 INFO:tasks.workunit.client.0.smithi019.stdout:1/363: write d8/d18/d64/f66 [221982,9650] 0 2024-02-22T15:50:30.503 INFO:tasks.workunit.client.0.smithi019.stdout:1/364: dread - d8/d2a/f5e zero size 2024-02-22T15:50:30.504 INFO:tasks.workunit.client.0.smithi019.stdout:1/365: truncate d8/d15/d34/f74 814122 0 2024-02-22T15:50:30.505 INFO:tasks.workunit.client.0.smithi019.stdout:1/366: chown d8/d18/f3b 7112 1 2024-02-22T15:50:30.508 INFO:tasks.workunit.client.0.smithi019.stdout:1/367: getdents d8 0 2024-02-22T15:50:30.510 INFO:tasks.workunit.client.0.smithi019.stdout:1/368: creat d8/f80 x:0 0 0 2024-02-22T15:50:30.520 INFO:tasks.workunit.client.0.smithi019.stdout:1/369: dread d8/d44/f4a [0,4194304] 0 2024-02-22T15:50:30.523 INFO:tasks.workunit.client.0.smithi019.stdout:1/370: read d8/d15/f1b [116639,110509] 0 2024-02-22T15:50:30.525 INFO:tasks.workunit.client.0.smithi019.stdout:1/371: mkdir d8/d2a/d75/d81 0 2024-02-22T15:50:30.528 INFO:tasks.workunit.client.0.smithi019.stdout:1/372: mkdir d8/d18/d64/d7f/d82 0 2024-02-22T15:50:30.528 INFO:tasks.workunit.client.0.smithi019.stdout:1/373: dread - d8/d2a/f5e zero size 2024-02-22T15:50:30.536 INFO:tasks.workunit.client.0.smithi019.stdout:1/374: dwrite d8/d23/d33/f6a [0,4194304] 0 2024-02-22T15:50:30.540 INFO:tasks.workunit.client.0.smithi019.stdout:1/375: creat d8/d18/d64/f83 x:0 0 0 2024-02-22T15:50:30.550 INFO:tasks.workunit.client.0.smithi019.stdout:1/376: write d8/f5a [1629593,75207] 0 2024-02-22T15:50:30.552 INFO:tasks.workunit.client.0.smithi019.stdout:1/377: mkdir d8/d15/d84 0 2024-02-22T15:50:30.554 INFO:tasks.workunit.client.0.smithi019.stdout:1/378: mkdir d8/d18/d85 0 2024-02-22T15:50:30.555 INFO:tasks.workunit.client.0.smithi019.stdout:1/379: unlink d8/d15/d5b/l60 0 2024-02-22T15:50:30.559 INFO:tasks.workunit.client.0.smithi019.stdout:1/380: rmdir d8/d18 39 2024-02-22T15:50:30.575 INFO:tasks.workunit.client.0.smithi019.stdout:1/381: dread d8/d15/d34/f4d [0,4194304] 0 2024-02-22T15:50:30.581 INFO:tasks.workunit.client.0.smithi019.stdout:1/382: dwrite d8/d15/d34/f71 [0,4194304] 0 2024-02-22T15:50:30.584 INFO:tasks.workunit.client.0.smithi019.stdout:1/383: truncate d8/f53 527079 0 2024-02-22T15:50:30.586 INFO:tasks.workunit.client.0.smithi019.stdout:2/217: sync 2024-02-22T15:50:30.589 INFO:tasks.workunit.client.0.smithi019.stdout:1/384: dread d8/d15/f68 [4194304,4194304] 0 2024-02-22T15:50:30.589 INFO:tasks.workunit.client.0.smithi019.stdout:1/385: chown c2 2520 1 2024-02-22T15:50:30.592 INFO:tasks.workunit.client.0.smithi019.stdout:2/218: dread d1/de/d15/d3d/f3e [0,4194304] 0 2024-02-22T15:50:30.594 INFO:tasks.workunit.client.0.smithi019.stdout:2/219: symlink d1/d17/d19/l48 0 2024-02-22T15:50:30.595 INFO:tasks.workunit.client.0.smithi019.stdout:1/386: link d8/d18/c38 d8/d18/d64/c86 0 2024-02-22T15:50:30.598 INFO:tasks.workunit.client.0.smithi019.stdout:2/220: dread d1/f14 [0,4194304] 0 2024-02-22T15:50:30.600 INFO:tasks.workunit.client.0.smithi019.stdout:1/387: dread d8/fd [4194304,4194304] 0 2024-02-22T15:50:30.602 INFO:tasks.workunit.client.0.smithi019.stdout:2/221: rename d1/lb to d1/d17/l49 0 2024-02-22T15:50:30.603 INFO:tasks.workunit.client.0.smithi019.stdout:1/388: mknod d8/d18/d72/c87 0 2024-02-22T15:50:30.605 INFO:tasks.workunit.client.0.smithi019.stdout:1/389: write d8/d15/d34/f71 [260114,106391] 0 2024-02-22T15:50:30.606 INFO:tasks.workunit.client.0.smithi019.stdout:2/222: rmdir d1/de 39 2024-02-22T15:50:30.606 INFO:tasks.workunit.client.0.smithi019.stdout:1/390: chown d8/d18/d64/f67 236328817 1 2024-02-22T15:50:30.606 INFO:tasks.workunit.client.0.smithi019.stdout:2/223: readlink d1/d17/l3f 0 2024-02-22T15:50:30.615 INFO:tasks.workunit.client.0.smithi019.stdout:1/391: getdents d8/d15/d34 0 2024-02-22T15:50:30.616 INFO:tasks.workunit.client.0.smithi019.stdout:2/224: dwrite d1/de/d15/d3d/f3e [0,4194304] 0 2024-02-22T15:50:30.619 INFO:tasks.workunit.client.0.smithi019.stdout:1/392: creat d8/f88 x:0 0 0 2024-02-22T15:50:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 -- 172.21.15.19:0/3360891155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 msgr2=0x7f59000fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 --2- 172.21.15.19:0/3360891155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f59000fc250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f58fc004880 tx=0x7f58fc009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 -- 172.21.15.19:0/3360891155 shutdown_connections 2024-02-22T15:50:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 --2- 172.21.15.19:0/3360891155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59000fcfb0 0x7f59000fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 --2- 172.21.15.19:0/3360891155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f59000fc250 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 -- 172.21.15.19:0/3360891155 >> 172.21.15.19:0/3360891155 conn(0x7f59000f7500 msgr2=0x7f59000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 -- 172.21.15.19:0/3360891155 shutdown_connections 2024-02-22T15:50:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 -- 172.21.15.19:0/3360891155 wait complete. 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.643+0000 7f5908195700 1 Processor -- start 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5908195700 1 -- start start 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5908195700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f590010d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5908195700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59000fcfb0 0x7f590010dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5908195700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590010e340 con 0x7f59000fbe40 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5908195700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590010ebe0 con 0x7f59000fcfb0 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5905f31700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f590010d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5905f31700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f590010d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53824/0 (socket says 172.21.15.19:53824) 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5905f31700 1 -- 172.21.15.19:0/1555967477 learned_addr learned my addr 172.21.15.19:0/1555967477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:50:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.644+0000 7f5905730700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59000fcfb0 0x7f590010dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:30.627 INFO:tasks.workunit.client.0.smithi019.stdout:2/225: dwrite d1/f2e [0,4194304] 0 2024-02-22T15:50:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f5905f31700 1 -- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59000fcfb0 msgr2=0x7f590010dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f5905f31700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59000fcfb0 0x7f590010dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f5905f31700 1 -- 172.21.15.19:0/1555967477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc005040 con 0x7f59000fbe40 2024-02-22T15:50:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f5905f31700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f590010d800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0134a0 tx=0x7f58fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:30.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58fc00e030 con 0x7f59000fbe40 2024-02-22T15:50:30.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590010ede0 con 0x7f59000fbe40 2024-02-22T15:50:30.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590010f320 con 0x7f59000fbe40 2024-02-22T15:50:30.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58fc0093c0 con 0x7f59000fbe40 2024-02-22T15:50:30.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.645+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58fc0123c0 con 0x7f59000fbe40 2024-02-22T15:50:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.648+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e40052e0 con 0x7f59000fbe40 2024-02-22T15:50:30.632 INFO:tasks.workunit.client.0.smithi019.stdout:2/226: creat d1/de/d33/f4a x:0 0 0 2024-02-22T15:50:30.632 INFO:tasks.workunit.client.0.smithi019.stdout:1/393: dwrite d8/d15/d34/f4d [0,4194304] 0 2024-02-22T15:50:30.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.650+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f58fc013c30 con 0x7f59000fbe40 2024-02-22T15:50:30.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.650+0000 7f58f67fc700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f58ec06f3a0 0x7f58ec071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:30.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.651+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f58fc08b1c0 con 0x7f59000fbe40 2024-02-22T15:50:30.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.651+0000 7f5905730700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f58ec06f3a0 0x7f58ec071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:30.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.652+0000 7f5905730700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f58ec06f3a0 0x7f58ec071850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009b80 tx=0x7f58f000b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:30.636 INFO:tasks.workunit.client.0.smithi019.stdout:2/227: write d1/d17/d1c/f22 [1115260,38000] 0 2024-02-22T15:50:30.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:30.659+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f58fc059280 con 0x7f59000fbe40 2024-02-22T15:50:30.641 INFO:tasks.workunit.client.0.smithi019.stdout:1/394: dwrite d8/d23/d33/f37 [0,4194304] 0 2024-02-22T15:50:30.645 INFO:tasks.workunit.client.0.smithi019.stdout:2/228: rename d1/de/d33 to d1/d17/d4b 0 2024-02-22T15:50:30.647 INFO:tasks.workunit.client.0.smithi019.stdout:1/395: stat d8/c57 0 2024-02-22T15:50:30.647 INFO:tasks.workunit.client.0.smithi019.stdout:2/229: readlink d1/de/l20 0 2024-02-22T15:50:30.649 INFO:tasks.workunit.client.0.smithi019.stdout:1/396: creat d8/d18/d64/f89 x:0 0 0 2024-02-22T15:50:30.652 INFO:tasks.workunit.client.0.smithi019.stdout:1/397: creat d8/d18/d85/f8a x:0 0 0 2024-02-22T15:50:30.653 INFO:tasks.workunit.client.0.smithi019.stdout:1/398: rename d8 to d8/d15/d84/d8b 22 2024-02-22T15:50:30.653 INFO:tasks.workunit.client.0.smithi019.stdout:2/230: dread d1/de/d15/f35 [0,4194304] 0 2024-02-22T15:50:30.654 INFO:tasks.workunit.client.0.smithi019.stdout:1/399: rename d8/d15 to d8/d15/d34/d8c 22 2024-02-22T15:50:30.656 INFO:tasks.workunit.client.0.smithi019.stdout:2/231: mknod d1/de/d15/d3d/c4c 0 2024-02-22T15:50:30.657 INFO:tasks.workunit.client.0.smithi019.stdout:1/400: chown d8/d15/f69 123590054 1 2024-02-22T15:50:30.660 INFO:tasks.workunit.client.0.smithi019.stdout:1/401: rename d8/d18/f3b to d8/d18/d85/f8d 0 2024-02-22T15:50:30.661 INFO:tasks.workunit.client.0.smithi019.stdout:1/402: stat d8/d23 0 2024-02-22T15:50:30.662 INFO:tasks.workunit.client.0.smithi019.stdout:1/403: write d8/f53 [655057,33346] 0 2024-02-22T15:50:30.663 INFO:tasks.workunit.client.0.smithi019.stdout:1/404: write d8/f53 [1601835,47867] 0 2024-02-22T15:50:30.664 INFO:tasks.workunit.client.0.smithi019.stdout:1/405: chown d8/d15 867184 1 2024-02-22T15:50:30.668 INFO:tasks.workunit.client.0.smithi019.stdout:1/406: mkdir d8/d2a/d75/d81/d8e 0 2024-02-22T15:50:30.670 INFO:tasks.workunit.client.0.smithi019.stdout:1/407: mknod d8/d18/d64/d7f/d82/c8f 0 2024-02-22T15:50:30.670 INFO:tasks.workunit.client.0.smithi019.stdout:1/408: stat d8/d15/d34 0 2024-02-22T15:50:30.671 INFO:tasks.workunit.client.0.smithi019.stdout:2/232: dread d1/de/f31 [0,4194304] 0 2024-02-22T15:50:30.673 INFO:tasks.workunit.client.0.smithi019.stdout:1/409: symlink d8/d2a/l90 0 2024-02-22T15:50:30.673 INFO:tasks.workunit.client.0.smithi019.stdout:2/233: rename d1 to d1/d4d 22 2024-02-22T15:50:30.674 INFO:tasks.workunit.client.0.smithi019.stdout:2/234: write d1/f26 [3068154,77983] 0 2024-02-22T15:50:30.675 INFO:tasks.workunit.client.0.smithi019.stdout:1/410: mknod d8/d15/d34/c91 0 2024-02-22T15:50:30.677 INFO:tasks.workunit.client.0.smithi019.stdout:1/411: write d8/d15/d34/f65 [873848,68656] 0 2024-02-22T15:50:30.679 INFO:tasks.workunit.client.0.smithi019.stdout:2/235: creat d1/de/f4e x:0 0 0 2024-02-22T15:50:30.679 INFO:tasks.workunit.client.0.smithi019.stdout:2/236: fsync d1/de/f31 0 2024-02-22T15:50:30.682 INFO:tasks.workunit.client.0.smithi019.stdout:2/237: creat d1/d17/d1c/f4f x:0 0 0 2024-02-22T15:50:30.685 INFO:tasks.workunit.client.0.smithi019.stdout:2/238: rename d1/de/d15/c1d to d1/d17/d38/c50 0 2024-02-22T15:50:30.690 INFO:tasks.workunit.client.0.smithi019.stdout:2/239: dwrite d1/f26 [0,4194304] 0 2024-02-22T15:50:30.705 INFO:tasks.workunit.client.0.smithi019.stdout:2/240: dwrite d1/de/ff [0,4194304] 0 2024-02-22T15:50:30.708 INFO:tasks.workunit.client.0.smithi019.stdout:2/241: mkdir d1/d17/d21/d51 0 2024-02-22T15:50:30.709 INFO:tasks.workunit.client.0.smithi019.stdout:2/242: dread d1/f1f [8388608,4194304] 0 2024-02-22T15:50:30.711 INFO:tasks.workunit.client.0.smithi019.stdout:2/243: mknod d1/d17/d4b/c52 0 2024-02-22T15:50:30.711 INFO:tasks.workunit.client.0.smithi019.stdout:2/244: chown d1/c42 224 1 2024-02-22T15:50:30.718 INFO:tasks.workunit.client.0.smithi019.stdout:2/245: dwrite d1/d17/d21/f36 [0,4194304] 0 2024-02-22T15:50:30.721 INFO:tasks.workunit.client.0.smithi019.stdout:2/246: unlink d1/f26 0 2024-02-22T15:50:30.723 INFO:tasks.workunit.client.0.smithi019.stdout:2/247: truncate d1/d17/d1c/f4f 507249 0 2024-02-22T15:50:30.726 INFO:tasks.workunit.client.0.smithi019.stdout:2/248: mknod d1/d46/c53 0 2024-02-22T15:50:30.737 INFO:tasks.workunit.client.0.smithi019.stdout:2/249: dread d1/de/f31 [0,4194304] 0 2024-02-22T15:50:30.744 INFO:tasks.workunit.client.0.smithi019.stdout:2/250: link d1/d17/d4b/c52 d1/d17/d38/c54 0 2024-02-22T15:50:30.755 INFO:tasks.workunit.client.0.smithi019.stdout:2/251: dread d1/d17/d1c/f34 [0,4194304] 0 2024-02-22T15:50:30.757 INFO:tasks.workunit.client.0.smithi019.stdout:2/252: write d1/f18 [3283161,36552] 0 2024-02-22T15:50:30.760 INFO:tasks.workunit.client.0.smithi019.stdout:2/253: fsync d1/de/d15/f35 0 2024-02-22T15:50:30.766 INFO:tasks.workunit.client.0.smithi019.stdout:2/254: dwrite d1/d17/d19/f2f [0,4194304] 0 2024-02-22T15:50:30.768 INFO:tasks.workunit.client.0.smithi019.stdout:2/255: creat d1/d46/f55 x:0 0 0 2024-02-22T15:50:30.775 INFO:tasks.workunit.client.0.smithi019.stdout:2/256: mknod d1/c56 0 2024-02-22T15:50:30.777 INFO:tasks.workunit.client.0.smithi019.stdout:2/257: rename d1/d17/d38/c54 to d1/d17/d4b/c57 0 2024-02-22T15:50:30.780 INFO:tasks.workunit.client.0.smithi019.stdout:2/258: rename d1/de/ff to d1/de/d15/f58 0 2024-02-22T15:50:30.781 INFO:tasks.workunit.client.0.smithi019.stdout:2/259: rename d1/d17 to d1/d17/d21/d59 22 2024-02-22T15:50:30.782 INFO:tasks.workunit.client.0.smithi019.stdout:2/260: read d1/d17/d1c/f4f [28401,83832] 0 2024-02-22T15:50:30.786 INFO:tasks.workunit.client.1.smithi062.stdout:4/5: sync 2024-02-22T15:50:30.787 INFO:tasks.workunit.client.0.smithi019.stdout:2/261: rename d1/f12 to d1/de/f5a 0 2024-02-22T15:50:30.788 INFO:tasks.workunit.client.1.smithi062.stdout:4/6: unlink f1 0 2024-02-22T15:50:30.788 INFO:tasks.workunit.client.0.smithi019.stdout:2/262: chown d1/d17/d1c/f4f 1185206044 1 2024-02-22T15:50:30.790 INFO:tasks.workunit.client.1.smithi062.stdout:4/7: write f2 [2110376,65373] 0 2024-02-22T15:50:30.792 INFO:tasks.workunit.client.1.smithi062.stdout:4/8: creat f3 x:0 0 0 2024-02-22T15:50:30.793 INFO:tasks.workunit.client.1.smithi062.stdout:4/9: mkdir d4 0 2024-02-22T15:50:30.796 INFO:tasks.workunit.client.1.smithi062.stdout:4/10: link l0 d4/l5 0 2024-02-22T15:50:30.797 INFO:tasks.workunit.client.1.smithi062.stdout:4/11: truncate f3 384560 0 2024-02-22T15:50:30.804 INFO:tasks.workunit.client.0.smithi019.stdout:4/241: sync 2024-02-22T15:50:30.804 INFO:tasks.workunit.client.0.smithi019.stdout:0/179: sync 2024-02-22T15:50:30.805 INFO:tasks.workunit.client.0.smithi019.stdout:8/293: sync 2024-02-22T15:50:30.805 INFO:tasks.workunit.client.0.smithi019.stdout:7/345: sync 2024-02-22T15:50:30.806 INFO:tasks.workunit.client.1.smithi062.stdout:4/12: getdents d4 0 2024-02-22T15:50:30.806 INFO:tasks.workunit.client.0.smithi019.stdout:8/294: dread - d9/d14/d17/f45 zero size 2024-02-22T15:50:30.806 INFO:tasks.workunit.client.0.smithi019.stdout:8/295: chown d9/c15 15 1 2024-02-22T15:50:30.807 INFO:tasks.workunit.client.1.smithi062.stdout:4/13: rmdir d4 39 2024-02-22T15:50:30.807 INFO:tasks.workunit.client.0.smithi019.stdout:0/180: mknod d4/d5/c3f 0 2024-02-22T15:50:30.808 INFO:tasks.workunit.client.0.smithi019.stdout:0/181: fsync d4/d22/d2c/f30 0 2024-02-22T15:50:30.808 INFO:tasks.workunit.client.1.smithi062.stdout:4/14: dread f3 [0,4194304] 0 2024-02-22T15:50:30.809 INFO:tasks.workunit.client.0.smithi019.stdout:7/346: mkdir d9/d3a/d49/d5d/d74 0 2024-02-22T15:50:30.809 INFO:tasks.workunit.client.0.smithi019.stdout:0/182: write d4/d9/f13 [400723,37958] 0 2024-02-22T15:50:30.810 INFO:tasks.workunit.client.0.smithi019.stdout:0/183: dread - d4/d9/f26 zero size 2024-02-22T15:50:30.811 INFO:tasks.workunit.client.1.smithi062.stdout:4/15: dread f2 [0,4194304] 0 2024-02-22T15:50:30.812 INFO:tasks.workunit.client.0.smithi019.stdout:0/184: chown d4/db/d24 3862 1 2024-02-22T15:50:30.812 INFO:tasks.workunit.client.1.smithi062.stdout:4/16: truncate f3 1021227 0 2024-02-22T15:50:30.812 INFO:tasks.workunit.client.0.smithi019.stdout:8/296: creat d9/d14/d17/d5d/f63 x:0 0 0 2024-02-22T15:50:30.813 INFO:tasks.workunit.client.0.smithi019.stdout:7/347: write f8 [48943,52980] 0 2024-02-22T15:50:30.814 INFO:tasks.workunit.client.0.smithi019.stdout:8/297: read - d9/f58 zero size 2024-02-22T15:50:30.815 INFO:tasks.workunit.client.0.smithi019.stdout:4/242: creat d2/db/d14/d23/f4d x:0 0 0 2024-02-22T15:50:30.816 INFO:tasks.workunit.client.0.smithi019.stdout:0/185: mkdir d4/db/d1b/d40 0 2024-02-22T15:50:30.817 INFO:tasks.workunit.client.0.smithi019.stdout:8/298: write d9/f32 [561295,62843] 0 2024-02-22T15:50:30.818 INFO:tasks.workunit.client.0.smithi019.stdout:4/243: chown d2/db/d14/d23/d21/l22 641 1 2024-02-22T15:50:30.819 INFO:tasks.workunit.client.0.smithi019.stdout:4/244: truncate d2/f32 5032378 0 2024-02-22T15:50:30.820 INFO:tasks.workunit.client.0.smithi019.stdout:8/299: read d9/f12 [182359,35645] 0 2024-02-22T15:50:30.823 INFO:tasks.workunit.client.0.smithi019.stdout:4/245: unlink d2/c12 0 2024-02-22T15:50:30.825 INFO:tasks.workunit.client.0.smithi019.stdout:7/348: dwrite d9/d3a/f60 [0,4194304] 0 2024-02-22T15:50:30.831 INFO:tasks.workunit.client.0.smithi019.stdout:8/300: dwrite d9/d14/d17/f4a [0,4194304] 0 2024-02-22T15:50:30.832 INFO:tasks.workunit.client.0.smithi019.stdout:7/349: rename d9/d4c/d5b/f73 to d9/d3a/f75 0 2024-02-22T15:50:30.834 INFO:tasks.workunit.client.0.smithi019.stdout:8/301: chown d9/d14/d17/d3d/d3f 733 1 2024-02-22T15:50:30.838 INFO:tasks.workunit.client.0.smithi019.stdout:8/302: stat d9/d14/d17/c5f 0 2024-02-22T15:50:30.841 INFO:tasks.workunit.client.0.smithi019.stdout:8/303: creat d9/f64 x:0 0 0 2024-02-22T15:50:30.843 INFO:tasks.workunit.client.0.smithi019.stdout:8/304: creat d9/f65 x:0 0 0 2024-02-22T15:50:30.844 INFO:tasks.workunit.client.0.smithi019.stdout:8/305: chown d9/d14/d1f/f33 43772 1 2024-02-22T15:50:30.845 INFO:tasks.workunit.client.0.smithi019.stdout:8/306: readlink d9/d14/d17/l2e 0 2024-02-22T15:50:30.847 INFO:tasks.workunit.client.0.smithi019.stdout:8/307: creat d9/f66 x:0 0 0 2024-02-22T15:50:30.848 INFO:tasks.workunit.client.0.smithi019.stdout:8/308: write d9/f66 [292355,117706] 0 2024-02-22T15:50:30.850 INFO:tasks.workunit.client.0.smithi019.stdout:8/309: write d9/d14/d17/d1c/f48 [4105017,19492] 0 2024-02-22T15:50:30.853 INFO:tasks.workunit.client.0.smithi019.stdout:8/310: mknod d9/d14/d1f/d5c/c67 0 2024-02-22T15:50:30.854 INFO:tasks.workunit.client.0.smithi019.stdout:8/311: creat d9/d14/d17/d5d/f68 x:0 0 0 2024-02-22T15:50:30.855 INFO:tasks.workunit.client.0.smithi019.stdout:4/246: truncate d2/f6 3937189 0 2024-02-22T15:50:30.857 INFO:tasks.workunit.client.0.smithi019.stdout:4/247: stat d2/c13 0 2024-02-22T15:50:30.860 INFO:tasks.workunit.client.0.smithi019.stdout:8/312: getdents d9/d14/d1f/d4c 0 2024-02-22T15:50:30.864 INFO:tasks.workunit.client.0.smithi019.stdout:8/313: creat d9/f69 x:0 0 0 2024-02-22T15:50:30.872 INFO:tasks.workunit.client.0.smithi019.stdout:8/314: dwrite d9/d14/d17/f25 [0,4194304] 0 2024-02-22T15:50:30.877 INFO:tasks.workunit.client.0.smithi019.stdout:8/315: truncate d9/d14/f34 966174 0 2024-02-22T15:50:30.878 INFO:tasks.workunit.client.0.smithi019.stdout:4/248: dwrite d2/f6 [4194304,4194304] 0 2024-02-22T15:50:30.879 INFO:tasks.workunit.client.0.smithi019.stdout:4/249: chown d2/db/d44 398274 1 2024-02-22T15:50:30.881 INFO:tasks.workunit.client.0.smithi019.stdout:8/316: unlink d9/c3c 0 2024-02-22T15:50:30.882 INFO:tasks.workunit.client.0.smithi019.stdout:4/250: symlink d2/d3f/l4e 0 2024-02-22T15:50:30.884 INFO:tasks.workunit.client.0.smithi019.stdout:4/251: chown d2/db/d14/d23/d21/f3a 0 1 2024-02-22T15:50:30.885 INFO:tasks.workunit.client.0.smithi019.stdout:4/252: readlink d2/db/d14/l3b 0 2024-02-22T15:50:30.888 INFO:tasks.workunit.client.0.smithi019.stdout:8/317: dwrite f5 [0,4194304] 0 2024-02-22T15:50:30.890 INFO:tasks.workunit.client.0.smithi019.stdout:8/318: write d9/d14/f3e [3947996,87665] 0 2024-02-22T15:50:30.890 INFO:tasks.workunit.client.0.smithi019.stdout:4/253: rename d2/d3f/l4e to d2/l4f 0 2024-02-22T15:50:30.893 INFO:tasks.workunit.client.0.smithi019.stdout:4/254: fsync d2/db/d14/d23/d21/f2c 0 2024-02-22T15:50:30.895 INFO:tasks.workunit.client.0.smithi019.stdout:8/319: creat d9/d14/f6a x:0 0 0 2024-02-22T15:50:30.896 INFO:tasks.workunit.client.0.smithi019.stdout:4/255: mknod d2/db/d14/d23/d21/c50 0 2024-02-22T15:50:30.899 INFO:tasks.workunit.client.0.smithi019.stdout:4/256: dread - d2/db/d14/d23/f28 zero size 2024-02-22T15:50:30.900 INFO:tasks.workunit.client.0.smithi019.stdout:8/320: rmdir d9/d14/d17/d5d 39 2024-02-22T15:50:30.902 INFO:tasks.workunit.client.0.smithi019.stdout:4/257: dread d2/db/d14/d23/d21/f29 [0,4194304] 0 2024-02-22T15:50:30.903 INFO:tasks.workunit.client.0.smithi019.stdout:8/321: symlink d9/d14/l6b 0 2024-02-22T15:50:30.904 INFO:tasks.workunit.client.0.smithi019.stdout:8/322: chown d9/d14/f38 1655893 1 2024-02-22T15:50:30.905 INFO:tasks.workunit.client.0.smithi019.stdout:4/258: getdents d2/db/d44 0 2024-02-22T15:50:30.905 INFO:tasks.workunit.client.0.smithi019.stdout:8/323: readlink d9/d14/d17/d3d/d3f/l4f 0 2024-02-22T15:50:30.907 INFO:tasks.workunit.client.0.smithi019.stdout:8/324: read - d9/d14/d1f/f4e zero size 2024-02-22T15:50:30.909 INFO:tasks.workunit.client.0.smithi019.stdout:8/325: mknod d9/d14/d17/d1c/c6c 0 2024-02-22T15:50:30.913 INFO:tasks.workunit.client.0.smithi019.stdout:4/259: dwrite d2/db/d14/d23/d21/f2d [4194304,4194304] 0 2024-02-22T15:50:30.915 INFO:tasks.workunit.client.0.smithi019.stdout:8/326: write d9/fc [1489723,88050] 0 2024-02-22T15:50:30.915 INFO:tasks.workunit.client.0.smithi019.stdout:4/260: write d2/db/d14/d23/f36 [120729,61427] 0 2024-02-22T15:50:30.916 INFO:tasks.workunit.client.0.smithi019.stdout:4/261: chown d2/db/d14/f18 3635 1 2024-02-22T15:50:30.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/262: chown d2/db/d14/d23 95 1 2024-02-22T15:50:30.919 INFO:tasks.workunit.client.0.smithi019.stdout:4/263: write d2/f6 [8530037,33942] 0 2024-02-22T15:50:30.919 INFO:tasks.workunit.client.0.smithi019.stdout:4/264: stat d2/db/c1d 0 2024-02-22T15:50:30.921 INFO:tasks.workunit.client.0.smithi019.stdout:4/265: creat d2/d3f/f51 x:0 0 0 2024-02-22T15:50:30.923 INFO:tasks.workunit.client.0.smithi019.stdout:4/266: mknod d2/d3f/c52 0 2024-02-22T15:50:30.927 INFO:tasks.workunit.client.0.smithi019.stdout:4/267: rename d2/f1f to d2/d3f/f53 0 2024-02-22T15:50:30.928 INFO:tasks.workunit.client.0.smithi019.stdout:4/268: write d2/d3f/d48/f3e [666423,33855] 0 2024-02-22T15:50:30.932 INFO:tasks.workunit.client.0.smithi019.stdout:4/269: getdents d2/db/d14 0 2024-02-22T15:50:30.935 INFO:tasks.workunit.client.0.smithi019.stdout:4/270: mkdir d2/db/d54 0 2024-02-22T15:50:30.937 INFO:tasks.workunit.client.0.smithi019.stdout:4/271: creat d2/db/d44/f55 x:0 0 0 2024-02-22T15:50:30.942 INFO:tasks.workunit.client.0.smithi019.stdout:4/272: truncate d2/d3f/f53 1690723 0 2024-02-22T15:50:30.943 INFO:tasks.workunit.client.0.smithi019.stdout:4/273: symlink d2/l56 0 2024-02-22T15:50:30.945 INFO:tasks.workunit.client.0.smithi019.stdout:4/274: mkdir d2/db/d57 0 2024-02-22T15:50:30.947 INFO:tasks.workunit.client.0.smithi019.stdout:4/275: mknod d2/db/d14/d23/c58 0 2024-02-22T15:50:30.948 INFO:tasks.workunit.client.0.smithi019.stdout:4/276: mkdir d2/db/d59 0 2024-02-22T15:50:30.949 INFO:tasks.workunit.client.0.smithi019.stdout:4/277: write d2/db/f26 [806644,124636] 0 2024-02-22T15:50:30.951 INFO:tasks.workunit.client.0.smithi019.stdout:4/278: read d2/db/d14/f18 [2270199,106060] 0 2024-02-22T15:50:30.954 INFO:tasks.workunit.client.0.smithi019.stdout:4/279: creat d2/db/d14/d23/d21/f5a x:0 0 0 2024-02-22T15:50:30.955 INFO:tasks.workunit.client.0.smithi019.stdout:4/280: write d2/db/d14/d23/d21/d35/f38 [852214,127770] 0 2024-02-22T15:50:30.961 INFO:tasks.workunit.client.0.smithi019.stdout:4/281: dwrite d2/db/d14/d23/f36 [0,4194304] 0 2024-02-22T15:50:30.962 INFO:tasks.workunit.client.0.smithi019.stdout:4/282: stat d2/lc 0 2024-02-22T15:50:30.968 INFO:tasks.workunit.client.0.smithi019.stdout:4/283: link d2/db/d14/d23/d21/f2c d2/d3f/d48/f5b 0 2024-02-22T15:50:30.969 INFO:tasks.workunit.client.0.smithi019.stdout:4/284: symlink d2/db/d44/l5c 0 2024-02-22T15:50:30.970 INFO:tasks.workunit.client.0.smithi019.stdout:4/285: mknod d2/db/d54/c5d 0 2024-02-22T15:50:30.984 INFO:tasks.workunit.client.0.smithi019.stdout:4/286: rename d2/d3f/f53 to d2/db/d14/f5e 0 2024-02-22T15:50:30.986 INFO:tasks.workunit.client.0.smithi019.stdout:4/287: symlink d2/db/d57/l5f 0 2024-02-22T15:50:30.988 INFO:tasks.workunit.client.0.smithi019.stdout:4/288: unlink d2/lc 0 2024-02-22T15:50:30.990 INFO:tasks.workunit.client.0.smithi019.stdout:4/289: symlink d2/db/d44/l60 0 2024-02-22T15:50:30.998 INFO:tasks.workunit.client.0.smithi019.stdout:4/290: rename d2/db/d57 to d2/db/d14/d61 0 2024-02-22T15:50:31.000 INFO:tasks.workunit.client.0.smithi019.stdout:4/291: symlink d2/db/d14/d23/d33/l62 0 2024-02-22T15:50:31.001 INFO:tasks.workunit.client.0.smithi019.stdout:6/251: sync 2024-02-22T15:50:31.001 INFO:tasks.workunit.client.0.smithi019.stdout:9/281: sync 2024-02-22T15:50:31.002 INFO:tasks.workunit.client.0.smithi019.stdout:4/292: write d2/db/d14/f19 [4413632,7260] 0 2024-02-22T15:50:31.002 INFO:tasks.workunit.client.0.smithi019.stdout:9/282: chown d2/d1f/d2e/d48/d36/d4a 322291293 1 2024-02-22T15:50:31.005 INFO:tasks.workunit.client.0.smithi019.stdout:4/293: mknod d2/c63 0 2024-02-22T15:50:31.006 INFO:tasks.workunit.client.0.smithi019.stdout:6/252: creat d6/d31/d43/f46 x:0 0 0 2024-02-22T15:50:31.008 INFO:tasks.workunit.client.0.smithi019.stdout:9/283: getdents d2/d7/dc 0 2024-02-22T15:50:31.009 INFO:tasks.workunit.client.0.smithi019.stdout:9/284: dread - d2/d1f/d2e/d48/f4d zero size 2024-02-22T15:50:31.009 INFO:tasks.workunit.client.0.smithi019.stdout:4/294: creat d2/db/d14/d23/d21/d35/f64 x:0 0 0 2024-02-22T15:50:31.011 INFO:tasks.workunit.client.0.smithi019.stdout:9/285: fsync d2/f4f 0 2024-02-22T15:50:31.013 INFO:tasks.workunit.client.0.smithi019.stdout:6/253: rename d6/f18 to d6/f47 0 2024-02-22T15:50:31.014 INFO:tasks.workunit.client.0.smithi019.stdout:9/286: mknod d2/c65 0 2024-02-22T15:50:31.018 INFO:tasks.workunit.client.0.smithi019.stdout:4/295: creat d2/db/d14/d23/f65 x:0 0 0 2024-02-22T15:50:31.021 INFO:tasks.workunit.client.0.smithi019.stdout:9/287: rename d2/d1f/d2e/d56 to d2/d1f/d2e/d48/d36/d4a/d66 0 2024-02-22T15:50:31.023 INFO:tasks.workunit.client.0.smithi019.stdout:4/296: mkdir d2/db/d14/d23/d33/d66 0 2024-02-22T15:50:31.027 INFO:tasks.workunit.client.0.smithi019.stdout:4/297: mkdir d2/db/d14/d23/d67 0 2024-02-22T15:50:31.028 INFO:tasks.workunit.client.0.smithi019.stdout:4/298: stat d2/db/d14/d23/d33/l62 0 2024-02-22T15:50:31.033 INFO:tasks.workunit.client.0.smithi019.stdout:6/254: truncate d6/f1d 902616 0 2024-02-22T15:50:31.034 INFO:tasks.workunit.client.0.smithi019.stdout:9/288: rmdir d2/d1f/d2e/d48/d36/d4a/d66 39 2024-02-22T15:50:31.036 INFO:tasks.workunit.client.0.smithi019.stdout:6/255: mkdir d6/d31/d48 0 2024-02-22T15:50:31.039 INFO:tasks.workunit.client.0.smithi019.stdout:4/299: write d2/db/d14/f5e [2309994,128442] 0 2024-02-22T15:50:31.043 INFO:tasks.workunit.client.0.smithi019.stdout:4/300: readlink d2/l4f 0 2024-02-22T15:50:31.045 INFO:tasks.workunit.client.0.smithi019.stdout:6/256: write d6/f14 [1506348,35920] 0 2024-02-22T15:50:31.046 INFO:tasks.workunit.client.0.smithi019.stdout:9/289: truncate d2/d7/f44 3064861 0 2024-02-22T15:50:31.049 INFO:tasks.workunit.client.0.smithi019.stdout:6/257: write d6/d31/d43/f46 [507234,55329] 0 2024-02-22T15:50:31.052 INFO:tasks.workunit.client.0.smithi019.stdout:4/301: dwrite d2/f6 [0,4194304] 0 2024-02-22T15:50:31.054 INFO:tasks.workunit.client.0.smithi019.stdout:6/258: rmdir d6 39 2024-02-22T15:50:31.056 INFO:tasks.workunit.client.0.smithi019.stdout:9/290: symlink d2/d1f/d2e/d48/d36/d4a/d66/l67 0 2024-02-22T15:50:31.062 INFO:tasks.workunit.client.0.smithi019.stdout:4/302: dread d2/d3f/d48/f5b [0,4194304] 0 2024-02-22T15:50:31.065 INFO:tasks.workunit.client.0.smithi019.stdout:6/259: rename d6/c15 to d6/d3e/c49 0 2024-02-22T15:50:31.068 INFO:tasks.workunit.client.0.smithi019.stdout:9/291: getdents d2/d7 0 2024-02-22T15:50:31.072 INFO:tasks.workunit.client.0.smithi019.stdout:4/303: dwrite d2/db/d14/d23/f65 [0,4194304] 0 2024-02-22T15:50:31.073 INFO:tasks.workunit.client.0.smithi019.stdout:9/292: mkdir d2/d7/dc/d27/d68 0 2024-02-22T15:50:31.073 INFO:tasks.workunit.client.0.smithi019.stdout:9/293: fsync d2/f4f 0 2024-02-22T15:50:31.074 INFO:tasks.workunit.client.0.smithi019.stdout:6/260: creat d6/d31/d48/f4a x:0 0 0 2024-02-22T15:50:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.096+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58e4005860 con 0x7f59000fbe40 2024-02-22T15:50:31.081 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:50:31.081 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:50:31.081 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:50:31.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:50:31.081 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:50:31.081 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 1, 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 13, 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:50:31.082 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:50:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.097+0000 7f58f67fc700 1 -- 172.21.15.19:0/1555967477 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+848 (secure 0 0 0) 0x7f58fc058e10 con 0x7f59000fbe40 2024-02-22T15:50:31.083 INFO:tasks.workunit.client.0.smithi019.stdout:6/261: mkdir d6/d4b 0 2024-02-22T15:50:31.083 INFO:tasks.workunit.client.0.smithi019.stdout:4/304: rename d2/db/d14/f19 to d2/db/d14/d23/d67/f68 0 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f58ec06f3a0 msgr2=0x7f58ec071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f58ec06f3a0 0x7f58ec071850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009b80 tx=0x7f58f000b040 comp rx=0 tx=0).stop 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 msgr2=0x7f590010d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f590010d800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0134a0 tx=0x7f58fc009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 shutdown_connections 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f58ec06f3a0 0x7f58ec071850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59000fbe40 0x7f590010d800 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 --2- 172.21.15.19:0/1555967477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59000fcfb0 0x7f590010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 >> 172.21.15.19:0/1555967477 conn(0x7f59000f7500 msgr2=0x7f5900065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 shutdown_connections 2024-02-22T15:50:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.101+0000 7f5908195700 1 -- 172.21.15.19:0/1555967477 wait complete. 2024-02-22T15:50:31.085 INFO:tasks.workunit.client.0.smithi019.stdout:6/262: rmdir d6 39 2024-02-22T15:50:31.085 INFO:tasks.workunit.client.0.smithi019.stdout:9/294: dwrite d2/fb [4194304,4194304] 0 2024-02-22T15:50:31.090 INFO:tasks.workunit.client.0.smithi019.stdout:4/305: unlink d2/c1a 0 2024-02-22T15:50:31.091 INFO:tasks.workunit.client.0.smithi019.stdout:9/295: dread d2/d1f/f47 [0,4194304] 0 2024-02-22T15:50:31.091 INFO:tasks.workunit.client.0.smithi019.stdout:4/306: dread - d2/db/d14/d23/d21/f5a zero size 2024-02-22T15:50:31.093 INFO:tasks.workunit.client.0.smithi019.stdout:6/263: creat d6/f4c x:0 0 0 2024-02-22T15:50:31.098 INFO:tasks.workunit.client.0.smithi019.stdout:6/264: unlink d6/l12 0 2024-02-22T15:50:31.100 INFO:tasks.workunit.client.0.smithi019.stdout:6/265: write d6/fb [2885083,91717] 0 2024-02-22T15:50:31.102 INFO:tasks.workunit.client.0.smithi019.stdout:9/296: dwrite d2/fb [0,4194304] 0 2024-02-22T15:50:31.103 INFO:tasks.workunit.client.0.smithi019.stdout:6/266: chown d6/f47 77100 1 2024-02-22T15:50:31.103 INFO:tasks.workunit.client.0.smithi019.stdout:4/307: dread d2/db/f25 [0,4194304] 0 2024-02-22T15:50:31.105 INFO:tasks.workunit.client.0.smithi019.stdout:6/267: write d6/d3e/f42 [298756,32669] 0 2024-02-22T15:50:31.105 INFO:tasks.workunit.client.0.smithi019.stdout:4/308: chown d2/d3f/f51 1 1 2024-02-22T15:50:31.106 INFO:tasks.workunit.client.0.smithi019.stdout:9/297: truncate d2/d7/dc/f49 1124848 0 2024-02-22T15:50:31.108 INFO:tasks.workunit.client.0.smithi019.stdout:4/309: chown d2/db/d14/d23/d21/d35/l37 178926 1 2024-02-22T15:50:31.110 INFO:tasks.workunit.client.0.smithi019.stdout:4/310: read d2/d3f/d48/f3e [254888,114886] 0 2024-02-22T15:50:31.111 INFO:tasks.workunit.client.0.smithi019.stdout:6/268: truncate d6/ff 634586 0 2024-02-22T15:50:31.111 INFO:tasks.workunit.client.0.smithi019.stdout:6/269: readlink d6/l45 0 2024-02-22T15:50:31.114 INFO:tasks.workunit.client.0.smithi019.stdout:6/270: unlink d6/l16 0 2024-02-22T15:50:31.117 INFO:tasks.workunit.client.0.smithi019.stdout:4/311: creat d2/db/f69 x:0 0 0 2024-02-22T15:50:31.118 INFO:tasks.workunit.client.0.smithi019.stdout:4/312: chown d2/db/d14/d23/d21/d35/d3d 3 1 2024-02-22T15:50:31.119 INFO:tasks.workunit.client.0.smithi019.stdout:6/271: creat d6/f4d x:0 0 0 2024-02-22T15:50:31.121 INFO:tasks.workunit.client.0.smithi019.stdout:4/313: write d2/db/d14/d23/d21/f3a [1469549,126420] 0 2024-02-22T15:50:31.125 INFO:tasks.workunit.client.0.smithi019.stdout:6/272: dread d6/f41 [0,4194304] 0 2024-02-22T15:50:31.126 INFO:tasks.workunit.client.0.smithi019.stdout:9/298: dread d2/d7/f44 [0,4194304] 0 2024-02-22T15:50:31.126 INFO:tasks.workunit.client.0.smithi019.stdout:6/273: write d6/f4d [618741,52709] 0 2024-02-22T15:50:31.126 INFO:tasks.workunit.client.0.smithi019.stdout:4/314: creat d2/db/f6a x:0 0 0 2024-02-22T15:50:31.130 INFO:tasks.workunit.client.0.smithi019.stdout:6/274: read d6/f4d [576049,1126] 0 2024-02-22T15:50:31.131 INFO:tasks.workunit.client.0.smithi019.stdout:4/315: read d2/db/d14/d23/f36 [3739766,69058] 0 2024-02-22T15:50:31.131 INFO:tasks.workunit.client.0.smithi019.stdout:9/299: truncate d2/d1f/d2e/d48/d45/f61 119025 0 2024-02-22T15:50:31.134 INFO:tasks.workunit.client.0.smithi019.stdout:6/275: fsync d6/f3c 0 2024-02-22T15:50:31.137 INFO:tasks.workunit.client.0.smithi019.stdout:6/276: truncate d6/d31/f32 802173 0 2024-02-22T15:50:31.140 INFO:tasks.workunit.client.0.smithi019.stdout:9/300: symlink d2/d7/dc/d27/d68/l69 0 2024-02-22T15:50:31.141 INFO:tasks.workunit.client.0.smithi019.stdout:6/277: creat d6/d3e/f4e x:0 0 0 2024-02-22T15:50:31.142 INFO:tasks.workunit.client.0.smithi019.stdout:4/316: dwrite d2/fa [4194304,4194304] 0 2024-02-22T15:50:31.152 INFO:tasks.workunit.client.0.smithi019.stdout:6/278: mknod d6/d3e/c4f 0 2024-02-22T15:50:31.152 INFO:tasks.workunit.client.0.smithi019.stdout:4/317: unlink d2/db/d14/d23/c45 0 2024-02-22T15:50:31.154 INFO:tasks.workunit.client.0.smithi019.stdout:4/318: fdatasync d2/db/d14/d23/f65 0 2024-02-22T15:50:31.155 INFO:tasks.workunit.client.0.smithi019.stdout:9/301: creat d2/d1f/d2e/d48/d36/d4a/f6a x:0 0 0 2024-02-22T15:50:31.156 INFO:tasks.workunit.client.0.smithi019.stdout:6/279: rename d6/fb to d6/d3e/f50 0 2024-02-22T15:50:31.160 INFO:tasks.workunit.client.0.smithi019.stdout:9/302: creat d2/d7/dc/d27/d68/f6b x:0 0 0 2024-02-22T15:50:31.162 INFO:tasks.workunit.client.0.smithi019.stdout:4/319: creat d2/d3f/f6b x:0 0 0 2024-02-22T15:50:31.164 INFO:tasks.workunit.client.0.smithi019.stdout:6/280: dread d6/f22 [0,4194304] 0 2024-02-22T15:50:31.164 INFO:tasks.workunit.client.0.smithi019.stdout:9/303: mknod d2/d1f/c6c 0 2024-02-22T15:50:31.165 INFO:tasks.workunit.client.0.smithi019.stdout:4/320: write d2/db/d14/d23/f28 [671531,80607] 0 2024-02-22T15:50:31.167 INFO:tasks.workunit.client.0.smithi019.stdout:4/321: write d2/db/d14/d23/f4d [41256,48625] 0 2024-02-22T15:50:31.169 INFO:tasks.workunit.client.0.smithi019.stdout:9/304: rename d2/l53 to d2/d7/dc/d27/d37/l6d 0 2024-02-22T15:50:31.175 INFO:tasks.workunit.client.0.smithi019.stdout:6/281: dwrite d6/f41 [0,4194304] 0 2024-02-22T15:50:31.175 INFO:tasks.workunit.client.0.smithi019.stdout:4/322: truncate d2/db/d14/d23/f65 693796 0 2024-02-22T15:50:31.176 INFO:tasks.workunit.client.0.smithi019.stdout:6/282: chown d6/f4c 31559 1 2024-02-22T15:50:31.176 INFO:tasks.workunit.client.0.smithi019.stdout:6/283: readlink d6/l27 0 2024-02-22T15:50:31.176 INFO:tasks.workunit.client.0.smithi019.stdout:3/298: sync 2024-02-22T15:50:31.176 INFO:tasks.workunit.client.0.smithi019.stdout:5/317: sync 2024-02-22T15:50:31.177 INFO:tasks.workunit.client.0.smithi019.stdout:9/305: rename d2/d7/d14/l3c to d2/d1f/d2e/d48/d45/l6e 0 2024-02-22T15:50:31.183 INFO:tasks.workunit.client.0.smithi019.stdout:9/306: creat d2/f6f x:0 0 0 2024-02-22T15:50:31.188 INFO:tasks.workunit.client.0.smithi019.stdout:5/318: creat d8/d11/f6e x:0 0 0 2024-02-22T15:50:31.191 INFO:tasks.workunit.client.0.smithi019.stdout:4/323: truncate d2/db/d14/d23/f2f 3058470 0 2024-02-22T15:50:31.192 INFO:tasks.workunit.client.0.smithi019.stdout:5/319: write d8/dd/d19/f29 [1118433,9997] 0 2024-02-22T15:50:31.193 INFO:tasks.workunit.client.0.smithi019.stdout:9/307: dread d2/d1f/d2e/d48/d36/f3b [0,4194304] 0 2024-02-22T15:50:31.194 INFO:tasks.workunit.client.0.smithi019.stdout:6/284: dwrite d6/d31/f35 [4194304,4194304] 0 2024-02-22T15:50:31.201 INFO:tasks.workunit.client.0.smithi019.stdout:5/320: symlink d8/dd/d19/d38/l6f 0 2024-02-22T15:50:31.201 INFO:tasks.workunit.client.0.smithi019.stdout:3/299: dwrite d9/d26/d32/f33 [4194304,4194304] 0 2024-02-22T15:50:31.203 INFO:tasks.workunit.client.0.smithi019.stdout:4/324: truncate d2/db/d14/d23/f28 1037095 0 2024-02-22T15:50:31.203 INFO:tasks.workunit.client.0.smithi019.stdout:5/321: dread - d8/f4e zero size 2024-02-22T15:50:31.206 INFO:tasks.workunit.client.0.smithi019.stdout:4/325: stat d2/db/cf 0 2024-02-22T15:50:31.207 INFO:tasks.workunit.client.0.smithi019.stdout:5/322: fdatasync d8/f4e 0 2024-02-22T15:50:31.211 INFO:tasks.workunit.client.0.smithi019.stdout:5/323: stat d8/d11/f6e 0 2024-02-22T15:50:31.214 INFO:tasks.workunit.client.0.smithi019.stdout:5/324: write f1 [1086946,128693] 0 2024-02-22T15:50:31.215 INFO:tasks.workunit.client.0.smithi019.stdout:6/285: truncate d6/f17 1921388 0 2024-02-22T15:50:31.218 INFO:tasks.workunit.client.0.smithi019.stdout:3/300: dread d9/d22/d27/f46 [4194304,4194304] 0 2024-02-22T15:50:31.218 INFO:tasks.workunit.client.0.smithi019.stdout:9/308: dwrite d2/d7/f44 [0,4194304] 0 2024-02-22T15:50:31.219 INFO:tasks.workunit.client.0.smithi019.stdout:6/286: creat d6/d31/d48/f51 x:0 0 0 2024-02-22T15:50:31.220 INFO:tasks.workunit.client.0.smithi019.stdout:5/325: symlink d8/d11/d33/l70 0 2024-02-22T15:50:31.224 INFO:tasks.workunit.client.0.smithi019.stdout:6/287: fsync d6/f47 0 2024-02-22T15:50:31.224 INFO:tasks.workunit.client.0.smithi019.stdout:6/288: readlink d6/l3a 0 2024-02-22T15:50:31.224 INFO:tasks.workunit.client.0.smithi019.stdout:9/309: mkdir d2/d7/d52/d70 0 2024-02-22T15:50:31.228 INFO:tasks.workunit.client.0.smithi019.stdout:5/326: write d8/dd/d19/d3a/d46/f61 [835946,86585] 0 2024-02-22T15:50:31.229 INFO:tasks.workunit.client.0.smithi019.stdout:3/301: link d9/c44 d9/d22/d49/c5e 0 2024-02-22T15:50:31.231 INFO:tasks.workunit.client.0.smithi019.stdout:6/289: rename d6/f25 to d6/d4b/f52 0 2024-02-22T15:50:31.233 INFO:tasks.workunit.client.0.smithi019.stdout:6/290: chown d6/d31/d37/l3d 13 1 2024-02-22T15:50:31.234 INFO:tasks.workunit.client.0.smithi019.stdout:6/291: fdatasync d6/f28 0 2024-02-22T15:50:31.235 INFO:tasks.workunit.client.0.smithi019.stdout:3/302: creat d9/d26/d2d/d39/f5f x:0 0 0 2024-02-22T15:50:31.236 INFO:tasks.workunit.client.0.smithi019.stdout:9/310: rename d2/d7/dc/f11 to d2/d1f/d2e/d48/d36/f71 0 2024-02-22T15:50:31.243 INFO:tasks.workunit.client.0.smithi019.stdout:3/303: dread d9/d26/d32/f43 [0,4194304] 0 2024-02-22T15:50:31.243 INFO:tasks.workunit.client.0.smithi019.stdout:6/292: symlink d6/l53 0 2024-02-22T15:50:31.246 INFO:tasks.workunit.client.0.smithi019.stdout:9/311: dwrite d2/f4f [0,4194304] 0 2024-02-22T15:50:31.251 INFO:tasks.workunit.client.0.smithi019.stdout:9/312: creat d2/d1f/d2e/d48/d45/d5a/f72 x:0 0 0 2024-02-22T15:50:31.253 INFO:tasks.workunit.client.0.smithi019.stdout:3/304: symlink d9/d11/l60 0 2024-02-22T15:50:31.256 INFO:tasks.workunit.client.0.smithi019.stdout:5/327: dread d8/ff [0,4194304] 0 2024-02-22T15:50:31.258 INFO:tasks.workunit.client.0.smithi019.stdout:9/313: getdents d2/d7/d52/d70 0 2024-02-22T15:50:31.258 INFO:tasks.workunit.client.0.smithi019.stdout:9/314: stat d2/d7/c8 0 2024-02-22T15:50:31.261 INFO:tasks.workunit.client.0.smithi019.stdout:6/293: write d6/f1d [1798391,108062] 0 2024-02-22T15:50:31.262 INFO:tasks.workunit.client.0.smithi019.stdout:5/328: dread d8/dd/d19/d3a/f43 [0,4194304] 0 2024-02-22T15:50:31.262 INFO:tasks.workunit.client.0.smithi019.stdout:6/294: readlink d6/l27 0 2024-02-22T15:50:31.263 INFO:tasks.workunit.client.0.smithi019.stdout:9/315: creat d2/d7/dc/d27/d68/f73 x:0 0 0 2024-02-22T15:50:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.282+0000 7f6941f26700 1 -- 172.21.15.19:0/4016037037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c104f90 msgr2=0x7f693c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.282+0000 7f6941f26700 1 --2- 172.21.15.19:0/4016037037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c104f90 0x7f693c107370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f692c004940 tx=0x7f692c009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 -- 172.21.15.19:0/4016037037 shutdown_connections 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 --2- 172.21.15.19:0/4016037037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 --2- 172.21.15.19:0/4016037037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c104f90 0x7f693c107370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 -- 172.21.15.19:0/4016037037 >> 172.21.15.19:0/4016037037 conn(0x7f693c06a2c0 msgr2=0x7f693c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 -- 172.21.15.19:0/4016037037 shutdown_connections 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 -- 172.21.15.19:0/4016037037 wait complete. 2024-02-22T15:50:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.283+0000 7f6941f26700 1 Processor -- start 2024-02-22T15:50:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f6941f26700 1 -- start start 2024-02-22T15:50:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f6941f26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f6941f26700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c114580 0x7f693c115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f6941f26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c114a40 con 0x7f693c1078b0 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f6941f26700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c114b80 con 0x7f693c114580 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f693b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f693b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53842/0 (socket says 172.21.15.19:53842) 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f693b7fe700 1 -- 172.21.15.19:0/4247535911 learned_addr learned my addr 172.21.15.19:0/4247535911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.284+0000 7f693affd700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c114580 0x7f693c115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:31.267 INFO:tasks.workunit.client.0.smithi019.stdout:6/295: creat d6/d4b/f54 x:0 0 0 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f693b7fe700 1 -- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c114580 msgr2=0x7f693c115420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f693b7fe700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c114580 0x7f693c115420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f693b7fe700 1 -- 172.21.15.19:0/4247535911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c005040 con 0x7f693c1078b0 2024-02-22T15:50:31.268 INFO:tasks.workunit.client.0.smithi019.stdout:9/316: write d2/d1f/d2e/d48/f4e [974575,5906] 0 2024-02-22T15:50:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f693b7fe700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c114040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b3b0 tx=0x7f692c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692c00e040 con 0x7f693c1078b0 2024-02-22T15:50:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f692c004290 con 0x7f693c1078b0 2024-02-22T15:50:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.285+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692c01c3f0 con 0x7f693c1078b0 2024-02-22T15:50:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.286+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c115a20 con 0x7f693c1078b0 2024-02-22T15:50:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.286+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c115f30 con 0x7f693c1078b0 2024-02-22T15:50:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.287+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693c10df20 con 0x7f693c1078b0 2024-02-22T15:50:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.288+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f692c013790 con 0x7f693c1078b0 2024-02-22T15:50:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.289+0000 7f6923fff700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f692406d260 0x7f692406f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:50:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.289+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f692c08a4c0 con 0x7f693c1078b0 2024-02-22T15:50:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.289+0000 7f693affd700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f692406d260 0x7f692406f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:50:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.289+0000 7f693affd700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f692406d260 0x7f692406f710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6930009fd0 tx=0x7f6930009f00 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:50:31.273 INFO:tasks.workunit.client.0.smithi019.stdout:6/296: getdents d6/d31/d37 0 2024-02-22T15:50:31.273 INFO:tasks.workunit.client.0.smithi019.stdout:5/329: truncate d8/dd/d19/d38/d54/f6b 1888926 0 2024-02-22T15:50:31.274 INFO:tasks.workunit.client.0.smithi019.stdout:6/297: readlink d6/l21 0 2024-02-22T15:50:31.275 INFO:tasks.workunit.client.0.smithi019.stdout:6/298: chown d6/f2a 443385563 1 2024-02-22T15:50:31.275 INFO:tasks.workunit.client.0.smithi019.stdout:3/305: dread f4 [8388608,4194304] 0 2024-02-22T15:50:31.275 INFO:tasks.workunit.client.0.smithi019.stdout:5/330: write d8/f4e [20633,8313] 0 2024-02-22T15:50:31.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.294+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f692c0583a0 con 0x7f693c1078b0 2024-02-22T15:50:31.276 INFO:tasks.workunit.client.0.smithi019.stdout:5/331: chown d8/d11/l1e 1279 1 2024-02-22T15:50:31.277 INFO:tasks.workunit.client.0.smithi019.stdout:3/306: fdatasync d9/d14/f1b 0 2024-02-22T15:50:31.277 INFO:tasks.workunit.client.0.smithi019.stdout:6/299: write d6/f39 [813071,86153] 0 2024-02-22T15:50:31.278 INFO:tasks.workunit.client.0.smithi019.stdout:9/317: dread d2/d1f/d2e/d48/f4e [0,4194304] 0 2024-02-22T15:50:31.279 INFO:tasks.workunit.client.0.smithi019.stdout:6/300: write d6/f1d [2259292,8058] 0 2024-02-22T15:50:31.281 INFO:tasks.workunit.client.0.smithi019.stdout:9/318: fsync d2/d1f/d2e/d48/d36/f71 0 2024-02-22T15:50:31.282 INFO:tasks.workunit.client.0.smithi019.stdout:5/332: fdatasync d8/f4e 0 2024-02-22T15:50:31.285 INFO:tasks.workunit.client.0.smithi019.stdout:9/319: link d2/c25 d2/c74 0 2024-02-22T15:50:31.286 INFO:tasks.workunit.client.0.smithi019.stdout:5/333: truncate d8/d11/d21/d2c/f31 563775 0 2024-02-22T15:50:31.288 INFO:tasks.workunit.client.0.smithi019.stdout:9/320: unlink d2/l41 0 2024-02-22T15:50:31.290 INFO:tasks.workunit.client.0.smithi019.stdout:5/334: read d8/f23 [146762,100855] 0 2024-02-22T15:50:31.290 INFO:tasks.workunit.client.0.smithi019.stdout:6/301: dwrite d6/f28 [0,4194304] 0 2024-02-22T15:50:31.292 INFO:tasks.workunit.client.0.smithi019.stdout:6/302: write d6/f2c [447052,14357] 0 2024-02-22T15:50:31.296 INFO:tasks.workunit.client.0.smithi019.stdout:6/303: write d6/f39 [1189886,18943] 0 2024-02-22T15:50:31.296 INFO:tasks.workunit.client.0.smithi019.stdout:6/304: readlink d6/l3a 0 2024-02-22T15:50:31.296 INFO:tasks.workunit.client.0.smithi019.stdout:5/335: mknod d8/dd/d19/d3a/c71 0 2024-02-22T15:50:31.298 INFO:tasks.workunit.client.0.smithi019.stdout:5/336: chown d8/dd/d19/d38/d47/l4b 40577 1 2024-02-22T15:50:31.299 INFO:tasks.workunit.client.0.smithi019.stdout:9/321: dread d2/f30 [0,4194304] 0 2024-02-22T15:50:31.300 INFO:tasks.workunit.client.0.smithi019.stdout:9/322: chown d2/d7/f12 313044 1 2024-02-22T15:50:31.301 INFO:tasks.workunit.client.0.smithi019.stdout:6/305: mkdir d6/d31/d43/d55 0 2024-02-22T15:50:31.303 INFO:tasks.workunit.client.0.smithi019.stdout:5/337: unlink d8/d11/f2f 0 2024-02-22T15:50:31.305 INFO:tasks.workunit.client.0.smithi019.stdout:6/306: unlink d6/f4c 0 2024-02-22T15:50:31.306 INFO:tasks.workunit.client.0.smithi019.stdout:9/323: creat d2/d1f/d2e/d57/d60/f75 x:0 0 0 2024-02-22T15:50:31.307 INFO:tasks.workunit.client.0.smithi019.stdout:5/338: dread d8/d11/d21/d2c/f31 [0,4194304] 0 2024-02-22T15:50:31.308 INFO:tasks.workunit.client.0.smithi019.stdout:6/307: unlink d6/fe 0 2024-02-22T15:50:31.308 INFO:tasks.workunit.client.0.smithi019.stdout:5/339: fsync d8/d11/f1c 0 2024-02-22T15:50:31.311 INFO:tasks.workunit.client.0.smithi019.stdout:9/324: unlink d2/d7/f12 0 2024-02-22T15:50:31.311 INFO:tasks.workunit.client.0.smithi019.stdout:5/340: symlink d8/d11/d21/l72 0 2024-02-22T15:50:31.313 INFO:tasks.workunit.client.0.smithi019.stdout:6/308: rename d6/d4b/f54 to d6/d31/d43/f56 0 2024-02-22T15:50:31.314 INFO:tasks.workunit.client.0.smithi019.stdout:6/309: readlink d6/l45 0 2024-02-22T15:50:31.314 INFO:tasks.workunit.client.0.smithi019.stdout:9/325: rmdir d2/d1f 39 2024-02-22T15:50:31.316 INFO:tasks.workunit.client.0.smithi019.stdout:9/326: dread - d2/d7/d14/f3f zero size 2024-02-22T15:50:31.319 INFO:tasks.workunit.client.0.smithi019.stdout:9/327: symlink d2/d7/d52/l76 0 2024-02-22T15:50:31.323 INFO:tasks.workunit.client.0.smithi019.stdout:6/310: dwrite d6/f1d [0,4194304] 0 2024-02-22T15:50:31.331 INFO:tasks.workunit.client.0.smithi019.stdout:5/341: dread d8/fe [0,4194304] 0 2024-02-22T15:50:31.332 INFO:tasks.workunit.client.0.smithi019.stdout:5/342: chown d8/dd/d19/d38/d47/f68 60 1 2024-02-22T15:50:31.338 INFO:tasks.workunit.client.0.smithi019.stdout:5/343: mknod d8/d11/c73 0 2024-02-22T15:50:31.338 INFO:tasks.workunit.client.0.smithi019.stdout:6/311: write d6/f17 [547311,56921] 0 2024-02-22T15:50:31.342 INFO:tasks.workunit.client.0.smithi019.stdout:5/344: fsync d8/d11/d21/d2c/f31 0 2024-02-22T15:50:31.343 INFO:tasks.workunit.client.0.smithi019.stdout:9/328: link d2/d7/dc/l1d d2/d1f/l77 0 2024-02-22T15:50:31.349 INFO:tasks.workunit.client.0.smithi019.stdout:9/329: symlink d2/d1f/d2e/d48/d36/d4a/l78 0 2024-02-22T15:50:31.350 INFO:tasks.workunit.client.0.smithi019.stdout:6/312: truncate d6/d4b/f52 918443 0 2024-02-22T15:50:31.351 INFO:tasks.workunit.client.0.smithi019.stdout:9/330: write d2/d7/dc/f24 [11186083,120425] 0 2024-02-22T15:50:31.351 INFO:tasks.workunit.client.0.smithi019.stdout:6/313: fdatasync d6/f44 0 2024-02-22T15:50:31.351 INFO:tasks.workunit.client.0.smithi019.stdout:2/263: sync 2024-02-22T15:50:31.351 INFO:tasks.workunit.client.0.smithi019.stdout:7/350: sync 2024-02-22T15:50:31.352 INFO:tasks.workunit.client.0.smithi019.stdout:0/186: sync 2024-02-22T15:50:31.352 INFO:tasks.workunit.client.0.smithi019.stdout:1/412: sync 2024-02-22T15:50:31.357 INFO:tasks.workunit.client.0.smithi019.stdout:9/331: chown d2/d7/dc/l23 45201 1 2024-02-22T15:50:31.358 INFO:tasks.workunit.client.0.smithi019.stdout:2/264: stat d1/f1f 0 2024-02-22T15:50:31.362 INFO:tasks.workunit.client.0.smithi019.stdout:0/187: mkdir d4/d5/dc/d41 0 2024-02-22T15:50:31.362 INFO:tasks.workunit.client.0.smithi019.stdout:6/314: truncate d6/d31/d43/f56 955890 0 2024-02-22T15:50:31.363 INFO:tasks.workunit.client.0.smithi019.stdout:7/351: write d9/d26/f32 [3632013,87416] 0 2024-02-22T15:50:31.363 INFO:tasks.workunit.client.0.smithi019.stdout:9/332: mkdir d2/d7/dc/d27/d31/d79 0 2024-02-22T15:50:31.367 INFO:tasks.workunit.client.0.smithi019.stdout:2/265: unlink d1/de/d15/d3d/l40 0 2024-02-22T15:50:31.371 INFO:tasks.workunit.client.0.smithi019.stdout:5/345: dwrite d8/d11/d33/f3c [0,4194304] 0 2024-02-22T15:50:31.372 INFO:tasks.workunit.client.0.smithi019.stdout:9/333: unlink d2/d1f/d2e/d48/d36/f3b 0 2024-02-22T15:50:31.377 INFO:tasks.workunit.client.0.smithi019.stdout:6/315: creat d6/f57 x:0 0 0 2024-02-22T15:50:31.377 INFO:tasks.workunit.client.1.smithi062.stdout:2/45: sync 2024-02-22T15:50:31.377 INFO:tasks.workunit.client.1.smithi062.stdout:4/17: sync 2024-02-22T15:50:31.377 INFO:tasks.workunit.client.1.smithi062.stdout:5/33: sync 2024-02-22T15:50:31.377 INFO:tasks.workunit.client.1.smithi062.stdout:0/40: sync 2024-02-22T15:50:31.377 INFO:tasks.workunit.client.1.smithi062.stdout:1/71: sync 2024-02-22T15:50:31.378 INFO:tasks.workunit.client.1.smithi062.stdout:7/67: sync 2024-02-22T15:50:31.378 INFO:tasks.workunit.client.1.smithi062.stdout:8/2: sync 2024-02-22T15:50:31.378 INFO:tasks.workunit.client.1.smithi062.stdout:6/76: sync 2024-02-22T15:50:31.378 INFO:tasks.workunit.client.1.smithi062.stdout:3/45: sync 2024-02-22T15:50:31.378 INFO:tasks.workunit.client.1.smithi062.stdout:9/26: sync 2024-02-22T15:50:31.378 INFO:tasks.workunit.client.0.smithi019.stdout:2/266: mknod d1/d17/c5b 0 2024-02-22T15:50:31.379 INFO:tasks.workunit.client.0.smithi019.stdout:5/346: chown d8/dd/d19/l2a 1 1 2024-02-22T15:50:31.381 INFO:tasks.workunit.client.1.smithi062.stdout:0/41: readlink d1/d3/d4/lb 0 2024-02-22T15:50:31.381 INFO:tasks.workunit.client.1.smithi062.stdout:1/72: stat d1/c2 0 2024-02-22T15:50:31.381 INFO:tasks.workunit.client.0.smithi019.stdout:6/316: chown d6/d31/d37/l3d 72792 1 2024-02-22T15:50:31.382 INFO:tasks.workunit.client.1.smithi062.stdout:0/42: chown d1/d3 58683156 1 2024-02-22T15:50:31.382 INFO:tasks.workunit.client.1.smithi062.stdout:3/46: rmdir d5 39 2024-02-22T15:50:31.382 INFO:tasks.workunit.client.0.smithi019.stdout:9/334: mknod d2/d7/dc/d27/d31/c7a 0 2024-02-22T15:50:31.383 INFO:tasks.workunit.client.0.smithi019.stdout:5/347: readlink d8/d11/d21/l2b 0 2024-02-22T15:50:31.383 INFO:tasks.workunit.client.0.smithi019.stdout:1/413: dwrite d8/d2a/f2c [4194304,4194304] 0 2024-02-22T15:50:31.384 INFO:tasks.workunit.client.0.smithi019.stdout:7/352: dread d9/d38/f3c [0,4194304] 0 2024-02-22T15:50:31.385 INFO:tasks.workunit.client.1.smithi062.stdout:7/68: chown d0/d14/d13 493 1 2024-02-22T15:50:31.388 INFO:tasks.workunit.client.0.smithi019.stdout:9/335: write d2/fb [3664949,83565] 0 2024-02-22T15:50:31.391 INFO:tasks.workunit.client.0.smithi019.stdout:5/348: chown d8/dd/d19/d3a/f3f 223658888 1 2024-02-22T15:50:31.392 INFO:tasks.workunit.client.1.smithi062.stdout:9/27: creat d4/fc x:0 0 0 2024-02-22T15:50:31.392 INFO:tasks.workunit.client.1.smithi062.stdout:5/34: rename d0/d3/lc to d0/d3/ld 0 2024-02-22T15:50:31.392 INFO:tasks.workunit.client.1.smithi062.stdout:0/43: read d1/d3/f6 [39490,47190] 0 2024-02-22T15:50:31.392 INFO:tasks.workunit.client.0.smithi019.stdout:0/188: getdents d4/db/d24 0 2024-02-22T15:50:31.392 INFO:tasks.workunit.client.0.smithi019.stdout:5/349: chown d8/dd/d19/d3a/d46/f61 9921563 1 2024-02-22T15:50:31.392 INFO:tasks.workunit.client.0.smithi019.stdout:2/267: symlink d1/de/l5c 0 2024-02-22T15:50:31.393 INFO:tasks.workunit.client.0.smithi019.stdout:6/317: mkdir d6/d58 0 2024-02-22T15:50:31.393 INFO:tasks.workunit.client.1.smithi062.stdout:8/3: write f0 [739794,114188] 0 2024-02-22T15:50:31.395 INFO:tasks.workunit.client.0.smithi019.stdout:1/414: creat d8/d15/d84/f92 x:0 0 0 2024-02-22T15:50:31.396 INFO:tasks.workunit.client.1.smithi062.stdout:4/18: dwrite f3 [0,4194304] 0 2024-02-22T15:50:31.396 INFO:tasks.workunit.client.1.smithi062.stdout:1/73: write d1/d6/de/d9/fd [4746786,48763] 0 2024-02-22T15:50:31.397 INFO:tasks.workunit.client.0.smithi019.stdout:5/350: creat d8/d11/d33/f74 x:0 0 0 2024-02-22T15:50:31.398 INFO:tasks.workunit.client.1.smithi062.stdout:2/46: write d1/fe [528494,39790] 0 2024-02-22T15:50:31.400 INFO:tasks.workunit.client.0.smithi019.stdout:0/189: symlink d4/d5/dc/d41/l42 0 2024-02-22T15:50:31.401 INFO:tasks.workunit.client.0.smithi019.stdout:1/415: creat d8/d23/d33/f93 x:0 0 0 2024-02-22T15:50:31.402 INFO:tasks.workunit.client.0.smithi019.stdout:5/351: creat d8/dd/f75 x:0 0 0 2024-02-22T15:50:31.402 INFO:tasks.workunit.client.1.smithi062.stdout:1/74: chown d1/d6 8326154 1 2024-02-22T15:50:31.403 INFO:tasks.workunit.client.0.smithi019.stdout:6/318: creat d6/f59 x:0 0 0 2024-02-22T15:50:31.403 INFO:tasks.workunit.client.0.smithi019.stdout:7/353: dread d9/d26/f2e [0,4194304] 0 2024-02-22T15:50:31.404 INFO:tasks.workunit.client.1.smithi062.stdout:1/75: chown d1/d6 1874396 1 2024-02-22T15:50:31.404 INFO:tasks.workunit.client.1.smithi062.stdout:1/76: chown d1/d6 19379625 1 2024-02-22T15:50:31.404 INFO:tasks.workunit.client.1.smithi062.stdout:9/28: dread d4/f8 [0,4194304] 0 2024-02-22T15:50:31.405 INFO:tasks.workunit.client.0.smithi019.stdout:2/268: dread d1/d17/d1c/f22 [0,4194304] 0 2024-02-22T15:50:31.407 INFO:tasks.workunit.client.0.smithi019.stdout:1/416: dread - d8/d23/d33/f93 zero size 2024-02-22T15:50:31.407 INFO:tasks.workunit.client.0.smithi019.stdout:5/352: readlink d8/d11/d33/l44 0 2024-02-22T15:50:31.407 INFO:tasks.workunit.client.1.smithi062.stdout:6/77: link d5/f1e d5/f1f 0 2024-02-22T15:50:31.408 INFO:tasks.workunit.client.1.smithi062.stdout:7/69: link d0/l7 d0/db/l15 0 2024-02-22T15:50:31.408 INFO:tasks.workunit.client.1.smithi062.stdout:3/47: dwrite f4 [0,4194304] 0 2024-02-22T15:50:31.409 INFO:tasks.workunit.client.1.smithi062.stdout:2/47: mkdir d1/d8/df 0 2024-02-22T15:50:31.410 INFO:tasks.workunit.client.1.smithi062.stdout:4/19: write f2 [3085811,80953] 0 2024-02-22T15:50:31.412 INFO:tasks.workunit.client.1.smithi062.stdout:6/78: creat d5/dd/f20 x:0 0 0 2024-02-22T15:50:31.413 INFO:tasks.workunit.client.0.smithi019.stdout:0/190: rename d4/d22/d2c/f30 to d4/db/f43 0 2024-02-22T15:50:31.414 INFO:tasks.workunit.client.0.smithi019.stdout:2/269: rmdir d1/d17/d21 39 2024-02-22T15:50:31.414 INFO:tasks.workunit.client.0.smithi019.stdout:1/417: mkdir d8/d18/d64/d7f/d94 0 2024-02-22T15:50:31.414 INFO:tasks.workunit.client.0.smithi019.stdout:6/319: truncate d6/ff 665693 0 2024-02-22T15:50:31.415 INFO:tasks.workunit.client.1.smithi062.stdout:8/4: getdents . 0 2024-02-22T15:50:31.417 INFO:tasks.workunit.client.1.smithi062.stdout:7/70: truncate d0/f10 788358 0 2024-02-22T15:50:31.418 INFO:tasks.workunit.client.1.smithi062.stdout:6/79: truncate f2 2038762 0 2024-02-22T15:50:31.418 INFO:tasks.workunit.client.0.smithi019.stdout:1/418: chown d8/d18/f21 0 1 2024-02-22T15:50:31.419 INFO:tasks.workunit.client.0.smithi019.stdout:5/353: dread d8/dd/d19/d3a/d46/f53 [0,4194304] 0 2024-02-22T15:50:31.419 INFO:tasks.workunit.client.1.smithi062.stdout:5/35: dwrite d0/f7 [0,4194304] 0 2024-02-22T15:50:31.420 INFO:tasks.workunit.client.1.smithi062.stdout:4/20: symlink d4/l6 0 2024-02-22T15:50:31.421 INFO:tasks.workunit.client.0.smithi019.stdout:9/336: dread d2/f6 [0,4194304] 0 2024-02-22T15:50:31.422 INFO:tasks.workunit.client.1.smithi062.stdout:1/77: getdents d1/d6/d10 0 2024-02-22T15:50:31.426 INFO:tasks.workunit.client.0.smithi019.stdout:1/419: creat d8/d18/d64/f95 x:0 0 0 2024-02-22T15:50:31.426 INFO:tasks.workunit.client.1.smithi062.stdout:8/5: creat f1 x:0 0 0 2024-02-22T15:50:31.427 INFO:tasks.workunit.client.0.smithi019.stdout:1/420: dread - d8/f88 zero size 2024-02-22T15:50:31.427 INFO:tasks.workunit.client.1.smithi062.stdout:8/6: read - f1 zero size 2024-02-22T15:50:31.429 INFO:tasks.workunit.client.0.smithi019.stdout:6/320: truncate d6/f20 3511210 0 2024-02-22T15:50:31.429 INFO:tasks.workunit.client.1.smithi062.stdout:8/7: read f0 [345283,43127] 0 2024-02-22T15:50:31.429 INFO:tasks.workunit.client.1.smithi062.stdout:4/21: dread f2 [0,4194304] 0 2024-02-22T15:50:31.430 INFO:tasks.workunit.client.1.smithi062.stdout:4/22: chown f3 6088329 1 2024-02-22T15:50:31.432 INFO:tasks.workunit.client.1.smithi062.stdout:6/80: rename d5/dd/f12 to d5/df/f21 0 2024-02-22T15:50:31.433 INFO:tasks.workunit.client.1.smithi062.stdout:2/48: write d1/f5 [95573,65335] 0 2024-02-22T15:50:31.433 INFO:tasks.workunit.client.1.smithi062.stdout:8/8: mknod c2 0 2024-02-22T15:50:31.434 INFO:tasks.workunit.client.0.smithi019.stdout:9/337: dread d2/d7/f44 [0,4194304] 0 2024-02-22T15:50:31.434 INFO:tasks.workunit.client.0.smithi019.stdout:5/354: write d8/dd/d19/d3a/d46/f53 [966569,70020] 0 2024-02-22T15:50:31.436 INFO:tasks.workunit.client.1.smithi062.stdout:2/49: creat d1/d8/f10 x:0 0 0 2024-02-22T15:50:31.436 INFO:tasks.workunit.client.0.smithi019.stdout:1/421: creat d8/d2a/d75/d81/f96 x:0 0 0 2024-02-22T15:50:31.438 INFO:tasks.workunit.client.1.smithi062.stdout:6/81: symlink d5/df/l22 0 2024-02-22T15:50:31.438 INFO:tasks.workunit.client.1.smithi062.stdout:8/9: mknod c3 0 2024-02-22T15:50:31.438 INFO:tasks.workunit.client.0.smithi019.stdout:0/191: getdents d4/d9 0 2024-02-22T15:50:31.440 INFO:tasks.workunit.client.1.smithi062.stdout:5/36: fsync d0/f7 0 2024-02-22T15:50:31.441 INFO:tasks.workunit.client.0.smithi019.stdout:2/270: dread d1/de/f16 [0,4194304] 0 2024-02-22T15:50:31.442 INFO:tasks.workunit.client.1.smithi062.stdout:0/44: dwrite d1/d3/f6 [4194304,4194304] 0 2024-02-22T15:50:31.443 INFO:tasks.workunit.client.1.smithi062.stdout:8/10: unlink f1 0 2024-02-22T15:50:31.443 INFO:tasks.workunit.client.0.smithi019.stdout:9/338: chown d2/d1f/d2e/d48/d45/f61 0 1 2024-02-22T15:50:31.443 INFO:tasks.workunit.client.1.smithi062.stdout:7/71: write d0/f5 [296998,108845] 0 2024-02-22T15:50:31.445 INFO:tasks.workunit.client.1.smithi062.stdout:6/82: creat d5/d15/f23 x:0 0 0 2024-02-22T15:50:31.446 INFO:tasks.workunit.client.1.smithi062.stdout:5/37: symlink d0/le 0 2024-02-22T15:50:31.447 INFO:tasks.workunit.client.0.smithi019.stdout:6/321: dwrite d6/d31/f32 [0,4194304] 0 2024-02-22T15:50:31.448 INFO:tasks.workunit.client.0.smithi019.stdout:1/422: mkdir d8/d2a/d97 0 2024-02-22T15:50:31.449 INFO:tasks.workunit.client.1.smithi062.stdout:8/11: mknod c4 0 2024-02-22T15:50:31.450 INFO:tasks.workunit.client.0.smithi019.stdout:9/339: rmdir d2/d1f/d2e/d48/d36/d4a 39 2024-02-22T15:50:31.452 INFO:tasks.workunit.client.1.smithi062.stdout:0/45: dread d1/f5 [0,4194304] 0 2024-02-22T15:50:31.452 INFO:tasks.workunit.client.0.smithi019.stdout:0/192: mkdir d4/db/d1b/d40/d44 0 2024-02-22T15:50:31.452 INFO:tasks.workunit.client.1.smithi062.stdout:5/38: creat d0/d3/ff x:0 0 0 2024-02-22T15:50:31.455 INFO:tasks.workunit.client.0.smithi019.stdout:6/322: mknod d6/d3e/c5a 0 2024-02-22T15:50:31.456 INFO:tasks.workunit.client.1.smithi062.stdout:5/39: readlink d0/d3/ld 0 2024-02-22T15:50:31.457 INFO:tasks.workunit.client.1.smithi062.stdout:8/12: dread f0 [0,4194304] 0 2024-02-22T15:50:31.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:31 smithi062 ceph-mon[48130]: pgmap v31: 65 pgs: 65 active+clean; 870 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 10 MiB/s rd, 82 MiB/s wr, 321 op/s 2024-02-22T15:50:31.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:31 smithi062 ceph-mon[48130]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:31.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:31 smithi062 ceph-mon[48130]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:31.458 INFO:tasks.workunit.client.1.smithi062.stdout:7/72: rename d0/d6/fd to d0/f16 0 2024-02-22T15:50:31.458 INFO:tasks.workunit.client.1.smithi062.stdout:2/50: write d1/f7 [993814,66922] 0 2024-02-22T15:50:31.459 INFO:tasks.workunit.client.0.smithi019.stdout:5/355: dwrite d8/dd/d19/d38/d47/f64 [0,4194304] 0 2024-02-22T15:50:31.460 INFO:tasks.workunit.client.1.smithi062.stdout:0/46: creat d1/dd/f10 x:0 0 0 2024-02-22T15:50:31.461 INFO:tasks.workunit.client.0.smithi019.stdout:2/271: creat d1/d17/d21/d51/f5d x:0 0 0 2024-02-22T15:50:31.463 INFO:tasks.workunit.client.0.smithi019.stdout:0/193: symlink d4/d5/dc/l45 0 2024-02-22T15:50:31.463 INFO:tasks.workunit.client.1.smithi062.stdout:6/83: rename d5/l9 to d5/d19/l24 0 2024-02-22T15:50:31.464 INFO:tasks.workunit.client.1.smithi062.stdout:7/73: creat d0/d14/f17 x:0 0 0 2024-02-22T15:50:31.465 INFO:tasks.workunit.client.1.smithi062.stdout:2/51: unlink d1/fe 0 2024-02-22T15:50:31.466 INFO:tasks.workunit.client.0.smithi019.stdout:5/356: creat d8/d11/d33/f76 x:0 0 0 2024-02-22T15:50:31.467 INFO:tasks.workunit.client.1.smithi062.stdout:0/47: symlink d1/d3/de/l11 0 2024-02-22T15:50:31.469 INFO:tasks.workunit.client.1.smithi062.stdout:6/84: read d5/f1a [2673801,124445] 0 2024-02-22T15:50:31.470 INFO:tasks.workunit.client.0.smithi019.stdout:5/357: write d8/dd/f5b [745354,10526] 0 2024-02-22T15:50:31.470 INFO:tasks.workunit.client.0.smithi019.stdout:2/272: getdents d1/de 0 2024-02-22T15:50:31.470 INFO:tasks.workunit.client.0.smithi019.stdout:0/194: creat d4/d32/f46 x:0 0 0 2024-02-22T15:50:31.471 INFO:tasks.workunit.client.1.smithi062.stdout:7/74: readlink d0/l7 0 2024-02-22T15:50:31.472 INFO:tasks.workunit.client.0.smithi019.stdout:6/323: dwrite d6/f44 [0,4194304] 0 2024-02-22T15:50:31.473 INFO:tasks.workunit.client.1.smithi062.stdout:2/52: rename d1/d8/fb to d1/d8/f11 0 2024-02-22T15:50:31.474 INFO:tasks.workunit.client.1.smithi062.stdout:0/48: rename d1/f5 to d1/f12 0 2024-02-22T15:50:31.476 INFO:tasks.workunit.client.0.smithi019.stdout:2/273: creat d1/d46/f5e x:0 0 0 2024-02-22T15:50:31.477 INFO:tasks.workunit.client.1.smithi062.stdout:8/13: write f0 [37730,115994] 0 2024-02-22T15:50:31.478 INFO:tasks.workunit.client.0.smithi019.stdout:5/358: dread d8/d11/d21/d2c/f31 [0,4194304] 0 2024-02-22T15:50:31.478 INFO:tasks.workunit.client.1.smithi062.stdout:8/14: chown c4 0 1 2024-02-22T15:50:31.479 INFO:tasks.workunit.client.1.smithi062.stdout:8/15: read f0 [824502,64161] 0 2024-02-22T15:50:31.479 INFO:tasks.workunit.client.1.smithi062.stdout:7/75: rename d0/db/le to d0/d14/l18 0 2024-02-22T15:50:31.480 INFO:tasks.workunit.client.1.smithi062.stdout:2/53: symlink d1/d8/l12 0 2024-02-22T15:50:31.481 INFO:tasks.workunit.client.1.smithi062.stdout:8/16: write f0 [1719646,59729] 0 2024-02-22T15:50:31.481 INFO:tasks.workunit.client.1.smithi062.stdout:0/49: dread - d1/d3/d4/f7 zero size 2024-02-22T15:50:31.483 INFO:tasks.workunit.client.0.smithi019.stdout:0/195: write f2 [246186,20877] 0 2024-02-22T15:50:31.486 INFO:tasks.workunit.client.1.smithi062.stdout:0/50: fsync d1/d3/de/ff 0 2024-02-22T15:50:31.487 INFO:tasks.workunit.client.0.smithi019.stdout:6/324: dwrite d6/f57 [0,4194304] 0 2024-02-22T15:50:31.488 INFO:tasks.workunit.client.1.smithi062.stdout:0/51: dread - d1/dd/f10 zero size 2024-02-22T15:50:31.488 INFO:tasks.workunit.client.1.smithi062.stdout:2/54: mknod d1/d8/df/c13 0 2024-02-22T15:50:31.492 INFO:tasks.workunit.client.0.smithi019.stdout:1/423: dwrite d8/d15/f20 [0,4194304] 0 2024-02-22T15:50:31.492 INFO:tasks.workunit.client.0.smithi019.stdout:0/196: write d4/d22/f36 [819136,18298] 0 2024-02-22T15:50:31.492 INFO:tasks.workunit.client.0.smithi019.stdout:6/325: rmdir d6 39 2024-02-22T15:50:31.493 INFO:tasks.workunit.client.1.smithi062.stdout:2/55: chown d1/d8/fa 942032 1 2024-02-22T15:50:31.494 INFO:tasks.workunit.client.1.smithi062.stdout:0/52: mkdir d1/d13 0 2024-02-22T15:50:31.494 INFO:tasks.workunit.client.0.smithi019.stdout:2/274: truncate d1/de/f16 2391484 0 2024-02-22T15:50:31.494 INFO:tasks.workunit.client.0.smithi019.stdout:5/359: getdents d8/d11/d21/d2c 0 2024-02-22T15:50:31.495 INFO:tasks.workunit.client.0.smithi019.stdout:0/197: write d4/f19 [899175,21705] 0 2024-02-22T15:50:31.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:31 smithi019 ceph-mon[67418]: pgmap v31: 65 pgs: 65 active+clean; 870 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 10 MiB/s rd, 82 MiB/s wr, 321 op/s 2024-02-22T15:50:31.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:31 smithi019 ceph-mon[67418]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:31.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:31 smithi019 ceph-mon[67418]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:31.496 INFO:tasks.workunit.client.1.smithi062.stdout:7/76: dwrite d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:31.497 INFO:tasks.workunit.client.0.smithi019.stdout:1/424: creat d8/d18/d85/f98 x:0 0 0 2024-02-22T15:50:31.497 INFO:tasks.workunit.client.0.smithi019.stdout:2/275: chown d1/d17/d1c/f34 841502146 1 2024-02-22T15:50:31.498 INFO:tasks.workunit.client.0.smithi019.stdout:0/198: chown d4/db/c16 28449816 1 2024-02-22T15:50:31.500 INFO:tasks.workunit.client.0.smithi019.stdout:6/326: creat d6/d4b/f5b x:0 0 0 2024-02-22T15:50:31.502 INFO:tasks.workunit.client.1.smithi062.stdout:0/53: chown d1/d3/f6 32360 1 2024-02-22T15:50:31.502 INFO:tasks.workunit.client.0.smithi019.stdout:2/276: mknod d1/d17/d1c/c5f 0 2024-02-22T15:50:31.503 INFO:tasks.workunit.client.0.smithi019.stdout:1/425: symlink d8/d23/d33/l99 0 2024-02-22T15:50:31.504 INFO:tasks.workunit.client.1.smithi062.stdout:2/56: rename d1/f7 to d1/d8/df/f14 0 2024-02-22T15:50:31.505 INFO:tasks.workunit.client.0.smithi019.stdout:0/199: mknod d4/d22/d2c/c47 0 2024-02-22T15:50:31.506 INFO:tasks.workunit.client.0.smithi019.stdout:1/426: readlink d8/d23/d33/l56 0 2024-02-22T15:50:31.507 INFO:tasks.workunit.client.1.smithi062.stdout:8/17: dwrite f0 [0,4194304] 0 2024-02-22T15:50:31.507 INFO:tasks.workunit.client.0.smithi019.stdout:5/360: rename d8/d11/d21/d30 to d8/dd/d19/d77 0 2024-02-22T15:50:31.508 INFO:tasks.workunit.client.1.smithi062.stdout:2/57: dread - d1/d8/f10 zero size 2024-02-22T15:50:31.508 INFO:tasks.workunit.client.0.smithi019.stdout:5/361: readlink d8/l4f 0 2024-02-22T15:50:31.510 INFO:tasks.workunit.client.1.smithi062.stdout:0/54: mkdir d1/d3/de/d14 0 2024-02-22T15:50:31.513 INFO:tasks.workunit.client.1.smithi062.stdout:7/77: dread d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:31.513 INFO:tasks.workunit.client.0.smithi019.stdout:0/200: fsync d4/f3e 0 2024-02-22T15:50:31.515 INFO:tasks.workunit.client.0.smithi019.stdout:6/327: truncate d6/f20 413171 0 2024-02-22T15:50:31.515 INFO:tasks.workunit.client.1.smithi062.stdout:8/18: write f0 [3395752,45798] 0 2024-02-22T15:50:31.516 INFO:tasks.workunit.client.1.smithi062.stdout:0/55: mknod d1/d13/c15 0 2024-02-22T15:50:31.520 INFO:tasks.workunit.client.1.smithi062.stdout:6/85: dwrite d5/f1e [0,4194304] 0 2024-02-22T15:50:31.521 INFO:tasks.workunit.client.1.smithi062.stdout:7/78: chown d0/f16 1 1 2024-02-22T15:50:31.523 INFO:tasks.workunit.client.0.smithi019.stdout:2/277: rename d1/de/d15/d3d to d1/d17/d19/d60 0 2024-02-22T15:50:31.523 INFO:tasks.workunit.client.0.smithi019.stdout:5/362: fsync d8/dd/d19/d38/d54/f6b 0 2024-02-22T15:50:31.524 INFO:tasks.workunit.client.1.smithi062.stdout:0/56: dread - d1/dd/f10 zero size 2024-02-22T15:50:31.524 INFO:tasks.workunit.client.1.smithi062.stdout:8/19: truncate f0 4629763 0 2024-02-22T15:50:31.525 INFO:tasks.workunit.client.0.smithi019.stdout:2/278: read - d1/d17/f47 zero size 2024-02-22T15:50:31.525 INFO:tasks.workunit.client.0.smithi019.stdout:1/427: dwrite d8/f5a [4194304,4194304] 0 2024-02-22T15:50:31.526 INFO:tasks.workunit.client.0.smithi019.stdout:5/363: write d8/d11/d33/f3c [822563,46780] 0 2024-02-22T15:50:31.529 INFO:tasks.workunit.client.1.smithi062.stdout:7/79: mkdir d0/d14/d19 0 2024-02-22T15:50:31.529 INFO:tasks.workunit.client.1.smithi062.stdout:0/57: creat d1/dd/f16 x:0 0 0 2024-02-22T15:50:31.529 INFO:tasks.workunit.client.0.smithi019.stdout:2/279: write d1/d17/d19/f2f [3170816,86413] 0 2024-02-22T15:50:31.529 INFO:tasks.workunit.client.0.smithi019.stdout:0/201: symlink d4/db/d1b/d40/d44/l48 0 2024-02-22T15:50:31.530 INFO:tasks.workunit.client.1.smithi062.stdout:7/80: chown d0/d14/l18 3719 1 2024-02-22T15:50:31.532 INFO:tasks.workunit.client.1.smithi062.stdout:0/58: write d1/d3/f6 [2368442,26451] 0 2024-02-22T15:50:31.534 INFO:tasks.workunit.client.1.smithi062.stdout:8/20: dread f0 [0,4194304] 0 2024-02-22T15:50:31.535 INFO:tasks.workunit.client.0.smithi019.stdout:1/428: rename d8/d15/d34/f6c to d8/d2a/d97/f9a 0 2024-02-22T15:50:31.535 INFO:tasks.workunit.client.0.smithi019.stdout:6/328: creat d6/d31/d43/d55/f5c x:0 0 0 2024-02-22T15:50:31.535 INFO:tasks.workunit.client.0.smithi019.stdout:5/364: mknod d8/c78 0 2024-02-22T15:50:31.536 INFO:tasks.workunit.client.1.smithi062.stdout:7/81: creat d0/d14/d19/f1a x:0 0 0 2024-02-22T15:50:31.536 INFO:tasks.workunit.client.1.smithi062.stdout:2/58: dread d1/d8/f11 [0,4194304] 0 2024-02-22T15:50:31.536 INFO:tasks.workunit.client.0.smithi019.stdout:0/202: mkdir d4/db/d49 0 2024-02-22T15:50:31.537 INFO:tasks.workunit.client.1.smithi062.stdout:8/21: fdatasync f0 0 2024-02-22T15:50:31.539 INFO:tasks.workunit.client.0.smithi019.stdout:1/429: chown d8/d2a/d75 17029 1 2024-02-22T15:50:31.539 INFO:tasks.workunit.client.1.smithi062.stdout:7/82: dread - d0/f16 zero size 2024-02-22T15:50:31.540 INFO:tasks.workunit.client.1.smithi062.stdout:0/59: rename d1/dd/f10 to d1/d3/de/d14/f17 0 2024-02-22T15:50:31.541 INFO:tasks.workunit.client.1.smithi062.stdout:0/60: chown d1/d3/de 4 1 2024-02-22T15:50:31.541 INFO:tasks.workunit.client.0.smithi019.stdout:6/329: write d6/f59 [654463,82928] 0 2024-02-22T15:50:31.543 INFO:tasks.workunit.client.1.smithi062.stdout:8/22: dread f0 [0,4194304] 0 2024-02-22T15:50:31.543 INFO:tasks.workunit.client.1.smithi062.stdout:0/61: dread - d1/d3/de/d14/f17 zero size 2024-02-22T15:50:31.543 INFO:tasks.workunit.client.1.smithi062.stdout:2/59: creat d1/d8/f15 x:0 0 0 2024-02-22T15:50:31.543 INFO:tasks.workunit.client.1.smithi062.stdout:7/83: rmdir d0/db 39 2024-02-22T15:50:31.544 INFO:tasks.workunit.client.0.smithi019.stdout:5/365: symlink d8/dd/d19/d77/l79 0 2024-02-22T15:50:31.544 INFO:tasks.workunit.client.0.smithi019.stdout:5/366: readlink d8/dd/l1a 0 2024-02-22T15:50:31.544 INFO:tasks.workunit.client.1.smithi062.stdout:7/84: chown d0/f16 33 1 2024-02-22T15:50:31.545 INFO:tasks.workunit.client.1.smithi062.stdout:8/23: symlink l5 0 2024-02-22T15:50:31.546 INFO:tasks.workunit.client.0.smithi019.stdout:1/430: mknod d8/d18/c9b 0 2024-02-22T15:50:31.546 INFO:tasks.workunit.client.1.smithi062.stdout:8/24: write f0 [489304,42003] 0 2024-02-22T15:50:31.546 INFO:tasks.workunit.client.1.smithi062.stdout:2/60: creat d1/f16 x:0 0 0 2024-02-22T15:50:31.547 INFO:tasks.workunit.client.1.smithi062.stdout:8/25: read f0 [1747814,43303] 0 2024-02-22T15:50:31.547 INFO:tasks.workunit.client.0.smithi019.stdout:1/431: dread - d8/d15/d34/f7e zero size 2024-02-22T15:50:31.549 INFO:tasks.workunit.client.0.smithi019.stdout:1/432: chown d8/d2a/d75/c7d 11234 1 2024-02-22T15:50:31.549 INFO:tasks.workunit.client.0.smithi019.stdout:0/203: truncate f1 908471 0 2024-02-22T15:50:31.549 INFO:tasks.workunit.client.1.smithi062.stdout:8/26: mknod c6 0 2024-02-22T15:50:31.550 INFO:tasks.workunit.client.1.smithi062.stdout:2/61: truncate d1/d8/fa 4673202 0 2024-02-22T15:50:31.550 INFO:tasks.workunit.client.0.smithi019.stdout:6/330: rename d6/la to d6/d4b/l5d 0 2024-02-22T15:50:31.550 INFO:tasks.workunit.client.1.smithi062.stdout:2/62: stat d1/lc 0 2024-02-22T15:50:31.551 INFO:tasks.workunit.client.0.smithi019.stdout:5/367: mkdir d8/dd/d19/d77/d7a 0 2024-02-22T15:50:31.552 INFO:tasks.workunit.client.1.smithi062.stdout:9/29: sync 2024-02-22T15:50:31.552 INFO:tasks.workunit.client.0.smithi019.stdout:5/368: fdatasync d8/d11/d33/f3c 0 2024-02-22T15:50:31.554 INFO:tasks.workunit.client.0.smithi019.stdout:1/433: mknod d8/d15/d34/c9c 0 2024-02-22T15:50:31.555 INFO:tasks.workunit.client.0.smithi019.stdout:1/434: chown d8/d18/d64/l77 62618909 1 2024-02-22T15:50:31.556 INFO:tasks.workunit.client.1.smithi062.stdout:9/30: rename d4/f9 to d4/fd 0 2024-02-22T15:50:31.556 INFO:tasks.workunit.client.1.smithi062.stdout:9/31: readlink - no filename 2024-02-22T15:50:31.556 INFO:tasks.workunit.client.0.smithi019.stdout:0/204: rename d4/f21 to d4/d23/f4a 0 2024-02-22T15:50:31.557 INFO:tasks.workunit.client.0.smithi019.stdout:6/331: mknod d6/d31/d48/c5e 0 2024-02-22T15:50:31.557 INFO:tasks.workunit.client.1.smithi062.stdout:8/27: fsync f0 0 2024-02-22T15:50:31.559 INFO:tasks.workunit.client.1.smithi062.stdout:8/28: write f0 [2083635,88850] 0 2024-02-22T15:50:31.560 INFO:tasks.workunit.client.1.smithi062.stdout:9/32: creat d4/fe x:0 0 0 2024-02-22T15:50:31.563 INFO:tasks.workunit.client.0.smithi019.stdout:0/205: rename d4/d5/l3c to d4/d22/d3d/l4b 0 2024-02-22T15:50:31.563 INFO:tasks.workunit.client.0.smithi019.stdout:2/280: write d1/f1f [7372525,37438] 0 2024-02-22T15:50:31.564 INFO:tasks.workunit.client.1.smithi062.stdout:0/62: write d1/d3/d4/f7 [352835,103949] 0 2024-02-22T15:50:31.566 INFO:tasks.workunit.client.1.smithi062.stdout:9/33: symlink d4/lf 0 2024-02-22T15:50:31.568 INFO:tasks.workunit.client.0.smithi019.stdout:2/281: fsync d1/d17/d19/f2f 0 2024-02-22T15:50:31.569 INFO:tasks.workunit.client.0.smithi019.stdout:6/332: rename d6/f14 to d6/d31/d43/f5f 0 2024-02-22T15:50:31.570 INFO:tasks.workunit.client.1.smithi062.stdout:6/86: dwrite d5/d19/f1c [0,4194304] 0 2024-02-22T15:50:31.570 INFO:tasks.workunit.client.0.smithi019.stdout:1/435: dwrite d8/d18/d64/f83 [0,4194304] 0 2024-02-22T15:50:31.571 INFO:tasks.workunit.client.0.smithi019.stdout:5/369: dwrite d8/dd/d19/d3a/d46/f61 [0,4194304] 0 2024-02-22T15:50:31.573 INFO:tasks.workunit.client.0.smithi019.stdout:1/436: stat d8/d15/d34/c91 0 2024-02-22T15:50:31.573 INFO:tasks.workunit.client.1.smithi062.stdout:9/34: truncate d4/fe 447554 0 2024-02-22T15:50:31.574 INFO:tasks.workunit.client.0.smithi019.stdout:2/282: write d1/de/d15/f58 [2268436,28607] 0 2024-02-22T15:50:31.576 INFO:tasks.workunit.client.0.smithi019.stdout:6/333: creat d6/d31/d37/f60 x:0 0 0 2024-02-22T15:50:31.577 INFO:tasks.workunit.client.1.smithi062.stdout:0/63: creat d1/d3/de/d14/f18 x:0 0 0 2024-02-22T15:50:31.578 INFO:tasks.workunit.client.0.smithi019.stdout:5/370: creat d8/dd/d19/d3a/d46/f7b x:0 0 0 2024-02-22T15:50:31.579 INFO:tasks.workunit.client.0.smithi019.stdout:5/371: dread - d8/d11/d33/f76 zero size 2024-02-22T15:50:31.581 INFO:tasks.workunit.client.0.smithi019.stdout:2/283: write d1/d17/d1c/f34 [1245716,49521] 0 2024-02-22T15:50:31.587 INFO:tasks.workunit.client.1.smithi062.stdout:8/29: dwrite f0 [0,4194304] 0 2024-02-22T15:50:31.588 INFO:tasks.workunit.client.0.smithi019.stdout:2/284: mknod d1/d17/c61 0 2024-02-22T15:50:31.589 INFO:tasks.workunit.client.0.smithi019.stdout:1/437: getdents d8/d18/d64 0 2024-02-22T15:50:31.589 INFO:tasks.workunit.client.1.smithi062.stdout:8/30: read f0 [1875871,114190] 0 2024-02-22T15:50:31.589 INFO:tasks.workunit.client.1.smithi062.stdout:0/64: chown d1/d3/de/ff 1555064 1 2024-02-22T15:50:31.590 INFO:tasks.workunit.client.1.smithi062.stdout:9/35: truncate d4/f5 475118 0 2024-02-22T15:50:31.591 INFO:tasks.workunit.client.1.smithi062.stdout:7/85: dwrite d0/f16 [0,4194304] 0 2024-02-22T15:50:31.592 INFO:tasks.workunit.client.0.smithi019.stdout:2/285: mknod d1/d46/c62 0 2024-02-22T15:50:31.593 INFO:tasks.workunit.client.0.smithi019.stdout:1/438: creat d8/d18/d85/f9d x:0 0 0 2024-02-22T15:50:31.593 INFO:tasks.workunit.client.1.smithi062.stdout:9/36: dread - d4/f7 zero size 2024-02-22T15:50:31.594 INFO:tasks.workunit.client.0.smithi019.stdout:5/372: dwrite d8/dd/d19/f29 [0,4194304] 0 2024-02-22T15:50:31.594 INFO:tasks.workunit.client.1.smithi062.stdout:6/87: link d5/l8 d5/d15/l25 0 2024-02-22T15:50:31.595 INFO:tasks.workunit.client.0.smithi019.stdout:2/286: write d1/f18 [1012238,51982] 0 2024-02-22T15:50:31.595 INFO:tasks.workunit.client.1.smithi062.stdout:2/63: dwrite d1/d8/f11 [0,4194304] 0 2024-02-22T15:50:31.595 INFO:tasks.workunit.client.1.smithi062.stdout:7/86: read d0/f5 [589930,809] 0 2024-02-22T15:50:31.597 INFO:tasks.workunit.client.1.smithi062.stdout:2/64: fdatasync d1/f16 0 2024-02-22T15:50:31.598 INFO:tasks.workunit.client.0.smithi019.stdout:1/439: chown d8/d15/l73 1 1 2024-02-22T15:50:31.599 INFO:tasks.workunit.client.1.smithi062.stdout:6/88: fsync d5/f7 0 2024-02-22T15:50:31.600 INFO:tasks.workunit.client.1.smithi062.stdout:0/65: rename d1/d3/d4/lb to d1/d3/l19 0 2024-02-22T15:50:31.600 INFO:tasks.workunit.client.0.smithi019.stdout:5/373: unlink d8/dd/d19/c4c 0 2024-02-22T15:50:31.601 INFO:tasks.workunit.client.1.smithi062.stdout:0/66: chown d1/d3/de/d14 96 1 2024-02-22T15:50:31.601 INFO:tasks.workunit.client.1.smithi062.stdout:2/65: symlink d1/d8/l17 0 2024-02-22T15:50:31.603 INFO:tasks.workunit.client.0.smithi019.stdout:5/374: write d8/dd/f67 [1341392,47026] 0 2024-02-22T15:50:31.604 INFO:tasks.workunit.client.0.smithi019.stdout:1/440: mkdir d8/d15/d34/d9e 0 2024-02-22T15:50:31.606 INFO:tasks.workunit.client.1.smithi062.stdout:6/89: dread f2 [0,4194304] 0 2024-02-22T15:50:31.607 INFO:tasks.workunit.client.1.smithi062.stdout:0/67: dread d1/d3/d4/f7 [0,4194304] 0 2024-02-22T15:50:31.607 INFO:tasks.workunit.client.0.smithi019.stdout:2/287: dread d1/d17/d1c/f34 [0,4194304] 0 2024-02-22T15:50:31.611 INFO:tasks.workunit.client.0.smithi019.stdout:2/288: chown d1/d17/f30 851031 1 2024-02-22T15:50:31.618 INFO:tasks.workunit.client.0.smithi019.stdout:5/375: rename d8/d11/d21/c5c to d8/d11/d21/d2c/c7c 0 2024-02-22T15:50:31.618 INFO:tasks.workunit.client.0.smithi019.stdout:6/334: truncate d6/f17 676758 0 2024-02-22T15:50:31.619 INFO:tasks.workunit.client.0.smithi019.stdout:2/289: rmdir d1/d17/d21/d51 39 2024-02-22T15:50:31.620 INFO:tasks.workunit.client.1.smithi062.stdout:0/68: dwrite d1/d3/d4/f7 [0,4194304] 0 2024-02-22T15:50:31.621 INFO:tasks.workunit.client.0.smithi019.stdout:5/376: dread - d8/d11/d21/d2c/f5d zero size 2024-02-22T15:50:31.621 INFO:tasks.workunit.client.0.smithi019.stdout:1/441: rename d8/d15/d34/f7e to d8/d44/f9f 0 2024-02-22T15:50:31.622 INFO:tasks.workunit.client.1.smithi062.stdout:9/37: dread d4/f5 [0,4194304] 0 2024-02-22T15:50:31.623 INFO:tasks.workunit.client.1.smithi062.stdout:7/87: write d0/f10 [1132874,119163] 0 2024-02-22T15:50:31.626 INFO:tasks.workunit.client.0.smithi019.stdout:2/290: rename d1/d17/d4b/l43 to d1/de/l63 0 2024-02-22T15:50:31.626 INFO:tasks.workunit.client.1.smithi062.stdout:8/31: fdatasync f0 0 2024-02-22T15:50:31.626 INFO:tasks.workunit.client.1.smithi062.stdout:2/66: dwrite d1/f6 [4194304,4194304] 0 2024-02-22T15:50:31.626 INFO:tasks.workunit.client.0.smithi019.stdout:5/377: write d8/f23 [376537,83650] 0 2024-02-22T15:50:31.629 INFO:tasks.workunit.client.0.smithi019.stdout:6/335: dwrite d6/f10 [0,4194304] 0 2024-02-22T15:50:31.630 INFO:tasks.workunit.client.1.smithi062.stdout:0/69: mkdir d1/d3/d4/d1a 0 2024-02-22T15:50:31.631 INFO:tasks.workunit.client.1.smithi062.stdout:2/67: mknod d1/d8/c18 0 2024-02-22T15:50:31.633 INFO:tasks.workunit.client.1.smithi062.stdout:7/88: rename d0/d14/d13/l12 to d0/d14/l1b 0 2024-02-22T15:50:31.634 INFO:tasks.workunit.client.1.smithi062.stdout:9/38: truncate d4/fd 564835 0 2024-02-22T15:50:31.635 INFO:tasks.workunit.client.0.smithi019.stdout:5/378: creat d8/d11/f7d x:0 0 0 2024-02-22T15:50:31.636 INFO:tasks.workunit.client.1.smithi062.stdout:0/70: readlink d1/d3/l19 0 2024-02-22T15:50:31.639 INFO:tasks.workunit.client.0.smithi019.stdout:5/379: write d8/dd/d19/d3a/f3f [254089,80447] 0 2024-02-22T15:50:31.639 INFO:tasks.workunit.client.1.smithi062.stdout:7/89: chown d0/d14/d13/fc 12 1 2024-02-22T15:50:31.639 INFO:tasks.workunit.client.1.smithi062.stdout:0/71: rmdir d1/d3 39 2024-02-22T15:50:31.643 INFO:tasks.workunit.client.1.smithi062.stdout:8/32: dwrite f0 [4194304,4194304] 0 2024-02-22T15:50:31.644 INFO:tasks.workunit.client.0.smithi019.stdout:5/380: fdatasync d8/dd/d19/d3a/d46/f61 0 2024-02-22T15:50:31.646 INFO:tasks.workunit.client.1.smithi062.stdout:8/33: symlink l7 0 2024-02-22T15:50:31.646 INFO:tasks.workunit.client.0.smithi019.stdout:5/381: chown d8/d11/d33 5 1 2024-02-22T15:50:31.647 INFO:tasks.workunit.client.0.smithi019.stdout:2/291: dwrite d1/d17/d1c/f22 [0,4194304] 0 2024-02-22T15:50:31.648 INFO:tasks.workunit.client.1.smithi062.stdout:8/34: mkdir d8 0 2024-02-22T15:50:31.649 INFO:tasks.workunit.client.1.smithi062.stdout:8/35: chown l7 5 1 2024-02-22T15:50:31.649 INFO:tasks.workunit.client.1.smithi062.stdout:2/68: dread d1/f6 [0,4194304] 0 2024-02-22T15:50:31.649 INFO:tasks.workunit.client.0.smithi019.stdout:2/292: read - d1/d17/f47 zero size 2024-02-22T15:50:31.650 INFO:tasks.workunit.client.1.smithi062.stdout:2/69: fdatasync d1/f5 0 2024-02-22T15:50:31.651 INFO:tasks.workunit.client.1.smithi062.stdout:8/36: mkdir d8/d9 0 2024-02-22T15:50:31.653 INFO:tasks.workunit.client.0.smithi019.stdout:2/293: dread - d1/d46/f55 zero size 2024-02-22T15:50:31.653 INFO:tasks.workunit.client.0.smithi019.stdout:5/382: dread d8/f23 [0,4194304] 0 2024-02-22T15:50:31.654 INFO:tasks.workunit.client.1.smithi062.stdout:8/37: mknod d8/ca 0 2024-02-22T15:50:31.655 INFO:tasks.workunit.client.0.smithi019.stdout:6/336: dwrite d6/d31/d43/f56 [0,4194304] 0 2024-02-22T15:50:31.658 INFO:tasks.workunit.client.0.smithi019.stdout:2/294: unlink d1/d17/l3f 0 2024-02-22T15:50:31.661 INFO:tasks.workunit.client.1.smithi062.stdout:2/70: dwrite d1/f16 [0,4194304] 0 2024-02-22T15:50:31.662 INFO:tasks.workunit.client.0.smithi019.stdout:6/337: symlink d6/d31/d48/l61 0 2024-02-22T15:50:31.662 INFO:tasks.workunit.client.1.smithi062.stdout:8/38: dread f0 [0,4194304] 0 2024-02-22T15:50:31.663 INFO:tasks.workunit.client.1.smithi062.stdout:2/71: write d1/f16 [1860487,34487] 0 2024-02-22T15:50:31.663 INFO:tasks.workunit.client.1.smithi062.stdout:8/39: read f0 [5294120,57135] 0 2024-02-22T15:50:31.664 INFO:tasks.workunit.client.0.smithi019.stdout:5/383: mkdir d8/dd/d19/d3a/d46/d7e 0 2024-02-22T15:50:31.666 INFO:tasks.workunit.client.0.smithi019.stdout:8/327: sync 2024-02-22T15:50:31.666 INFO:tasks.workunit.client.1.smithi062.stdout:8/40: write f0 [2490215,39932] 0 2024-02-22T15:50:31.668 INFO:tasks.workunit.client.0.smithi019.stdout:6/338: rename d6/f47 to d6/d31/d37/f62 0 2024-02-22T15:50:31.670 INFO:tasks.workunit.client.0.smithi019.stdout:8/328: unlink d9/f65 0 2024-02-22T15:50:31.673 INFO:tasks.workunit.client.0.smithi019.stdout:2/295: dwrite d1/d17/f47 [0,4194304] 0 2024-02-22T15:50:31.673 INFO:tasks.workunit.client.0.smithi019.stdout:5/384: getdents d8/dd/d19/d77 0 2024-02-22T15:50:31.678 INFO:tasks.workunit.client.1.smithi062.stdout:9/39: write f2 [1003440,128129] 0 2024-02-22T15:50:31.680 INFO:tasks.workunit.client.1.smithi062.stdout:9/40: dread d4/f5 [0,4194304] 0 2024-02-22T15:50:31.682 INFO:tasks.workunit.client.0.smithi019.stdout:5/385: mknod d8/d11/d33/c7f 0 2024-02-22T15:50:31.685 INFO:tasks.workunit.client.0.smithi019.stdout:2/296: mknod d1/de/c64 0 2024-02-22T15:50:31.689 INFO:tasks.workunit.client.0.smithi019.stdout:5/386: getdents d8/dd/d19 0 2024-02-22T15:50:31.692 INFO:tasks.workunit.client.0.smithi019.stdout:2/297: link c0 d1/d17/d21/d51/c65 0 2024-02-22T15:50:31.694 INFO:tasks.workunit.client.1.smithi062.stdout:0/72: dwrite d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:31.694 INFO:tasks.workunit.client.0.smithi019.stdout:2/298: mkdir d1/d17/d1c/d66 0 2024-02-22T15:50:31.695 INFO:tasks.workunit.client.1.smithi062.stdout:0/73: chown d1/d3/d4/f7 4685 1 2024-02-22T15:50:31.697 INFO:tasks.workunit.client.0.smithi019.stdout:2/299: mkdir d1/d17/d67 0 2024-02-22T15:50:31.699 INFO:tasks.workunit.client.0.smithi019.stdout:2/300: creat d1/d17/d1c/d66/f68 x:0 0 0 2024-02-22T15:50:31.700 INFO:tasks.workunit.client.1.smithi062.stdout:0/74: truncate d1/d3/de/ff 1446968 0 2024-02-22T15:50:31.700 INFO:tasks.workunit.client.1.smithi062.stdout:0/75: chown d1/d13 1725 1 2024-02-22T15:50:31.701 INFO:tasks.workunit.client.0.smithi019.stdout:2/301: truncate d1/d17/d4b/f4a 395838 0 2024-02-22T15:50:31.702 INFO:tasks.workunit.client.1.smithi062.stdout:0/76: mknod d1/dd/c1b 0 2024-02-22T15:50:31.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.720+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f693c10c0a0 con 0x7f692406d260 2024-02-22T15:50:31.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.722+0000 7f6923fff700 1 -- 172.21.15.19:0/4247535911 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f693c10c0a0 con 0x7f692406d260 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "1/2 daemons upgraded", 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:50:31.704 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:50:31.705 INFO:tasks.workunit.client.1.smithi062.stdout:0/77: creat d1/dd/f1c x:0 0 0 2024-02-22T15:50:31.705 INFO:tasks.workunit.client.1.smithi062.stdout:0/78: dread - d1/d3/de/d14/f18 zero size 2024-02-22T15:50:31.705 INFO:tasks.workunit.client.1.smithi062.stdout:0/79: readlink d1/d3/l19 0 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f692406d260 msgr2=0x7f692406f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f692406d260 0x7f692406f710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6930009fd0 tx=0x7f6930009f00 comp rx=0 tx=0).stop 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 msgr2=0x7f693c114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c114040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f692c00b3b0 tx=0x7f692c009e30 comp rx=0 tx=0).stop 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 shutdown_connections 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f692406d260 0x7f692406f710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f693c1078b0 0x7f693c114040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 --2- 172.21.15.19:0/4247535911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f693c114580 0x7f693c115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 >> 172.21.15.19:0/4247535911 conn(0x7f693c06a2c0 msgr2=0x7f693c06c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:50:31.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 shutdown_connections 2024-02-22T15:50:31.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:50:31.724+0000 7f6941f26700 1 -- 172.21.15.19:0/4247535911 wait complete. 2024-02-22T15:50:31.710 INFO:tasks.workunit.client.1.smithi062.stdout:0/80: write d1/d3/de/ff [798868,59611] 0 2024-02-22T15:50:31.712 INFO:tasks.workunit.client.1.smithi062.stdout:0/81: dread d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:31.713 INFO:tasks.workunit.client.1.smithi062.stdout:0/82: stat d1/d3/de/ff 0 2024-02-22T15:50:31.721 INFO:tasks.workunit.client.1.smithi062.stdout:0/83: dwrite d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:31.735 INFO:tasks.workunit.client.1.smithi062.stdout:1/78: sync 2024-02-22T15:50:31.736 INFO:tasks.workunit.client.1.smithi062.stdout:4/23: sync 2024-02-22T15:50:31.736 INFO:tasks.workunit.client.1.smithi062.stdout:3/48: sync 2024-02-22T15:50:31.736 INFO:tasks.workunit.client.1.smithi062.stdout:1/79: chown d1/ca 463393222 1 2024-02-22T15:50:31.737 INFO:tasks.workunit.client.1.smithi062.stdout:4/24: readlink d4/l5 0 2024-02-22T15:50:31.739 INFO:tasks.workunit.client.1.smithi062.stdout:3/49: rename d5/fe to d5/f10 0 2024-02-22T15:50:31.742 INFO:tasks.workunit.client.1.smithi062.stdout:1/80: rename d1/d6/d10/l16 to d1/d6/d10/l18 0 2024-02-22T15:50:31.742 INFO:tasks.workunit.client.1.smithi062.stdout:1/81: chown d1/d6 0 1 2024-02-22T15:50:31.746 INFO:tasks.workunit.client.1.smithi062.stdout:8/41: fsync f0 0 2024-02-22T15:50:31.748 INFO:tasks.workunit.client.1.smithi062.stdout:3/50: dread d5/f10 [0,4194304] 0 2024-02-22T15:50:31.750 INFO:tasks.workunit.client.1.smithi062.stdout:8/42: rename c2 to d8/d9/cb 0 2024-02-22T15:50:31.750 INFO:tasks.workunit.client.1.smithi062.stdout:3/51: chown d5/c9 7 1 2024-02-22T15:50:31.751 INFO:tasks.workunit.client.1.smithi062.stdout:9/41: dread d4/f8 [0,4194304] 0 2024-02-22T15:50:31.753 INFO:tasks.workunit.client.1.smithi062.stdout:3/52: chown d5/cf 142869 1 2024-02-22T15:50:31.754 INFO:tasks.workunit.client.1.smithi062.stdout:9/42: creat d4/f10 x:0 0 0 2024-02-22T15:50:31.755 INFO:tasks.workunit.client.1.smithi062.stdout:2/72: fdatasync d1/f16 0 2024-02-22T15:50:31.756 INFO:tasks.workunit.client.0.smithi019.stdout:6/339: truncate d6/d31/d37/f62 1304884 0 2024-02-22T15:50:31.756 INFO:tasks.workunit.client.1.smithi062.stdout:1/82: dwrite d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:31.756 INFO:tasks.workunit.client.1.smithi062.stdout:3/53: symlink d5/l11 0 2024-02-22T15:50:31.757 INFO:tasks.workunit.client.1.smithi062.stdout:9/43: dread - d4/f10 zero size 2024-02-22T15:50:31.759 INFO:tasks.workunit.client.1.smithi062.stdout:3/54: chown d5/la 16 1 2024-02-22T15:50:31.760 INFO:tasks.workunit.client.1.smithi062.stdout:8/43: dread f0 [4194304,4194304] 0 2024-02-22T15:50:31.761 INFO:tasks.workunit.client.1.smithi062.stdout:8/44: stat c3 0 2024-02-22T15:50:31.761 INFO:tasks.workunit.client.1.smithi062.stdout:2/73: write d1/f6 [7809667,111033] 0 2024-02-22T15:50:31.763 INFO:tasks.workunit.client.1.smithi062.stdout:3/55: mknod d5/c12 0 2024-02-22T15:50:31.763 INFO:tasks.workunit.client.1.smithi062.stdout:9/44: dread d4/fd [0,4194304] 0 2024-02-22T15:50:31.763 INFO:tasks.workunit.client.1.smithi062.stdout:3/56: stat f3 0 2024-02-22T15:50:31.764 INFO:tasks.workunit.client.1.smithi062.stdout:8/45: mknod d8/cc 0 2024-02-22T15:50:31.765 INFO:tasks.workunit.client.0.smithi019.stdout:2/302: unlink d1/de/d15/l25 0 2024-02-22T15:50:31.765 INFO:tasks.workunit.client.1.smithi062.stdout:8/46: fsync f0 0 2024-02-22T15:50:31.766 INFO:tasks.workunit.client.1.smithi062.stdout:9/45: symlink d4/l11 0 2024-02-22T15:50:31.769 INFO:tasks.workunit.client.1.smithi062.stdout:1/83: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:31.771 INFO:tasks.workunit.client.0.smithi019.stdout:2/303: dwrite d1/f1f [0,4194304] 0 2024-02-22T15:50:31.772 INFO:tasks.workunit.client.0.smithi019.stdout:2/304: chown d1/l27 92718 1 2024-02-22T15:50:31.773 INFO:tasks.workunit.client.1.smithi062.stdout:1/84: mknod d1/d6/d7/c19 0 2024-02-22T15:50:31.775 INFO:tasks.workunit.client.0.smithi019.stdout:2/305: truncate d1/de/f31 4052876 0 2024-02-22T15:50:31.778 INFO:tasks.workunit.client.1.smithi062.stdout:8/47: dwrite f0 [0,4194304] 0 2024-02-22T15:50:31.781 INFO:tasks.workunit.client.0.smithi019.stdout:2/306: rename d1/de/l5c to d1/d17/d38/l69 0 2024-02-22T15:50:31.782 INFO:tasks.workunit.client.0.smithi019.stdout:2/307: truncate d1/d17/d21/f41 261294 0 2024-02-22T15:50:31.785 INFO:tasks.workunit.client.0.smithi019.stdout:2/308: creat d1/d17/d21/d51/f6a x:0 0 0 2024-02-22T15:50:31.789 INFO:tasks.workunit.client.0.smithi019.stdout:6/340: dread d6/d31/f36 [0,4194304] 0 2024-02-22T15:50:31.790 INFO:tasks.workunit.client.1.smithi062.stdout:2/74: dwrite d1/d8/f10 [0,4194304] 0 2024-02-22T15:50:31.792 INFO:tasks.workunit.client.0.smithi019.stdout:6/341: fsync d6/ff 0 2024-02-22T15:50:31.793 INFO:tasks.workunit.client.0.smithi019.stdout:6/342: dread - d6/d31/f3b zero size 2024-02-22T15:50:31.794 INFO:tasks.workunit.client.1.smithi062.stdout:9/46: dwrite d4/f5 [0,4194304] 0 2024-02-22T15:50:31.799 INFO:tasks.workunit.client.1.smithi062.stdout:2/75: write d1/d8/f11 [7247042,32886] 0 2024-02-22T15:50:31.800 INFO:tasks.workunit.client.1.smithi062.stdout:3/57: dwrite f3 [0,4194304] 0 2024-02-22T15:50:31.800 INFO:tasks.workunit.client.1.smithi062.stdout:1/85: dread d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:31.800 INFO:tasks.workunit.client.1.smithi062.stdout:8/48: dread f0 [0,4194304] 0 2024-02-22T15:50:31.802 INFO:tasks.workunit.client.0.smithi019.stdout:2/309: getdents d1/d17/d21/d51 0 2024-02-22T15:50:31.803 INFO:tasks.workunit.client.1.smithi062.stdout:9/47: dread d4/fd [0,4194304] 0 2024-02-22T15:50:31.804 INFO:tasks.workunit.client.0.smithi019.stdout:6/343: dwrite d6/f41 [4194304,4194304] 0 2024-02-22T15:50:31.806 INFO:tasks.workunit.client.1.smithi062.stdout:9/48: chown d4/fd 6 1 2024-02-22T15:50:31.807 INFO:tasks.workunit.client.1.smithi062.stdout:1/86: creat d1/d6/f1a x:0 0 0 2024-02-22T15:50:31.808 INFO:tasks.workunit.client.1.smithi062.stdout:9/49: creat d4/f12 x:0 0 0 2024-02-22T15:50:31.809 INFO:tasks.workunit.client.1.smithi062.stdout:2/76: dread d1/d8/f10 [0,4194304] 0 2024-02-22T15:50:31.809 INFO:tasks.workunit.client.1.smithi062.stdout:1/87: chown d1/d6 16209 1 2024-02-22T15:50:31.810 INFO:tasks.workunit.client.1.smithi062.stdout:9/50: creat d4/f13 x:0 0 0 2024-02-22T15:50:31.811 INFO:tasks.workunit.client.1.smithi062.stdout:2/77: write d1/f5 [292616,73343] 0 2024-02-22T15:50:31.812 INFO:tasks.workunit.client.1.smithi062.stdout:1/88: symlink d1/d6/d10/l1b 0 2024-02-22T15:50:31.812 INFO:tasks.workunit.client.0.smithi019.stdout:6/344: write d6/f2c [808032,58224] 0 2024-02-22T15:50:31.812 INFO:tasks.workunit.client.0.smithi019.stdout:2/310: unlink d1/de/l20 0 2024-02-22T15:50:31.814 INFO:tasks.workunit.client.1.smithi062.stdout:9/51: creat d4/f14 x:0 0 0 2024-02-22T15:50:31.815 INFO:tasks.workunit.client.0.smithi019.stdout:2/311: truncate d1/d46/f55 733994 0 2024-02-22T15:50:31.817 INFO:tasks.workunit.client.0.smithi019.stdout:6/345: dwrite d6/f1c [0,4194304] 0 2024-02-22T15:50:31.821 INFO:tasks.workunit.client.0.smithi019.stdout:2/312: symlink d1/d17/d21/d51/l6b 0 2024-02-22T15:50:31.823 INFO:tasks.workunit.client.0.smithi019.stdout:2/313: readlink d1/ld 0 2024-02-22T15:50:31.825 INFO:tasks.workunit.client.0.smithi019.stdout:6/346: dwrite d6/d3e/f42 [0,4194304] 0 2024-02-22T15:50:31.827 INFO:tasks.workunit.client.0.smithi019.stdout:6/347: write d6/f2a [466639,111733] 0 2024-02-22T15:50:31.828 INFO:tasks.workunit.client.0.smithi019.stdout:2/314: mknod d1/d17/d21/d51/c6c 0 2024-02-22T15:50:31.830 INFO:tasks.workunit.client.0.smithi019.stdout:4/326: sync 2024-02-22T15:50:31.830 INFO:tasks.workunit.client.0.smithi019.stdout:3/307: sync 2024-02-22T15:50:31.831 INFO:tasks.workunit.client.0.smithi019.stdout:7/354: sync 2024-02-22T15:50:31.832 INFO:tasks.workunit.client.0.smithi019.stdout:7/355: rename d9 to d9/d3a/d76 22 2024-02-22T15:50:31.834 INFO:tasks.workunit.client.0.smithi019.stdout:7/356: dread - d9/d38/d41/d45/f70 zero size 2024-02-22T15:50:31.834 INFO:tasks.workunit.client.0.smithi019.stdout:6/348: dread d6/d31/d43/f56 [0,4194304] 0 2024-02-22T15:50:31.835 INFO:tasks.workunit.client.0.smithi019.stdout:4/327: symlink d2/db/d14/d23/d21/d35/l6c 0 2024-02-22T15:50:31.839 INFO:tasks.workunit.client.0.smithi019.stdout:7/357: truncate d9/d26/f29 822605 0 2024-02-22T15:50:31.839 INFO:tasks.workunit.client.0.smithi019.stdout:3/308: dread d9/d22/d27/f46 [4194304,4194304] 0 2024-02-22T15:50:31.843 INFO:tasks.workunit.client.1.smithi062.stdout:5/40: sync 2024-02-22T15:50:31.844 INFO:tasks.workunit.client.0.smithi019.stdout:6/349: getdents d6 0 2024-02-22T15:50:31.844 INFO:tasks.workunit.client.0.smithi019.stdout:6/350: stat d6/l24 0 2024-02-22T15:50:31.844 INFO:tasks.workunit.client.0.smithi019.stdout:7/358: mkdir d9/d26/d77 0 2024-02-22T15:50:31.845 INFO:tasks.workunit.client.0.smithi019.stdout:4/328: mknod d2/db/d14/d23/d4b/c6d 0 2024-02-22T15:50:31.845 INFO:tasks.workunit.client.0.smithi019.stdout:6/351: write d6/f1d [750322,68993] 0 2024-02-22T15:50:31.848 INFO:tasks.workunit.client.0.smithi019.stdout:7/359: creat d9/d38/d41/f78 x:0 0 0 2024-02-22T15:50:31.849 INFO:tasks.workunit.client.1.smithi062.stdout:3/58: fsync f3 0 2024-02-22T15:50:31.849 INFO:tasks.workunit.client.0.smithi019.stdout:4/329: mkdir d2/d3f/d6e 0 2024-02-22T15:50:31.850 INFO:tasks.workunit.client.0.smithi019.stdout:4/330: chown d2 50820 1 2024-02-22T15:50:31.853 INFO:tasks.workunit.client.0.smithi019.stdout:7/360: mkdir d9/d79 0 2024-02-22T15:50:31.853 INFO:tasks.workunit.client.0.smithi019.stdout:6/352: dwrite d6/f1d [0,4194304] 0 2024-02-22T15:50:31.854 INFO:tasks.workunit.client.1.smithi062.stdout:8/49: dwrite f0 [8388608,4194304] 0 2024-02-22T15:50:31.854 INFO:tasks.workunit.client.1.smithi062.stdout:9/52: dread - d4/f10 zero size 2024-02-22T15:50:31.856 INFO:tasks.workunit.client.0.smithi019.stdout:4/331: mknod d2/db/d14/d23/d33/c6f 0 2024-02-22T15:50:31.856 INFO:tasks.workunit.client.0.smithi019.stdout:6/353: mkdir d6/d31/d63 0 2024-02-22T15:50:31.857 INFO:tasks.workunit.client.1.smithi062.stdout:1/89: getdents d1/d6/de 0 2024-02-22T15:50:31.857 INFO:tasks.workunit.client.0.smithi019.stdout:7/361: rmdir d9/d38/d41/d45/d61/d6a 39 2024-02-22T15:50:31.858 INFO:tasks.workunit.client.0.smithi019.stdout:7/362: dread - d9/d38/d41/f78 zero size 2024-02-22T15:50:31.864 INFO:tasks.workunit.client.1.smithi062.stdout:2/78: truncate d1/f5 269469 0 2024-02-22T15:50:31.864 INFO:tasks.workunit.client.1.smithi062.stdout:9/53: write d4/f8 [408719,113706] 0 2024-02-22T15:50:31.865 INFO:tasks.workunit.client.0.smithi019.stdout:6/354: unlink d6/d3e/f3f 0 2024-02-22T15:50:31.865 INFO:tasks.workunit.client.0.smithi019.stdout:4/332: link d2/db/d14/d23/l30 d2/db/d14/d23/d33/d66/l70 0 2024-02-22T15:50:31.868 INFO:tasks.workunit.client.1.smithi062.stdout:1/90: write d1/d6/de/f17 [3471493,107092] 0 2024-02-22T15:50:31.869 INFO:tasks.workunit.client.0.smithi019.stdout:6/355: mknod d6/c64 0 2024-02-22T15:50:31.869 INFO:tasks.workunit.client.0.smithi019.stdout:7/363: dwrite d9/f19 [0,4194304] 0 2024-02-22T15:50:31.871 INFO:tasks.workunit.client.1.smithi062.stdout:5/41: dwrite d0/f7 [0,4194304] 0 2024-02-22T15:50:31.871 INFO:tasks.workunit.client.0.smithi019.stdout:4/333: symlink d2/d3f/d48/l71 0 2024-02-22T15:50:31.872 INFO:tasks.workunit.client.1.smithi062.stdout:8/50: dread f0 [4194304,4194304] 0 2024-02-22T15:50:31.872 INFO:tasks.workunit.client.1.smithi062.stdout:2/79: truncate d1/d8/df/f14 1803531 0 2024-02-22T15:50:31.873 INFO:tasks.workunit.client.0.smithi019.stdout:7/364: mknod d9/d3a/d49/c7a 0 2024-02-22T15:50:31.874 INFO:tasks.workunit.client.1.smithi062.stdout:3/59: dwrite f3 [0,4194304] 0 2024-02-22T15:50:31.880 INFO:tasks.workunit.client.1.smithi062.stdout:2/80: mkdir d1/d8/d19 0 2024-02-22T15:50:31.880 INFO:tasks.workunit.client.0.smithi019.stdout:4/334: dwrite d2/db/f17 [4194304,4194304] 0 2024-02-22T15:50:31.884 INFO:tasks.workunit.client.1.smithi062.stdout:8/51: rename d8/ca to d8/cd 0 2024-02-22T15:50:31.886 INFO:tasks.workunit.client.0.smithi019.stdout:4/335: rename c1 to d2/db/d44/c72 0 2024-02-22T15:50:31.886 INFO:tasks.workunit.client.1.smithi062.stdout:8/52: readlink l7 0 2024-02-22T15:50:31.887 INFO:tasks.workunit.client.0.smithi019.stdout:6/356: mknod d6/d4b/c65 0 2024-02-22T15:50:31.888 INFO:tasks.workunit.client.1.smithi062.stdout:2/81: creat d1/d8/f1a x:0 0 0 2024-02-22T15:50:31.889 INFO:tasks.workunit.client.0.smithi019.stdout:7/365: dwrite d9/d3a/f75 [0,4194304] 0 2024-02-22T15:50:31.889 INFO:tasks.workunit.client.0.smithi019.stdout:7/366: chown d9/d26 967 1 2024-02-22T15:50:31.893 INFO:tasks.workunit.client.0.smithi019.stdout:7/367: mknod d9/d3a/d49/c7b 0 2024-02-22T15:50:31.893 INFO:tasks.workunit.client.0.smithi019.stdout:4/336: creat d2/db/d14/d61/f73 x:0 0 0 2024-02-22T15:50:31.894 INFO:tasks.workunit.client.0.smithi019.stdout:6/357: creat d6/d58/f66 x:0 0 0 2024-02-22T15:50:31.894 INFO:tasks.workunit.client.1.smithi062.stdout:9/54: dwrite d4/f12 [0,4194304] 0 2024-02-22T15:50:31.898 INFO:tasks.workunit.client.0.smithi019.stdout:7/368: truncate d9/d3a/d49/d5d/f72 1036229 0 2024-02-22T15:50:31.900 INFO:tasks.workunit.client.0.smithi019.stdout:7/369: rename d9/d26 to d9/d26/d77/d7c 22 2024-02-22T15:50:31.900 INFO:tasks.workunit.client.0.smithi019.stdout:4/337: link d2/db/d14/d23/f36 d2/db/d14/d23/d33/f74 0 2024-02-22T15:50:31.901 INFO:tasks.workunit.client.1.smithi062.stdout:2/82: symlink d1/d8/df/l1b 0 2024-02-22T15:50:31.903 INFO:tasks.workunit.client.0.smithi019.stdout:7/370: unlink d9/d3a/d49/f71 0 2024-02-22T15:50:31.905 INFO:tasks.workunit.client.1.smithi062.stdout:1/91: dwrite d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:31.905 INFO:tasks.workunit.client.0.smithi019.stdout:4/338: fdatasync d2/db/d14/d23/d33/f74 0 2024-02-22T15:50:31.905 INFO:tasks.workunit.client.1.smithi062.stdout:5/42: dwrite d0/f7 [0,4194304] 0 2024-02-22T15:50:31.906 INFO:tasks.workunit.client.0.smithi019.stdout:7/371: symlink d9/d3a/d49/d5d/l7d 0 2024-02-22T15:50:31.906 INFO:tasks.workunit.client.1.smithi062.stdout:9/55: rmdir d4 39 2024-02-22T15:50:31.907 INFO:tasks.workunit.client.0.smithi019.stdout:7/372: write d9/f1b [9321380,62560] 0 2024-02-22T15:50:31.910 INFO:tasks.workunit.client.0.smithi019.stdout:4/339: write d2/d3f/d48/f5b [502629,85900] 0 2024-02-22T15:50:31.912 INFO:tasks.workunit.client.0.smithi019.stdout:7/373: symlink d9/d38/d41/d45/d61/l7e 0 2024-02-22T15:50:31.913 INFO:tasks.workunit.client.1.smithi062.stdout:1/92: creat d1/d6/d10/f1c x:0 0 0 2024-02-22T15:50:31.913 INFO:tasks.workunit.client.1.smithi062.stdout:2/83: readlink l0 0 2024-02-22T15:50:31.913 INFO:tasks.workunit.client.1.smithi062.stdout:9/56: readlink d4/l11 0 2024-02-22T15:50:31.913 INFO:tasks.workunit.client.1.smithi062.stdout:5/43: write d0/d3/da/fb [2085564,126724] 0 2024-02-22T15:50:31.916 INFO:tasks.workunit.client.1.smithi062.stdout:9/57: dread - d4/fc zero size 2024-02-22T15:50:31.917 INFO:tasks.workunit.client.1.smithi062.stdout:1/93: truncate d1/d6/de/d9/fd 5052717 0 2024-02-22T15:50:31.917 INFO:tasks.workunit.client.0.smithi019.stdout:4/340: rename d2/c3 to d2/db/d14/d23/d33/c75 0 2024-02-22T15:50:31.918 INFO:tasks.workunit.client.0.smithi019.stdout:7/374: symlink d9/d3a/l7f 0 2024-02-22T15:50:31.919 INFO:tasks.workunit.client.1.smithi062.stdout:1/94: dread - d1/d6/d10/f1c zero size 2024-02-22T15:50:31.920 INFO:tasks.workunit.client.0.smithi019.stdout:4/341: write d2/db/d14/d23/f28 [325021,125908] 0 2024-02-22T15:50:31.920 INFO:tasks.workunit.client.0.smithi019.stdout:7/375: mknod d9/d3a/c80 0 2024-02-22T15:50:31.922 INFO:tasks.workunit.client.1.smithi062.stdout:1/95: chown d1/d6/f1a 105571 1 2024-02-22T15:50:31.922 INFO:tasks.workunit.client.1.smithi062.stdout:6/90: sync 2024-02-22T15:50:31.922 INFO:tasks.workunit.client.1.smithi062.stdout:1/96: dread - d1/d6/f1a zero size 2024-02-22T15:50:31.923 INFO:tasks.workunit.client.1.smithi062.stdout:1/97: dread - d1/d6/d10/f1c zero size 2024-02-22T15:50:31.923 INFO:tasks.workunit.client.1.smithi062.stdout:9/58: mkdir d4/d15 0 2024-02-22T15:50:31.924 INFO:tasks.workunit.client.0.smithi019.stdout:7/376: dread d9/d3a/d49/d5d/f72 [0,4194304] 0 2024-02-22T15:50:31.924 INFO:tasks.workunit.client.1.smithi062.stdout:2/84: rmdir d1/d8/d19 0 2024-02-22T15:50:31.925 INFO:tasks.workunit.client.1.smithi062.stdout:9/59: chown d4/f7 604133025 1 2024-02-22T15:50:31.926 INFO:tasks.workunit.client.0.smithi019.stdout:4/342: getdents d2/db/d14/d23/d21/d35 0 2024-02-22T15:50:31.926 INFO:tasks.workunit.client.1.smithi062.stdout:5/44: link d0/f2 d0/d3/f10 0 2024-02-22T15:50:31.927 INFO:tasks.workunit.client.0.smithi019.stdout:4/343: chown d2/db/d14/d23/d21/d35/c49 8 1 2024-02-22T15:50:31.927 INFO:tasks.workunit.client.1.smithi062.stdout:1/98: creat d1/d6/d7/f1d x:0 0 0 2024-02-22T15:50:31.928 INFO:tasks.workunit.client.0.smithi019.stdout:4/344: write d2/f6 [525200,85561] 0 2024-02-22T15:50:31.928 INFO:tasks.workunit.client.0.smithi019.stdout:7/377: symlink d9/d38/l81 0 2024-02-22T15:50:31.930 INFO:tasks.workunit.client.1.smithi062.stdout:5/45: mknod d0/c11 0 2024-02-22T15:50:31.932 INFO:tasks.workunit.client.0.smithi019.stdout:7/378: mknod d9/d26/d77/c82 0 2024-02-22T15:50:31.932 INFO:tasks.workunit.client.1.smithi062.stdout:6/91: getdents d5 0 2024-02-22T15:50:31.934 INFO:tasks.workunit.client.0.smithi019.stdout:7/379: mknod d9/c83 0 2024-02-22T15:50:31.934 INFO:tasks.workunit.client.1.smithi062.stdout:1/99: symlink d1/l1e 0 2024-02-22T15:50:31.935 INFO:tasks.workunit.client.1.smithi062.stdout:2/85: dwrite d1/d8/f11 [0,4194304] 0 2024-02-22T15:50:31.936 INFO:tasks.workunit.client.1.smithi062.stdout:1/100: fdatasync d1/d6/d10/f1c 0 2024-02-22T15:50:31.941 INFO:tasks.workunit.client.1.smithi062.stdout:2/86: chown d1/d8/fa 342636201 1 2024-02-22T15:50:31.942 INFO:tasks.workunit.client.1.smithi062.stdout:1/101: truncate d1/d6/f1a 900949 0 2024-02-22T15:50:31.943 INFO:tasks.workunit.client.0.smithi019.stdout:6/358: write d6/f4d [217210,129834] 0 2024-02-22T15:50:31.944 INFO:tasks.workunit.client.1.smithi062.stdout:6/92: link f0 d5/df/f26 0 2024-02-22T15:50:31.946 INFO:tasks.workunit.client.1.smithi062.stdout:3/60: write d5/f6 [93841,114578] 0 2024-02-22T15:50:31.947 INFO:tasks.workunit.client.0.smithi019.stdout:4/345: rmdir d2 39 2024-02-22T15:50:31.947 INFO:tasks.workunit.client.1.smithi062.stdout:5/46: dwrite d0/d3/da/fb [0,4194304] 0 2024-02-22T15:50:31.948 INFO:tasks.workunit.client.1.smithi062.stdout:5/47: chown d0/le 1 1 2024-02-22T15:50:31.949 INFO:tasks.workunit.client.0.smithi019.stdout:6/359: mkdir d6/d31/d43/d55/d67 0 2024-02-22T15:50:31.950 INFO:tasks.workunit.client.1.smithi062.stdout:8/53: truncate f0 10516344 0 2024-02-22T15:50:31.953 INFO:tasks.workunit.client.0.smithi019.stdout:6/360: chown d6/f17 31383926 1 2024-02-22T15:50:31.955 INFO:tasks.workunit.client.0.smithi019.stdout:4/346: read d2/d3f/d48/f3e [221913,57076] 0 2024-02-22T15:50:31.956 INFO:tasks.workunit.client.1.smithi062.stdout:9/60: getdents d4 0 2024-02-22T15:50:31.956 INFO:tasks.workunit.client.1.smithi062.stdout:1/102: symlink d1/d6/d10/l1f 0 2024-02-22T15:50:31.958 INFO:tasks.workunit.client.1.smithi062.stdout:2/87: creat d1/d8/f1c x:0 0 0 2024-02-22T15:50:31.958 INFO:tasks.workunit.client.1.smithi062.stdout:9/61: write d4/f5 [3954485,3469] 0 2024-02-22T15:50:31.959 INFO:tasks.workunit.client.0.smithi019.stdout:6/361: unlink d6/f4d 0 2024-02-22T15:50:31.959 INFO:tasks.workunit.client.0.smithi019.stdout:7/380: truncate d9/d26/f2e 111855 0 2024-02-22T15:50:31.960 INFO:tasks.workunit.client.1.smithi062.stdout:6/93: stat d5/df/f21 0 2024-02-22T15:50:31.960 INFO:tasks.workunit.client.1.smithi062.stdout:5/48: fsync d0/f9 0 2024-02-22T15:50:31.960 INFO:tasks.workunit.client.1.smithi062.stdout:2/88: readlink l0 0 2024-02-22T15:50:31.961 INFO:tasks.workunit.client.0.smithi019.stdout:9/340: sync 2024-02-22T15:50:31.961 INFO:tasks.workunit.client.0.smithi019.stdout:4/347: mkdir d2/d3f/d48/d76 0 2024-02-22T15:50:31.962 INFO:tasks.workunit.client.1.smithi062.stdout:3/61: dwrite d5/f6 [0,4194304] 0 2024-02-22T15:50:31.963 INFO:tasks.workunit.client.0.smithi019.stdout:4/348: write d2/fd [935476,77097] 0 2024-02-22T15:50:31.963 INFO:tasks.workunit.client.1.smithi062.stdout:1/103: creat d1/d6/d14/f20 x:0 0 0 2024-02-22T15:50:31.964 INFO:tasks.workunit.client.0.smithi019.stdout:9/341: creat d2/d1f/d2e/f7b x:0 0 0 2024-02-22T15:50:31.966 INFO:tasks.workunit.client.0.smithi019.stdout:4/349: write d2/db/d14/d23/d21/f29 [219464,98089] 0 2024-02-22T15:50:31.966 INFO:tasks.workunit.client.1.smithi062.stdout:2/89: mknod d1/c1d 0 2024-02-22T15:50:31.967 INFO:tasks.workunit.client.0.smithi019.stdout:6/362: rename d6/d4b/c65 to d6/d31/d43/d55/c68 0 2024-02-22T15:50:31.967 INFO:tasks.workunit.client.1.smithi062.stdout:5/49: creat d0/d3/da/f12 x:0 0 0 2024-02-22T15:50:31.968 INFO:tasks.workunit.client.1.smithi062.stdout:0/84: sync 2024-02-22T15:50:31.969 INFO:tasks.workunit.client.1.smithi062.stdout:4/25: sync 2024-02-22T15:50:31.969 INFO:tasks.workunit.client.1.smithi062.stdout:7/90: sync 2024-02-22T15:50:31.969 INFO:tasks.workunit.client.0.smithi019.stdout:7/381: getdents d9/d4c/d5b 0 2024-02-22T15:50:31.969 INFO:tasks.workunit.client.1.smithi062.stdout:9/62: truncate d4/f7 834865 0 2024-02-22T15:50:31.970 INFO:tasks.workunit.client.1.smithi062.stdout:6/94: fdatasync d5/df/f10 0 2024-02-22T15:50:31.970 INFO:tasks.workunit.client.1.smithi062.stdout:3/62: creat d5/f13 x:0 0 0 2024-02-22T15:50:31.972 INFO:tasks.workunit.client.0.smithi019.stdout:6/363: rename d6/d31/d63 to d6/d31/d69 0 2024-02-22T15:50:31.972 INFO:tasks.workunit.client.0.smithi019.stdout:9/342: rmdir d2/d1f/d2e/d48/d36/d4a/d66 39 2024-02-22T15:50:31.973 INFO:tasks.workunit.client.0.smithi019.stdout:7/382: write d9/d38/f3c [2840112,124200] 0 2024-02-22T15:50:31.974 INFO:tasks.workunit.client.1.smithi062.stdout:5/50: write d0/f7 [3125099,29437] 0 2024-02-22T15:50:31.976 INFO:tasks.workunit.client.0.smithi019.stdout:9/343: rename d2/d7/dc/d27/d31/c7a to d2/d1f/d2e/c7c 0 2024-02-22T15:50:31.977 INFO:tasks.workunit.client.0.smithi019.stdout:7/383: mknod d9/d3a/c84 0 2024-02-22T15:50:31.978 INFO:tasks.workunit.client.1.smithi062.stdout:1/104: write d1/d6/d10/f1c [671738,47894] 0 2024-02-22T15:50:31.978 INFO:tasks.workunit.client.1.smithi062.stdout:4/26: read f3 [3674992,82482] 0 2024-02-22T15:50:31.979 INFO:tasks.workunit.client.1.smithi062.stdout:0/85: chown d1/d3/d4/d1a 0 1 2024-02-22T15:50:31.981 INFO:tasks.workunit.client.1.smithi062.stdout:4/27: read f3 [3008195,128330] 0 2024-02-22T15:50:31.982 INFO:tasks.workunit.client.1.smithi062.stdout:0/86: write d1/d3/de/d14/f18 [284597,85027] 0 2024-02-22T15:50:31.982 INFO:tasks.workunit.client.1.smithi062.stdout:2/90: truncate d1/d8/f1c 192946 0 2024-02-22T15:50:31.983 INFO:tasks.workunit.client.1.smithi062.stdout:9/63: truncate d4/fd 635980 0 2024-02-22T15:50:31.984 INFO:tasks.workunit.client.0.smithi019.stdout:4/350: dread d2/db/d14/f5e [0,4194304] 0 2024-02-22T15:50:31.985 INFO:tasks.workunit.client.1.smithi062.stdout:6/95: fdatasync d5/df/f26 0 2024-02-22T15:50:31.985 INFO:tasks.workunit.client.1.smithi062.stdout:9/64: write d4/f12 [4414917,101180] 0 2024-02-22T15:50:31.988 INFO:tasks.workunit.client.1.smithi062.stdout:9/65: write d4/f12 [2417384,116063] 0 2024-02-22T15:50:31.988 INFO:tasks.workunit.client.1.smithi062.stdout:8/54: write f0 [5476205,82285] 0 2024-02-22T15:50:31.989 INFO:tasks.workunit.client.0.smithi019.stdout:4/351: dread d2/db/d14/d23/f28 [0,4194304] 0 2024-02-22T15:50:31.990 INFO:tasks.workunit.client.1.smithi062.stdout:5/51: mkdir d0/d3/da/d13 0 2024-02-22T15:50:31.990 INFO:tasks.workunit.client.0.smithi019.stdout:4/352: fsync d2/db/d14/d23/d21/f29 0 2024-02-22T15:50:31.992 INFO:tasks.workunit.client.0.smithi019.stdout:7/384: rmdir d9/d3a 39 2024-02-22T15:50:31.992 INFO:tasks.workunit.client.1.smithi062.stdout:5/52: chown d0/f7 3 1 2024-02-22T15:50:31.994 INFO:tasks.workunit.client.1.smithi062.stdout:7/91: mknod d0/d14/d19/c1c 0 2024-02-22T15:50:31.994 INFO:tasks.workunit.client.1.smithi062.stdout:6/96: write d5/d15/f23 [922368,64399] 0 2024-02-22T15:50:31.995 INFO:tasks.workunit.client.1.smithi062.stdout:3/63: getdents d5 0 2024-02-22T15:50:31.995 INFO:tasks.workunit.client.0.smithi019.stdout:4/353: rename d2/l43 to d2/db/d14/d23/d21/d42/l77 0 2024-02-22T15:50:31.996 INFO:tasks.workunit.client.0.smithi019.stdout:6/364: truncate d6/d3e/f42 496983 0 2024-02-22T15:50:31.997 INFO:tasks.workunit.client.0.smithi019.stdout:7/385: dread - d9/d3a/f6b zero size 2024-02-22T15:50:31.998 INFO:tasks.workunit.client.1.smithi062.stdout:8/55: dread f0 [8388608,4194304] 0 2024-02-22T15:50:31.999 INFO:tasks.workunit.client.0.smithi019.stdout:7/386: write d9/d38/f59 [865188,56045] 0 2024-02-22T15:50:32.002 INFO:tasks.workunit.client.0.smithi019.stdout:9/344: dwrite d2/d7/f1b [4194304,4194304] 0 2024-02-22T15:50:32.002 INFO:tasks.workunit.client.1.smithi062.stdout:8/56: chown c6 75 1 2024-02-22T15:50:32.003 INFO:tasks.workunit.client.1.smithi062.stdout:0/87: mkdir d1/d3/d4/d1a/d1d 0 2024-02-22T15:50:32.004 INFO:tasks.workunit.client.1.smithi062.stdout:7/92: write d0/f5 [4884300,86014] 0 2024-02-22T15:50:32.004 INFO:tasks.workunit.client.1.smithi062.stdout:3/64: mknod d5/c14 0 2024-02-22T15:50:32.005 INFO:tasks.workunit.client.0.smithi019.stdout:7/387: rename d9/d26/c62 to d9/d79/c85 0 2024-02-22T15:50:32.006 INFO:tasks.workunit.client.1.smithi062.stdout:9/66: symlink d4/d15/l16 0 2024-02-22T15:50:32.006 INFO:tasks.workunit.client.0.smithi019.stdout:4/354: link d2/db/d44/c72 d2/db/d54/c78 0 2024-02-22T15:50:32.007 INFO:tasks.workunit.client.1.smithi062.stdout:6/97: creat d5/dd/f27 x:0 0 0 2024-02-22T15:50:32.008 INFO:tasks.workunit.client.0.smithi019.stdout:9/345: mkdir d2/d1f/d2e/d7d 0 2024-02-22T15:50:32.009 INFO:tasks.workunit.client.0.smithi019.stdout:4/355: write d2/db/d14/d23/d21/f2c [760878,121226] 0 2024-02-22T15:50:32.009 INFO:tasks.workunit.client.0.smithi019.stdout:7/388: creat d9/d3a/f86 x:0 0 0 2024-02-22T15:50:32.010 INFO:tasks.workunit.client.1.smithi062.stdout:1/105: dwrite d1/d6/d10/f1c [0,4194304] 0 2024-02-22T15:50:32.010 INFO:tasks.workunit.client.0.smithi019.stdout:9/346: chown d2/d1f/d2e/d48/d45/f61 120589 1 2024-02-22T15:50:32.011 INFO:tasks.workunit.client.0.smithi019.stdout:4/356: stat d2/db/d14/d23/f36 0 2024-02-22T15:50:32.012 INFO:tasks.workunit.client.0.smithi019.stdout:6/365: getdents d6/d58 0 2024-02-22T15:50:32.014 INFO:tasks.workunit.client.1.smithi062.stdout:6/98: fsync d5/f1e 0 2024-02-22T15:50:32.014 INFO:tasks.workunit.client.1.smithi062.stdout:9/67: chown d4/f8 1423692372 1 2024-02-22T15:50:32.014 INFO:tasks.workunit.client.1.smithi062.stdout:4/28: dwrite f2 [0,4194304] 0 2024-02-22T15:50:32.014 INFO:tasks.workunit.client.1.smithi062.stdout:4/29: readlink l0 0 2024-02-22T15:50:32.015 INFO:tasks.workunit.client.1.smithi062.stdout:8/57: write f0 [5502742,9607] 0 2024-02-22T15:50:32.015 INFO:tasks.workunit.client.0.smithi019.stdout:6/366: rmdir d6/d31/d37 39 2024-02-22T15:50:32.016 INFO:tasks.workunit.client.1.smithi062.stdout:5/53: write d0/f2 [3189605,81608] 0 2024-02-22T15:50:32.016 INFO:tasks.workunit.client.1.smithi062.stdout:7/93: readlink d0/d14/l1b 0 2024-02-22T15:50:32.017 INFO:tasks.workunit.client.1.smithi062.stdout:1/106: chown d1/d6/d14/f20 0 1 2024-02-22T15:50:32.018 INFO:tasks.workunit.client.1.smithi062.stdout:0/88: truncate d1/dd/f16 525917 0 2024-02-22T15:50:32.019 INFO:tasks.workunit.client.1.smithi062.stdout:9/68: creat d4/d15/f17 x:0 0 0 2024-02-22T15:50:32.020 INFO:tasks.workunit.client.0.smithi019.stdout:6/367: fdatasync d6/d31/d43/d55/f5c 0 2024-02-22T15:50:32.021 INFO:tasks.workunit.client.0.smithi019.stdout:7/389: dwrite d9/d38/f52 [0,4194304] 0 2024-02-22T15:50:32.021 INFO:tasks.workunit.client.1.smithi062.stdout:5/54: write d0/f7 [4328833,7070] 0 2024-02-22T15:50:32.022 INFO:tasks.workunit.client.1.smithi062.stdout:4/30: unlink f3 0 2024-02-22T15:50:32.023 INFO:tasks.workunit.client.0.smithi019.stdout:4/357: link d2/db/d14/d23/d21/l22 d2/d3f/l79 0 2024-02-22T15:50:32.025 INFO:tasks.workunit.client.0.smithi019.stdout:9/347: dread d2/d7/f44 [0,4194304] 0 2024-02-22T15:50:32.026 INFO:tasks.workunit.client.0.smithi019.stdout:7/390: write d9/fa [33255,85848] 0 2024-02-22T15:50:32.027 INFO:tasks.workunit.client.0.smithi019.stdout:6/368: chown d6/d31/d43/d55/d67 15 1 2024-02-22T15:50:32.028 INFO:tasks.workunit.client.1.smithi062.stdout:9/69: symlink d4/l18 0 2024-02-22T15:50:32.028 INFO:tasks.workunit.client.1.smithi062.stdout:7/94: dread - d0/d14/d19/f1a zero size 2024-02-22T15:50:32.028 INFO:tasks.workunit.client.0.smithi019.stdout:9/348: write d2/d1f/d2e/d48/d45/f61 [293216,124935] 0 2024-02-22T15:50:32.030 INFO:tasks.workunit.client.1.smithi062.stdout:2/91: dwrite d1/d8/f10 [0,4194304] 0 2024-02-22T15:50:32.031 INFO:tasks.workunit.client.1.smithi062.stdout:9/70: write d4/fe [653078,29080] 0 2024-02-22T15:50:32.032 INFO:tasks.workunit.client.1.smithi062.stdout:6/99: rename d5/d15/f23 to d5/d19/f28 0 2024-02-22T15:50:32.033 INFO:tasks.workunit.client.1.smithi062.stdout:1/107: mkdir d1/d6/de/d21 0 2024-02-22T15:50:32.033 INFO:tasks.workunit.client.1.smithi062.stdout:4/31: rename d4 to d4/d7 22 2024-02-22T15:50:32.034 INFO:tasks.workunit.client.1.smithi062.stdout:7/95: write d0/d14/d19/f1a [170030,74190] 0 2024-02-22T15:50:32.038 INFO:tasks.workunit.client.1.smithi062.stdout:1/108: dread d1/d6/f1a [0,4194304] 0 2024-02-22T15:50:32.041 INFO:tasks.workunit.client.1.smithi062.stdout:1/109: write d1/d6/d14/f20 [745796,21565] 0 2024-02-22T15:50:32.042 INFO:tasks.workunit.client.1.smithi062.stdout:9/71: truncate d4/fd 429386 0 2024-02-22T15:50:32.045 INFO:tasks.workunit.client.1.smithi062.stdout:9/72: write d4/f14 [825751,56447] 0 2024-02-22T15:50:32.045 INFO:tasks.workunit.client.1.smithi062.stdout:7/96: read d0/f10 [673194,62322] 0 2024-02-22T15:50:32.045 INFO:tasks.workunit.client.1.smithi062.stdout:3/65: truncate d5/f6 4023455 0 2024-02-22T15:50:32.045 INFO:tasks.workunit.client.1.smithi062.stdout:1/110: dread d1/d6/f1a [0,4194304] 0 2024-02-22T15:50:32.046 INFO:tasks.workunit.client.1.smithi062.stdout:3/66: chown d5/cf 388517 1 2024-02-22T15:50:32.047 INFO:tasks.workunit.client.1.smithi062.stdout:9/73: chown d4/fc 64 1 2024-02-22T15:50:32.050 INFO:tasks.workunit.client.1.smithi062.stdout:4/32: dread f2 [0,4194304] 0 2024-02-22T15:50:32.050 INFO:tasks.workunit.client.1.smithi062.stdout:0/89: write d1/f12 [5198247,129291] 0 2024-02-22T15:50:32.053 INFO:tasks.workunit.client.1.smithi062.stdout:8/58: truncate f0 8624551 0 2024-02-22T15:50:32.053 INFO:tasks.workunit.client.1.smithi062.stdout:7/97: dread d0/f10 [0,4194304] 0 2024-02-22T15:50:32.053 INFO:tasks.workunit.client.1.smithi062.stdout:9/74: symlink d4/d15/l19 0 2024-02-22T15:50:32.053 INFO:tasks.workunit.client.1.smithi062.stdout:1/111: rename d1/d6/de/lb to d1/d6/d7/l22 0 2024-02-22T15:50:32.054 INFO:tasks.workunit.client.1.smithi062.stdout:0/90: write d1/f12 [2063356,55764] 0 2024-02-22T15:50:32.054 INFO:tasks.workunit.client.1.smithi062.stdout:9/75: readlink d4/l18 0 2024-02-22T15:50:32.055 INFO:tasks.workunit.client.1.smithi062.stdout:7/98: stat d0/d14/d13 0 2024-02-22T15:50:32.055 INFO:tasks.workunit.client.1.smithi062.stdout:6/100: write d5/fb [1198320,115569] 0 2024-02-22T15:50:32.057 INFO:tasks.workunit.client.1.smithi062.stdout:2/92: write d1/f5 [779584,19719] 0 2024-02-22T15:50:32.058 INFO:tasks.workunit.client.1.smithi062.stdout:7/99: mknod d0/d14/c1d 0 2024-02-22T15:50:32.060 INFO:tasks.workunit.client.1.smithi062.stdout:8/59: chown d8/cd 863394601 1 2024-02-22T15:50:32.061 INFO:tasks.workunit.client.1.smithi062.stdout:9/76: dread - d4/f10 zero size 2024-02-22T15:50:32.061 INFO:tasks.workunit.client.1.smithi062.stdout:6/101: stat d5/d15/l16 0 2024-02-22T15:50:32.061 INFO:tasks.workunit.client.1.smithi062.stdout:8/60: stat c3 0 2024-02-22T15:50:32.062 INFO:tasks.workunit.client.1.smithi062.stdout:1/112: symlink d1/d6/l23 0 2024-02-22T15:50:32.062 INFO:tasks.workunit.client.1.smithi062.stdout:0/91: mkdir d1/d3/d4/d1e 0 2024-02-22T15:50:32.064 INFO:tasks.workunit.client.1.smithi062.stdout:3/67: write d5/f10 [816227,109669] 0 2024-02-22T15:50:32.065 INFO:tasks.workunit.client.1.smithi062.stdout:2/93: mkdir d1/d1e 0 2024-02-22T15:50:32.066 INFO:tasks.workunit.client.1.smithi062.stdout:3/68: readlink d5/l7 0 2024-02-22T15:50:32.066 INFO:tasks.workunit.client.1.smithi062.stdout:9/77: write d4/f14 [1398016,6147] 0 2024-02-22T15:50:32.068 INFO:tasks.workunit.client.1.smithi062.stdout:4/33: dread f2 [0,4194304] 0 2024-02-22T15:50:32.068 INFO:tasks.workunit.client.1.smithi062.stdout:7/100: mkdir d0/d14/d1e 0 2024-02-22T15:50:32.070 INFO:tasks.workunit.client.1.smithi062.stdout:6/102: mknod d5/d15/c29 0 2024-02-22T15:50:32.076 INFO:tasks.workunit.client.1.smithi062.stdout:3/69: creat d5/f15 x:0 0 0 2024-02-22T15:50:32.079 INFO:tasks.workunit.client.1.smithi062.stdout:2/94: dread d1/d8/f11 [4194304,4194304] 0 2024-02-22T15:50:32.079 INFO:tasks.workunit.client.1.smithi062.stdout:3/70: readlink d5/l7 0 2024-02-22T15:50:32.080 INFO:tasks.workunit.client.1.smithi062.stdout:1/113: mkdir d1/d6/de/d21/d24 0 2024-02-22T15:50:32.081 INFO:tasks.workunit.client.1.smithi062.stdout:4/34: write f2 [4826900,122443] 0 2024-02-22T15:50:32.082 INFO:tasks.workunit.client.1.smithi062.stdout:7/101: dread d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:32.083 INFO:tasks.workunit.client.1.smithi062.stdout:9/78: dwrite f2 [0,4194304] 0 2024-02-22T15:50:32.083 INFO:tasks.workunit.client.1.smithi062.stdout:0/92: mkdir d1/d3/d4/d1a/d1d/d1f 0 2024-02-22T15:50:32.088 INFO:tasks.workunit.client.1.smithi062.stdout:3/71: mkdir d5/d16 0 2024-02-22T15:50:32.090 INFO:tasks.workunit.client.1.smithi062.stdout:6/103: dread d5/f1f [0,4194304] 0 2024-02-22T15:50:32.092 INFO:tasks.workunit.client.1.smithi062.stdout:4/35: creat d4/f8 x:0 0 0 2024-02-22T15:50:32.093 INFO:tasks.workunit.client.1.smithi062.stdout:2/95: truncate d1/f5 1009818 0 2024-02-22T15:50:32.093 INFO:tasks.workunit.client.1.smithi062.stdout:9/79: truncate d4/f10 338160 0 2024-02-22T15:50:32.093 INFO:tasks.workunit.client.1.smithi062.stdout:3/72: symlink d5/l17 0 2024-02-22T15:50:32.094 INFO:tasks.workunit.client.1.smithi062.stdout:6/104: fsync d5/df/f10 0 2024-02-22T15:50:32.095 INFO:tasks.workunit.client.1.smithi062.stdout:4/36: mknod d4/c9 0 2024-02-22T15:50:32.096 INFO:tasks.workunit.client.1.smithi062.stdout:9/80: write d4/f13 [820680,70481] 0 2024-02-22T15:50:32.096 INFO:tasks.workunit.client.1.smithi062.stdout:4/37: readlink l0 0 2024-02-22T15:50:32.097 INFO:tasks.workunit.client.1.smithi062.stdout:7/102: creat d0/db/f1f x:0 0 0 2024-02-22T15:50:32.101 INFO:tasks.workunit.client.1.smithi062.stdout:2/96: write d1/f16 [1446426,102444] 0 2024-02-22T15:50:32.101 INFO:tasks.workunit.client.1.smithi062.stdout:6/105: mkdir d5/df/d2a 0 2024-02-22T15:50:32.101 INFO:tasks.workunit.client.1.smithi062.stdout:4/38: readlink d4/l6 0 2024-02-22T15:50:32.103 INFO:tasks.workunit.client.1.smithi062.stdout:1/114: dread d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:32.105 INFO:tasks.workunit.client.1.smithi062.stdout:7/103: creat d0/d14/f20 x:0 0 0 2024-02-22T15:50:32.107 INFO:tasks.workunit.client.1.smithi062.stdout:6/106: write d5/f1e [4926237,58668] 0 2024-02-22T15:50:32.107 INFO:tasks.workunit.client.0.smithi019.stdout:7/391: creat d9/d26/f87 x:0 0 0 2024-02-22T15:50:32.108 INFO:tasks.workunit.client.0.smithi019.stdout:6/369: mkdir d6/d58/d6a 0 2024-02-22T15:50:32.109 INFO:tasks.workunit.client.1.smithi062.stdout:0/93: truncate d1/d3/d4/f7 3448010 0 2024-02-22T15:50:32.109 INFO:tasks.workunit.client.1.smithi062.stdout:7/104: dread - d0/db/f1f zero size 2024-02-22T15:50:32.112 INFO:tasks.workunit.client.1.smithi062.stdout:2/97: mknod d1/d8/df/c1f 0 2024-02-22T15:50:32.114 INFO:tasks.workunit.client.1.smithi062.stdout:3/73: dwrite d5/f15 [0,4194304] 0 2024-02-22T15:50:32.116 INFO:tasks.workunit.client.1.smithi062.stdout:1/115: creat d1/d6/de/d9/f25 x:0 0 0 2024-02-22T15:50:32.118 INFO:tasks.workunit.client.0.smithi019.stdout:7/392: dwrite d9/d38/f4e [0,4194304] 0 2024-02-22T15:50:32.118 INFO:tasks.workunit.client.0.smithi019.stdout:6/370: write d6/d4b/f52 [1363749,100434] 0 2024-02-22T15:50:32.119 INFO:tasks.workunit.client.1.smithi062.stdout:8/61: write f0 [9215060,25946] 0 2024-02-22T15:50:32.119 INFO:tasks.workunit.client.1.smithi062.stdout:6/107: symlink d5/dd/l2b 0 2024-02-22T15:50:32.120 INFO:tasks.workunit.client.1.smithi062.stdout:8/62: fsync f0 0 2024-02-22T15:50:32.123 INFO:tasks.workunit.client.0.smithi019.stdout:9/349: dwrite d2/f30 [4194304,4194304] 0 2024-02-22T15:50:32.123 INFO:tasks.workunit.client.0.smithi019.stdout:7/393: write d9/f19 [2210928,44593] 0 2024-02-22T15:50:32.125 INFO:tasks.workunit.client.1.smithi062.stdout:1/116: read d1/d6/d14/f20 [674478,110924] 0 2024-02-22T15:50:32.127 INFO:tasks.workunit.client.0.smithi019.stdout:9/350: chown d2/d7/d14/l3a 90 1 2024-02-22T15:50:32.127 INFO:tasks.workunit.client.1.smithi062.stdout:0/94: unlink d1/dd/c1b 0 2024-02-22T15:50:32.129 INFO:tasks.workunit.client.1.smithi062.stdout:7/105: mknod d0/d14/d19/c21 0 2024-02-22T15:50:32.129 INFO:tasks.workunit.client.0.smithi019.stdout:6/371: unlink d6/f59 0 2024-02-22T15:50:32.129 INFO:tasks.workunit.client.1.smithi062.stdout:4/39: dwrite d4/f8 [0,4194304] 0 2024-02-22T15:50:32.130 INFO:tasks.workunit.client.0.smithi019.stdout:7/394: creat d9/d3a/d49/d5d/d74/f88 x:0 0 0 2024-02-22T15:50:32.130 INFO:tasks.workunit.client.0.smithi019.stdout:4/358: dread d2/db/d14/f18 [0,4194304] 0 2024-02-22T15:50:32.131 INFO:tasks.workunit.client.1.smithi062.stdout:8/63: dread f0 [4194304,4194304] 0 2024-02-22T15:50:32.132 INFO:tasks.workunit.client.0.smithi019.stdout:9/351: mkdir d2/d1f/d2e/d57/d7e 0 2024-02-22T15:50:32.132 INFO:tasks.workunit.client.1.smithi062.stdout:9/81: link d4/d15/l16 d4/d15/l1a 0 2024-02-22T15:50:32.134 INFO:tasks.workunit.client.1.smithi062.stdout:6/108: creat d5/dd/f2c x:0 0 0 2024-02-22T15:50:32.135 INFO:tasks.workunit.client.1.smithi062.stdout:1/117: readlink d1/d6/d10/l18 0 2024-02-22T15:50:32.135 INFO:tasks.workunit.client.1.smithi062.stdout:0/95: symlink d1/d3/l20 0 2024-02-22T15:50:32.135 INFO:tasks.workunit.client.1.smithi062.stdout:3/74: fsync d5/f6 0 2024-02-22T15:50:32.135 INFO:tasks.workunit.client.0.smithi019.stdout:4/359: mknod d2/d3f/d6e/c7a 0 2024-02-22T15:50:32.138 INFO:tasks.workunit.client.1.smithi062.stdout:9/82: readlink d4/d15/l1a 0 2024-02-22T15:50:32.139 INFO:tasks.workunit.client.1.smithi062.stdout:7/106: chown d0/d14/d19/c21 363690985 1 2024-02-22T15:50:32.140 INFO:tasks.workunit.client.1.smithi062.stdout:0/96: chown d1/d3/f6 43 1 2024-02-22T15:50:32.141 INFO:tasks.workunit.client.0.smithi019.stdout:4/360: write d2/db/d14/d23/d21/f2c [1763509,33627] 0 2024-02-22T15:50:32.141 INFO:tasks.workunit.client.1.smithi062.stdout:3/75: mknod d5/c18 0 2024-02-22T15:50:32.142 INFO:tasks.workunit.client.1.smithi062.stdout:8/64: dread f0 [0,4194304] 0 2024-02-22T15:50:32.143 INFO:tasks.workunit.client.1.smithi062.stdout:0/97: fsync d1/d3/de/ff 0 2024-02-22T15:50:32.143 INFO:tasks.workunit.client.1.smithi062.stdout:8/65: chown c3 3 1 2024-02-22T15:50:32.144 INFO:tasks.workunit.client.1.smithi062.stdout:1/118: creat d1/d6/de/d9/f26 x:0 0 0 2024-02-22T15:50:32.144 INFO:tasks.workunit.client.0.smithi019.stdout:7/395: dwrite d9/d3a/f75 [0,4194304] 0 2024-02-22T15:50:32.145 INFO:tasks.workunit.client.0.smithi019.stdout:0/206: sync 2024-02-22T15:50:32.146 INFO:tasks.workunit.client.1.smithi062.stdout:2/98: write d1/d8/fa [4879027,61629] 0 2024-02-22T15:50:32.146 INFO:tasks.workunit.client.0.smithi019.stdout:6/372: dwrite d6/d31/d37/f60 [0,4194304] 0 2024-02-22T15:50:32.149 INFO:tasks.workunit.client.0.smithi019.stdout:7/396: symlink d9/d3a/d49/l89 0 2024-02-22T15:50:32.149 INFO:tasks.workunit.client.0.smithi019.stdout:4/361: rmdir d2/db/d14/d23/d21/d42 39 2024-02-22T15:50:32.150 INFO:tasks.workunit.client.0.smithi019.stdout:0/207: mknod d4/d5/dc/d41/c4c 0 2024-02-22T15:50:32.150 INFO:tasks.workunit.client.0.smithi019.stdout:4/362: dread - d2/db/f69 zero size 2024-02-22T15:50:32.151 INFO:tasks.workunit.client.1.smithi062.stdout:8/66: creat d8/d9/fe x:0 0 0 2024-02-22T15:50:32.152 INFO:tasks.workunit.client.0.smithi019.stdout:6/373: unlink d6/d31/d43/d55/f5c 0 2024-02-22T15:50:32.152 INFO:tasks.workunit.client.1.smithi062.stdout:9/83: dread d4/f12 [0,4194304] 0 2024-02-22T15:50:32.154 INFO:tasks.workunit.client.1.smithi062.stdout:9/84: fsync d4/f5 0 2024-02-22T15:50:32.156 INFO:tasks.workunit.client.1.smithi062.stdout:6/109: link d5/f1e d5/d19/f2d 0 2024-02-22T15:50:32.157 INFO:tasks.workunit.client.1.smithi062.stdout:7/107: mknod d0/c22 0 2024-02-22T15:50:32.160 INFO:tasks.workunit.client.1.smithi062.stdout:0/98: dwrite d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:32.162 INFO:tasks.workunit.client.1.smithi062.stdout:0/99: readlink d1/d3/de/l11 0 2024-02-22T15:50:32.164 INFO:tasks.workunit.client.1.smithi062.stdout:8/67: creat d8/ff x:0 0 0 2024-02-22T15:50:32.168 INFO:tasks.workunit.client.1.smithi062.stdout:4/40: truncate f2 1137250 0 2024-02-22T15:50:32.168 INFO:tasks.workunit.client.1.smithi062.stdout:7/108: fsync d0/d14/f17 0 2024-02-22T15:50:32.169 INFO:tasks.workunit.client.1.smithi062.stdout:1/119: rename d1/d6/de/d9/f26 to d1/f27 0 2024-02-22T15:50:32.170 INFO:tasks.workunit.client.1.smithi062.stdout:2/99: dread d1/d8/df/f14 [0,4194304] 0 2024-02-22T15:50:32.171 INFO:tasks.workunit.client.1.smithi062.stdout:1/120: dread - d1/f27 zero size 2024-02-22T15:50:32.174 INFO:tasks.workunit.client.1.smithi062.stdout:4/41: symlink d4/la 0 2024-02-22T15:50:32.176 INFO:tasks.workunit.client.1.smithi062.stdout:4/42: write d4/f8 [419236,134] 0 2024-02-22T15:50:32.177 INFO:tasks.workunit.client.1.smithi062.stdout:1/121: creat d1/d6/d7/f28 x:0 0 0 2024-02-22T15:50:32.178 INFO:tasks.workunit.client.1.smithi062.stdout:7/109: dread d0/f16 [0,4194304] 0 2024-02-22T15:50:32.179 INFO:tasks.workunit.client.1.smithi062.stdout:8/68: truncate f0 8335026 0 2024-02-22T15:50:32.181 INFO:tasks.workunit.client.1.smithi062.stdout:4/43: symlink d4/lb 0 2024-02-22T15:50:32.182 INFO:tasks.workunit.client.1.smithi062.stdout:6/110: getdents d5/d15 0 2024-02-22T15:50:32.183 INFO:tasks.workunit.client.1.smithi062.stdout:7/110: creat d0/d6/f23 x:0 0 0 2024-02-22T15:50:32.184 INFO:tasks.workunit.client.1.smithi062.stdout:6/111: write d5/dd/f2c [608558,69833] 0 2024-02-22T15:50:32.185 INFO:tasks.workunit.client.1.smithi062.stdout:1/122: unlink d1/d6/d7/f1d 0 2024-02-22T15:50:32.187 INFO:tasks.workunit.client.1.smithi062.stdout:8/69: symlink d8/d9/l10 0 2024-02-22T15:50:32.188 INFO:tasks.workunit.client.1.smithi062.stdout:4/44: creat d4/fc x:0 0 0 2024-02-22T15:50:32.189 INFO:tasks.workunit.client.1.smithi062.stdout:6/112: creat d5/d19/f2e x:0 0 0 2024-02-22T15:50:32.192 INFO:tasks.workunit.client.1.smithi062.stdout:1/123: creat d1/d6/d7/f29 x:0 0 0 2024-02-22T15:50:32.193 INFO:tasks.workunit.client.1.smithi062.stdout:1/124: chown d1/d6/l23 3338986 1 2024-02-22T15:50:32.194 INFO:tasks.workunit.client.1.smithi062.stdout:8/70: rename c4 to d8/c11 0 2024-02-22T15:50:32.194 INFO:tasks.workunit.client.1.smithi062.stdout:8/71: read - d8/d9/fe zero size 2024-02-22T15:50:32.195 INFO:tasks.workunit.client.1.smithi062.stdout:4/45: creat d4/fd x:0 0 0 2024-02-22T15:50:32.195 INFO:tasks.workunit.client.0.smithi019.stdout:0/208: unlink d4/f27 0 2024-02-22T15:50:32.197 INFO:tasks.workunit.client.1.smithi062.stdout:8/72: write d8/ff [306850,25948] 0 2024-02-22T15:50:32.199 INFO:tasks.workunit.client.1.smithi062.stdout:6/113: getdents d5/df/d2a 0 2024-02-22T15:50:32.200 INFO:tasks.workunit.client.0.smithi019.stdout:0/209: rename d4/d5/dc/c31 to d4/db/d1b/d40/c4d 0 2024-02-22T15:50:32.201 INFO:tasks.workunit.client.1.smithi062.stdout:4/46: creat d4/fe x:0 0 0 2024-02-22T15:50:32.201 INFO:tasks.workunit.client.1.smithi062.stdout:6/114: readlink d5/df/l22 0 2024-02-22T15:50:32.202 INFO:tasks.workunit.client.0.smithi019.stdout:7/397: creat d9/f8a x:0 0 0 2024-02-22T15:50:32.202 INFO:tasks.workunit.client.1.smithi062.stdout:1/125: creat d1/d6/de/d21/d24/f2a x:0 0 0 2024-02-22T15:50:32.207 INFO:tasks.workunit.client.0.smithi019.stdout:4/363: mknod d2/db/d14/d23/d21/d35/d3d/c7b 0 2024-02-22T15:50:32.208 INFO:tasks.workunit.client.1.smithi062.stdout:4/47: unlink d4/c9 0 2024-02-22T15:50:32.208 INFO:tasks.workunit.client.1.smithi062.stdout:6/115: creat d5/dd/f2f x:0 0 0 2024-02-22T15:50:32.209 INFO:tasks.workunit.client.0.smithi019.stdout:4/364: dread - d2/db/f69 zero size 2024-02-22T15:50:32.209 INFO:tasks.workunit.client.0.smithi019.stdout:0/210: mknod d4/db/d1b/c4e 0 2024-02-22T15:50:32.209 INFO:tasks.workunit.client.0.smithi019.stdout:4/365: fsync d2/db/f26 0 2024-02-22T15:50:32.209 INFO:tasks.workunit.client.0.smithi019.stdout:0/211: chown d4/d5/dc 5 1 2024-02-22T15:50:32.210 INFO:tasks.workunit.client.1.smithi062.stdout:6/116: chown d5/dd/f2f 8033185 1 2024-02-22T15:50:32.210 INFO:tasks.workunit.client.0.smithi019.stdout:0/212: truncate d4/d9/f26 304459 0 2024-02-22T15:50:32.210 INFO:tasks.workunit.client.0.smithi019.stdout:7/398: mknod d9/d38/c8b 0 2024-02-22T15:50:32.211 INFO:tasks.workunit.client.1.smithi062.stdout:9/85: dwrite d4/f7 [0,4194304] 0 2024-02-22T15:50:32.212 INFO:tasks.workunit.client.0.smithi019.stdout:7/399: fdatasync d9/f8a 0 2024-02-22T15:50:32.212 INFO:tasks.workunit.client.1.smithi062.stdout:2/100: truncate d1/d8/f11 387184 0 2024-02-22T15:50:32.214 INFO:tasks.workunit.client.1.smithi062.stdout:0/100: dwrite d1/d3/d4/f7 [0,4194304] 0 2024-02-22T15:50:32.217 INFO:tasks.workunit.client.0.smithi019.stdout:0/213: dread d4/d9/f11 [0,4194304] 0 2024-02-22T15:50:32.224 INFO:tasks.workunit.client.0.smithi019.stdout:4/366: dwrite d2/db/d14/d23/f28 [0,4194304] 0 2024-02-22T15:50:32.226 INFO:tasks.workunit.client.1.smithi062.stdout:8/73: getdents d8/d9 0 2024-02-22T15:50:32.227 INFO:tasks.workunit.client.1.smithi062.stdout:7/111: truncate d0/f16 98470 0 2024-02-22T15:50:32.228 INFO:tasks.workunit.client.1.smithi062.stdout:0/101: unlink d1/dd/f16 0 2024-02-22T15:50:32.229 INFO:tasks.workunit.client.0.smithi019.stdout:4/367: chown d2/d3f/l47 168966145 1 2024-02-22T15:50:32.229 INFO:tasks.workunit.client.1.smithi062.stdout:6/117: truncate d5/df/f17 1166100 0 2024-02-22T15:50:32.229 INFO:tasks.workunit.client.1.smithi062.stdout:7/112: truncate d0/f5 5098618 0 2024-02-22T15:50:32.230 INFO:tasks.workunit.client.0.smithi019.stdout:6/374: dwrite d6/f17 [0,4194304] 0 2024-02-22T15:50:32.230 INFO:tasks.workunit.client.1.smithi062.stdout:1/126: write d1/d6/d14/f20 [1105952,52817] 0 2024-02-22T15:50:32.231 INFO:tasks.workunit.client.1.smithi062.stdout:6/118: write d5/dd/f2f [209428,41413] 0 2024-02-22T15:50:32.232 INFO:tasks.workunit.client.1.smithi062.stdout:7/113: dread - d0/db/f1f zero size 2024-02-22T15:50:32.235 INFO:tasks.workunit.client.1.smithi062.stdout:1/127: truncate d1/d6/de/d21/d24/f2a 499991 0 2024-02-22T15:50:32.236 INFO:tasks.workunit.client.0.smithi019.stdout:0/214: link d4/db/d1b/f1c d4/d22/d3d/f4f 0 2024-02-22T15:50:32.238 INFO:tasks.workunit.client.1.smithi062.stdout:9/86: link d4/fc d4/f1b 0 2024-02-22T15:50:32.238 INFO:tasks.workunit.client.0.smithi019.stdout:6/375: write d6/f2d [4669231,43637] 0 2024-02-22T15:50:32.240 INFO:tasks.workunit.client.1.smithi062.stdout:8/74: fdatasync f0 0 2024-02-22T15:50:32.242 INFO:tasks.workunit.client.0.smithi019.stdout:0/215: creat d4/d22/d3d/f50 x:0 0 0 2024-02-22T15:50:32.242 INFO:tasks.workunit.client.1.smithi062.stdout:1/128: truncate d1/d6/f1a 1350946 0 2024-02-22T15:50:32.244 INFO:tasks.workunit.client.1.smithi062.stdout:0/102: mkdir d1/d21 0 2024-02-22T15:50:32.244 INFO:tasks.workunit.client.1.smithi062.stdout:9/87: mkdir d4/d1c 0 2024-02-22T15:50:32.247 INFO:tasks.workunit.client.1.smithi062.stdout:8/75: unlink d8/ff 0 2024-02-22T15:50:32.247 INFO:tasks.workunit.client.0.smithi019.stdout:7/400: truncate d9/f34 508100 0 2024-02-22T15:50:32.248 INFO:tasks.workunit.client.0.smithi019.stdout:6/376: getdents d6/d31/d43 0 2024-02-22T15:50:32.253 INFO:tasks.workunit.client.1.smithi062.stdout:7/114: link d0/d6/f23 d0/f24 0 2024-02-22T15:50:32.253 INFO:tasks.workunit.client.1.smithi062.stdout:0/103: mkdir d1/d3/d4/d1a/d1d/d1f/d22 0 2024-02-22T15:50:32.254 INFO:tasks.workunit.client.0.smithi019.stdout:4/368: dread d2/db/d14/d23/f65 [0,4194304] 0 2024-02-22T15:50:32.256 INFO:tasks.workunit.client.0.smithi019.stdout:4/369: creat d2/d3f/f7c x:0 0 0 2024-02-22T15:50:32.257 INFO:tasks.workunit.client.1.smithi062.stdout:7/115: dread - d0/f24 zero size 2024-02-22T15:50:32.262 INFO:tasks.workunit.client.0.smithi019.stdout:7/401: truncate d9/d26/f32 2905725 0 2024-02-22T15:50:32.262 INFO:tasks.workunit.client.0.smithi019.stdout:4/370: creat d2/db/d14/d23/d33/f7d x:0 0 0 2024-02-22T15:50:32.263 INFO:tasks.workunit.client.0.smithi019.stdout:4/371: stat d2/d3f/d6e 0 2024-02-22T15:50:32.263 INFO:tasks.workunit.client.0.smithi019.stdout:7/402: write d9/d3a/f86 [215228,95445] 0 2024-02-22T15:50:32.267 INFO:tasks.workunit.client.1.smithi062.stdout:9/88: dwrite d4/fc [0,4194304] 0 2024-02-22T15:50:32.268 INFO:tasks.workunit.client.1.smithi062.stdout:1/129: dwrite d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:32.270 INFO:tasks.workunit.client.1.smithi062.stdout:8/76: dwrite d8/d9/fe [0,4194304] 0 2024-02-22T15:50:32.270 INFO:tasks.workunit.client.1.smithi062.stdout:4/48: dread f2 [0,4194304] 0 2024-02-22T15:50:32.273 INFO:tasks.workunit.client.0.smithi019.stdout:7/403: dwrite d9/f8a [0,4194304] 0 2024-02-22T15:50:32.274 INFO:tasks.workunit.client.1.smithi062.stdout:4/49: chown d4 2968000 1 2024-02-22T15:50:32.275 INFO:tasks.workunit.client.1.smithi062.stdout:4/50: dread - d4/fe zero size 2024-02-22T15:50:32.275 INFO:tasks.workunit.client.1.smithi062.stdout:9/89: rmdir d4 39 2024-02-22T15:50:32.280 INFO:tasks.workunit.client.1.smithi062.stdout:1/130: mkdir d1/d6/d7/d2b 0 2024-02-22T15:50:32.283 INFO:tasks.workunit.client.1.smithi062.stdout:7/116: write d0/d14/d13/fc [2262723,65194] 0 2024-02-22T15:50:32.283 INFO:tasks.workunit.client.1.smithi062.stdout:9/90: rmdir d4 39 2024-02-22T15:50:32.286 INFO:tasks.workunit.client.1.smithi062.stdout:9/91: stat d4/f5 0 2024-02-22T15:50:32.287 INFO:tasks.workunit.client.1.smithi062.stdout:7/117: creat d0/d14/f25 x:0 0 0 2024-02-22T15:50:32.288 INFO:tasks.workunit.client.0.smithi019.stdout:7/404: dwrite d9/d38/d41/d45/d61/d6a/f6d [0,4194304] 0 2024-02-22T15:50:32.292 INFO:tasks.workunit.client.0.smithi019.stdout:7/405: write d9/f24 [1009411,72409] 0 2024-02-22T15:50:32.292 INFO:tasks.workunit.client.1.smithi062.stdout:6/119: dwrite f2 [0,4194304] 0 2024-02-22T15:50:32.293 INFO:tasks.workunit.client.1.smithi062.stdout:4/51: dwrite d4/fe [0,4194304] 0 2024-02-22T15:50:32.294 INFO:tasks.workunit.client.1.smithi062.stdout:6/120: fsync f0 0 2024-02-22T15:50:32.295 INFO:tasks.workunit.client.1.smithi062.stdout:6/121: stat d5/f7 0 2024-02-22T15:50:32.295 INFO:tasks.workunit.client.1.smithi062.stdout:5/55: sync 2024-02-22T15:50:32.298 INFO:tasks.workunit.client.0.smithi019.stdout:7/406: dread d9/f8a [0,4194304] 0 2024-02-22T15:50:32.301 INFO:tasks.workunit.client.1.smithi062.stdout:9/92: creat d4/d1c/f1d x:0 0 0 2024-02-22T15:50:32.301 INFO:tasks.workunit.client.0.smithi019.stdout:7/407: getdents d9/d26/d77 0 2024-02-22T15:50:32.304 INFO:tasks.workunit.client.0.smithi019.stdout:7/408: creat d9/d38/d41/d45/f8c x:0 0 0 2024-02-22T15:50:32.304 INFO:tasks.workunit.client.1.smithi062.stdout:4/52: mkdir d4/df 0 2024-02-22T15:50:32.304 INFO:tasks.workunit.client.1.smithi062.stdout:7/118: dread d0/f4 [0,4194304] 0 2024-02-22T15:50:32.305 INFO:tasks.workunit.client.1.smithi062.stdout:9/93: stat d4/fc 0 2024-02-22T15:50:32.307 INFO:tasks.workunit.client.1.smithi062.stdout:9/94: chown d4/fe 1252 1 2024-02-22T15:50:32.308 INFO:tasks.workunit.client.1.smithi062.stdout:6/122: creat d5/df/f30 x:0 0 0 2024-02-22T15:50:32.309 INFO:tasks.workunit.client.1.smithi062.stdout:7/119: symlink d0/db/l26 0 2024-02-22T15:50:32.309 INFO:tasks.workunit.client.1.smithi062.stdout:8/77: write f0 [8555571,70833] 0 2024-02-22T15:50:32.310 INFO:tasks.workunit.client.1.smithi062.stdout:4/53: truncate d4/fd 116066 0 2024-02-22T15:50:32.311 INFO:tasks.workunit.client.1.smithi062.stdout:8/78: readlink l5 0 2024-02-22T15:50:32.313 INFO:tasks.workunit.client.1.smithi062.stdout:6/123: chown c4 249842723 1 2024-02-22T15:50:32.314 INFO:tasks.workunit.client.1.smithi062.stdout:4/54: write d4/fd [1026942,109610] 0 2024-02-22T15:50:32.315 INFO:tasks.workunit.client.1.smithi062.stdout:5/56: dwrite d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:32.316 INFO:tasks.workunit.client.1.smithi062.stdout:6/124: fsync d5/dd/f20 0 2024-02-22T15:50:32.316 INFO:tasks.workunit.client.1.smithi062.stdout:4/55: stat d4/lb 0 2024-02-22T15:50:32.320 INFO:tasks.workunit.client.1.smithi062.stdout:7/120: rename d0/d14/d19/c1c to d0/d14/d13/c27 0 2024-02-22T15:50:32.320 INFO:tasks.workunit.client.1.smithi062.stdout:5/57: write d0/f2 [3664952,71765] 0 2024-02-22T15:50:32.324 INFO:tasks.workunit.client.1.smithi062.stdout:4/56: rename d4/l6 to d4/l10 0 2024-02-22T15:50:32.325 INFO:tasks.workunit.client.1.smithi062.stdout:8/79: dwrite f0 [0,4194304] 0 2024-02-22T15:50:32.325 INFO:tasks.workunit.client.1.smithi062.stdout:5/58: write d0/d3/da/fb [1648204,15964] 0 2024-02-22T15:50:32.329 INFO:tasks.workunit.client.1.smithi062.stdout:4/57: truncate d4/fc 82961 0 2024-02-22T15:50:32.330 INFO:tasks.workunit.client.0.smithi019.stdout:7/409: write d9/d38/f3e [373236,66403] 0 2024-02-22T15:50:32.332 INFO:tasks.workunit.client.1.smithi062.stdout:7/121: fsync d0/f24 0 2024-02-22T15:50:32.334 INFO:tasks.workunit.client.0.smithi019.stdout:7/410: rename d9/d38/d41/c51 to d9/d3a/c8d 0 2024-02-22T15:50:32.334 INFO:tasks.workunit.client.1.smithi062.stdout:6/125: getdents d5/d19 0 2024-02-22T15:50:32.335 INFO:tasks.workunit.client.0.smithi019.stdout:7/411: write d9/f8a [3249778,70550] 0 2024-02-22T15:50:32.336 INFO:tasks.workunit.client.0.smithi019.stdout:7/412: write d9/f8a [1703098,39789] 0 2024-02-22T15:50:32.338 INFO:tasks.workunit.client.0.smithi019.stdout:7/413: creat d9/d26/d77/f8e x:0 0 0 2024-02-22T15:50:32.341 INFO:tasks.workunit.client.0.smithi019.stdout:7/414: unlink d9/d3a/c3f 0 2024-02-22T15:50:32.342 INFO:tasks.workunit.client.1.smithi062.stdout:7/122: getdents d0/db 0 2024-02-22T15:50:32.343 INFO:tasks.workunit.client.1.smithi062.stdout:6/126: dread d5/df/f26 [0,4194304] 0 2024-02-22T15:50:32.344 INFO:tasks.workunit.client.0.smithi019.stdout:7/415: fdatasync d9/d26/f29 0 2024-02-22T15:50:32.345 INFO:tasks.workunit.client.1.smithi062.stdout:8/80: dwrite f0 [4194304,4194304] 0 2024-02-22T15:50:32.345 INFO:tasks.workunit.client.1.smithi062.stdout:7/123: chown d0/d14/f25 398151173 1 2024-02-22T15:50:32.346 INFO:tasks.workunit.client.1.smithi062.stdout:5/59: dwrite d0/d3/da/fb [0,4194304] 0 2024-02-22T15:50:32.346 INFO:tasks.workunit.client.0.smithi019.stdout:7/416: mkdir d9/d3a/d49/d5d/d8f 0 2024-02-22T15:50:32.349 INFO:tasks.workunit.client.0.smithi019.stdout:7/417: rename d9/d38/d41/f5f to d9/d38/d41/f90 0 2024-02-22T15:50:32.352 INFO:tasks.workunit.client.0.smithi019.stdout:7/418: symlink d9/d4c/l91 0 2024-02-22T15:50:32.356 INFO:tasks.workunit.client.1.smithi062.stdout:6/127: write d5/df/f21 [1221042,89173] 0 2024-02-22T15:50:32.356 INFO:tasks.workunit.client.1.smithi062.stdout:7/124: dread d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:32.361 INFO:tasks.workunit.client.1.smithi062.stdout:6/128: write d5/dd/f2f [1213604,58611] 0 2024-02-22T15:50:32.361 INFO:tasks.workunit.client.1.smithi062.stdout:7/125: mkdir d0/d14/d28 0 2024-02-22T15:50:32.363 INFO:tasks.workunit.client.1.smithi062.stdout:8/81: dwrite f0 [8388608,4194304] 0 2024-02-22T15:50:32.363 INFO:tasks.workunit.client.1.smithi062.stdout:5/60: dwrite d0/d3/ff [0,4194304] 0 2024-02-22T15:50:32.366 INFO:tasks.workunit.client.0.smithi019.stdout:7/419: getdents d9/d3a/d49/d5d 0 2024-02-22T15:50:32.367 INFO:tasks.workunit.client.0.smithi019.stdout:7/420: dread - d9/d3a/d49/d5d/d74/f88 zero size 2024-02-22T15:50:32.367 INFO:tasks.workunit.client.0.smithi019.stdout:7/421: chown d9/d3a/d49/d5d/d8f 42226 1 2024-02-22T15:50:32.370 INFO:tasks.workunit.client.0.smithi019.stdout:7/422: fdatasync d9/f1d 0 2024-02-22T15:50:32.372 INFO:tasks.workunit.client.0.smithi019.stdout:7/423: chown d9/d38/d41/f78 0 1 2024-02-22T15:50:32.373 INFO:tasks.workunit.client.0.smithi019.stdout:1/442: sync 2024-02-22T15:50:32.374 INFO:tasks.workunit.client.0.smithi019.stdout:5/387: sync 2024-02-22T15:50:32.374 INFO:tasks.workunit.client.0.smithi019.stdout:8/329: sync 2024-02-22T15:50:32.375 INFO:tasks.workunit.client.0.smithi019.stdout:7/424: mkdir d9/d38/d41/d45/d92 0 2024-02-22T15:50:32.377 INFO:tasks.workunit.client.0.smithi019.stdout:1/443: mkdir d8/d2a/d75/da0 0 2024-02-22T15:50:32.380 INFO:tasks.workunit.client.0.smithi019.stdout:8/330: mkdir d9/d14/d1f/d6d 0 2024-02-22T15:50:32.382 INFO:tasks.workunit.client.0.smithi019.stdout:8/331: write d9/d14/d17/f25 [3206245,126333] 0 2024-02-22T15:50:32.383 INFO:tasks.workunit.client.0.smithi019.stdout:7/425: dwrite f6 [0,4194304] 0 2024-02-22T15:50:32.385 INFO:tasks.workunit.client.0.smithi019.stdout:8/332: dread - d9/d14/d17/f28 zero size 2024-02-22T15:50:32.390 INFO:tasks.workunit.client.0.smithi019.stdout:8/333: mknod d9/d14/c6e 0 2024-02-22T15:50:32.392 INFO:tasks.workunit.client.0.smithi019.stdout:8/334: chown d9/fc 188124250 1 2024-02-22T15:50:32.393 INFO:tasks.workunit.client.0.smithi019.stdout:5/388: dwrite d8/f2d [0,4194304] 0 2024-02-22T15:50:32.399 INFO:tasks.workunit.client.0.smithi019.stdout:5/389: rename d8/d11/d33/l70 to d8/dd/d19/d77/l80 0 2024-02-22T15:50:32.402 INFO:tasks.workunit.client.0.smithi019.stdout:7/426: dread d9/f16 [0,4194304] 0 2024-02-22T15:50:32.403 INFO:tasks.workunit.client.0.smithi019.stdout:5/390: creat d8/dd/d19/f81 x:0 0 0 2024-02-22T15:50:32.405 INFO:tasks.workunit.client.0.smithi019.stdout:8/335: dwrite d9/d14/d17/f25 [0,4194304] 0 2024-02-22T15:50:32.405 INFO:tasks.workunit.client.0.smithi019.stdout:5/391: stat d8/l10 0 2024-02-22T15:50:32.405 INFO:tasks.workunit.client.0.smithi019.stdout:1/444: dread d8/d15/f63 [0,4194304] 0 2024-02-22T15:50:32.409 INFO:tasks.workunit.client.0.smithi019.stdout:8/336: mkdir d9/d14/d6f 0 2024-02-22T15:50:32.412 INFO:tasks.workunit.client.0.smithi019.stdout:8/337: write d9/f66 [1025604,67612] 0 2024-02-22T15:50:32.414 INFO:tasks.workunit.client.0.smithi019.stdout:1/445: symlink d8/d44/d5d/la1 0 2024-02-22T15:50:32.414 INFO:tasks.workunit.client.0.smithi019.stdout:7/427: dwrite d9/f16 [0,4194304] 0 2024-02-22T15:50:32.417 INFO:tasks.workunit.client.0.smithi019.stdout:8/338: unlink d9/f58 0 2024-02-22T15:50:32.418 INFO:tasks.workunit.client.0.smithi019.stdout:1/446: creat d8/d18/d64/d7f/fa2 x:0 0 0 2024-02-22T15:50:32.420 INFO:tasks.workunit.client.0.smithi019.stdout:7/428: creat d9/d38/d41/d45/d61/d6a/f93 x:0 0 0 2024-02-22T15:50:32.423 INFO:tasks.workunit.client.0.smithi019.stdout:5/392: dwrite d8/f28 [0,4194304] 0 2024-02-22T15:50:32.425 INFO:tasks.workunit.client.0.smithi019.stdout:1/447: creat d8/d15/d34/fa3 x:0 0 0 2024-02-22T15:50:32.425 INFO:tasks.workunit.client.0.smithi019.stdout:7/429: symlink d9/d38/d41/d45/l94 0 2024-02-22T15:50:32.427 INFO:tasks.workunit.client.0.smithi019.stdout:5/393: symlink d8/dd/d19/d77/l82 0 2024-02-22T15:50:32.429 INFO:tasks.workunit.client.0.smithi019.stdout:5/394: truncate d8/d11/d33/f3c 4226233 0 2024-02-22T15:50:32.433 INFO:tasks.workunit.client.1.smithi062.stdout:8/82: creat d8/f12 x:0 0 0 2024-02-22T15:50:32.433 INFO:tasks.workunit.client.1.smithi062.stdout:8/83: chown f0 24162212 1 2024-02-22T15:50:32.434 INFO:tasks.workunit.client.1.smithi062.stdout:6/129: truncate d5/dd/fe 809521 0 2024-02-22T15:50:32.434 INFO:tasks.workunit.client.0.smithi019.stdout:5/395: readlink d8/dd/d19/d38/d54/l6c 0 2024-02-22T15:50:32.438 INFO:tasks.workunit.client.1.smithi062.stdout:5/61: rename d0/d3/da/f12 to d0/d3/f14 0 2024-02-22T15:50:32.438 INFO:tasks.workunit.client.0.smithi019.stdout:5/396: creat d8/dd/d19/d3a/d46/d7e/f83 x:0 0 0 2024-02-22T15:50:32.438 INFO:tasks.workunit.client.0.smithi019.stdout:7/430: stat d9/d26/f32 0 2024-02-22T15:50:32.439 INFO:tasks.workunit.client.1.smithi062.stdout:6/130: dread - d5/dd/f20 zero size 2024-02-22T15:50:32.439 INFO:tasks.workunit.client.1.smithi062.stdout:7/126: getdents d0/d14/d13 0 2024-02-22T15:50:32.440 INFO:tasks.workunit.client.1.smithi062.stdout:6/131: dread - d5/dd/f20 zero size 2024-02-22T15:50:32.440 INFO:tasks.workunit.client.1.smithi062.stdout:7/127: chown d0/d14/d19 1082376 1 2024-02-22T15:50:32.441 INFO:tasks.workunit.client.0.smithi019.stdout:5/397: write d8/dd/d19/f81 [668642,122795] 0 2024-02-22T15:50:32.441 INFO:tasks.workunit.client.0.smithi019.stdout:1/448: unlink d8/d2a/f2c 0 2024-02-22T15:50:32.441 INFO:tasks.workunit.client.1.smithi062.stdout:7/128: chown d0/f5 1 1 2024-02-22T15:50:32.443 INFO:tasks.workunit.client.0.smithi019.stdout:8/339: truncate d9/d14/d1f/d20/f61 377566 0 2024-02-22T15:50:32.445 INFO:tasks.workunit.client.1.smithi062.stdout:8/84: dread f0 [4194304,4194304] 0 2024-02-22T15:50:32.446 INFO:tasks.workunit.client.1.smithi062.stdout:8/85: symlink d8/d9/l13 0 2024-02-22T15:50:32.447 INFO:tasks.workunit.client.0.smithi019.stdout:5/398: link d8/d11/c1d d8/dd/d19/d77/c84 0 2024-02-22T15:50:32.449 INFO:tasks.workunit.client.0.smithi019.stdout:8/340: creat d9/d14/d17/f70 x:0 0 0 2024-02-22T15:50:32.449 INFO:tasks.workunit.client.1.smithi062.stdout:8/86: fdatasync d8/f12 0 2024-02-22T15:50:32.450 INFO:tasks.workunit.client.0.smithi019.stdout:5/399: truncate d8/dd/d19/d3a/f5e 249564 0 2024-02-22T15:50:32.450 INFO:tasks.workunit.client.1.smithi062.stdout:5/62: write d0/f9 [867856,33192] 0 2024-02-22T15:50:32.451 INFO:tasks.workunit.client.1.smithi062.stdout:6/132: dread d5/df/f21 [0,4194304] 0 2024-02-22T15:50:32.452 INFO:tasks.workunit.client.0.smithi019.stdout:5/400: write d8/fe [4312757,98360] 0 2024-02-22T15:50:32.453 INFO:tasks.workunit.client.1.smithi062.stdout:8/87: mknod d8/c14 0 2024-02-22T15:50:32.455 INFO:tasks.workunit.client.1.smithi062.stdout:7/129: dwrite d0/d14/d19/f1a [0,4194304] 0 2024-02-22T15:50:32.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:32 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1555967477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:50:32.457 INFO:tasks.workunit.client.1.smithi062.stdout:5/63: symlink d0/d3/da/d13/l15 0 2024-02-22T15:50:32.458 INFO:tasks.workunit.client.1.smithi062.stdout:5/64: stat d0/f9 0 2024-02-22T15:50:32.458 INFO:tasks.workunit.client.1.smithi062.stdout:6/133: rename d5/dd/f2f to d5/df/d2a/f31 0 2024-02-22T15:50:32.459 INFO:tasks.workunit.client.1.smithi062.stdout:7/130: creat d0/d14/f29 x:0 0 0 2024-02-22T15:50:32.460 INFO:tasks.workunit.client.1.smithi062.stdout:6/134: read d5/df/f17 [1063742,74920] 0 2024-02-22T15:50:32.462 INFO:tasks.workunit.client.1.smithi062.stdout:8/88: rename d8/d9/l13 to d8/d9/l15 0 2024-02-22T15:50:32.463 INFO:tasks.workunit.client.1.smithi062.stdout:5/65: unlink d0/f2 0 2024-02-22T15:50:32.464 INFO:tasks.workunit.client.1.smithi062.stdout:3/76: sync 2024-02-22T15:50:32.465 INFO:tasks.workunit.client.1.smithi062.stdout:7/131: creat d0/d14/d13/f2a x:0 0 0 2024-02-22T15:50:32.468 INFO:tasks.workunit.client.1.smithi062.stdout:8/89: write f0 [10771698,23489] 0 2024-02-22T15:50:32.471 INFO:tasks.workunit.client.1.smithi062.stdout:8/90: write d8/d9/fe [1479193,124657] 0 2024-02-22T15:50:32.473 INFO:tasks.workunit.client.1.smithi062.stdout:3/77: getdents d5 0 2024-02-22T15:50:32.474 INFO:tasks.workunit.client.1.smithi062.stdout:6/135: dwrite d5/dd/f27 [0,4194304] 0 2024-02-22T15:50:32.475 INFO:tasks.workunit.client.0.smithi019.stdout:8/341: dread d9/d14/d17/d1c/f24 [0,4194304] 0 2024-02-22T15:50:32.476 INFO:tasks.workunit.client.1.smithi062.stdout:8/91: write f0 [8123705,68552] 0 2024-02-22T15:50:32.477 INFO:tasks.workunit.client.0.smithi019.stdout:7/431: dread d9/d3a/f60 [0,4194304] 0 2024-02-22T15:50:32.477 INFO:tasks.workunit.client.0.smithi019.stdout:8/342: fdatasync d9/d14/f19 0 2024-02-22T15:50:32.478 INFO:tasks.workunit.client.0.smithi019.stdout:7/432: chown d9/d4c/d5b/f64 30 1 2024-02-22T15:50:32.479 INFO:tasks.workunit.client.1.smithi062.stdout:5/66: link d0/f7 d0/f16 0 2024-02-22T15:50:32.483 INFO:tasks.workunit.client.1.smithi062.stdout:8/92: creat d8/d9/f16 x:0 0 0 2024-02-22T15:50:32.484 INFO:tasks.workunit.client.1.smithi062.stdout:3/78: creat d5/d16/f19 x:0 0 0 2024-02-22T15:50:32.484 INFO:tasks.workunit.client.1.smithi062.stdout:6/136: dread d5/f7 [0,4194304] 0 2024-02-22T15:50:32.485 INFO:tasks.workunit.client.1.smithi062.stdout:6/137: fsync f2 0 2024-02-22T15:50:32.488 INFO:tasks.workunit.client.1.smithi062.stdout:7/132: rmdir d0/d14/d13 39 2024-02-22T15:50:32.491 INFO:tasks.workunit.client.1.smithi062.stdout:7/133: chown d0/l7 1135 1 2024-02-22T15:50:32.491 INFO:tasks.workunit.client.0.smithi019.stdout:7/433: creat d9/d26/d77/f95 x:0 0 0 2024-02-22T15:50:32.493 INFO:tasks.workunit.client.1.smithi062.stdout:3/79: dread f3 [0,4194304] 0 2024-02-22T15:50:32.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:32 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1555967477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:50:32.495 INFO:tasks.workunit.client.1.smithi062.stdout:6/138: mkdir d5/df/d32 0 2024-02-22T15:50:32.496 INFO:tasks.workunit.client.1.smithi062.stdout:3/80: fsync f4 0 2024-02-22T15:50:32.499 INFO:tasks.workunit.client.1.smithi062.stdout:8/93: dwrite d8/d9/f16 [0,4194304] 0 2024-02-22T15:50:32.499 INFO:tasks.workunit.client.1.smithi062.stdout:6/139: rmdir d5/dd 39 2024-02-22T15:50:32.500 INFO:tasks.workunit.client.0.smithi019.stdout:8/343: link d9/d14/d1f/d20/l2c d9/d14/d1f/d4c/l71 0 2024-02-22T15:50:32.501 INFO:tasks.workunit.client.0.smithi019.stdout:1/449: truncate d8/d18/f27 3598351 0 2024-02-22T15:50:32.502 INFO:tasks.workunit.client.0.smithi019.stdout:2/315: sync 2024-02-22T15:50:32.502 INFO:tasks.workunit.client.0.smithi019.stdout:9/352: sync 2024-02-22T15:50:32.502 INFO:tasks.workunit.client.0.smithi019.stdout:3/309: sync 2024-02-22T15:50:32.502 INFO:tasks.workunit.client.0.smithi019.stdout:0/216: sync 2024-02-22T15:50:32.502 INFO:tasks.workunit.client.1.smithi062.stdout:7/134: mknod d0/d14/c2b 0 2024-02-22T15:50:32.503 INFO:tasks.workunit.client.1.smithi062.stdout:3/81: mkdir d5/d1a 0 2024-02-22T15:50:32.503 INFO:tasks.workunit.client.1.smithi062.stdout:7/135: fsync d0/d14/f29 0 2024-02-22T15:50:32.510 INFO:tasks.workunit.client.1.smithi062.stdout:8/94: dwrite d8/d9/f16 [0,4194304] 0 2024-02-22T15:50:32.511 INFO:tasks.workunit.client.0.smithi019.stdout:0/217: creat d4/d22/f51 x:0 0 0 2024-02-22T15:50:32.512 INFO:tasks.workunit.client.1.smithi062.stdout:3/82: creat d5/d1a/f1b x:0 0 0 2024-02-22T15:50:32.513 INFO:tasks.workunit.client.0.smithi019.stdout:3/310: symlink d9/d22/d58/l61 0 2024-02-22T15:50:32.514 INFO:tasks.workunit.client.1.smithi062.stdout:3/83: truncate d5/f13 891178 0 2024-02-22T15:50:32.515 INFO:tasks.workunit.client.0.smithi019.stdout:9/353: symlink d2/d1f/d2e/l7f 0 2024-02-22T15:50:32.517 INFO:tasks.workunit.client.1.smithi062.stdout:7/136: rename d0/f4 to d0/d14/f2c 0 2024-02-22T15:50:32.522 INFO:tasks.workunit.client.1.smithi062.stdout:6/140: truncate d5/d19/f1c 268816 0 2024-02-22T15:50:32.522 INFO:tasks.workunit.client.0.smithi019.stdout:1/450: dread d8/d44/f45 [0,4194304] 0 2024-02-22T15:50:32.523 INFO:tasks.workunit.client.0.smithi019.stdout:0/218: dwrite d4/d23/f28 [0,4194304] 0 2024-02-22T15:50:32.524 INFO:tasks.workunit.client.0.smithi019.stdout:0/219: readlink d4/l12 0 2024-02-22T15:50:32.525 INFO:tasks.workunit.client.1.smithi062.stdout:3/84: link d5/c14 d5/d1a/c1c 0 2024-02-22T15:50:32.526 INFO:tasks.workunit.client.1.smithi062.stdout:3/85: chown d5/cf 54654484 1 2024-02-22T15:50:32.526 INFO:tasks.workunit.client.1.smithi062.stdout:6/141: readlink d5/d15/l25 0 2024-02-22T15:50:32.527 INFO:tasks.workunit.client.1.smithi062.stdout:6/142: chown c4 1 1 2024-02-22T15:50:32.527 INFO:tasks.workunit.client.1.smithi062.stdout:7/137: rename d0/f16 to d0/db/f2d 0 2024-02-22T15:50:32.529 INFO:tasks.workunit.client.1.smithi062.stdout:6/143: read - d5/dd/f20 zero size 2024-02-22T15:50:32.531 INFO:tasks.workunit.client.0.smithi019.stdout:9/354: symlink d2/d7/dc/d27/d31/d79/l80 0 2024-02-22T15:50:32.531 INFO:tasks.workunit.client.0.smithi019.stdout:1/451: rmdir d8/d18/d64 39 2024-02-22T15:50:32.532 INFO:tasks.workunit.client.1.smithi062.stdout:7/138: truncate d0/d14/d13/f2a 208739 0 2024-02-22T15:50:32.532 INFO:tasks.workunit.client.0.smithi019.stdout:1/452: readlink d8/d15/l73 0 2024-02-22T15:50:32.532 INFO:tasks.workunit.client.1.smithi062.stdout:8/95: truncate d8/d9/fe 2667786 0 2024-02-22T15:50:32.533 INFO:tasks.workunit.client.1.smithi062.stdout:7/139: fdatasync d0/f5 0 2024-02-22T15:50:32.533 INFO:tasks.workunit.client.1.smithi062.stdout:6/144: rmdir d5/df/d2a 39 2024-02-22T15:50:32.533 INFO:tasks.workunit.client.0.smithi019.stdout:9/355: mknod d2/d7/d14/c81 0 2024-02-22T15:50:32.534 INFO:tasks.workunit.client.1.smithi062.stdout:7/140: fsync d0/f10 0 2024-02-22T15:50:32.534 INFO:tasks.workunit.client.1.smithi062.stdout:8/96: chown d8/d9/l10 7562 1 2024-02-22T15:50:32.535 INFO:tasks.workunit.client.1.smithi062.stdout:6/145: fsync d5/df/f17 0 2024-02-22T15:50:32.536 INFO:tasks.workunit.client.1.smithi062.stdout:8/97: creat d8/f17 x:0 0 0 2024-02-22T15:50:32.537 INFO:tasks.workunit.client.1.smithi062.stdout:7/141: mkdir d0/d6/d2e 0 2024-02-22T15:50:32.539 INFO:tasks.workunit.client.1.smithi062.stdout:6/146: mkdir d5/dd/d33 0 2024-02-22T15:50:32.539 INFO:tasks.workunit.client.1.smithi062.stdout:6/147: readlink d5/l8 0 2024-02-22T15:50:32.540 INFO:tasks.workunit.client.1.smithi062.stdout:8/98: creat d8/d9/f18 x:0 0 0 2024-02-22T15:50:32.541 INFO:tasks.workunit.client.1.smithi062.stdout:6/148: write d5/d19/f2e [632877,97803] 0 2024-02-22T15:50:32.541 INFO:tasks.workunit.client.1.smithi062.stdout:6/149: fdatasync f0 0 2024-02-22T15:50:32.542 INFO:tasks.workunit.client.1.smithi062.stdout:8/99: mknod d8/c19 0 2024-02-22T15:50:32.543 INFO:tasks.workunit.client.0.smithi019.stdout:0/220: mkdir d4/d5/d52 0 2024-02-22T15:50:32.544 INFO:tasks.workunit.client.1.smithi062.stdout:6/150: write d5/dd/f27 [3379349,60543] 0 2024-02-22T15:50:32.544 INFO:tasks.workunit.client.1.smithi062.stdout:7/142: dread d0/f5 [0,4194304] 0 2024-02-22T15:50:32.546 INFO:tasks.workunit.client.0.smithi019.stdout:0/221: symlink d4/d5/l53 0 2024-02-22T15:50:32.546 INFO:tasks.workunit.client.1.smithi062.stdout:6/151: write d5/df/f30 [969843,14541] 0 2024-02-22T15:50:32.546 INFO:tasks.workunit.client.1.smithi062.stdout:7/143: fdatasync d0/d14/f20 0 2024-02-22T15:50:32.546 INFO:tasks.workunit.client.1.smithi062.stdout:8/100: mknod d8/d9/c1a 0 2024-02-22T15:50:32.547 INFO:tasks.workunit.client.0.smithi019.stdout:2/316: rename d1/d17/d19 to d1/d17/d6d 0 2024-02-22T15:50:32.547 INFO:tasks.workunit.client.1.smithi062.stdout:8/101: chown f0 28 1 2024-02-22T15:50:32.548 INFO:tasks.workunit.client.1.smithi062.stdout:7/144: fsync d0/d14/f29 0 2024-02-22T15:50:32.549 INFO:tasks.workunit.client.0.smithi019.stdout:2/317: read d1/d46/f55 [266142,122883] 0 2024-02-22T15:50:32.549 INFO:tasks.workunit.client.0.smithi019.stdout:0/222: rename d4/d5/dc/l45 to d4/d5/d7/l54 0 2024-02-22T15:50:32.552 INFO:tasks.workunit.client.0.smithi019.stdout:3/311: dread f4 [0,4194304] 0 2024-02-22T15:50:32.552 INFO:tasks.workunit.client.0.smithi019.stdout:2/318: readlink d1/de/l13 0 2024-02-22T15:50:32.553 INFO:tasks.workunit.client.0.smithi019.stdout:2/319: chown d1/d17/d1c/f34 782689168 1 2024-02-22T15:50:32.554 INFO:tasks.workunit.client.0.smithi019.stdout:0/223: creat d4/db/d1b/f55 x:0 0 0 2024-02-22T15:50:32.554 INFO:tasks.workunit.client.1.smithi062.stdout:8/102: dread f0 [0,4194304] 0 2024-02-22T15:50:32.555 INFO:tasks.workunit.client.1.smithi062.stdout:6/152: rename d5/d19/f2e to d5/df/d32/f34 0 2024-02-22T15:50:32.557 INFO:tasks.workunit.client.0.smithi019.stdout:2/320: creat d1/d17/d21/d51/f6e x:0 0 0 2024-02-22T15:50:32.559 INFO:tasks.workunit.client.1.smithi062.stdout:7/145: mkdir d0/d6/d2e/d2f 0 2024-02-22T15:50:32.561 INFO:tasks.workunit.client.0.smithi019.stdout:3/312: mknod d9/d11/c62 0 2024-02-22T15:50:32.563 INFO:tasks.workunit.client.1.smithi062.stdout:7/146: fsync d0/f24 0 2024-02-22T15:50:32.563 INFO:tasks.workunit.client.1.smithi062.stdout:6/153: dread d5/f1f [0,4194304] 0 2024-02-22T15:50:32.565 INFO:tasks.workunit.client.1.smithi062.stdout:6/154: chown d5/df/f26 42095 1 2024-02-22T15:50:32.565 INFO:tasks.workunit.client.0.smithi019.stdout:2/321: unlink d1/d46/c62 0 2024-02-22T15:50:32.566 INFO:tasks.workunit.client.1.smithi062.stdout:6/155: chown d5/l8 160 1 2024-02-22T15:50:32.566 INFO:tasks.workunit.client.0.smithi019.stdout:7/434: truncate f6 3699304 0 2024-02-22T15:50:32.566 INFO:tasks.workunit.client.1.smithi062.stdout:6/156: stat d5/df/f30 0 2024-02-22T15:50:32.566 INFO:tasks.workunit.client.1.smithi062.stdout:7/147: mkdir d0/d14/d19/d30 0 2024-02-22T15:50:32.567 INFO:tasks.workunit.client.1.smithi062.stdout:6/157: chown d5/d15/l16 320 1 2024-02-22T15:50:32.568 INFO:tasks.workunit.client.1.smithi062.stdout:7/148: symlink d0/d14/d19/l31 0 2024-02-22T15:50:32.570 INFO:tasks.workunit.client.0.smithi019.stdout:7/435: unlink d9/d38/l55 0 2024-02-22T15:50:32.571 INFO:tasks.workunit.client.0.smithi019.stdout:7/436: readlink d9/l20 0 2024-02-22T15:50:32.572 INFO:tasks.workunit.client.0.smithi019.stdout:7/437: chown d9/d38/d41/d45/d61/d6a/f6d 2985703 1 2024-02-22T15:50:32.573 INFO:tasks.workunit.client.1.smithi062.stdout:7/149: dread d0/d14/d19/f1a [0,4194304] 0 2024-02-22T15:50:32.575 INFO:tasks.workunit.client.1.smithi062.stdout:7/150: unlink d0/db/l26 0 2024-02-22T15:50:32.575 INFO:tasks.workunit.client.1.smithi062.stdout:2/101: sync 2024-02-22T15:50:32.575 INFO:tasks.workunit.client.1.smithi062.stdout:0/104: sync 2024-02-22T15:50:32.578 INFO:tasks.workunit.client.0.smithi019.stdout:1/453: truncate d8/d44/f4a 3870638 0 2024-02-22T15:50:32.578 INFO:tasks.workunit.client.0.smithi019.stdout:7/438: dwrite d9/d38/f52 [0,4194304] 0 2024-02-22T15:50:32.579 INFO:tasks.workunit.client.1.smithi062.stdout:7/151: creat d0/d14/d1e/f32 x:0 0 0 2024-02-22T15:50:32.579 INFO:tasks.workunit.client.1.smithi062.stdout:2/102: mkdir d1/d1e/d20 0 2024-02-22T15:50:32.581 INFO:tasks.workunit.client.0.smithi019.stdout:2/322: unlink d1/lc 0 2024-02-22T15:50:32.582 INFO:tasks.workunit.client.1.smithi062.stdout:7/152: mknod d0/d6/d2e/d2f/c33 0 2024-02-22T15:50:32.582 INFO:tasks.workunit.client.1.smithi062.stdout:0/105: dread d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:32.582 INFO:tasks.workunit.client.1.smithi062.stdout:2/103: truncate d1/d8/f15 1020382 0 2024-02-22T15:50:32.583 INFO:tasks.workunit.client.0.smithi019.stdout:3/313: dread d9/d11/fd [4194304,4194304] 0 2024-02-22T15:50:32.583 INFO:tasks.workunit.client.1.smithi062.stdout:7/153: creat d0/db/f34 x:0 0 0 2024-02-22T15:50:32.584 INFO:tasks.workunit.client.0.smithi019.stdout:1/454: mkdir d8/d15/da4 0 2024-02-22T15:50:32.584 INFO:tasks.workunit.client.1.smithi062.stdout:0/106: creat d1/d3/d4/f23 x:0 0 0 2024-02-22T15:50:32.585 INFO:tasks.workunit.client.0.smithi019.stdout:0/224: creat d4/f56 x:0 0 0 2024-02-22T15:50:32.588 INFO:tasks.workunit.client.0.smithi019.stdout:3/314: write d9/d22/d27/f46 [4695110,101070] 0 2024-02-22T15:50:32.589 INFO:tasks.workunit.client.0.smithi019.stdout:2/323: creat d1/d17/d21/f6f x:0 0 0 2024-02-22T15:50:32.590 INFO:tasks.workunit.client.0.smithi019.stdout:0/225: write d4/d32/f46 [696679,77462] 0 2024-02-22T15:50:32.590 INFO:tasks.workunit.client.1.smithi062.stdout:0/107: dread d1/d3/f6 [0,4194304] 0 2024-02-22T15:50:32.593 INFO:tasks.workunit.client.1.smithi062.stdout:0/108: creat d1/d21/f24 x:0 0 0 2024-02-22T15:50:32.593 INFO:tasks.workunit.client.0.smithi019.stdout:0/226: read d4/d9/f11 [166776,108852] 0 2024-02-22T15:50:32.594 INFO:tasks.workunit.client.0.smithi019.stdout:7/439: dwrite d9/d4c/d5b/f65 [0,4194304] 0 2024-02-22T15:50:32.595 INFO:tasks.workunit.client.1.smithi062.stdout:0/109: mknod d1/dd/c25 0 2024-02-22T15:50:32.596 INFO:tasks.workunit.client.0.smithi019.stdout:3/315: symlink d9/d26/d3f/l63 0 2024-02-22T15:50:32.596 INFO:tasks.workunit.client.0.smithi019.stdout:2/324: mkdir d1/d17/d4b/d70 0 2024-02-22T15:50:32.597 INFO:tasks.workunit.client.0.smithi019.stdout:0/227: creat d4/d22/d2c/f57 x:0 0 0 2024-02-22T15:50:32.597 INFO:tasks.workunit.client.1.smithi062.stdout:7/154: rename d0/d14/d1e to d0/d6/d2e/d35 0 2024-02-22T15:50:32.597 INFO:tasks.workunit.client.1.smithi062.stdout:0/110: write d1/d3/de/d14/f18 [667081,123121] 0 2024-02-22T15:50:32.598 INFO:tasks.workunit.client.0.smithi019.stdout:7/440: creat d9/d4c/d5b/f96 x:0 0 0 2024-02-22T15:50:32.598 INFO:tasks.workunit.client.1.smithi062.stdout:7/155: chown d0/f24 2599 1 2024-02-22T15:50:32.599 INFO:tasks.workunit.client.0.smithi019.stdout:7/441: chown d9/l5c 23257987 1 2024-02-22T15:50:32.599 INFO:tasks.workunit.client.0.smithi019.stdout:2/325: chown d1/de/f16 7357857 1 2024-02-22T15:50:32.600 INFO:tasks.workunit.client.0.smithi019.stdout:3/316: unlink d9/d22/d58/c5a 0 2024-02-22T15:50:32.601 INFO:tasks.workunit.client.0.smithi019.stdout:3/317: chown d9/d11 87207675 1 2024-02-22T15:50:32.601 INFO:tasks.workunit.client.1.smithi062.stdout:2/104: truncate d1/f16 3339238 0 2024-02-22T15:50:32.601 INFO:tasks.workunit.client.1.smithi062.stdout:0/111: creat d1/d3/de/d14/f26 x:0 0 0 2024-02-22T15:50:32.602 INFO:tasks.workunit.client.0.smithi019.stdout:7/442: symlink d9/d4c/l97 0 2024-02-22T15:50:32.603 INFO:tasks.workunit.client.0.smithi019.stdout:0/228: creat d4/d5/d52/f58 x:0 0 0 2024-02-22T15:50:32.604 INFO:tasks.workunit.client.1.smithi062.stdout:7/156: rename d0/d6/d2e/d35/f32 to d0/d14/d13/f36 0 2024-02-22T15:50:32.604 INFO:tasks.workunit.client.1.smithi062.stdout:7/157: chown d0/d14/d13 425273 1 2024-02-22T15:50:32.604 INFO:tasks.workunit.client.0.smithi019.stdout:3/318: mknod d9/d22/d58/c64 0 2024-02-22T15:50:32.605 INFO:tasks.workunit.client.1.smithi062.stdout:2/105: mknod d1/d8/df/c21 0 2024-02-22T15:50:32.607 INFO:tasks.workunit.client.0.smithi019.stdout:3/319: mkdir d9/d65 0 2024-02-22T15:50:32.608 INFO:tasks.workunit.client.1.smithi062.stdout:2/106: unlink d1/d8/df/l1b 0 2024-02-22T15:50:32.608 INFO:tasks.workunit.client.1.smithi062.stdout:2/107: readlink l0 0 2024-02-22T15:50:32.608 INFO:tasks.workunit.client.0.smithi019.stdout:2/326: dread d1/d17/d6d/f37 [0,4194304] 0 2024-02-22T15:50:32.612 INFO:tasks.workunit.client.0.smithi019.stdout:7/443: dwrite d9/d38/d41/f90 [0,4194304] 0 2024-02-22T15:50:32.612 INFO:tasks.workunit.client.1.smithi062.stdout:0/112: dwrite d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:32.614 INFO:tasks.workunit.client.1.smithi062.stdout:7/158: dwrite d0/d14/f2c [0,4194304] 0 2024-02-22T15:50:32.615 INFO:tasks.workunit.client.0.smithi019.stdout:7/444: unlink d9/f8a 0 2024-02-22T15:50:32.617 INFO:tasks.workunit.client.1.smithi062.stdout:7/159: mkdir d0/d14/d19/d30/d37 0 2024-02-22T15:50:32.622 INFO:tasks.workunit.client.1.smithi062.stdout:0/113: dwrite d1/f12 [0,4194304] 0 2024-02-22T15:50:32.624 INFO:tasks.workunit.client.0.smithi019.stdout:7/445: dwrite d9/d38/d41/d45/d61/d6a/f93 [0,4194304] 0 2024-02-22T15:50:32.627 INFO:tasks.workunit.client.1.smithi062.stdout:7/160: dread d0/d14/f2c [0,4194304] 0 2024-02-22T15:50:32.632 INFO:tasks.workunit.client.1.smithi062.stdout:7/161: truncate d0/d14/d13/fc 5154413 0 2024-02-22T15:50:32.633 INFO:tasks.workunit.client.1.smithi062.stdout:0/114: dread d1/d3/d4/f7 [0,4194304] 0 2024-02-22T15:50:32.636 INFO:tasks.workunit.client.1.smithi062.stdout:0/115: stat d1/d3/de/d14/f18 0 2024-02-22T15:50:32.639 INFO:tasks.workunit.client.1.smithi062.stdout:2/108: dwrite d1/d8/f11 [0,4194304] 0 2024-02-22T15:50:32.640 INFO:tasks.workunit.client.1.smithi062.stdout:2/109: readlink d1/d8/l9 0 2024-02-22T15:50:32.640 INFO:tasks.workunit.client.1.smithi062.stdout:0/116: symlink d1/d3/d4/d1a/l27 0 2024-02-22T15:50:32.644 INFO:tasks.workunit.client.1.smithi062.stdout:0/117: mknod d1/d13/c28 0 2024-02-22T15:50:32.644 INFO:tasks.workunit.client.1.smithi062.stdout:0/118: write d1/d21/f24 [915020,68955] 0 2024-02-22T15:50:32.646 INFO:tasks.workunit.client.1.smithi062.stdout:0/119: write d1/d3/de/d14/f18 [940401,60607] 0 2024-02-22T15:50:32.648 INFO:tasks.workunit.client.1.smithi062.stdout:0/120: readlink d1/d3/l19 0 2024-02-22T15:50:32.648 INFO:tasks.workunit.client.1.smithi062.stdout:7/162: fdatasync d0/d14/f2c 0 2024-02-22T15:50:32.651 INFO:tasks.workunit.client.1.smithi062.stdout:0/121: symlink d1/d3/l29 0 2024-02-22T15:50:32.655 INFO:tasks.workunit.client.1.smithi062.stdout:2/110: truncate d1/d8/f15 678130 0 2024-02-22T15:50:32.657 INFO:tasks.workunit.client.1.smithi062.stdout:0/122: link d1/d3/d4/f7 d1/d3/de/d14/f2a 0 2024-02-22T15:50:32.658 INFO:tasks.workunit.client.1.smithi062.stdout:2/111: mknod d1/d1e/c22 0 2024-02-22T15:50:32.660 INFO:tasks.workunit.client.0.smithi019.stdout:1/455: rename d8/f79 to d8/d15/d84/fa5 0 2024-02-22T15:50:32.664 INFO:tasks.workunit.client.1.smithi062.stdout:2/112: link d1/d8/df/c13 d1/d1e/c23 0 2024-02-22T15:50:32.665 INFO:tasks.workunit.client.1.smithi062.stdout:0/123: dwrite d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:32.667 INFO:tasks.workunit.client.0.smithi019.stdout:8/344: unlink d9/d14/d17/f25 0 2024-02-22T15:50:32.667 INFO:tasks.workunit.client.1.smithi062.stdout:2/113: creat d1/d1e/d20/f24 x:0 0 0 2024-02-22T15:50:32.668 INFO:tasks.workunit.client.0.smithi019.stdout:2/327: rename d1/d17/d6d to d1/d17/d21/d71 0 2024-02-22T15:50:32.669 INFO:tasks.workunit.client.0.smithi019.stdout:2/328: fsync d1/d17/d4b/f4a 0 2024-02-22T15:50:32.670 INFO:tasks.workunit.client.0.smithi019.stdout:1/456: creat d8/d15/fa6 x:0 0 0 2024-02-22T15:50:32.670 INFO:tasks.workunit.client.1.smithi062.stdout:2/114: creat d1/d8/df/f25 x:0 0 0 2024-02-22T15:50:32.671 INFO:tasks.workunit.client.1.smithi062.stdout:0/124: dread d1/d3/de/d14/f2a [0,4194304] 0 2024-02-22T15:50:32.671 INFO:tasks.workunit.client.1.smithi062.stdout:2/115: read - d1/d8/df/f25 zero size 2024-02-22T15:50:32.673 INFO:tasks.workunit.client.0.smithi019.stdout:8/345: mknod d9/d14/d1f/d20/c72 0 2024-02-22T15:50:32.674 INFO:tasks.workunit.client.1.smithi062.stdout:2/116: write d1/d8/f11 [1737062,109374] 0 2024-02-22T15:50:32.680 INFO:tasks.workunit.client.0.smithi019.stdout:1/457: mkdir d8/d2a/da7 0 2024-02-22T15:50:32.682 INFO:tasks.workunit.client.1.smithi062.stdout:0/125: dwrite d1/d3/de/d14/f18 [0,4194304] 0 2024-02-22T15:50:32.682 INFO:tasks.workunit.client.0.smithi019.stdout:1/458: read d8/d23/f55 [78841,27848] 0 2024-02-22T15:50:32.683 INFO:tasks.workunit.client.1.smithi062.stdout:0/126: stat d1/f12 0 2024-02-22T15:50:32.683 INFO:tasks.workunit.client.0.smithi019.stdout:8/346: mknod d9/d14/d17/c73 0 2024-02-22T15:50:32.684 INFO:tasks.workunit.client.1.smithi062.stdout:2/117: dwrite d1/d8/fa [0,4194304] 0 2024-02-22T15:50:32.692 INFO:tasks.workunit.client.1.smithi062.stdout:0/127: dread d1/d3/de/d14/f2a [0,4194304] 0 2024-02-22T15:50:32.693 INFO:tasks.workunit.client.0.smithi019.stdout:8/347: write d9/d14/d17/d5d/f63 [541574,946] 0 2024-02-22T15:50:32.695 INFO:tasks.workunit.client.1.smithi062.stdout:2/118: creat d1/d8/df/f26 x:0 0 0 2024-02-22T15:50:32.698 INFO:tasks.workunit.client.0.smithi019.stdout:3/320: chown d9/d26/d32/c25 430 1 2024-02-22T15:50:32.700 INFO:tasks.workunit.client.0.smithi019.stdout:3/321: chown d9/d26/d32/l1c 145001222 1 2024-02-22T15:50:32.701 INFO:tasks.workunit.client.0.smithi019.stdout:0/229: dwrite d4/db/d1b/f1f [0,4194304] 0 2024-02-22T15:50:32.701 INFO:tasks.workunit.client.1.smithi062.stdout:2/119: dread d1/d8/fa [0,4194304] 0 2024-02-22T15:50:32.702 INFO:tasks.workunit.client.1.smithi062.stdout:0/128: dwrite d1/d3/f6 [4194304,4194304] 0 2024-02-22T15:50:32.703 INFO:tasks.workunit.client.1.smithi062.stdout:9/95: sync 2024-02-22T15:50:32.703 INFO:tasks.workunit.client.1.smithi062.stdout:1/131: sync 2024-02-22T15:50:32.706 INFO:tasks.workunit.client.0.smithi019.stdout:0/230: mkdir d4/d23/d59 0 2024-02-22T15:50:32.708 INFO:tasks.workunit.client.1.smithi062.stdout:1/132: dread d1/d6/de/d21/d24/f2a [0,4194304] 0 2024-02-22T15:50:32.709 INFO:tasks.workunit.client.0.smithi019.stdout:7/446: dwrite d9/f21 [0,4194304] 0 2024-02-22T15:50:32.709 INFO:tasks.workunit.client.0.smithi019.stdout:3/322: mkdir d9/d26/d3f/d59/d66 0 2024-02-22T15:50:32.711 INFO:tasks.workunit.client.1.smithi062.stdout:0/129: dread d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:32.711 INFO:tasks.workunit.client.1.smithi062.stdout:1/133: mknod d1/d6/d14/c2c 0 2024-02-22T15:50:32.713 INFO:tasks.workunit.client.1.smithi062.stdout:0/130: mknod d1/d3/de/c2b 0 2024-02-22T15:50:32.714 INFO:tasks.workunit.client.0.smithi019.stdout:6/377: sync 2024-02-22T15:50:32.714 INFO:tasks.workunit.client.0.smithi019.stdout:4/372: sync 2024-02-22T15:50:32.715 INFO:tasks.workunit.client.0.smithi019.stdout:4/373: chown d2/db/d14/d23/d4b/c6d 57835 1 2024-02-22T15:50:32.716 INFO:tasks.workunit.client.1.smithi062.stdout:1/134: unlink d1/l3 0 2024-02-22T15:50:32.716 INFO:tasks.workunit.client.0.smithi019.stdout:8/348: getdents d9/d14/d1f/d20 0 2024-02-22T15:50:32.717 INFO:tasks.workunit.client.0.smithi019.stdout:4/374: fdatasync d2/db/d14/d23/d21/f5a 0 2024-02-22T15:50:32.718 INFO:tasks.workunit.client.1.smithi062.stdout:9/96: link d4/fd d4/d1c/f1e 0 2024-02-22T15:50:32.718 INFO:tasks.workunit.client.0.smithi019.stdout:3/323: stat d9/d11/f1e 0 2024-02-22T15:50:32.719 INFO:tasks.workunit.client.1.smithi062.stdout:9/97: write d4/fe [1704475,129416] 0 2024-02-22T15:50:32.719 INFO:tasks.workunit.client.1.smithi062.stdout:0/131: mkdir d1/d3/d4/d1a/d1d/d1f/d22/d2c 0 2024-02-22T15:50:32.720 INFO:tasks.workunit.client.1.smithi062.stdout:0/132: chown d1/d3/d4/d1e 1020688 1 2024-02-22T15:50:32.721 INFO:tasks.workunit.client.1.smithi062.stdout:0/133: dread - d1/dd/f1c zero size 2024-02-22T15:50:32.721 INFO:tasks.workunit.client.0.smithi019.stdout:2/329: truncate d1/d17/f47 3461616 0 2024-02-22T15:50:32.722 INFO:tasks.workunit.client.1.smithi062.stdout:0/134: write d1/f12 [3926980,11327] 0 2024-02-22T15:50:32.723 INFO:tasks.workunit.client.1.smithi062.stdout:0/135: chown d1/d21/f24 2659 1 2024-02-22T15:50:32.724 INFO:tasks.workunit.client.1.smithi062.stdout:0/136: mknod d1/dd/c2d 0 2024-02-22T15:50:32.727 INFO:tasks.workunit.client.0.smithi019.stdout:8/349: mknod d9/d14/d1f/d6d/c74 0 2024-02-22T15:50:32.729 INFO:tasks.workunit.client.0.smithi019.stdout:8/350: chown d9/d14/d17/l1b 30724 1 2024-02-22T15:50:32.730 INFO:tasks.workunit.client.1.smithi062.stdout:9/98: read d4/fe [1159739,102285] 0 2024-02-22T15:50:32.730 INFO:tasks.workunit.client.0.smithi019.stdout:6/378: creat d6/d31/f6b x:0 0 0 2024-02-22T15:50:32.732 INFO:tasks.workunit.client.1.smithi062.stdout:9/99: symlink d4/d1c/l1f 0 2024-02-22T15:50:32.734 INFO:tasks.workunit.client.0.smithi019.stdout:7/447: write d9/d26/f29 [522066,72227] 0 2024-02-22T15:50:32.735 INFO:tasks.workunit.client.0.smithi019.stdout:6/379: dread - d6/f3c zero size 2024-02-22T15:50:32.735 INFO:tasks.workunit.client.1.smithi062.stdout:9/100: chown d4/d15/l1a 122591 1 2024-02-22T15:50:32.736 INFO:tasks.workunit.client.1.smithi062.stdout:9/101: read - d4/d15/f17 zero size 2024-02-22T15:50:32.737 INFO:tasks.workunit.client.0.smithi019.stdout:7/448: write d9/f19 [4741228,43420] 0 2024-02-22T15:50:32.740 INFO:tasks.workunit.client.1.smithi062.stdout:9/102: mkdir d4/d15/d20 0 2024-02-22T15:50:32.740 INFO:tasks.workunit.client.0.smithi019.stdout:3/324: link c8 d9/d11/c67 0 2024-02-22T15:50:32.741 INFO:tasks.workunit.client.1.smithi062.stdout:0/137: truncate d1/f12 3505696 0 2024-02-22T15:50:32.741 INFO:tasks.workunit.client.0.smithi019.stdout:6/380: symlink d6/d31/d37/l6c 0 2024-02-22T15:50:32.741 INFO:tasks.workunit.client.0.smithi019.stdout:4/375: write d2/db/d14/d23/f2f [2521989,46269] 0 2024-02-22T15:50:32.742 INFO:tasks.workunit.client.0.smithi019.stdout:4/376: write d2/db/d44/f55 [837331,757] 0 2024-02-22T15:50:32.744 INFO:tasks.workunit.client.0.smithi019.stdout:2/330: link d1/d17/d21/f41 d1/d17/d21/d51/f72 0 2024-02-22T15:50:32.744 INFO:tasks.workunit.client.0.smithi019.stdout:4/377: chown d2/db/d14/d23/d21/d35/l6c 11131414 1 2024-02-22T15:50:32.745 INFO:tasks.workunit.client.0.smithi019.stdout:7/449: chown d9/c2a 57987 1 2024-02-22T15:50:32.745 INFO:tasks.workunit.client.1.smithi062.stdout:9/103: dread d4/fc [0,4194304] 0 2024-02-22T15:50:32.746 INFO:tasks.workunit.client.1.smithi062.stdout:0/138: dread d1/d3/f6 [4194304,4194304] 0 2024-02-22T15:50:32.747 INFO:tasks.workunit.client.0.smithi019.stdout:1/459: write d8/d44/f4a [801851,77107] 0 2024-02-22T15:50:32.747 INFO:tasks.workunit.client.0.smithi019.stdout:6/381: mkdir d6/d4b/d6d 0 2024-02-22T15:50:32.748 INFO:tasks.workunit.client.1.smithi062.stdout:9/104: mknod d4/d15/d20/c21 0 2024-02-22T15:50:32.748 INFO:tasks.workunit.client.1.smithi062.stdout:0/139: rmdir d1/d13 39 2024-02-22T15:50:32.749 INFO:tasks.workunit.client.0.smithi019.stdout:4/378: symlink d2/db/d54/l7e 0 2024-02-22T15:50:32.749 INFO:tasks.workunit.client.1.smithi062.stdout:9/105: truncate d4/f14 2037378 0 2024-02-22T15:50:32.751 INFO:tasks.workunit.client.1.smithi062.stdout:0/140: unlink d1/d3/d4/f7 0 2024-02-22T15:50:32.752 INFO:tasks.workunit.client.1.smithi062.stdout:0/141: chown d1/d3/l19 2 1 2024-02-22T15:50:32.752 INFO:tasks.workunit.client.1.smithi062.stdout:9/106: creat d4/f22 x:0 0 0 2024-02-22T15:50:32.753 INFO:tasks.workunit.client.0.smithi019.stdout:7/450: mkdir d9/d98 0 2024-02-22T15:50:32.754 INFO:tasks.workunit.client.1.smithi062.stdout:9/107: creat d4/d15/f23 x:0 0 0 2024-02-22T15:50:32.754 INFO:tasks.workunit.client.0.smithi019.stdout:4/379: readlink d2/db/d14/l24 0 2024-02-22T15:50:32.755 INFO:tasks.workunit.client.1.smithi062.stdout:9/108: write f2 [1017619,28669] 0 2024-02-22T15:50:32.758 INFO:tasks.workunit.client.0.smithi019.stdout:7/451: write d9/d38/d41/f78 [329850,65097] 0 2024-02-22T15:50:32.759 INFO:tasks.workunit.client.1.smithi062.stdout:9/109: unlink d4/f22 0 2024-02-22T15:50:32.760 INFO:tasks.workunit.client.1.smithi062.stdout:0/142: dwrite d1/dd/f1c [0,4194304] 0 2024-02-22T15:50:32.760 INFO:tasks.workunit.client.0.smithi019.stdout:4/380: rmdir d2/db/d14/d23/d21 39 2024-02-22T15:50:32.762 INFO:tasks.workunit.client.0.smithi019.stdout:6/382: dwrite d6/d31/d48/f4a [0,4194304] 0 2024-02-22T15:50:32.762 INFO:tasks.workunit.client.0.smithi019.stdout:3/325: truncate f3 11579500 0 2024-02-22T15:50:32.764 INFO:tasks.workunit.client.0.smithi019.stdout:1/460: truncate d8/d15/f17 342795 0 2024-02-22T15:50:32.766 INFO:tasks.workunit.client.0.smithi019.stdout:3/326: stat d9/d14/f3c 0 2024-02-22T15:50:32.766 INFO:tasks.workunit.client.1.smithi062.stdout:0/143: dread d1/d3/f6 [4194304,4194304] 0 2024-02-22T15:50:32.767 INFO:tasks.workunit.client.0.smithi019.stdout:7/452: rename d9/d4c/d5b/l63 to d9/d38/d41/l99 0 2024-02-22T15:50:32.767 INFO:tasks.workunit.client.0.smithi019.stdout:1/461: chown d8/d2a/c48 19831 1 2024-02-22T15:50:32.768 INFO:tasks.workunit.client.0.smithi019.stdout:2/331: dwrite d1/d17/d21/d51/f72 [0,4194304] 0 2024-02-22T15:50:32.770 INFO:tasks.workunit.client.0.smithi019.stdout:1/462: stat d8/d44/c4e 0 2024-02-22T15:50:32.770 INFO:tasks.workunit.client.1.smithi062.stdout:0/144: mknod d1/d3/d4/d1a/d1d/d1f/c2e 0 2024-02-22T15:50:32.771 INFO:tasks.workunit.client.0.smithi019.stdout:1/463: chown d8/d44/d5d/l52 1569 1 2024-02-22T15:50:32.773 INFO:tasks.workunit.client.0.smithi019.stdout:4/381: unlink d2/db/d14/d23/d21/c39 0 2024-02-22T15:50:32.774 INFO:tasks.workunit.client.1.smithi062.stdout:0/145: creat d1/d13/f2f x:0 0 0 2024-02-22T15:50:32.775 INFO:tasks.workunit.client.0.smithi019.stdout:7/453: mkdir d9/d3a/d49/d5d/d74/d9a 0 2024-02-22T15:50:32.776 INFO:tasks.workunit.client.0.smithi019.stdout:4/382: chown d2/db/d14/d23/d21/d35/f38 5531828 1 2024-02-22T15:50:32.777 INFO:tasks.workunit.client.1.smithi062.stdout:0/146: mknod d1/d3/d4/d1e/c30 0 2024-02-22T15:50:32.778 INFO:tasks.workunit.client.0.smithi019.stdout:4/383: dread - d2/d3f/f7c zero size 2024-02-22T15:50:32.780 INFO:tasks.workunit.client.0.smithi019.stdout:2/332: rename d1/d17/d38/l69 to d1/d17/d4b/l73 0 2024-02-22T15:50:32.780 INFO:tasks.workunit.client.0.smithi019.stdout:4/384: write d2/db/f17 [460689,117905] 0 2024-02-22T15:50:32.781 INFO:tasks.workunit.client.0.smithi019.stdout:6/383: getdents d6/d31/d43/d55 0 2024-02-22T15:50:32.783 INFO:tasks.workunit.client.0.smithi019.stdout:4/385: write d2/db/d14/d23/d33/f7d [820532,108767] 0 2024-02-22T15:50:32.784 INFO:tasks.workunit.client.0.smithi019.stdout:3/327: dread d9/d26/d2d/f3a [0,4194304] 0 2024-02-22T15:50:32.785 INFO:tasks.workunit.client.1.smithi062.stdout:0/147: write d1/d3/de/d14/f17 [152001,35011] 0 2024-02-22T15:50:32.785 INFO:tasks.workunit.client.0.smithi019.stdout:1/464: creat d8/d18/d64/d7f/d82/fa8 x:0 0 0 2024-02-22T15:50:32.786 INFO:tasks.workunit.client.1.smithi062.stdout:0/148: chown d1/d3/de/d14 2 1 2024-02-22T15:50:32.787 INFO:tasks.workunit.client.0.smithi019.stdout:3/328: fdatasync d9/d26/d32/f2c 0 2024-02-22T15:50:32.787 INFO:tasks.workunit.client.0.smithi019.stdout:7/454: fsync d9/f34 0 2024-02-22T15:50:32.787 INFO:tasks.workunit.client.1.smithi062.stdout:0/149: read d1/f12 [1468932,56934] 0 2024-02-22T15:50:32.789 INFO:tasks.workunit.client.1.smithi062.stdout:0/150: creat d1/d13/f31 x:0 0 0 2024-02-22T15:50:32.790 INFO:tasks.workunit.client.0.smithi019.stdout:6/384: symlink d6/d3e/l6e 0 2024-02-22T15:50:32.790 INFO:tasks.workunit.client.1.smithi062.stdout:0/151: read - d1/d3/de/d14/f26 zero size 2024-02-22T15:50:32.792 INFO:tasks.workunit.client.0.smithi019.stdout:4/386: creat d2/db/d54/f7f x:0 0 0 2024-02-22T15:50:32.792 INFO:tasks.workunit.client.1.smithi062.stdout:0/152: symlink d1/dd/l32 0 2024-02-22T15:50:32.793 INFO:tasks.workunit.client.0.smithi019.stdout:4/387: write d2/fa [3423177,7108] 0 2024-02-22T15:50:32.795 INFO:tasks.workunit.client.1.smithi062.stdout:0/153: symlink d1/d3/d4/l33 0 2024-02-22T15:50:32.795 INFO:tasks.workunit.client.0.smithi019.stdout:1/465: rmdir d8/d44 39 2024-02-22T15:50:32.796 INFO:tasks.workunit.client.0.smithi019.stdout:1/466: chown d8/d18/d85/f9d 0 1 2024-02-22T15:50:32.799 INFO:tasks.workunit.client.0.smithi019.stdout:7/455: creat d9/d26/d77/f9b x:0 0 0 2024-02-22T15:50:32.799 INFO:tasks.workunit.client.0.smithi019.stdout:1/467: read d8/d2a/f3a [58413,110378] 0 2024-02-22T15:50:32.800 INFO:tasks.workunit.client.0.smithi019.stdout:6/385: dread d6/d4b/f52 [0,4194304] 0 2024-02-22T15:50:32.800 INFO:tasks.workunit.client.0.smithi019.stdout:1/468: stat d8/d15/d34/c49 0 2024-02-22T15:50:32.800 INFO:tasks.workunit.client.0.smithi019.stdout:2/333: unlink d1/ld 0 2024-02-22T15:50:32.802 INFO:tasks.workunit.client.0.smithi019.stdout:1/469: write d8/d15/d34/fa3 [154344,47553] 0 2024-02-22T15:50:32.803 INFO:tasks.workunit.client.0.smithi019.stdout:1/470: chown d8/d23/d33/f93 50 1 2024-02-22T15:50:32.805 INFO:tasks.workunit.client.0.smithi019.stdout:4/388: mknod d2/d3f/d6e/c80 0 2024-02-22T15:50:32.805 INFO:tasks.workunit.client.1.smithi062.stdout:0/154: dwrite d1/d3/de/d14/f2a [0,4194304] 0 2024-02-22T15:50:32.806 INFO:tasks.workunit.client.0.smithi019.stdout:1/471: dread d8/d2a/f3a [0,4194304] 0 2024-02-22T15:50:32.806 INFO:tasks.workunit.client.1.smithi062.stdout:0/155: rmdir d1/d3/d4/d1e 39 2024-02-22T15:50:32.808 INFO:tasks.workunit.client.0.smithi019.stdout:6/386: fdatasync d6/f2c 0 2024-02-22T15:50:32.810 INFO:tasks.workunit.client.1.smithi062.stdout:0/156: creat d1/d3/d4/d1a/f34 x:0 0 0 2024-02-22T15:50:32.810 INFO:tasks.workunit.client.1.smithi062.stdout:0/157: chown d1/d3/f6 213489 1 2024-02-22T15:50:32.811 INFO:tasks.workunit.client.1.smithi062.stdout:0/158: fsync d1/d3/d4/f23 0 2024-02-22T15:50:32.813 INFO:tasks.workunit.client.1.smithi062.stdout:0/159: creat d1/d3/de/f35 x:0 0 0 2024-02-22T15:50:32.815 INFO:tasks.workunit.client.1.smithi062.stdout:0/160: creat d1/dd/f36 x:0 0 0 2024-02-22T15:50:32.817 INFO:tasks.workunit.client.1.smithi062.stdout:0/161: dread d1/d3/de/d14/f17 [0,4194304] 0 2024-02-22T15:50:32.819 INFO:tasks.workunit.client.0.smithi019.stdout:6/387: mkdir d6/d58/d6f 0 2024-02-22T15:50:32.819 INFO:tasks.workunit.client.0.smithi019.stdout:7/456: symlink d9/d3a/d49/d5d/d74/d9a/l9c 0 2024-02-22T15:50:32.819 INFO:tasks.workunit.client.0.smithi019.stdout:7/457: readlink d9/d4c/l91 0 2024-02-22T15:50:32.819 INFO:tasks.workunit.client.0.smithi019.stdout:2/334: unlink d1/d17/d21/d51/c65 0 2024-02-22T15:50:32.820 INFO:tasks.workunit.client.0.smithi019.stdout:7/458: dread - d9/d4c/f68 zero size 2024-02-22T15:50:32.820 INFO:tasks.workunit.client.1.smithi062.stdout:0/162: rmdir d1/d3/d4/d1a/d1d/d1f/d22 39 2024-02-22T15:50:32.821 INFO:tasks.workunit.client.1.smithi062.stdout:0/163: dread - d1/d3/d4/d1a/f34 zero size 2024-02-22T15:50:32.821 INFO:tasks.workunit.client.1.smithi062.stdout:0/164: dread - d1/dd/f36 zero size 2024-02-22T15:50:32.821 INFO:tasks.workunit.client.0.smithi019.stdout:7/459: chown d9/d4c/f68 12 1 2024-02-22T15:50:32.821 INFO:tasks.workunit.client.0.smithi019.stdout:7/460: fdatasync d9/f1b 0 2024-02-22T15:50:32.822 INFO:tasks.workunit.client.0.smithi019.stdout:7/461: stat d9/d26/d77/f9b 0 2024-02-22T15:50:32.823 INFO:tasks.workunit.client.0.smithi019.stdout:7/462: dread - d9/d38/f5a zero size 2024-02-22T15:50:32.825 INFO:tasks.workunit.client.0.smithi019.stdout:2/335: symlink d1/d17/d38/l74 0 2024-02-22T15:50:32.825 INFO:tasks.workunit.client.1.smithi062.stdout:0/165: rename d1/d3/de/f35 to d1/d3/d4/d1a/d1d/d1f/f37 0 2024-02-22T15:50:32.826 INFO:tasks.workunit.client.0.smithi019.stdout:3/329: link d9/d22/d58/l61 d9/l68 0 2024-02-22T15:50:32.827 INFO:tasks.workunit.client.0.smithi019.stdout:4/389: creat d2/db/d14/f81 x:0 0 0 2024-02-22T15:50:32.830 INFO:tasks.workunit.client.0.smithi019.stdout:7/463: mknod d9/d26/d77/c9d 0 2024-02-22T15:50:32.830 INFO:tasks.workunit.client.0.smithi019.stdout:7/464: stat d9/c53 0 2024-02-22T15:50:32.831 INFO:tasks.workunit.client.0.smithi019.stdout:7/465: fsync d9/d38/f3e 0 2024-02-22T15:50:32.835 INFO:tasks.workunit.client.0.smithi019.stdout:4/390: chown d2/c10 15 1 2024-02-22T15:50:32.836 INFO:tasks.workunit.client.0.smithi019.stdout:1/472: link d8/d18/c38 d8/d18/d64/d7f/d82/ca9 0 2024-02-22T15:50:32.838 INFO:tasks.workunit.client.0.smithi019.stdout:6/388: rmdir d6/d58/d6f 0 2024-02-22T15:50:32.841 INFO:tasks.workunit.client.0.smithi019.stdout:2/336: rename d1/d46/c53 to d1/d17/d1c/d66/c75 0 2024-02-22T15:50:32.842 INFO:tasks.workunit.client.0.smithi019.stdout:4/391: creat d2/db/d14/d23/d4b/f82 x:0 0 0 2024-02-22T15:50:32.843 INFO:tasks.workunit.client.0.smithi019.stdout:6/389: readlink d6/l19 0 2024-02-22T15:50:32.846 INFO:tasks.workunit.client.0.smithi019.stdout:7/466: rmdir d9/d98 0 2024-02-22T15:50:32.848 INFO:tasks.workunit.client.0.smithi019.stdout:3/330: dread d9/d26/d32/f43 [0,4194304] 0 2024-02-22T15:50:32.848 INFO:tasks.workunit.client.0.smithi019.stdout:4/392: rename d2/c63 to d2/db/d14/d23/d21/d35/d3d/c83 0 2024-02-22T15:50:32.850 INFO:tasks.workunit.client.0.smithi019.stdout:7/467: unlink d9/d3a/c47 0 2024-02-22T15:50:32.852 INFO:tasks.workunit.client.0.smithi019.stdout:6/390: dwrite d6/d58/f66 [0,4194304] 0 2024-02-22T15:50:32.852 INFO:tasks.workunit.client.0.smithi019.stdout:4/393: symlink d2/db/d14/d23/d21/d35/l84 0 2024-02-22T15:50:32.854 INFO:tasks.workunit.client.0.smithi019.stdout:7/468: write d9/d38/d41/d45/d61/d6a/f93 [2922961,19829] 0 2024-02-22T15:50:32.857 INFO:tasks.workunit.client.0.smithi019.stdout:4/394: truncate d2/db/d14/d23/d21/d35/f38 2004345 0 2024-02-22T15:50:32.857 INFO:tasks.workunit.client.0.smithi019.stdout:3/331: unlink d9/c13 0 2024-02-22T15:50:32.857 INFO:tasks.workunit.client.0.smithi019.stdout:4/395: fsync d2/db/f17 0 2024-02-22T15:50:32.860 INFO:tasks.workunit.client.0.smithi019.stdout:2/337: rename d1/d17/d21/d71/d60/f3e to d1/f76 0 2024-02-22T15:50:32.863 INFO:tasks.workunit.client.0.smithi019.stdout:7/469: creat d9/d3a/d49/d5d/d74/f9e x:0 0 0 2024-02-22T15:50:32.865 INFO:tasks.workunit.client.0.smithi019.stdout:7/470: write d9/d4c/f66 [1167400,14118] 0 2024-02-22T15:50:32.868 INFO:tasks.workunit.client.0.smithi019.stdout:7/471: write d9/d4c/f68 [535715,93564] 0 2024-02-22T15:50:32.869 INFO:tasks.workunit.client.0.smithi019.stdout:4/396: unlink d2/db/f69 0 2024-02-22T15:50:32.872 INFO:tasks.workunit.client.0.smithi019.stdout:2/338: creat d1/d17/d21/d51/f77 x:0 0 0 2024-02-22T15:50:32.876 INFO:tasks.workunit.client.0.smithi019.stdout:6/391: getdents d6/d31 0 2024-02-22T15:50:32.877 INFO:tasks.workunit.client.0.smithi019.stdout:7/472: dwrite d9/d38/d41/f90 [0,4194304] 0 2024-02-22T15:50:32.880 INFO:tasks.workunit.client.0.smithi019.stdout:6/392: mknod d6/d4b/c70 0 2024-02-22T15:50:32.883 INFO:tasks.workunit.client.0.smithi019.stdout:7/473: chown d9/d79/c85 514715 1 2024-02-22T15:50:32.884 INFO:tasks.workunit.client.0.smithi019.stdout:6/393: mkdir d6/d4b/d71 0 2024-02-22T15:50:32.888 INFO:tasks.workunit.client.0.smithi019.stdout:6/394: symlink d6/l72 0 2024-02-22T15:50:32.889 INFO:tasks.workunit.client.0.smithi019.stdout:2/339: truncate d1/f76 382434 0 2024-02-22T15:50:32.890 INFO:tasks.workunit.client.0.smithi019.stdout:4/397: dread d2/db/f17 [0,4194304] 0 2024-02-22T15:50:32.891 INFO:tasks.workunit.client.1.smithi062.stdout:4/58: sync 2024-02-22T15:50:32.891 INFO:tasks.workunit.client.1.smithi062.stdout:5/67: sync 2024-02-22T15:50:32.891 INFO:tasks.workunit.client.1.smithi062.stdout:3/86: sync 2024-02-22T15:50:32.891 INFO:tasks.workunit.client.0.smithi019.stdout:6/395: read - d6/d3e/f4e zero size 2024-02-22T15:50:32.892 INFO:tasks.workunit.client.1.smithi062.stdout:4/59: stat d4/l5 0 2024-02-22T15:50:32.892 INFO:tasks.workunit.client.1.smithi062.stdout:3/87: read - d5/d1a/f1b zero size 2024-02-22T15:50:32.892 INFO:tasks.workunit.client.0.smithi019.stdout:4/398: chown d2/d3f/d48/l71 68843 1 2024-02-22T15:50:32.893 INFO:tasks.workunit.client.0.smithi019.stdout:7/474: dwrite d9/d3a/d49/d5d/d74/f88 [0,4194304] 0 2024-02-22T15:50:32.894 INFO:tasks.workunit.client.1.smithi062.stdout:4/60: rmdir d4 39 2024-02-22T15:50:32.895 INFO:tasks.workunit.client.0.smithi019.stdout:7/475: stat d9/d38/f3c 0 2024-02-22T15:50:32.897 INFO:tasks.workunit.client.0.smithi019.stdout:4/399: truncate d2/db/d14/d23/f65 72585 0 2024-02-22T15:50:32.898 INFO:tasks.workunit.client.1.smithi062.stdout:3/88: rename d5/f13 to d5/f1d 0 2024-02-22T15:50:32.898 INFO:tasks.workunit.client.0.smithi019.stdout:7/476: creat d9/d3a/d49/d5d/f9f x:0 0 0 2024-02-22T15:50:32.899 INFO:tasks.workunit.client.0.smithi019.stdout:2/340: write d1/de/f16 [3426577,39238] 0 2024-02-22T15:50:32.900 INFO:tasks.workunit.client.0.smithi019.stdout:7/477: fsync d9/f16 0 2024-02-22T15:50:32.901 INFO:tasks.workunit.client.1.smithi062.stdout:3/89: truncate d5/f8 493464 0 2024-02-22T15:50:32.903 INFO:tasks.workunit.client.0.smithi019.stdout:7/478: truncate d9/d38/d41/f78 1196533 0 2024-02-22T15:50:32.903 INFO:tasks.workunit.client.1.smithi062.stdout:5/68: dread d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:32.903 INFO:tasks.workunit.client.1.smithi062.stdout:4/61: rename d4/fe to d4/f11 0 2024-02-22T15:50:32.907 INFO:tasks.workunit.client.1.smithi062.stdout:3/90: creat d5/d1a/f1e x:0 0 0 2024-02-22T15:50:32.908 INFO:tasks.workunit.client.1.smithi062.stdout:3/91: stat d5/cf 0 2024-02-22T15:50:32.912 INFO:tasks.workunit.client.1.smithi062.stdout:5/69: dwrite d0/f9 [0,4194304] 0 2024-02-22T15:50:32.912 INFO:tasks.workunit.client.0.smithi019.stdout:7/479: dwrite d9/d3a/f86 [0,4194304] 0 2024-02-22T15:50:32.913 INFO:tasks.workunit.client.0.smithi019.stdout:4/400: truncate d2/db/d14/f5e 641289 0 2024-02-22T15:50:32.914 INFO:tasks.workunit.client.1.smithi062.stdout:3/92: dread d5/f10 [0,4194304] 0 2024-02-22T15:50:32.916 INFO:tasks.workunit.client.0.smithi019.stdout:4/401: mkdir d2/db/d14/d23/d21/d35/d85 0 2024-02-22T15:50:32.918 INFO:tasks.workunit.client.1.smithi062.stdout:3/93: rename d5/c9 to d5/d16/c1f 0 2024-02-22T15:50:32.920 INFO:tasks.workunit.client.0.smithi019.stdout:7/480: symlink d9/la0 0 2024-02-22T15:50:32.922 INFO:tasks.workunit.client.1.smithi062.stdout:4/62: getdents d4 0 2024-02-22T15:50:32.924 INFO:tasks.workunit.client.1.smithi062.stdout:4/63: mknod d4/df/c12 0 2024-02-22T15:50:32.925 INFO:tasks.workunit.client.1.smithi062.stdout:4/64: dread f2 [0,4194304] 0 2024-02-22T15:50:32.926 INFO:tasks.workunit.client.0.smithi019.stdout:2/341: write d1/f76 [1080808,83315] 0 2024-02-22T15:50:32.927 INFO:tasks.workunit.client.0.smithi019.stdout:7/481: mkdir d9/d38/d41/d45/d92/da1 0 2024-02-22T15:50:32.929 INFO:tasks.workunit.client.1.smithi062.stdout:4/65: unlink d4/fc 0 2024-02-22T15:50:32.929 INFO:tasks.workunit.client.0.smithi019.stdout:2/342: stat d1/de/f1e 0 2024-02-22T15:50:32.931 INFO:tasks.workunit.client.1.smithi062.stdout:4/66: mkdir d4/d13 0 2024-02-22T15:50:32.932 INFO:tasks.workunit.client.0.smithi019.stdout:4/402: dread d2/fd [0,4194304] 0 2024-02-22T15:50:32.933 INFO:tasks.workunit.client.1.smithi062.stdout:4/67: creat d4/df/f14 x:0 0 0 2024-02-22T15:50:32.933 INFO:tasks.workunit.client.1.smithi062.stdout:4/68: chown d4/df/f14 6624 1 2024-02-22T15:50:32.934 INFO:tasks.workunit.client.0.smithi019.stdout:2/343: getdents d1/de 0 2024-02-22T15:50:32.935 INFO:tasks.workunit.client.1.smithi062.stdout:4/69: creat d4/d13/f15 x:0 0 0 2024-02-22T15:50:32.935 INFO:tasks.workunit.client.0.smithi019.stdout:7/482: dwrite d9/d38/f4e [0,4194304] 0 2024-02-22T15:50:32.936 INFO:tasks.workunit.client.0.smithi019.stdout:7/483: stat d9/fa 0 2024-02-22T15:50:32.936 INFO:tasks.workunit.client.0.smithi019.stdout:2/344: chown d1/d17/d1c/f22 0 1 2024-02-22T15:50:32.936 INFO:tasks.workunit.client.0.smithi019.stdout:7/484: chown d9/d3a/c84 66201887 1 2024-02-22T15:50:32.939 INFO:tasks.workunit.client.0.smithi019.stdout:4/403: link d2/d3f/f51 d2/db/d14/d23/d4b/f86 0 2024-02-22T15:50:32.940 INFO:tasks.workunit.client.0.smithi019.stdout:2/345: unlink d1/de/d15/f35 0 2024-02-22T15:50:32.940 INFO:tasks.workunit.client.0.smithi019.stdout:7/485: dread - d9/d4c/d5b/f64 zero size 2024-02-22T15:50:32.942 INFO:tasks.workunit.client.1.smithi062.stdout:4/70: dwrite d4/df/f14 [0,4194304] 0 2024-02-22T15:50:32.943 INFO:tasks.workunit.client.0.smithi019.stdout:4/404: read d2/db/d14/d23/f36 [2180792,21858] 0 2024-02-22T15:50:32.943 INFO:tasks.workunit.client.1.smithi062.stdout:4/71: write d4/f8 [3965571,35720] 0 2024-02-22T15:50:32.944 INFO:tasks.workunit.client.1.smithi062.stdout:5/70: fdatasync d0/f9 0 2024-02-22T15:50:32.944 INFO:tasks.workunit.client.0.smithi019.stdout:7/486: unlink d9/d26/c5e 0 2024-02-22T15:50:32.947 INFO:tasks.workunit.client.0.smithi019.stdout:7/487: write d9/d38/f4e [1414069,108404] 0 2024-02-22T15:50:32.950 INFO:tasks.workunit.client.0.smithi019.stdout:7/488: truncate d9/d38/f69 760968 0 2024-02-22T15:50:32.953 INFO:tasks.workunit.client.1.smithi062.stdout:3/94: link d5/d16/c1f d5/d1a/c20 0 2024-02-22T15:50:32.954 INFO:tasks.workunit.client.0.smithi019.stdout:7/489: write d9/f19 [4842000,127460] 0 2024-02-22T15:50:32.954 INFO:tasks.workunit.client.0.smithi019.stdout:2/346: dwrite d1/d17/d21/d51/f5d [0,4194304] 0 2024-02-22T15:50:32.957 INFO:tasks.workunit.client.0.smithi019.stdout:4/405: rename d2/db/d54/f7f to d2/db/d14/f87 0 2024-02-22T15:50:32.957 INFO:tasks.workunit.client.0.smithi019.stdout:2/347: truncate d1/d17/d21/d51/f6e 48392 0 2024-02-22T15:50:32.961 INFO:tasks.workunit.client.0.smithi019.stdout:7/490: mknod d9/d38/d41/d45/d92/da1/ca2 0 2024-02-22T15:50:32.966 INFO:tasks.workunit.client.0.smithi019.stdout:7/491: unlink d9/d38/c8b 0 2024-02-22T15:50:32.966 INFO:tasks.workunit.client.0.smithi019.stdout:2/348: creat d1/f78 x:0 0 0 2024-02-22T15:50:32.966 INFO:tasks.workunit.client.1.smithi062.stdout:3/95: dwrite d5/f10 [0,4194304] 0 2024-02-22T15:50:32.967 INFO:tasks.workunit.client.1.smithi062.stdout:3/96: write f4 [5792170,113082] 0 2024-02-22T15:50:32.969 INFO:tasks.workunit.client.0.smithi019.stdout:7/492: creat d9/d3a/d49/d5d/d8f/fa3 x:0 0 0 2024-02-22T15:50:32.971 INFO:tasks.workunit.client.0.smithi019.stdout:2/349: link d1/d46/f5e d1/d17/d4b/f79 0 2024-02-22T15:50:32.976 INFO:tasks.workunit.client.0.smithi019.stdout:2/350: symlink d1/d17/d21/l7a 0 2024-02-22T15:50:32.979 INFO:tasks.workunit.client.0.smithi019.stdout:7/493: dwrite d9/d4c/d5b/f64 [0,4194304] 0 2024-02-22T15:50:32.980 INFO:tasks.workunit.client.0.smithi019.stdout:2/351: rename d1/f78 to d1/d17/d4b/f7b 0 2024-02-22T15:50:32.980 INFO:tasks.workunit.client.1.smithi062.stdout:3/97: rmdir d5/d16 39 2024-02-22T15:50:32.982 INFO:tasks.workunit.client.0.smithi019.stdout:2/352: dread - d1/d17/d1c/d66/f68 zero size 2024-02-22T15:50:32.983 INFO:tasks.workunit.client.0.smithi019.stdout:7/494: creat d9/d38/d41/d45/d61/fa4 x:0 0 0 2024-02-22T15:50:32.984 INFO:tasks.workunit.client.0.smithi019.stdout:2/353: creat d1/d17/d21/d51/f7c x:0 0 0 2024-02-22T15:50:32.989 INFO:tasks.workunit.client.0.smithi019.stdout:2/354: mknod d1/d17/d21/d71/d60/c7d 0 2024-02-22T15:50:32.990 INFO:tasks.workunit.client.1.smithi062.stdout:3/98: dwrite d5/f15 [4194304,4194304] 0 2024-02-22T15:50:32.991 INFO:tasks.workunit.client.0.smithi019.stdout:7/495: getdents d9/d3a 0 2024-02-22T15:50:32.994 INFO:tasks.workunit.client.1.smithi062.stdout:3/99: mkdir d5/d16/d21 0 2024-02-22T15:50:32.994 INFO:tasks.workunit.client.0.smithi019.stdout:2/355: mknod d1/de/c7e 0 2024-02-22T15:50:32.994 INFO:tasks.workunit.client.1.smithi062.stdout:3/100: dread - d5/d1a/f1e zero size 2024-02-22T15:50:32.995 INFO:tasks.workunit.client.1.smithi062.stdout:3/101: chown d5/l17 1 1 2024-02-22T15:50:32.998 INFO:tasks.workunit.client.1.smithi062.stdout:3/102: creat d5/d16/d21/f22 x:0 0 0 2024-02-22T15:50:32.999 INFO:tasks.workunit.client.1.smithi062.stdout:3/103: stat d5/d16/d21/f22 0 2024-02-22T15:50:33.002 INFO:tasks.workunit.client.1.smithi062.stdout:3/104: symlink d5/d16/l23 0 2024-02-22T15:50:33.004 INFO:tasks.workunit.client.0.smithi019.stdout:2/356: rmdir d1/d17/d21 39 2024-02-22T15:50:33.006 INFO:tasks.workunit.client.0.smithi019.stdout:2/357: mkdir d1/d17/d1c/d7f 0 2024-02-22T15:50:33.012 INFO:tasks.workunit.client.0.smithi019.stdout:2/358: dwrite d1/d17/f30 [0,4194304] 0 2024-02-22T15:50:33.014 INFO:tasks.workunit.client.0.smithi019.stdout:2/359: dread d1/d17/d4b/f4a [0,4194304] 0 2024-02-22T15:50:33.017 INFO:tasks.workunit.client.0.smithi019.stdout:2/360: write d1/f2e [3020045,33754] 0 2024-02-22T15:50:33.019 INFO:tasks.workunit.client.0.smithi019.stdout:2/361: stat d1/d17/d4b/c52 0 2024-02-22T15:50:33.023 INFO:tasks.workunit.client.1.smithi062.stdout:3/105: dread f4 [4194304,4194304] 0 2024-02-22T15:50:33.024 INFO:tasks.workunit.client.0.smithi019.stdout:2/362: rename d1/c2a to d1/de/d15/c80 0 2024-02-22T15:50:33.025 INFO:tasks.workunit.client.1.smithi062.stdout:3/106: fsync f4 0 2024-02-22T15:50:33.025 INFO:tasks.workunit.client.0.smithi019.stdout:2/363: chown d1/d46/f5e 2 1 2024-02-22T15:50:33.025 INFO:tasks.workunit.client.1.smithi062.stdout:3/107: dread - d5/d1a/f1b zero size 2024-02-22T15:50:33.026 INFO:tasks.workunit.client.0.smithi019.stdout:2/364: chown d1/d17/d1c/d66/f68 445 1 2024-02-22T15:50:33.028 INFO:tasks.workunit.client.0.smithi019.stdout:2/365: mknod d1/d17/d21/d71/c81 0 2024-02-22T15:50:33.032 INFO:tasks.workunit.client.1.smithi062.stdout:8/103: sync 2024-02-22T15:50:33.032 INFO:tasks.workunit.client.1.smithi062.stdout:6/158: sync 2024-02-22T15:50:33.033 INFO:tasks.workunit.client.0.smithi019.stdout:2/366: link d1/d17/d21/d71/l2b d1/l82 0 2024-02-22T15:50:33.033 INFO:tasks.workunit.client.1.smithi062.stdout:6/159: rmdir d5/dd 39 2024-02-22T15:50:33.034 INFO:tasks.workunit.client.1.smithi062.stdout:8/104: creat d8/d9/f1b x:0 0 0 2024-02-22T15:50:33.035 INFO:tasks.workunit.client.1.smithi062.stdout:6/160: write d5/df/f26 [2427305,81567] 0 2024-02-22T15:50:33.037 INFO:tasks.workunit.client.1.smithi062.stdout:6/161: write d5/df/f10 [3315286,52652] 0 2024-02-22T15:50:33.038 INFO:tasks.workunit.client.0.smithi019.stdout:2/367: write d1/d46/f5e [570952,16856] 0 2024-02-22T15:50:33.039 INFO:tasks.workunit.client.0.smithi019.stdout:2/368: fdatasync d1/d17/d21/d51/f77 0 2024-02-22T15:50:33.041 INFO:tasks.workunit.client.1.smithi062.stdout:6/162: fdatasync d5/df/d32/f34 0 2024-02-22T15:50:33.041 INFO:tasks.workunit.client.0.smithi019.stdout:2/369: creat d1/d17/d1c/d66/f83 x:0 0 0 2024-02-22T15:50:33.043 INFO:tasks.workunit.client.1.smithi062.stdout:6/163: creat d5/df/d32/f35 x:0 0 0 2024-02-22T15:50:33.043 INFO:tasks.workunit.client.1.smithi062.stdout:8/105: rmdir d8 39 2024-02-22T15:50:33.044 INFO:tasks.workunit.client.1.smithi062.stdout:3/108: dread f4 [0,4194304] 0 2024-02-22T15:50:33.046 INFO:tasks.workunit.client.0.smithi019.stdout:2/370: write d1/f14 [1011889,20451] 0 2024-02-22T15:50:33.046 INFO:tasks.workunit.client.1.smithi062.stdout:8/106: dread - d8/f17 zero size 2024-02-22T15:50:33.046 INFO:tasks.workunit.client.1.smithi062.stdout:6/164: dread d5/df/f17 [0,4194304] 0 2024-02-22T15:50:33.047 INFO:tasks.workunit.client.1.smithi062.stdout:8/107: write d8/d9/f18 [786053,61171] 0 2024-02-22T15:50:33.050 INFO:tasks.workunit.client.1.smithi062.stdout:8/108: read d8/d9/f18 [231005,117172] 0 2024-02-22T15:50:33.051 INFO:tasks.workunit.client.1.smithi062.stdout:6/165: rename d5/dd/f27 to d5/d15/f36 0 2024-02-22T15:50:33.055 INFO:tasks.workunit.client.0.smithi019.stdout:2/371: dwrite d1/de/f31 [4194304,4194304] 0 2024-02-22T15:50:33.057 INFO:tasks.workunit.client.1.smithi062.stdout:3/109: dwrite d5/f8 [0,4194304] 0 2024-02-22T15:50:33.057 INFO:tasks.workunit.client.0.smithi019.stdout:2/372: symlink d1/d17/d1c/l84 0 2024-02-22T15:50:33.058 INFO:tasks.workunit.client.1.smithi062.stdout:6/166: dread d5/f7 [4194304,4194304] 0 2024-02-22T15:50:33.060 INFO:tasks.workunit.client.1.smithi062.stdout:3/110: mknod d5/d16/c24 0 2024-02-22T15:50:33.061 INFO:tasks.workunit.client.1.smithi062.stdout:6/167: creat d5/d15/f37 x:0 0 0 2024-02-22T15:50:33.063 INFO:tasks.workunit.client.1.smithi062.stdout:6/168: write d5/df/f26 [3812687,76029] 0 2024-02-22T15:50:33.065 INFO:tasks.workunit.client.1.smithi062.stdout:6/169: dread - d5/dd/f13 zero size 2024-02-22T15:50:33.066 INFO:tasks.workunit.client.1.smithi062.stdout:6/170: chown d5/d15/f36 81 1 2024-02-22T15:50:33.069 INFO:tasks.workunit.client.1.smithi062.stdout:6/171: creat d5/f38 x:0 0 0 2024-02-22T15:50:33.080 INFO:tasks.workunit.client.1.smithi062.stdout:6/172: getdents d5/d15 0 2024-02-22T15:50:33.081 INFO:tasks.workunit.client.1.smithi062.stdout:6/173: write d5/f1f [1001667,119101] 0 2024-02-22T15:50:33.083 INFO:tasks.workunit.client.1.smithi062.stdout:6/174: symlink d5/df/d32/l39 0 2024-02-22T15:50:33.090 INFO:tasks.workunit.client.1.smithi062.stdout:6/175: dwrite d5/d19/f2d [4194304,4194304] 0 2024-02-22T15:50:33.090 INFO:tasks.workunit.client.1.smithi062.stdout:6/176: chown d5/df 12203 1 2024-02-22T15:50:33.091 INFO:tasks.workunit.client.1.smithi062.stdout:6/177: readlink d5/df/l14 0 2024-02-22T15:50:33.091 INFO:tasks.workunit.client.1.smithi062.stdout:6/178: dread - d5/f6 zero size 2024-02-22T15:50:33.100 INFO:tasks.workunit.client.1.smithi062.stdout:6/179: dwrite d5/f1a [0,4194304] 0 2024-02-22T15:50:33.101 INFO:tasks.workunit.client.1.smithi062.stdout:6/180: chown d5/dd/f20 131575307 1 2024-02-22T15:50:33.102 INFO:tasks.workunit.client.1.smithi062.stdout:6/181: read d5/d19/f2d [4754791,97160] 0 2024-02-22T15:50:33.107 INFO:tasks.workunit.client.1.smithi062.stdout:6/182: link d5/df/d32/l39 d5/df/d2a/l3a 0 2024-02-22T15:50:33.108 INFO:tasks.workunit.client.1.smithi062.stdout:6/183: read d5/f1a [1290716,70309] 0 2024-02-22T15:50:33.110 INFO:tasks.workunit.client.1.smithi062.stdout:6/184: creat d5/df/d2a/f3b x:0 0 0 2024-02-22T15:50:33.112 INFO:tasks.workunit.client.1.smithi062.stdout:6/185: symlink d5/d15/l3c 0 2024-02-22T15:50:33.114 INFO:tasks.workunit.client.1.smithi062.stdout:6/186: creat d5/df/d32/f3d x:0 0 0 2024-02-22T15:50:33.117 INFO:tasks.workunit.client.1.smithi062.stdout:6/187: getdents d5 0 2024-02-22T15:50:33.118 INFO:tasks.workunit.client.1.smithi062.stdout:6/188: chown d5/df/f10 4096389 1 2024-02-22T15:50:33.120 INFO:tasks.workunit.client.1.smithi062.stdout:6/189: symlink d5/df/l3e 0 2024-02-22T15:50:33.121 INFO:tasks.workunit.client.1.smithi062.stdout:6/190: write d5/df/f10 [3045353,66935] 0 2024-02-22T15:50:33.125 INFO:tasks.workunit.client.1.smithi062.stdout:6/191: getdents d5 0 2024-02-22T15:50:33.126 INFO:tasks.workunit.client.1.smithi062.stdout:6/192: readlink d5/df/d32/l39 0 2024-02-22T15:50:33.127 INFO:tasks.workunit.client.1.smithi062.stdout:6/193: truncate f2 4661549 0 2024-02-22T15:50:33.128 INFO:tasks.workunit.client.1.smithi062.stdout:6/194: write d5/df/f30 [1847719,15173] 0 2024-02-22T15:50:33.128 INFO:tasks.workunit.client.0.smithi019.stdout:5/401: sync 2024-02-22T15:50:33.128 INFO:tasks.workunit.client.0.smithi019.stdout:9/356: sync 2024-02-22T15:50:33.129 INFO:tasks.workunit.client.0.smithi019.stdout:0/231: sync 2024-02-22T15:50:33.129 INFO:tasks.workunit.client.0.smithi019.stdout:0/232: readlink d4/l12 0 2024-02-22T15:50:33.129 INFO:tasks.workunit.client.0.smithi019.stdout:9/357: chown d2/d1f/d2e/d48/d36 54864348 1 2024-02-22T15:50:33.130 INFO:tasks.workunit.client.1.smithi062.stdout:6/195: dread d5/df/f21 [0,4194304] 0 2024-02-22T15:50:33.131 INFO:tasks.workunit.client.0.smithi019.stdout:5/402: creat d8/d11/d21/f85 x:0 0 0 2024-02-22T15:50:33.133 INFO:tasks.workunit.client.0.smithi019.stdout:9/358: creat d2/d7/d14/f82 x:0 0 0 2024-02-22T15:50:33.133 INFO:tasks.workunit.client.1.smithi062.stdout:6/196: link d5/df/l14 d5/d19/l3f 0 2024-02-22T15:50:33.136 INFO:tasks.workunit.client.0.smithi019.stdout:0/233: link d4/d5/d7/c25 d4/d5/d52/c5a 0 2024-02-22T15:50:33.138 INFO:tasks.workunit.client.1.smithi062.stdout:6/197: truncate f2 874691 0 2024-02-22T15:50:33.140 INFO:tasks.workunit.client.0.smithi019.stdout:0/234: mkdir d4/d5/dc/d41/d5b 0 2024-02-22T15:50:33.141 INFO:tasks.workunit.client.1.smithi062.stdout:6/198: getdents d5/dd 0 2024-02-22T15:50:33.146 INFO:tasks.workunit.client.0.smithi019.stdout:0/235: rename d4/db/d1b/f55 to d4/f5c 0 2024-02-22T15:50:33.147 INFO:tasks.workunit.client.1.smithi062.stdout:6/199: dread d5/df/f30 [0,4194304] 0 2024-02-22T15:50:33.147 INFO:tasks.workunit.client.1.smithi062.stdout:6/200: chown f0 1164970401 1 2024-02-22T15:50:33.148 INFO:tasks.workunit.client.1.smithi062.stdout:6/201: stat d5/df/f21 0 2024-02-22T15:50:33.149 INFO:tasks.workunit.client.1.smithi062.stdout:6/202: write d5/df/d2a/f3b [51953,34483] 0 2024-02-22T15:50:33.152 INFO:tasks.workunit.client.0.smithi019.stdout:0/236: truncate d4/d9/f26 1003356 0 2024-02-22T15:50:33.154 INFO:tasks.workunit.client.1.smithi062.stdout:6/203: mkdir d5/df/d40 0 2024-02-22T15:50:33.154 INFO:tasks.workunit.client.0.smithi019.stdout:0/237: write d4/d5/d52/f58 [381005,62908] 0 2024-02-22T15:50:33.155 INFO:tasks.workunit.client.1.smithi062.stdout:6/204: truncate d5/df/d2a/f31 2225157 0 2024-02-22T15:50:33.155 INFO:tasks.workunit.client.0.smithi019.stdout:9/359: dwrite d2/d1f/d2e/f42 [0,4194304] 0 2024-02-22T15:50:33.156 INFO:tasks.workunit.client.1.smithi062.stdout:6/205: read f0 [2418814,34839] 0 2024-02-22T15:50:33.157 INFO:tasks.workunit.client.0.smithi019.stdout:9/360: dread - d2/d1f/d2e/d48/f4d zero size 2024-02-22T15:50:33.158 INFO:tasks.workunit.client.0.smithi019.stdout:9/361: dread - d2/d7/d14/f3f zero size 2024-02-22T15:50:33.160 INFO:tasks.workunit.client.1.smithi062.stdout:6/206: getdents d5/df/d2a 0 2024-02-22T15:50:33.161 INFO:tasks.workunit.client.0.smithi019.stdout:9/362: truncate d2/d1f/d2e/f7b 542625 0 2024-02-22T15:50:33.163 INFO:tasks.workunit.client.1.smithi062.stdout:9/110: sync 2024-02-22T15:50:33.163 INFO:tasks.workunit.client.1.smithi062.stdout:2/120: sync 2024-02-22T15:50:33.163 INFO:tasks.workunit.client.1.smithi062.stdout:7/163: sync 2024-02-22T15:50:33.163 INFO:tasks.workunit.client.1.smithi062.stdout:1/135: sync 2024-02-22T15:50:33.164 INFO:tasks.workunit.client.1.smithi062.stdout:0/166: sync 2024-02-22T15:50:33.164 INFO:tasks.workunit.client.0.smithi019.stdout:9/363: unlink d2/d1f/d2e/d48/f4d 0 2024-02-22T15:50:33.165 INFO:tasks.workunit.client.1.smithi062.stdout:6/207: dread d5/f1a [0,4194304] 0 2024-02-22T15:50:33.166 INFO:tasks.workunit.client.1.smithi062.stdout:1/136: read - d1/d6/de/d9/f25 zero size 2024-02-22T15:50:33.166 INFO:tasks.workunit.client.1.smithi062.stdout:9/111: creat d4/d15/f24 x:0 0 0 2024-02-22T15:50:33.167 INFO:tasks.workunit.client.1.smithi062.stdout:7/164: truncate d0/d14/f29 535963 0 2024-02-22T15:50:33.168 INFO:tasks.workunit.client.1.smithi062.stdout:6/208: mkdir d5/d19/d41 0 2024-02-22T15:50:33.169 INFO:tasks.workunit.client.1.smithi062.stdout:7/165: stat d0/d14/f17 0 2024-02-22T15:50:33.169 INFO:tasks.workunit.client.1.smithi062.stdout:0/167: chown d1/d3/d4/d1a/d1d/d1f/c2e 26165377 1 2024-02-22T15:50:33.170 INFO:tasks.workunit.client.1.smithi062.stdout:1/137: fdatasync d1/d6/d7/f28 0 2024-02-22T15:50:33.171 INFO:tasks.workunit.client.0.smithi019.stdout:9/364: dwrite d2/d7/dc/d27/d68/f6b [0,4194304] 0 2024-02-22T15:50:33.171 INFO:tasks.workunit.client.1.smithi062.stdout:7/166: readlink d0/l7 0 2024-02-22T15:50:33.172 INFO:tasks.workunit.client.0.smithi019.stdout:0/238: write d4/d22/d3d/f4f [957503,74333] 0 2024-02-22T15:50:33.175 INFO:tasks.workunit.client.1.smithi062.stdout:0/168: write d1/d3/de/d14/f26 [576951,98524] 0 2024-02-22T15:50:33.176 INFO:tasks.workunit.client.0.smithi019.stdout:9/365: creat d2/d1f/d2e/d48/d36/f83 x:0 0 0 2024-02-22T15:50:33.177 INFO:tasks.workunit.client.0.smithi019.stdout:0/239: dread d4/db/d1b/f2f [0,4194304] 0 2024-02-22T15:50:33.178 INFO:tasks.workunit.client.1.smithi062.stdout:2/121: dwrite d1/f5 [0,4194304] 0 2024-02-22T15:50:33.178 INFO:tasks.workunit.client.0.smithi019.stdout:0/240: write d4/d22/f51 [228803,93461] 0 2024-02-22T15:50:33.179 INFO:tasks.workunit.client.1.smithi062.stdout:1/138: truncate d1/d6/de/d9/f25 1015828 0 2024-02-22T15:50:33.181 INFO:tasks.workunit.client.1.smithi062.stdout:0/169: write d1/d13/f2f [249946,61354] 0 2024-02-22T15:50:33.182 INFO:tasks.workunit.client.1.smithi062.stdout:2/122: creat d1/d1e/f27 x:0 0 0 2024-02-22T15:50:33.183 INFO:tasks.workunit.client.0.smithi019.stdout:0/241: creat d4/db/d1b/d40/f5d x:0 0 0 2024-02-22T15:50:33.184 INFO:tasks.workunit.client.1.smithi062.stdout:7/167: rename d0/f5 to d0/d14/f38 0 2024-02-22T15:50:33.184 INFO:tasks.workunit.client.0.smithi019.stdout:9/366: dwrite d2/f6f [0,4194304] 0 2024-02-22T15:50:33.186 INFO:tasks.workunit.client.1.smithi062.stdout:1/139: mknod d1/d6/d7/d2b/c2d 0 2024-02-22T15:50:33.186 INFO:tasks.workunit.client.1.smithi062.stdout:9/112: dwrite d4/f7 [0,4194304] 0 2024-02-22T15:50:33.188 INFO:tasks.workunit.client.0.smithi019.stdout:0/242: rename d4/d9/c1e to d4/d22/d3d/c5e 0 2024-02-22T15:50:33.190 INFO:tasks.workunit.client.1.smithi062.stdout:0/170: write d1/d3/d4/d1a/f34 [912393,119567] 0 2024-02-22T15:50:33.191 INFO:tasks.workunit.client.1.smithi062.stdout:2/123: write d1/d8/f1c [695724,117815] 0 2024-02-22T15:50:33.191 INFO:tasks.workunit.client.1.smithi062.stdout:6/209: rename d5/df/d32/l39 to d5/dd/d33/l42 0 2024-02-22T15:50:33.194 INFO:tasks.workunit.client.1.smithi062.stdout:0/171: write d1/d3/d4/f23 [892087,72629] 0 2024-02-22T15:50:33.194 INFO:tasks.workunit.client.1.smithi062.stdout:2/124: mkdir d1/d28 0 2024-02-22T15:50:33.195 INFO:tasks.workunit.client.0.smithi019.stdout:9/367: rename d2/d7/dc/l23 to d2/l84 0 2024-02-22T15:50:33.195 INFO:tasks.workunit.client.1.smithi062.stdout:1/140: creat d1/f2e x:0 0 0 2024-02-22T15:50:33.200 INFO:tasks.workunit.client.1.smithi062.stdout:1/141: stat d1/d6/de/d9 0 2024-02-22T15:50:33.201 INFO:tasks.workunit.client.0.smithi019.stdout:9/368: rename d2/d7/dc/d27/d37/l6d to d2/d1f/d2e/d48/d36/l85 0 2024-02-22T15:50:33.204 INFO:tasks.workunit.client.0.smithi019.stdout:9/369: rename d2/d1f/l59 to d2/d7/l86 0 2024-02-22T15:50:33.205 INFO:tasks.workunit.client.1.smithi062.stdout:1/142: creat d1/d6/de/d21/f2f x:0 0 0 2024-02-22T15:50:33.205 INFO:tasks.workunit.client.0.smithi019.stdout:9/370: write d2/d7/d14/f82 [204218,4103] 0 2024-02-22T15:50:33.206 INFO:tasks.workunit.client.0.smithi019.stdout:9/371: chown d2/d1f/d2e/d48/f4e 22439 1 2024-02-22T15:50:33.206 INFO:tasks.workunit.client.0.smithi019.stdout:9/372: chown d2/c15 7 1 2024-02-22T15:50:33.207 INFO:tasks.workunit.client.1.smithi062.stdout:6/210: dwrite d5/f1e [4194304,4194304] 0 2024-02-22T15:50:33.207 INFO:tasks.workunit.client.1.smithi062.stdout:0/172: dread d1/d21/f24 [0,4194304] 0 2024-02-22T15:50:33.210 INFO:tasks.workunit.client.0.smithi019.stdout:9/373: rename d2/d7/d52/d70 to d2/d7/dc/d27/d87 0 2024-02-22T15:50:33.211 INFO:tasks.workunit.client.1.smithi062.stdout:7/168: write d0/d6/f23 [366471,37232] 0 2024-02-22T15:50:33.211 INFO:tasks.workunit.client.1.smithi062.stdout:0/173: write d1/d3/de/d14/f17 [670258,5355] 0 2024-02-22T15:50:33.213 INFO:tasks.workunit.client.1.smithi062.stdout:7/169: readlink d0/db/l15 0 2024-02-22T15:50:33.213 INFO:tasks.workunit.client.1.smithi062.stdout:6/211: mkdir d5/d19/d41/d43 0 2024-02-22T15:50:33.215 INFO:tasks.workunit.client.0.smithi019.stdout:0/243: truncate d4/d23/f28 2040886 0 2024-02-22T15:50:33.216 INFO:tasks.workunit.client.0.smithi019.stdout:9/374: rmdir d2/d1f/d2e/d57/d7e 0 2024-02-22T15:50:33.218 INFO:tasks.workunit.client.1.smithi062.stdout:1/143: dread d1/d6/d10/f1c [0,4194304] 0 2024-02-22T15:50:33.219 INFO:tasks.workunit.client.1.smithi062.stdout:2/125: truncate d1/d8/f11 1506594 0 2024-02-22T15:50:33.219 INFO:tasks.workunit.client.0.smithi019.stdout:0/244: rmdir d4/db/d24/d37 0 2024-02-22T15:50:33.220 INFO:tasks.workunit.client.1.smithi062.stdout:0/174: dread d1/d13/f2f [0,4194304] 0 2024-02-22T15:50:33.221 INFO:tasks.workunit.client.0.smithi019.stdout:9/375: truncate d2/d1f/d2e/d48/d36/f71 3576377 0 2024-02-22T15:50:33.221 INFO:tasks.workunit.client.1.smithi062.stdout:7/170: symlink d0/d14/d19/d30/d37/l39 0 2024-02-22T15:50:33.222 INFO:tasks.workunit.client.0.smithi019.stdout:0/245: creat d4/d22/f5f x:0 0 0 2024-02-22T15:50:33.224 INFO:tasks.workunit.client.1.smithi062.stdout:2/126: truncate d1/d1e/f27 153270 0 2024-02-22T15:50:33.227 INFO:tasks.workunit.client.1.smithi062.stdout:6/212: dwrite d5/df/d32/f3d [0,4194304] 0 2024-02-22T15:50:33.229 INFO:tasks.workunit.client.0.smithi019.stdout:0/246: mknod d4/db/d49/c60 0 2024-02-22T15:50:33.229 INFO:tasks.workunit.client.0.smithi019.stdout:9/376: dread d2/d1f/d2e/f42 [0,4194304] 0 2024-02-22T15:50:33.229 INFO:tasks.workunit.client.0.smithi019.stdout:9/377: fsync d2/d1f/f47 0 2024-02-22T15:50:33.229 INFO:tasks.workunit.client.0.smithi019.stdout:0/247: mknod d4/db/d1b/d40/c61 0 2024-02-22T15:50:33.229 INFO:tasks.workunit.client.0.smithi019.stdout:9/378: chown d2/d7/dc/d27/d68 602403404 1 2024-02-22T15:50:33.229 INFO:tasks.workunit.client.1.smithi062.stdout:7/171: creat d0/d6/d2e/d2f/f3a x:0 0 0 2024-02-22T15:50:33.231 INFO:tasks.workunit.client.1.smithi062.stdout:7/172: rename d0 to d0/db/d3b 22 2024-02-22T15:50:33.233 INFO:tasks.workunit.client.0.smithi019.stdout:9/379: mkdir d2/d1f/d2e/d88 0 2024-02-22T15:50:33.234 INFO:tasks.workunit.client.1.smithi062.stdout:2/127: unlink d1/d8/f11 0 2024-02-22T15:50:33.236 INFO:tasks.workunit.client.1.smithi062.stdout:6/213: mkdir d5/df/d40/d44 0 2024-02-22T15:50:33.237 INFO:tasks.workunit.client.1.smithi062.stdout:7/173: dread d0/f24 [0,4194304] 0 2024-02-22T15:50:33.237 INFO:tasks.workunit.client.0.smithi019.stdout:9/380: fdatasync d2/d1f/d2e/d48/d36/f83 0 2024-02-22T15:50:33.239 INFO:tasks.workunit.client.1.smithi062.stdout:2/128: chown d1/lc 23521 1 2024-02-22T15:50:33.239 INFO:tasks.workunit.client.1.smithi062.stdout:1/144: dwrite d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:33.239 INFO:tasks.workunit.client.0.smithi019.stdout:9/381: truncate d2/f19 4379751 0 2024-02-22T15:50:33.241 INFO:tasks.workunit.client.1.smithi062.stdout:0/175: dwrite d1/d21/f24 [0,4194304] 0 2024-02-22T15:50:33.241 INFO:tasks.workunit.client.0.smithi019.stdout:0/248: dwrite d4/d22/d3d/f50 [0,4194304] 0 2024-02-22T15:50:33.241 INFO:tasks.workunit.client.0.smithi019.stdout:9/382: fsync d2/d7/dc/d27/d68/f73 0 2024-02-22T15:50:33.242 INFO:tasks.workunit.client.1.smithi062.stdout:2/129: chown d1/lc 3182 1 2024-02-22T15:50:33.242 INFO:tasks.workunit.client.1.smithi062.stdout:0/176: read d1/d21/f24 [33768,2317] 0 2024-02-22T15:50:33.244 INFO:tasks.workunit.client.1.smithi062.stdout:2/130: read d1/d8/f1c [75954,19024] 0 2024-02-22T15:50:33.245 INFO:tasks.workunit.client.1.smithi062.stdout:2/131: stat d1 0 2024-02-22T15:50:33.245 INFO:tasks.workunit.client.0.smithi019.stdout:0/249: rmdir d4/d22 39 2024-02-22T15:50:33.245 INFO:tasks.workunit.client.1.smithi062.stdout:7/174: symlink d0/d14/d28/l3c 0 2024-02-22T15:50:33.246 INFO:tasks.workunit.client.1.smithi062.stdout:2/132: chown d1/c1d 1676800 1 2024-02-22T15:50:33.247 INFO:tasks.workunit.client.1.smithi062.stdout:7/175: dread - d0/db/f1f zero size 2024-02-22T15:50:33.247 INFO:tasks.workunit.client.1.smithi062.stdout:7/176: read d0/f10 [618457,86014] 0 2024-02-22T15:50:33.248 INFO:tasks.workunit.client.0.smithi019.stdout:9/383: rename d2/d7/dc/d27/d37 to d2/d1f/d2e/d48/d89 0 2024-02-22T15:50:33.248 INFO:tasks.workunit.client.1.smithi062.stdout:0/177: unlink d1/dd/f1c 0 2024-02-22T15:50:33.249 INFO:tasks.workunit.client.1.smithi062.stdout:1/145: creat d1/f30 x:0 0 0 2024-02-22T15:50:33.249 INFO:tasks.workunit.client.0.smithi019.stdout:0/250: creat d4/d5/d52/f62 x:0 0 0 2024-02-22T15:50:33.250 INFO:tasks.workunit.client.1.smithi062.stdout:7/177: chown d0/db/f34 89782 1 2024-02-22T15:50:33.251 INFO:tasks.workunit.client.1.smithi062.stdout:0/178: write d1/d3/de/d14/f17 [1457954,12470] 0 2024-02-22T15:50:33.251 INFO:tasks.workunit.client.1.smithi062.stdout:7/178: chown d0/cf 121149026 1 2024-02-22T15:50:33.254 INFO:tasks.workunit.client.1.smithi062.stdout:7/179: dread d0/d14/f29 [0,4194304] 0 2024-02-22T15:50:33.254 INFO:tasks.workunit.client.0.smithi019.stdout:9/384: mknod d2/d1f/d2e/d48/d36/d4a/c8a 0 2024-02-22T15:50:33.256 INFO:tasks.workunit.client.0.smithi019.stdout:0/251: dread d4/d22/f51 [0,4194304] 0 2024-02-22T15:50:33.257 INFO:tasks.workunit.client.1.smithi062.stdout:1/146: rmdir d1/d6/de/d9 39 2024-02-22T15:50:33.259 INFO:tasks.workunit.client.0.smithi019.stdout:9/385: readlink d2/d1f/d2e/d48/d45/l6e 0 2024-02-22T15:50:33.260 INFO:tasks.workunit.client.0.smithi019.stdout:0/252: chown d4/db/d1b/d40/c4d 20991154 1 2024-02-22T15:50:33.260 INFO:tasks.workunit.client.0.smithi019.stdout:9/386: chown d2/d7/dc/d27 48098830 1 2024-02-22T15:50:33.261 INFO:tasks.workunit.client.0.smithi019.stdout:0/253: chown d4/d5/d7/l54 123 1 2024-02-22T15:50:33.261 INFO:tasks.workunit.client.1.smithi062.stdout:0/179: write d1/d3/de/d14/f18 [2459038,27945] 0 2024-02-22T15:50:33.262 INFO:tasks.workunit.client.0.smithi019.stdout:0/254: chown d4/d5/dc/d41/d5b 21 1 2024-02-22T15:50:33.263 INFO:tasks.workunit.client.1.smithi062.stdout:2/133: link d1/d8/df/f14 d1/d1e/f29 0 2024-02-22T15:50:33.264 INFO:tasks.workunit.client.0.smithi019.stdout:0/255: unlink d4/d9/f17 0 2024-02-22T15:50:33.264 INFO:tasks.workunit.client.1.smithi062.stdout:6/214: getdents d5/d19 0 2024-02-22T15:50:33.266 INFO:tasks.workunit.client.0.smithi019.stdout:0/256: mkdir d4/db/d1b/d63 0 2024-02-22T15:50:33.266 INFO:tasks.workunit.client.0.smithi019.stdout:0/257: dread - d4/d5/f39 zero size 2024-02-22T15:50:33.267 INFO:tasks.workunit.client.1.smithi062.stdout:6/215: rmdir d5 39 2024-02-22T15:50:33.269 INFO:tasks.workunit.client.0.smithi019.stdout:0/258: unlink d4/db/c18 0 2024-02-22T15:50:33.270 INFO:tasks.workunit.client.0.smithi019.stdout:0/259: chown d4/d5/d7/l54 0 1 2024-02-22T15:50:33.271 INFO:tasks.workunit.client.1.smithi062.stdout:0/180: mkdir d1/d38 0 2024-02-22T15:50:33.271 INFO:tasks.workunit.client.1.smithi062.stdout:2/134: write d1/d8/f10 [33058,57308] 0 2024-02-22T15:50:33.273 INFO:tasks.workunit.client.0.smithi019.stdout:9/387: truncate d2/f6f 1288083 0 2024-02-22T15:50:33.274 INFO:tasks.workunit.client.1.smithi062.stdout:7/180: dwrite d0/db/f1f [0,4194304] 0 2024-02-22T15:50:33.277 INFO:tasks.workunit.client.0.smithi019.stdout:0/260: dwrite d4/f3e [0,4194304] 0 2024-02-22T15:50:33.277 INFO:tasks.workunit.client.1.smithi062.stdout:1/147: dwrite d1/d6/de/d21/f2f [0,4194304] 0 2024-02-22T15:50:33.277 INFO:tasks.workunit.client.1.smithi062.stdout:0/181: write d1/d3/d4/f23 [197802,74311] 0 2024-02-22T15:50:33.277 INFO:tasks.workunit.client.1.smithi062.stdout:2/135: truncate d1/d8/df/f26 269047 0 2024-02-22T15:50:33.278 INFO:tasks.workunit.client.1.smithi062.stdout:6/216: symlink d5/d19/d41/l45 0 2024-02-22T15:50:33.279 INFO:tasks.workunit.client.0.smithi019.stdout:0/261: symlink d4/d22/l64 0 2024-02-22T15:50:33.282 INFO:tasks.workunit.client.1.smithi062.stdout:2/136: write d1/d8/df/f25 [740208,70088] 0 2024-02-22T15:50:33.286 INFO:tasks.workunit.client.1.smithi062.stdout:0/182: creat d1/d3/f39 x:0 0 0 2024-02-22T15:50:33.286 INFO:tasks.workunit.client.1.smithi062.stdout:6/217: rmdir d5/dd/d33 39 2024-02-22T15:50:33.286 INFO:tasks.workunit.client.1.smithi062.stdout:7/181: mknod d0/c3d 0 2024-02-22T15:50:33.286 INFO:tasks.workunit.client.0.smithi019.stdout:9/388: link d2/d1f/d2e/d48/d89/l38 d2/d1f/d2e/d48/d36/l8b 0 2024-02-22T15:50:33.289 INFO:tasks.workunit.client.0.smithi019.stdout:0/262: rmdir d4/d23/d59 0 2024-02-22T15:50:33.289 INFO:tasks.workunit.client.1.smithi062.stdout:1/148: creat d1/d6/de/f31 x:0 0 0 2024-02-22T15:50:33.292 INFO:tasks.workunit.client.1.smithi062.stdout:2/137: truncate d1/f6 1906575 0 2024-02-22T15:50:33.292 INFO:tasks.workunit.client.1.smithi062.stdout:2/138: read - d1/d8/f1a zero size 2024-02-22T15:50:33.292 INFO:tasks.workunit.client.1.smithi062.stdout:7/182: creat d0/d14/d13/f3e x:0 0 0 2024-02-22T15:50:33.292 INFO:tasks.workunit.client.0.smithi019.stdout:0/263: getdents d4/d5/dc/d41/d5b 0 2024-02-22T15:50:33.292 INFO:tasks.workunit.client.1.smithi062.stdout:6/218: write d5/df/d2a/f31 [1061023,105843] 0 2024-02-22T15:50:33.293 INFO:tasks.workunit.client.1.smithi062.stdout:2/139: rename d1 to d1/d2a 22 2024-02-22T15:50:33.293 INFO:tasks.workunit.client.0.smithi019.stdout:9/389: dwrite d2/d7/d14/f82 [0,4194304] 0 2024-02-22T15:50:33.294 INFO:tasks.workunit.client.1.smithi062.stdout:7/183: dread - d0/d14/d13/f36 zero size 2024-02-22T15:50:33.295 INFO:tasks.workunit.client.1.smithi062.stdout:2/140: truncate d1/d1e/d20/f24 165563 0 2024-02-22T15:50:33.295 INFO:tasks.workunit.client.1.smithi062.stdout:0/183: link d1/d3/de/ff d1/d3/de/f3a 0 2024-02-22T15:50:33.296 INFO:tasks.workunit.client.1.smithi062.stdout:1/149: creat d1/f32 x:0 0 0 2024-02-22T15:50:33.301 INFO:tasks.workunit.client.1.smithi062.stdout:6/219: creat d5/dd/d33/f46 x:0 0 0 2024-02-22T15:50:33.302 INFO:tasks.workunit.client.1.smithi062.stdout:0/184: mkdir d1/d3/d4/d1a/d1d/d3b 0 2024-02-22T15:50:33.304 INFO:tasks.workunit.client.1.smithi062.stdout:2/141: dread d1/d8/df/f14 [0,4194304] 0 2024-02-22T15:50:33.306 INFO:tasks.workunit.client.1.smithi062.stdout:0/185: readlink d1/d3/l20 0 2024-02-22T15:50:33.307 INFO:tasks.workunit.client.1.smithi062.stdout:2/142: write d1/f5 [2815848,59543] 0 2024-02-22T15:50:33.308 INFO:tasks.workunit.client.1.smithi062.stdout:7/184: dwrite d0/db/f1f [0,4194304] 0 2024-02-22T15:50:33.309 INFO:tasks.workunit.client.1.smithi062.stdout:6/220: write d5/df/d32/f34 [1450826,126167] 0 2024-02-22T15:50:33.311 INFO:tasks.workunit.client.1.smithi062.stdout:7/185: fdatasync d0/d14/f17 0 2024-02-22T15:50:33.313 INFO:tasks.workunit.client.1.smithi062.stdout:6/221: mkdir d5/df/d32/d47 0 2024-02-22T15:50:33.315 INFO:tasks.workunit.client.1.smithi062.stdout:4/72: sync 2024-02-22T15:50:33.315 INFO:tasks.workunit.client.1.smithi062.stdout:5/71: sync 2024-02-22T15:50:33.315 INFO:tasks.workunit.client.1.smithi062.stdout:1/150: dwrite d1/d6/d10/f1c [4194304,4194304] 0 2024-02-22T15:50:33.317 INFO:tasks.workunit.client.1.smithi062.stdout:5/72: chown d0/f7 1669 1 2024-02-22T15:50:33.317 INFO:tasks.workunit.client.1.smithi062.stdout:1/151: chown d1/c2 48384 1 2024-02-22T15:50:33.321 INFO:tasks.workunit.client.1.smithi062.stdout:1/152: write d1/f2e [277965,81556] 0 2024-02-22T15:50:33.323 INFO:tasks.workunit.client.1.smithi062.stdout:7/186: mknod d0/d14/d19/c3f 0 2024-02-22T15:50:33.326 INFO:tasks.workunit.client.1.smithi062.stdout:0/186: link d1/d3/d4/l33 d1/d3/de/d14/l3c 0 2024-02-22T15:50:33.326 INFO:tasks.workunit.client.1.smithi062.stdout:7/187: read - d0/db/f34 zero size 2024-02-22T15:50:33.327 INFO:tasks.workunit.client.1.smithi062.stdout:6/222: dwrite d5/fb [0,4194304] 0 2024-02-22T15:50:33.328 INFO:tasks.workunit.client.1.smithi062.stdout:2/143: truncate d1/d8/fa 1197040 0 2024-02-22T15:50:33.328 INFO:tasks.workunit.client.1.smithi062.stdout:0/187: chown d1/d3/de/d14 1 1 2024-02-22T15:50:33.330 INFO:tasks.workunit.client.1.smithi062.stdout:1/153: creat d1/d6/de/d21/f33 x:0 0 0 2024-02-22T15:50:33.331 INFO:tasks.workunit.client.1.smithi062.stdout:5/73: write d0/d3/da/fb [688744,110397] 0 2024-02-22T15:50:33.331 INFO:tasks.workunit.client.1.smithi062.stdout:0/188: chown d1/d3/f6 256345 1 2024-02-22T15:50:33.332 INFO:tasks.workunit.client.1.smithi062.stdout:1/154: readlink d1/d6/l15 0 2024-02-22T15:50:33.333 INFO:tasks.workunit.client.1.smithi062.stdout:6/223: creat d5/dd/d33/f48 x:0 0 0 2024-02-22T15:50:33.335 INFO:tasks.workunit.client.1.smithi062.stdout:6/224: chown d5/dd/f20 0 1 2024-02-22T15:50:33.340 INFO:tasks.workunit.client.1.smithi062.stdout:1/155: read d1/d6/de/f17 [1451042,2975] 0 2024-02-22T15:50:33.340 INFO:tasks.workunit.client.1.smithi062.stdout:0/189: write d1/d21/f24 [1174381,20546] 0 2024-02-22T15:50:33.340 INFO:tasks.workunit.client.1.smithi062.stdout:5/74: getdents d0/d3 0 2024-02-22T15:50:33.340 INFO:tasks.workunit.client.1.smithi062.stdout:2/144: dread d1/d1e/f29 [0,4194304] 0 2024-02-22T15:50:33.340 INFO:tasks.workunit.client.1.smithi062.stdout:6/225: readlink d5/df/l14 0 2024-02-22T15:50:33.340 INFO:tasks.workunit.client.1.smithi062.stdout:5/75: fsync d0/f9 0 2024-02-22T15:50:33.341 INFO:tasks.workunit.client.1.smithi062.stdout:1/156: unlink d1/d6/d14/f20 0 2024-02-22T15:50:33.341 INFO:tasks.workunit.client.1.smithi062.stdout:6/226: dread - d5/dd/f13 zero size 2024-02-22T15:50:33.342 INFO:tasks.workunit.client.0.smithi019.stdout:8/351: sync 2024-02-22T15:50:33.345 INFO:tasks.workunit.client.1.smithi062.stdout:2/145: rename d1/d1e/f29 to d1/d1e/d20/f2b 0 2024-02-22T15:50:33.346 INFO:tasks.workunit.client.0.smithi019.stdout:8/352: chown d9/d14/d17/c54 74624976 1 2024-02-22T15:50:33.347 INFO:tasks.workunit.client.1.smithi062.stdout:6/227: creat d5/dd/d33/f49 x:0 0 0 2024-02-22T15:50:33.349 INFO:tasks.workunit.client.0.smithi019.stdout:8/353: chown d9/d14/d17/d3d 1 1 2024-02-22T15:50:33.352 INFO:tasks.workunit.client.0.smithi019.stdout:0/264: dwrite d4/db/d1b/f2f [0,4194304] 0 2024-02-22T15:50:33.353 INFO:tasks.workunit.client.0.smithi019.stdout:0/265: fdatasync d4/d22/f36 0 2024-02-22T15:50:33.355 INFO:tasks.workunit.client.1.smithi062.stdout:0/190: dwrite d1/d3/d4/d1a/d1d/d1f/f37 [0,4194304] 0 2024-02-22T15:50:33.356 INFO:tasks.workunit.client.1.smithi062.stdout:6/228: rename d5/d19 to d5/df/d40/d44/d4a 0 2024-02-22T15:50:33.357 INFO:tasks.workunit.client.0.smithi019.stdout:0/266: write d4/d22/f5f [394886,47419] 0 2024-02-22T15:50:33.357 INFO:tasks.workunit.client.1.smithi062.stdout:1/157: dwrite d1/d6/d7/f29 [0,4194304] 0 2024-02-22T15:50:33.357 INFO:tasks.workunit.client.0.smithi019.stdout:8/354: read d9/d14/d17/d1c/f1e [1057221,60539] 0 2024-02-22T15:50:33.359 INFO:tasks.workunit.client.0.smithi019.stdout:0/267: chown d4/d5/lf 2 1 2024-02-22T15:50:33.367 INFO:tasks.workunit.client.1.smithi062.stdout:6/229: truncate d5/dd/f20 626900 0 2024-02-22T15:50:33.368 INFO:tasks.workunit.client.0.smithi019.stdout:8/355: mknod d9/d14/d1f/d4c/c75 0 2024-02-22T15:50:33.369 INFO:tasks.workunit.client.0.smithi019.stdout:8/356: chown d9/d14/d17/l55 1 1 2024-02-22T15:50:33.369 INFO:tasks.workunit.client.1.smithi062.stdout:6/230: truncate d5/df/f10 4493850 0 2024-02-22T15:50:33.370 INFO:tasks.workunit.client.1.smithi062.stdout:5/76: fdatasync d0/d3/da/fb 0 2024-02-22T15:50:33.370 INFO:tasks.workunit.client.1.smithi062.stdout:4/73: truncate d4/f8 3569497 0 2024-02-22T15:50:33.374 INFO:tasks.workunit.client.0.smithi019.stdout:9/390: fsync d2/f19 0 2024-02-22T15:50:33.375 INFO:tasks.workunit.client.1.smithi062.stdout:6/231: dread d5/f1e [0,4194304] 0 2024-02-22T15:50:33.376 INFO:tasks.workunit.client.0.smithi019.stdout:0/268: mkdir d4/db/d49/d65 0 2024-02-22T15:50:33.376 INFO:tasks.workunit.client.1.smithi062.stdout:5/77: fsync d0/f16 0 2024-02-22T15:50:33.377 INFO:tasks.workunit.client.0.smithi019.stdout:8/357: rename d9/c15 to d9/d14/d1f/d4c/c76 0 2024-02-22T15:50:33.379 INFO:tasks.workunit.client.0.smithi019.stdout:0/269: write d4/db/f43 [922552,22442] 0 2024-02-22T15:50:33.380 INFO:tasks.workunit.client.0.smithi019.stdout:8/358: creat d9/d14/d17/d3d/f77 x:0 0 0 2024-02-22T15:50:33.381 INFO:tasks.workunit.client.0.smithi019.stdout:0/270: dread - d4/d23/f4a zero size 2024-02-22T15:50:33.381 INFO:tasks.workunit.client.0.smithi019.stdout:9/391: getdents d2/d7/d52 0 2024-02-22T15:50:33.382 INFO:tasks.workunit.client.1.smithi062.stdout:5/78: dread d0/d3/ff [0,4194304] 0 2024-02-22T15:50:33.382 INFO:tasks.workunit.client.1.smithi062.stdout:5/79: chown d0/d3/f14 1 1 2024-02-22T15:50:33.383 INFO:tasks.workunit.client.0.smithi019.stdout:0/271: symlink d4/d5/dc/l66 0 2024-02-22T15:50:33.385 INFO:tasks.workunit.client.1.smithi062.stdout:4/74: link l0 d4/df/l16 0 2024-02-22T15:50:33.385 INFO:tasks.workunit.client.0.smithi019.stdout:0/272: dread d4/d22/f51 [0,4194304] 0 2024-02-22T15:50:33.387 INFO:tasks.workunit.client.0.smithi019.stdout:9/392: rename d2/d1f/d2e/d48/d36/d4a to d2/d1f/d2e/d57/d60/d8c 0 2024-02-22T15:50:33.387 INFO:tasks.workunit.client.0.smithi019.stdout:8/359: dread d9/f32 [0,4194304] 0 2024-02-22T15:50:33.389 INFO:tasks.workunit.client.1.smithi062.stdout:1/158: getdents d1/d6/d10 0 2024-02-22T15:50:33.389 INFO:tasks.workunit.client.0.smithi019.stdout:0/273: mkdir d4/d67 0 2024-02-22T15:50:33.390 INFO:tasks.workunit.client.0.smithi019.stdout:0/274: fsync d4/db/d1b/f1f 0 2024-02-22T15:50:33.391 INFO:tasks.workunit.client.0.smithi019.stdout:9/393: mknod d2/d7/dc/c8d 0 2024-02-22T15:50:33.392 INFO:tasks.workunit.client.1.smithi062.stdout:2/146: dwrite d1/d8/df/f14 [0,4194304] 0 2024-02-22T15:50:33.393 INFO:tasks.workunit.client.0.smithi019.stdout:8/360: rename d9/d14/d17/d3d/f77 to d9/d14/d17/d3d/f78 0 2024-02-22T15:50:33.395 INFO:tasks.workunit.client.0.smithi019.stdout:0/275: mkdir d4/d22/d2c/d68 0 2024-02-22T15:50:33.396 INFO:tasks.workunit.client.1.smithi062.stdout:2/147: chown d1/d8/f1c 133075 1 2024-02-22T15:50:33.396 INFO:tasks.workunit.client.1.smithi062.stdout:1/159: mknod d1/d6/de/d9/c34 0 2024-02-22T15:50:33.396 INFO:tasks.workunit.client.1.smithi062.stdout:4/75: chown d4/f8 961 1 2024-02-22T15:50:33.397 INFO:tasks.workunit.client.0.smithi019.stdout:9/394: mknod d2/d1f/d2e/d48/d89/c8e 0 2024-02-22T15:50:33.399 INFO:tasks.workunit.client.1.smithi062.stdout:1/160: write d1/d6/de/f31 [74983,99091] 0 2024-02-22T15:50:33.400 INFO:tasks.workunit.client.1.smithi062.stdout:0/191: write d1/d3/de/f3a [604144,109349] 0 2024-02-22T15:50:33.402 INFO:tasks.workunit.client.0.smithi019.stdout:0/276: rename d4/d5/dc/c2a to d4/d5/d7/c69 0 2024-02-22T15:50:33.402 INFO:tasks.workunit.client.1.smithi062.stdout:5/80: write d0/f7 [3770518,112218] 0 2024-02-22T15:50:33.403 INFO:tasks.workunit.client.1.smithi062.stdout:6/232: truncate d5/df/d40/d44/d4a/f2d 3109110 0 2024-02-22T15:50:33.404 INFO:tasks.workunit.client.1.smithi062.stdout:4/76: mknod d4/df/c17 0 2024-02-22T15:50:33.404 INFO:tasks.workunit.client.1.smithi062.stdout:2/148: rmdir d1/d1e/d20 39 2024-02-22T15:50:33.407 INFO:tasks.workunit.client.0.smithi019.stdout:9/395: dread - d2/d1f/d2e/d57/d60/d8c/d66/f64 zero size 2024-02-22T15:50:33.407 INFO:tasks.workunit.client.0.smithi019.stdout:0/277: write d4/d9/f11 [624025,92005] 0 2024-02-22T15:50:33.407 INFO:tasks.workunit.client.0.smithi019.stdout:0/278: readlink d4/d5/l53 0 2024-02-22T15:50:33.408 INFO:tasks.workunit.client.1.smithi062.stdout:0/192: creat d1/d3/d4/d1a/f3d x:0 0 0 2024-02-22T15:50:33.412 INFO:tasks.workunit.client.0.smithi019.stdout:0/279: creat d4/d22/d2c/d68/f6a x:0 0 0 2024-02-22T15:50:33.412 INFO:tasks.workunit.client.1.smithi062.stdout:2/149: chown d1/d1e/c23 3 1 2024-02-22T15:50:33.413 INFO:tasks.workunit.client.1.smithi062.stdout:1/161: link d1/d6/de/d9/c34 d1/d6/de/d21/c35 0 2024-02-22T15:50:33.414 INFO:tasks.workunit.client.0.smithi019.stdout:9/396: dwrite d2/d7/dc/f24 [12582912,4194304] 0 2024-02-22T15:50:33.418 INFO:tasks.workunit.client.1.smithi062.stdout:2/150: creat d1/f2c x:0 0 0 2024-02-22T15:50:33.419 INFO:tasks.workunit.client.1.smithi062.stdout:5/81: getdents d0/d3/da 0 2024-02-22T15:50:33.422 INFO:tasks.workunit.client.1.smithi062.stdout:0/193: dwrite d1/dd/f36 [0,4194304] 0 2024-02-22T15:50:33.422 INFO:tasks.workunit.client.0.smithi019.stdout:9/397: symlink d2/d1f/d2e/d88/l8f 0 2024-02-22T15:50:33.422 INFO:tasks.workunit.client.0.smithi019.stdout:8/361: rmdir d9/d14/d17 39 2024-02-22T15:50:33.422 INFO:tasks.workunit.client.1.smithi062.stdout:1/162: mkdir d1/d6/d36 0 2024-02-22T15:50:33.423 INFO:tasks.workunit.client.0.smithi019.stdout:0/280: dwrite d4/f19 [0,4194304] 0 2024-02-22T15:50:33.424 INFO:tasks.workunit.client.1.smithi062.stdout:1/163: write d1/f2e [555312,77475] 0 2024-02-22T15:50:33.425 INFO:tasks.workunit.client.1.smithi062.stdout:0/194: write d1/d3/f39 [227112,22649] 0 2024-02-22T15:50:33.426 INFO:tasks.workunit.client.1.smithi062.stdout:4/77: fsync d4/f8 0 2024-02-22T15:50:33.427 INFO:tasks.workunit.client.0.smithi019.stdout:9/398: mknod d2/c90 0 2024-02-22T15:50:33.431 INFO:tasks.workunit.client.1.smithi062.stdout:5/82: symlink d0/l17 0 2024-02-22T15:50:33.431 INFO:tasks.workunit.client.1.smithi062.stdout:0/195: write d1/d3/f6 [156701,110848] 0 2024-02-22T15:50:33.433 INFO:tasks.workunit.client.1.smithi062.stdout:0/196: stat d1/dd/c25 0 2024-02-22T15:50:33.433 INFO:tasks.workunit.client.1.smithi062.stdout:4/78: symlink d4/d13/l18 0 2024-02-22T15:50:33.435 INFO:tasks.workunit.client.1.smithi062.stdout:2/151: fsync d1/f16 0 2024-02-22T15:50:33.435 INFO:tasks.workunit.client.0.smithi019.stdout:8/362: symlink d9/d14/d1f/d20/l79 0 2024-02-22T15:50:33.437 INFO:tasks.workunit.client.1.smithi062.stdout:1/164: mkdir d1/d6/d36/d37 0 2024-02-22T15:50:33.437 INFO:tasks.workunit.client.1.smithi062.stdout:5/83: creat d0/d3/da/f18 x:0 0 0 2024-02-22T15:50:33.437 INFO:tasks.workunit.client.0.smithi019.stdout:8/363: mknod d9/d14/c7a 0 2024-02-22T15:50:33.438 INFO:tasks.workunit.client.1.smithi062.stdout:6/233: dread f2 [0,4194304] 0 2024-02-22T15:50:33.439 INFO:tasks.workunit.client.1.smithi062.stdout:5/84: chown d0/d3/da/d13 886 1 2024-02-22T15:50:33.439 INFO:tasks.workunit.client.1.smithi062.stdout:0/197: mkdir d1/d21/d3e 0 2024-02-22T15:50:33.442 INFO:tasks.workunit.client.1.smithi062.stdout:4/79: dread f2 [0,4194304] 0 2024-02-22T15:50:33.443 INFO:tasks.workunit.client.1.smithi062.stdout:2/152: truncate d1/d8/fa 1258205 0 2024-02-22T15:50:33.444 INFO:tasks.workunit.client.1.smithi062.stdout:2/153: chown d1/f2c 30 1 2024-02-22T15:50:33.445 INFO:tasks.workunit.client.1.smithi062.stdout:6/234: rename d5/l18 to d5/df/l4b 0 2024-02-22T15:50:33.446 INFO:tasks.workunit.client.0.smithi019.stdout:8/364: dwrite d9/d14/d17/f50 [0,4194304] 0 2024-02-22T15:50:33.446 INFO:tasks.workunit.client.1.smithi062.stdout:4/80: mknod d4/df/c19 0 2024-02-22T15:50:33.448 INFO:tasks.workunit.client.0.smithi019.stdout:0/281: rmdir d4 39 2024-02-22T15:50:33.449 INFO:tasks.workunit.client.1.smithi062.stdout:4/81: readlink d4/lb 0 2024-02-22T15:50:33.450 INFO:tasks.workunit.client.1.smithi062.stdout:5/85: dwrite d0/f7 [4194304,4194304] 0 2024-02-22T15:50:33.450 INFO:tasks.workunit.client.1.smithi062.stdout:2/154: unlink d1/d8/df/c13 0 2024-02-22T15:50:33.451 INFO:tasks.workunit.client.1.smithi062.stdout:4/82: truncate d4/d13/f15 844963 0 2024-02-22T15:50:33.453 INFO:tasks.workunit.client.0.smithi019.stdout:9/399: write d2/f6 [4847434,24491] 0 2024-02-22T15:50:33.453 INFO:tasks.workunit.client.0.smithi019.stdout:0/282: write d4/d22/f51 [1285759,100044] 0 2024-02-22T15:50:33.454 INFO:tasks.workunit.client.1.smithi062.stdout:5/86: truncate d0/d3/da/f18 565444 0 2024-02-22T15:50:33.455 INFO:tasks.workunit.client.0.smithi019.stdout:8/365: dwrite d9/d14/d17/d3d/f59 [0,4194304] 0 2024-02-22T15:50:33.457 INFO:tasks.workunit.client.0.smithi019.stdout:9/400: read d2/d1f/f47 [345906,86762] 0 2024-02-22T15:50:33.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:33 smithi062 ceph-mon[48130]: pgmap v32: 65 pgs: 65 active+clean; 870 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 9.6 MiB/s rd, 81 MiB/s wr, 202 op/s 2024-02-22T15:50:33.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:33 smithi062 ceph-mon[48130]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:33.461 INFO:tasks.workunit.client.0.smithi019.stdout:8/366: readlink d9/d14/d17/d1c/l2b 0 2024-02-22T15:50:33.463 INFO:tasks.workunit.client.0.smithi019.stdout:0/283: getdents d4/d5 0 2024-02-22T15:50:33.464 INFO:tasks.workunit.client.1.smithi062.stdout:0/198: rename d1/d3/d4/d1a/d1d/d1f/d22 to d1/d3/de/d3f 0 2024-02-22T15:50:33.464 INFO:tasks.workunit.client.1.smithi062.stdout:2/155: dwrite d1/d1e/f27 [0,4194304] 0 2024-02-22T15:50:33.465 INFO:tasks.workunit.client.0.smithi019.stdout:8/367: symlink d9/d14/d17/d3d/l7b 0 2024-02-22T15:50:33.466 INFO:tasks.workunit.client.0.smithi019.stdout:8/368: write d9/d14/d17/d3d/f78 [627188,34741] 0 2024-02-22T15:50:33.467 INFO:tasks.workunit.client.0.smithi019.stdout:0/284: getdents d4/d5/d7 0 2024-02-22T15:50:33.468 INFO:tasks.workunit.client.0.smithi019.stdout:8/369: fsync f6 0 2024-02-22T15:50:33.468 INFO:tasks.workunit.client.1.smithi062.stdout:0/199: fdatasync d1/d3/f6 0 2024-02-22T15:50:33.469 INFO:tasks.workunit.client.0.smithi019.stdout:8/370: creat d9/d14/d1f/d6d/f7c x:0 0 0 2024-02-22T15:50:33.470 INFO:tasks.workunit.client.1.smithi062.stdout:6/235: fsync d5/df/d40/d44/d4a/f1c 0 2024-02-22T15:50:33.472 INFO:tasks.workunit.client.1.smithi062.stdout:4/83: link d4/d13/l18 d4/l1a 0 2024-02-22T15:50:33.472 INFO:tasks.workunit.client.1.smithi062.stdout:4/84: stat d4/df/c19 0 2024-02-22T15:50:33.473 INFO:tasks.workunit.client.1.smithi062.stdout:2/156: creat d1/d1e/f2d x:0 0 0 2024-02-22T15:50:33.474 INFO:tasks.workunit.client.1.smithi062.stdout:0/200: symlink d1/d3/de/d3f/l40 0 2024-02-22T15:50:33.475 INFO:tasks.workunit.client.1.smithi062.stdout:4/85: chown d4/l10 184 1 2024-02-22T15:50:33.475 INFO:tasks.workunit.client.1.smithi062.stdout:6/236: mknod d5/df/d32/c4c 0 2024-02-22T15:50:33.475 INFO:tasks.workunit.client.1.smithi062.stdout:2/157: read - d1/d8/f1a zero size 2024-02-22T15:50:33.477 INFO:tasks.workunit.client.1.smithi062.stdout:0/201: creat d1/d13/f41 x:0 0 0 2024-02-22T15:50:33.480 INFO:tasks.workunit.client.1.smithi062.stdout:6/237: creat d5/df/d40/d44/d4a/d41/f4d x:0 0 0 2024-02-22T15:50:33.482 INFO:tasks.workunit.client.1.smithi062.stdout:2/158: truncate d1/d8/f15 1100942 0 2024-02-22T15:50:33.485 INFO:tasks.workunit.client.1.smithi062.stdout:4/86: dread d4/df/f14 [0,4194304] 0 2024-02-22T15:50:33.488 INFO:tasks.workunit.client.1.smithi062.stdout:2/159: dread d1/d1e/d20/f24 [0,4194304] 0 2024-02-22T15:50:33.491 INFO:tasks.workunit.client.1.smithi062.stdout:0/202: dwrite d1/dd/f36 [0,4194304] 0 2024-02-22T15:50:33.491 INFO:tasks.workunit.client.1.smithi062.stdout:4/87: mkdir d4/df/d1b 0 2024-02-22T15:50:33.492 INFO:tasks.workunit.client.1.smithi062.stdout:0/203: write d1/d3/de/d14/f26 [1610692,52548] 0 2024-02-22T15:50:33.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:33 smithi019 ceph-mon[67418]: pgmap v32: 65 pgs: 65 active+clean; 870 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 9.6 MiB/s rd, 81 MiB/s wr, 202 op/s 2024-02-22T15:50:33.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:33 smithi019 ceph-mon[67418]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:50:33.496 INFO:tasks.workunit.client.1.smithi062.stdout:2/160: fdatasync d1/d8/f1c 0 2024-02-22T15:50:33.497 INFO:tasks.workunit.client.1.smithi062.stdout:6/238: dwrite d5/df/f17 [0,4194304] 0 2024-02-22T15:50:33.500 INFO:tasks.workunit.client.1.smithi062.stdout:4/88: readlink l0 0 2024-02-22T15:50:33.500 INFO:tasks.workunit.client.1.smithi062.stdout:6/239: readlink d5/df/l14 0 2024-02-22T15:50:33.501 INFO:tasks.workunit.client.1.smithi062.stdout:6/240: dread - d5/dd/d33/f46 zero size 2024-02-22T15:50:33.502 INFO:tasks.workunit.client.1.smithi062.stdout:0/204: mknod d1/d3/d4/d1a/d1d/c42 0 2024-02-22T15:50:33.503 INFO:tasks.workunit.client.1.smithi062.stdout:5/87: dread d0/f7 [0,4194304] 0 2024-02-22T15:50:33.507 INFO:tasks.workunit.client.1.smithi062.stdout:4/89: mkdir d4/d1c 0 2024-02-22T15:50:33.509 INFO:tasks.workunit.client.1.smithi062.stdout:6/241: creat d5/df/d32/f4e x:0 0 0 2024-02-22T15:50:33.512 INFO:tasks.workunit.client.1.smithi062.stdout:5/88: write d0/d3/f14 [643123,16310] 0 2024-02-22T15:50:33.513 INFO:tasks.workunit.client.1.smithi062.stdout:8/109: sync 2024-02-22T15:50:33.513 INFO:tasks.workunit.client.1.smithi062.stdout:3/111: sync 2024-02-22T15:50:33.514 INFO:tasks.workunit.client.1.smithi062.stdout:8/110: dread - d8/d9/f1b zero size 2024-02-22T15:50:33.515 INFO:tasks.workunit.client.1.smithi062.stdout:6/242: dread d5/fb [0,4194304] 0 2024-02-22T15:50:33.518 INFO:tasks.workunit.client.1.smithi062.stdout:3/112: creat d5/d1a/f25 x:0 0 0 2024-02-22T15:50:33.519 INFO:tasks.workunit.client.1.smithi062.stdout:3/113: write d5/f15 [4645032,123208] 0 2024-02-22T15:50:33.519 INFO:tasks.workunit.client.1.smithi062.stdout:5/89: truncate d0/d3/ff 2350496 0 2024-02-22T15:50:33.520 INFO:tasks.workunit.client.1.smithi062.stdout:4/90: symlink d4/d1c/l1d 0 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.0.smithi019.stdout:2/373: sync 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.0.smithi019.stdout:1/473: sync 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.0.smithi019.stdout:3/332: sync 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.0.smithi019.stdout:4/406: sync 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.0.smithi019.stdout:7/496: sync 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.0.smithi019.stdout:6/396: sync 2024-02-22T15:50:33.522 INFO:tasks.workunit.client.1.smithi062.stdout:8/111: rename d8/d9/l10 to d8/d9/l1c 0 2024-02-22T15:50:33.525 INFO:tasks.workunit.client.1.smithi062.stdout:2/161: dread d1/d8/df/f25 [0,4194304] 0 2024-02-22T15:50:33.526 INFO:tasks.workunit.client.0.smithi019.stdout:8/371: mkdir d9/d14/d17/d3d/d3f/d7d 0 2024-02-22T15:50:33.528 INFO:tasks.workunit.client.1.smithi062.stdout:6/243: symlink d5/df/d32/d47/l4f 0 2024-02-22T15:50:33.528 INFO:tasks.workunit.client.1.smithi062.stdout:0/205: dread d1/d3/d4/d1a/f34 [0,4194304] 0 2024-02-22T15:50:33.528 INFO:tasks.workunit.client.1.smithi062.stdout:5/90: dwrite d0/f9 [0,4194304] 0 2024-02-22T15:50:33.528 INFO:tasks.workunit.client.1.smithi062.stdout:4/91: chown d4/l1a 803676 1 2024-02-22T15:50:33.529 INFO:tasks.workunit.client.0.smithi019.stdout:6/397: symlink d6/d31/d48/l73 0 2024-02-22T15:50:33.529 INFO:tasks.workunit.client.0.smithi019.stdout:4/407: write d2/db/d14/d23/d21/f5a [494015,24254] 0 2024-02-22T15:50:33.529 INFO:tasks.workunit.client.0.smithi019.stdout:8/372: fsync d9/ff 0 2024-02-22T15:50:33.529 INFO:tasks.workunit.client.1.smithi062.stdout:2/162: write d1/d8/df/f14 [3665437,124547] 0 2024-02-22T15:50:33.530 INFO:tasks.workunit.client.1.smithi062.stdout:8/112: mknod d8/c1d 0 2024-02-22T15:50:33.531 INFO:tasks.workunit.client.1.smithi062.stdout:5/91: read d0/d3/f10 [3284820,108486] 0 2024-02-22T15:50:33.531 INFO:tasks.workunit.client.1.smithi062.stdout:2/163: read - d1/d8/f1a zero size 2024-02-22T15:50:33.533 INFO:tasks.workunit.client.1.smithi062.stdout:5/92: chown d0/d3/da/fb 502386 1 2024-02-22T15:50:33.533 INFO:tasks.workunit.client.1.smithi062.stdout:0/206: unlink d1/dd/c2d 0 2024-02-22T15:50:33.533 INFO:tasks.workunit.client.0.smithi019.stdout:4/408: dread - d2/db/f6a zero size 2024-02-22T15:50:33.534 INFO:tasks.workunit.client.0.smithi019.stdout:2/374: chown d1/de/d15/c80 28 1 2024-02-22T15:50:33.534 INFO:tasks.workunit.client.0.smithi019.stdout:0/285: dread d4/db/d1b/f1c [0,4194304] 0 2024-02-22T15:50:33.534 INFO:tasks.workunit.client.1.smithi062.stdout:4/92: symlink d4/df/l1e 0 2024-02-22T15:50:33.534 INFO:tasks.workunit.client.0.smithi019.stdout:3/333: creat d9/d26/d3f/d59/d66/f69 x:0 0 0 2024-02-22T15:50:33.534 INFO:tasks.workunit.client.0.smithi019.stdout:6/398: chown d6/l23 2 1 2024-02-22T15:50:33.539 INFO:tasks.workunit.client.0.smithi019.stdout:6/399: symlink d6/d31/d37/l74 0 2024-02-22T15:50:33.539 INFO:tasks.workunit.client.0.smithi019.stdout:4/409: readlink d2/db/d44/l60 0 2024-02-22T15:50:33.540 INFO:tasks.workunit.client.1.smithi062.stdout:5/93: dwrite d0/f9 [4194304,4194304] 0 2024-02-22T15:50:33.540 INFO:tasks.workunit.client.0.smithi019.stdout:0/286: fsync d4/d5/d52/f62 0 2024-02-22T15:50:33.541 INFO:tasks.workunit.client.0.smithi019.stdout:2/375: symlink d1/de/l85 0 2024-02-22T15:50:33.542 INFO:tasks.workunit.client.0.smithi019.stdout:4/410: readlink d2/db/d44/l60 0 2024-02-22T15:50:33.543 INFO:tasks.workunit.client.0.smithi019.stdout:6/400: symlink d6/d3e/l75 0 2024-02-22T15:50:33.544 INFO:tasks.workunit.client.0.smithi019.stdout:2/376: fsync d1/d17/d21/f41 0 2024-02-22T15:50:33.547 INFO:tasks.workunit.client.0.smithi019.stdout:3/334: creat d9/f6a x:0 0 0 2024-02-22T15:50:33.549 INFO:tasks.workunit.client.0.smithi019.stdout:7/497: dread d9/f1d [0,4194304] 0 2024-02-22T15:50:33.550 INFO:tasks.workunit.client.0.smithi019.stdout:4/411: rename d2/db/d14/d23/d33/c6f to d2/db/d54/c88 0 2024-02-22T15:50:33.551 INFO:tasks.workunit.client.0.smithi019.stdout:7/498: chown d9/d3a/l7f 66214768 1 2024-02-22T15:50:33.552 INFO:tasks.workunit.client.0.smithi019.stdout:2/377: mkdir d1/d86 0 2024-02-22T15:50:33.553 INFO:tasks.workunit.client.0.smithi019.stdout:3/335: dread - d9/d22/f4f zero size 2024-02-22T15:50:33.558 INFO:tasks.workunit.client.0.smithi019.stdout:3/336: write d9/d11/f4d [371685,82099] 0 2024-02-22T15:50:33.559 INFO:tasks.workunit.client.0.smithi019.stdout:2/378: creat d1/d17/d21/f87 x:0 0 0 2024-02-22T15:50:33.560 INFO:tasks.workunit.client.0.smithi019.stdout:1/474: dread d8/d2a/f2b [0,4194304] 0 2024-02-22T15:50:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:7/499: symlink d9/d38/la5 0 2024-02-22T15:50:33.561 INFO:tasks.workunit.client.0.smithi019.stdout:3/337: write d9/d26/f55 [132358,97155] 0 2024-02-22T15:50:33.562 INFO:tasks.workunit.client.0.smithi019.stdout:2/379: creat d1/d17/d1c/f88 x:0 0 0 2024-02-22T15:50:33.564 INFO:tasks.workunit.client.0.smithi019.stdout:3/338: chown c8 1900 1 2024-02-22T15:50:33.565 INFO:tasks.workunit.client.0.smithi019.stdout:2/380: chown d1/c56 0 1 2024-02-22T15:50:33.566 INFO:tasks.workunit.client.0.smithi019.stdout:7/500: creat d9/d38/d41/d45/d92/fa6 x:0 0 0 2024-02-22T15:50:33.569 INFO:tasks.workunit.client.0.smithi019.stdout:1/475: truncate d8/d23/f55 983159 0 2024-02-22T15:50:33.571 INFO:tasks.workunit.client.0.smithi019.stdout:7/501: mknod d9/d38/d41/d45/d92/da1/ca7 0 2024-02-22T15:50:33.571 INFO:tasks.workunit.client.0.smithi019.stdout:2/381: write d1/de/f4e [217878,124997] 0 2024-02-22T15:50:33.573 INFO:tasks.workunit.client.0.smithi019.stdout:1/476: symlink d8/d23/d33/laa 0 2024-02-22T15:50:33.574 INFO:tasks.workunit.client.1.smithi062.stdout:8/113: mknod d8/c1e 0 2024-02-22T15:50:33.575 INFO:tasks.workunit.client.0.smithi019.stdout:0/287: write f1 [1355642,127187] 0 2024-02-22T15:50:33.576 INFO:tasks.workunit.client.1.smithi062.stdout:2/164: read d1/d8/fa [1199101,30792] 0 2024-02-22T15:50:33.576 INFO:tasks.workunit.client.0.smithi019.stdout:4/412: write d2/db/d14/f18 [87191,94114] 0 2024-02-22T15:50:33.577 INFO:tasks.workunit.client.0.smithi019.stdout:2/382: mknod d1/de/d15/c89 0 2024-02-22T15:50:33.577 INFO:tasks.workunit.client.0.smithi019.stdout:8/373: dread d9/d14/d1f/f33 [0,4194304] 0 2024-02-22T15:50:33.577 INFO:tasks.workunit.client.1.smithi062.stdout:2/165: chown d1/d8/f1a 0 1 2024-02-22T15:50:33.577 INFO:tasks.workunit.client.1.smithi062.stdout:0/207: mkdir d1/d13/d43 0 2024-02-22T15:50:33.578 INFO:tasks.workunit.client.1.smithi062.stdout:2/166: chown d1/d8/f1c 1 1 2024-02-22T15:50:33.578 INFO:tasks.workunit.client.0.smithi019.stdout:3/339: dwrite d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:33.579 INFO:tasks.workunit.client.1.smithi062.stdout:0/208: stat d1/d3/de/d14/f2a 0 2024-02-22T15:50:33.579 INFO:tasks.workunit.client.0.smithi019.stdout:0/288: read - d4/d5/f39 zero size 2024-02-22T15:50:33.580 INFO:tasks.workunit.client.0.smithi019.stdout:1/477: symlink d8/d15/d34/d9e/lab 0 2024-02-22T15:50:33.581 INFO:tasks.workunit.client.0.smithi019.stdout:8/374: dread - d9/d14/f6a zero size 2024-02-22T15:50:33.581 INFO:tasks.workunit.client.0.smithi019.stdout:7/502: getdents d9/d3a 0 2024-02-22T15:50:33.582 INFO:tasks.workunit.client.1.smithi062.stdout:3/114: rmdir d5/d1a 39 2024-02-22T15:50:33.582 INFO:tasks.workunit.client.0.smithi019.stdout:0/289: chown d4/db/d1b/d40/d44 8360662 1 2024-02-22T15:50:33.582 INFO:tasks.workunit.client.0.smithi019.stdout:4/413: chown d2/db/d14/d23/d21/d35/f38 99 1 2024-02-22T15:50:33.583 INFO:tasks.workunit.client.0.smithi019.stdout:8/375: truncate d9/f69 569243 0 2024-02-22T15:50:33.583 INFO:tasks.workunit.client.0.smithi019.stdout:7/503: chown d9/d38/d41/f78 3783 1 2024-02-22T15:50:33.583 INFO:tasks.workunit.client.0.smithi019.stdout:2/383: mknod d1/d17/d21/c8a 0 2024-02-22T15:50:33.584 INFO:tasks.workunit.client.0.smithi019.stdout:0/290: write f2 [1738172,5695] 0 2024-02-22T15:50:33.586 INFO:tasks.workunit.client.0.smithi019.stdout:0/291: readlink d4/d5/l2e 0 2024-02-22T15:50:33.587 INFO:tasks.workunit.client.0.smithi019.stdout:3/340: creat d9/d26/d3f/d59/f6b x:0 0 0 2024-02-22T15:50:33.587 INFO:tasks.workunit.client.1.smithi062.stdout:4/93: getdents d4/df/d1b 0 2024-02-22T15:50:33.587 INFO:tasks.workunit.client.0.smithi019.stdout:4/414: rmdir d2/db/d44 39 2024-02-22T15:50:33.589 INFO:tasks.workunit.client.1.smithi062.stdout:2/167: mknod d1/c2e 0 2024-02-22T15:50:33.589 INFO:tasks.workunit.client.0.smithi019.stdout:8/376: read d9/d14/d1f/d20/f61 [100056,95007] 0 2024-02-22T15:50:33.589 INFO:tasks.workunit.client.1.smithi062.stdout:4/94: write d4/d13/f15 [1235448,68874] 0 2024-02-22T15:50:33.590 INFO:tasks.workunit.client.0.smithi019.stdout:7/504: readlink d9/d38/d41/l99 0 2024-02-22T15:50:33.591 INFO:tasks.workunit.client.0.smithi019.stdout:0/292: write d4/f3e [3513898,18180] 0 2024-02-22T15:50:33.593 INFO:tasks.workunit.client.1.smithi062.stdout:2/168: read - d1/d8/f1a zero size 2024-02-22T15:50:33.593 INFO:tasks.workunit.client.0.smithi019.stdout:3/341: creat d9/d26/d3f/f6c x:0 0 0 2024-02-22T15:50:33.594 INFO:tasks.workunit.client.1.smithi062.stdout:3/115: mkdir d5/d26 0 2024-02-22T15:50:33.594 INFO:tasks.workunit.client.1.smithi062.stdout:8/114: dwrite f0 [4194304,4194304] 0 2024-02-22T15:50:33.595 INFO:tasks.workunit.client.1.smithi062.stdout:4/95: mknod d4/d1c/c1f 0 2024-02-22T15:50:33.595 INFO:tasks.workunit.client.1.smithi062.stdout:2/169: dread - d1/d8/f1a zero size 2024-02-22T15:50:33.596 INFO:tasks.workunit.client.0.smithi019.stdout:3/342: stat d9/d22/f4f 0 2024-02-22T15:50:33.596 INFO:tasks.workunit.client.0.smithi019.stdout:8/377: truncate d9/d14/d1f/d6d/f7c 33388 0 2024-02-22T15:50:33.598 INFO:tasks.workunit.client.0.smithi019.stdout:2/384: dwrite d1/d17/d21/d51/f6a [0,4194304] 0 2024-02-22T15:50:33.599 INFO:tasks.workunit.client.1.smithi062.stdout:5/94: getdents d0/d3/da/d13 0 2024-02-22T15:50:33.599 INFO:tasks.workunit.client.0.smithi019.stdout:8/378: fdatasync d9/d14/d17/f28 0 2024-02-22T15:50:33.600 INFO:tasks.workunit.client.1.smithi062.stdout:3/116: dread - d5/d1a/f1b zero size 2024-02-22T15:50:33.600 INFO:tasks.workunit.client.0.smithi019.stdout:7/505: symlink d9/d38/d41/d45/d61/la8 0 2024-02-22T15:50:33.601 INFO:tasks.workunit.client.0.smithi019.stdout:0/293: rmdir d4/d5/dc/d41/d5b 0 2024-02-22T15:50:33.602 INFO:tasks.workunit.client.1.smithi062.stdout:5/95: write d0/d3/da/fb [4515772,33111] 0 2024-02-22T15:50:33.604 INFO:tasks.workunit.client.1.smithi062.stdout:4/96: rename d4/df/d1b to d4/d13/d20 0 2024-02-22T15:50:33.605 INFO:tasks.workunit.client.1.smithi062.stdout:2/170: dread d1/f6 [0,4194304] 0 2024-02-22T15:50:33.606 INFO:tasks.workunit.client.1.smithi062.stdout:8/115: creat d8/d9/f1f x:0 0 0 2024-02-22T15:50:33.608 INFO:tasks.workunit.client.0.smithi019.stdout:7/506: dread d9/d3a/f86 [0,4194304] 0 2024-02-22T15:50:33.608 INFO:tasks.workunit.client.1.smithi062.stdout:5/96: mknod d0/d3/da/c19 0 2024-02-22T15:50:33.610 INFO:tasks.workunit.client.1.smithi062.stdout:2/171: mknod d1/d8/df/c2f 0 2024-02-22T15:50:33.611 INFO:tasks.workunit.client.1.smithi062.stdout:5/97: rename d0 to d0/d3/da/d13/d1a 22 2024-02-22T15:50:33.612 INFO:tasks.workunit.client.0.smithi019.stdout:8/379: mkdir d9/d14/d6f/d7e 0 2024-02-22T15:50:33.612 INFO:tasks.workunit.client.1.smithi062.stdout:3/117: creat d5/d26/f27 x:0 0 0 2024-02-22T15:50:33.613 INFO:tasks.workunit.client.0.smithi019.stdout:7/507: fsync d9/d4c/d5b/f65 0 2024-02-22T15:50:33.614 INFO:tasks.workunit.client.1.smithi062.stdout:5/98: write d0/f9 [214897,12637] 0 2024-02-22T15:50:33.614 INFO:tasks.workunit.client.0.smithi019.stdout:0/294: dwrite d4/d22/f5f [0,4194304] 0 2024-02-22T15:50:33.616 INFO:tasks.workunit.client.1.smithi062.stdout:2/172: chown d1/d1e/d20 1 1 2024-02-22T15:50:33.616 INFO:tasks.workunit.client.1.smithi062.stdout:5/99: chown d0/f9 137579 1 2024-02-22T15:50:33.617 INFO:tasks.workunit.client.1.smithi062.stdout:3/118: symlink d5/d16/l28 0 2024-02-22T15:50:33.619 INFO:tasks.workunit.client.0.smithi019.stdout:4/415: write d2/db/f25 [550924,130937] 0 2024-02-22T15:50:33.619 INFO:tasks.workunit.client.1.smithi062.stdout:3/119: read f3 [3225979,15184] 0 2024-02-22T15:50:33.620 INFO:tasks.workunit.client.1.smithi062.stdout:8/116: dwrite d8/d9/f1b [0,4194304] 0 2024-02-22T15:50:33.620 INFO:tasks.workunit.client.1.smithi062.stdout:3/120: chown d5/d16 651 1 2024-02-22T15:50:33.621 INFO:tasks.workunit.client.1.smithi062.stdout:2/173: write d1/d1e/f27 [3587615,65440] 0 2024-02-22T15:50:33.622 INFO:tasks.workunit.client.0.smithi019.stdout:8/380: creat d9/d14/d1f/d4c/f7f x:0 0 0 2024-02-22T15:50:33.622 INFO:tasks.workunit.client.1.smithi062.stdout:5/100: creat d0/d3/da/d13/f1b x:0 0 0 2024-02-22T15:50:33.624 INFO:tasks.workunit.client.1.smithi062.stdout:8/117: creat d8/f20 x:0 0 0 2024-02-22T15:50:33.625 INFO:tasks.workunit.client.1.smithi062.stdout:2/174: chown d1/d8/df/f25 159 1 2024-02-22T15:50:33.626 INFO:tasks.workunit.client.1.smithi062.stdout:8/118: chown d8/cd 518548 1 2024-02-22T15:50:33.627 INFO:tasks.workunit.client.1.smithi062.stdout:5/101: mknod d0/d3/da/c1c 0 2024-02-22T15:50:33.628 INFO:tasks.workunit.client.0.smithi019.stdout:2/385: dwrite d1/de/f1e [4194304,4194304] 0 2024-02-22T15:50:33.630 INFO:tasks.workunit.client.0.smithi019.stdout:0/295: mknod d4/db/d1b/d40/c6b 0 2024-02-22T15:50:33.631 INFO:tasks.workunit.client.0.smithi019.stdout:3/343: write f4 [9268877,118272] 0 2024-02-22T15:50:33.631 INFO:tasks.workunit.client.0.smithi019.stdout:4/416: readlink d2/db/d14/d23/d33/d66/l70 0 2024-02-22T15:50:33.631 INFO:tasks.workunit.client.1.smithi062.stdout:3/121: dwrite d5/f10 [0,4194304] 0 2024-02-22T15:50:33.635 INFO:tasks.workunit.client.0.smithi019.stdout:8/381: creat d9/d14/d1f/f80 x:0 0 0 2024-02-22T15:50:33.636 INFO:tasks.workunit.client.0.smithi019.stdout:4/417: fdatasync d2/db/d14/d23/d4b/f82 0 2024-02-22T15:50:33.638 INFO:tasks.workunit.client.0.smithi019.stdout:3/344: mkdir d9/d22/d27/d6d 0 2024-02-22T15:50:33.640 INFO:tasks.workunit.client.0.smithi019.stdout:4/418: chown d2/db/d14/d23/d21/c50 3899984 1 2024-02-22T15:50:33.640 INFO:tasks.workunit.client.0.smithi019.stdout:2/386: dread d1/de/f31 [4194304,4194304] 0 2024-02-22T15:50:33.643 INFO:tasks.workunit.client.0.smithi019.stdout:7/508: truncate d9/d3a/f75 1764046 0 2024-02-22T15:50:33.644 INFO:tasks.workunit.client.0.smithi019.stdout:8/382: symlink d9/d14/d1f/d20/l81 0 2024-02-22T15:50:33.646 INFO:tasks.workunit.client.1.smithi062.stdout:4/97: dwrite f2 [0,4194304] 0 2024-02-22T15:50:33.647 INFO:tasks.workunit.client.1.smithi062.stdout:5/102: dwrite d0/f16 [4194304,4194304] 0 2024-02-22T15:50:33.647 INFO:tasks.workunit.client.0.smithi019.stdout:4/419: symlink d2/db/d14/d61/l89 0 2024-02-22T15:50:33.648 INFO:tasks.workunit.client.0.smithi019.stdout:3/345: creat d9/d26/d3f/d5c/f6e x:0 0 0 2024-02-22T15:50:33.649 INFO:tasks.workunit.client.0.smithi019.stdout:0/296: dwrite d4/d32/f46 [0,4194304] 0 2024-02-22T15:50:33.649 INFO:tasks.workunit.client.1.smithi062.stdout:2/175: getdents d1/d8 0 2024-02-22T15:50:33.649 INFO:tasks.workunit.client.0.smithi019.stdout:8/383: stat d9/d14/d17/d3d/c49 0 2024-02-22T15:50:33.650 INFO:tasks.workunit.client.1.smithi062.stdout:4/98: rename d4/f8 to d4/df/f21 0 2024-02-22T15:50:33.651 INFO:tasks.workunit.client.1.smithi062.stdout:2/176: rmdir d1/d1e 39 2024-02-22T15:50:33.656 INFO:tasks.workunit.client.0.smithi019.stdout:0/297: symlink d4/d9/l6c 0 2024-02-22T15:50:33.658 INFO:tasks.workunit.client.0.smithi019.stdout:2/387: dwrite d1/d46/f5e [0,4194304] 0 2024-02-22T15:50:33.659 INFO:tasks.workunit.client.1.smithi062.stdout:4/99: dread d4/d13/f15 [0,4194304] 0 2024-02-22T15:50:33.660 INFO:tasks.workunit.client.1.smithi062.stdout:4/100: fdatasync d4/fd 0 2024-02-22T15:50:33.661 INFO:tasks.workunit.client.0.smithi019.stdout:7/509: dread d9/f34 [0,4194304] 0 2024-02-22T15:50:33.661 INFO:tasks.workunit.client.1.smithi062.stdout:4/101: write d4/fd [1688682,42442] 0 2024-02-22T15:50:33.664 INFO:tasks.workunit.client.0.smithi019.stdout:2/388: unlink d1/d46/f55 0 2024-02-22T15:50:33.664 INFO:tasks.workunit.client.0.smithi019.stdout:5/403: sync 2024-02-22T15:50:33.665 INFO:tasks.workunit.client.0.smithi019.stdout:5/404: fdatasync d8/d11/d33/f76 0 2024-02-22T15:50:33.666 INFO:tasks.workunit.client.0.smithi019.stdout:7/510: chown d9/d3a/l7f 0 1 2024-02-22T15:50:33.670 INFO:tasks.workunit.client.0.smithi019.stdout:7/511: write d9/f1d [898340,129005] 0 2024-02-22T15:50:33.674 INFO:tasks.workunit.client.0.smithi019.stdout:5/405: unlink d8/d11/f6e 0 2024-02-22T15:50:33.674 INFO:tasks.workunit.client.0.smithi019.stdout:2/389: mknod d1/d17/d21/d71/d60/c8b 0 2024-02-22T15:50:33.675 INFO:tasks.workunit.client.0.smithi019.stdout:7/512: unlink d9/d38/f3e 0 2024-02-22T15:50:33.678 INFO:tasks.workunit.client.0.smithi019.stdout:2/390: creat d1/de/d15/f8c x:0 0 0 2024-02-22T15:50:33.680 INFO:tasks.workunit.client.0.smithi019.stdout:2/391: write d1/f14 [1256400,92419] 0 2024-02-22T15:50:33.680 INFO:tasks.workunit.client.0.smithi019.stdout:7/513: dread d9/f24 [0,4194304] 0 2024-02-22T15:50:33.683 INFO:tasks.workunit.client.0.smithi019.stdout:7/514: mknod d9/d3a/d49/d5d/d74/d9a/ca9 0 2024-02-22T15:50:33.684 INFO:tasks.workunit.client.0.smithi019.stdout:2/392: mknod d1/d17/d67/c8d 0 2024-02-22T15:50:33.686 INFO:tasks.workunit.client.0.smithi019.stdout:7/515: truncate d9/d3a/f86 1051215 0 2024-02-22T15:50:33.689 INFO:tasks.workunit.client.0.smithi019.stdout:8/384: getdents d9/d14/d1f 0 2024-02-22T15:50:33.693 INFO:tasks.workunit.client.0.smithi019.stdout:4/420: rmdir d2/db/d14 39 2024-02-22T15:50:33.697 INFO:tasks.workunit.client.0.smithi019.stdout:8/385: rename d9/d14/d17/c26 to d9/d14/d17/d3d/c82 0 2024-02-22T15:50:33.699 INFO:tasks.workunit.client.1.smithi062.stdout:7/188: sync 2024-02-22T15:50:33.699 INFO:tasks.workunit.client.1.smithi062.stdout:9/113: sync 2024-02-22T15:50:33.700 INFO:tasks.workunit.client.1.smithi062.stdout:7/189: write d0/d14/d13/fc [4012082,28394] 0 2024-02-22T15:50:33.701 INFO:tasks.workunit.client.0.smithi019.stdout:8/386: mkdir d9/d14/d17/d3d/d83 0 2024-02-22T15:50:33.701 INFO:tasks.workunit.client.0.smithi019.stdout:0/298: truncate d4/d32/f46 3142879 0 2024-02-22T15:50:33.702 INFO:tasks.workunit.client.0.smithi019.stdout:2/393: dread d1/f2e [0,4194304] 0 2024-02-22T15:50:33.703 INFO:tasks.workunit.client.1.smithi062.stdout:9/114: dread d4/f1b [0,4194304] 0 2024-02-22T15:50:33.703 INFO:tasks.workunit.client.0.smithi019.stdout:4/421: chown d2/db/d14/d23/d21/l22 55214168 1 2024-02-22T15:50:33.703 INFO:tasks.workunit.client.0.smithi019.stdout:0/299: dread - d4/d22/d2c/d68/f6a zero size 2024-02-22T15:50:33.705 INFO:tasks.workunit.client.0.smithi019.stdout:5/406: truncate d8/d11/f3e 1169603 0 2024-02-22T15:50:33.705 INFO:tasks.workunit.client.0.smithi019.stdout:0/300: fdatasync d4/d5/d52/f62 0 2024-02-22T15:50:33.711 INFO:tasks.workunit.client.0.smithi019.stdout:8/387: dwrite d9/d14/d17/f45 [0,4194304] 0 2024-02-22T15:50:33.711 INFO:tasks.workunit.client.1.smithi062.stdout:5/103: creat d0/d3/da/f1d x:0 0 0 2024-02-22T15:50:33.712 INFO:tasks.workunit.client.0.smithi019.stdout:7/516: unlink d9/d3a/f86 0 2024-02-22T15:50:33.712 INFO:tasks.workunit.client.1.smithi062.stdout:5/104: chown d0/f7 43114281 1 2024-02-22T15:50:33.713 INFO:tasks.workunit.client.0.smithi019.stdout:2/394: link d1/de/l85 d1/d17/d1c/l8e 0 2024-02-22T15:50:33.713 INFO:tasks.workunit.client.1.smithi062.stdout:5/105: readlink d0/d3/ld 0 2024-02-22T15:50:33.715 INFO:tasks.workunit.client.0.smithi019.stdout:0/301: mkdir d4/d22/d2c/d68/d6d 0 2024-02-22T15:50:33.718 INFO:tasks.workunit.client.0.smithi019.stdout:2/395: fsync d1/d17/d21/d51/f6a 0 2024-02-22T15:50:33.718 INFO:tasks.workunit.client.1.smithi062.stdout:8/119: write d8/d9/fe [589572,31447] 0 2024-02-22T15:50:33.719 INFO:tasks.workunit.client.1.smithi062.stdout:4/102: unlink d4/df/c17 0 2024-02-22T15:50:33.719 INFO:tasks.workunit.client.0.smithi019.stdout:8/388: symlink d9/d14/d1f/l84 0 2024-02-22T15:50:33.721 INFO:tasks.workunit.client.1.smithi062.stdout:3/122: truncate d5/f8 781690 0 2024-02-22T15:50:33.722 INFO:tasks.workunit.client.0.smithi019.stdout:5/407: dwrite d8/dd/d19/d3a/d46/f7b [0,4194304] 0 2024-02-22T15:50:33.722 INFO:tasks.workunit.client.1.smithi062.stdout:7/190: creat d0/d14/d13/f40 x:0 0 0 2024-02-22T15:50:33.723 INFO:tasks.workunit.client.1.smithi062.stdout:9/115: creat d4/d15/d20/f25 x:0 0 0 2024-02-22T15:50:33.723 INFO:tasks.workunit.client.1.smithi062.stdout:2/177: mknod d1/c30 0 2024-02-22T15:50:33.724 INFO:tasks.workunit.client.0.smithi019.stdout:7/517: truncate f6 1437464 0 2024-02-22T15:50:33.724 INFO:tasks.workunit.client.1.smithi062.stdout:9/116: dread - d4/d15/f17 zero size 2024-02-22T15:50:33.724 INFO:tasks.workunit.client.1.smithi062.stdout:7/191: fsync d0/d14/d13/fc 0 2024-02-22T15:50:33.725 INFO:tasks.workunit.client.1.smithi062.stdout:5/106: symlink d0/d3/da/l1e 0 2024-02-22T15:50:33.727 INFO:tasks.workunit.client.1.smithi062.stdout:8/120: creat d8/f21 x:0 0 0 2024-02-22T15:50:33.727 INFO:tasks.workunit.client.0.smithi019.stdout:0/302: symlink d4/db/d1b/d40/d44/l6e 0 2024-02-22T15:50:33.727 INFO:tasks.workunit.client.0.smithi019.stdout:2/396: mknod d1/d17/d38/c8f 0 2024-02-22T15:50:33.727 INFO:tasks.workunit.client.0.smithi019.stdout:7/518: write d9/d38/d41/f48 [954265,118555] 0 2024-02-22T15:50:33.728 INFO:tasks.workunit.client.0.smithi019.stdout:0/303: fsync d4/f3e 0 2024-02-22T15:50:33.729 INFO:tasks.workunit.client.1.smithi062.stdout:5/107: write d0/d3/da/fb [207393,64376] 0 2024-02-22T15:50:33.729 INFO:tasks.workunit.client.1.smithi062.stdout:3/123: creat d5/d16/d21/f29 x:0 0 0 2024-02-22T15:50:33.729 INFO:tasks.workunit.client.1.smithi062.stdout:7/192: chown d0/d14/f25 0 1 2024-02-22T15:50:33.730 INFO:tasks.workunit.client.0.smithi019.stdout:5/408: mkdir d8/dd/d19/d86 0 2024-02-22T15:50:33.733 INFO:tasks.workunit.client.1.smithi062.stdout:8/121: creat d8/f22 x:0 0 0 2024-02-22T15:50:33.733 INFO:tasks.workunit.client.1.smithi062.stdout:9/117: creat d4/f26 x:0 0 0 2024-02-22T15:50:33.735 INFO:tasks.workunit.client.1.smithi062.stdout:5/108: symlink d0/d3/da/d13/l1f 0 2024-02-22T15:50:33.735 INFO:tasks.workunit.client.1.smithi062.stdout:9/118: stat d4/d15/l16 0 2024-02-22T15:50:33.735 INFO:tasks.workunit.client.1.smithi062.stdout:8/122: dread - d8/f12 zero size 2024-02-22T15:50:33.737 INFO:tasks.workunit.client.0.smithi019.stdout:0/304: creat d4/d23/f6f x:0 0 0 2024-02-22T15:50:33.737 INFO:tasks.workunit.client.0.smithi019.stdout:7/519: dread d9/d38/d41/f90 [0,4194304] 0 2024-02-22T15:50:33.738 INFO:tasks.workunit.client.1.smithi062.stdout:7/193: creat d0/d14/d13/f41 x:0 0 0 2024-02-22T15:50:33.738 INFO:tasks.workunit.client.1.smithi062.stdout:2/178: link d1/f16 d1/d8/df/f31 0 2024-02-22T15:50:33.739 INFO:tasks.workunit.client.1.smithi062.stdout:9/119: dread - d4/d15/d20/f25 zero size 2024-02-22T15:50:33.739 INFO:tasks.workunit.client.0.smithi019.stdout:8/389: creat d9/d14/d17/f85 x:0 0 0 2024-02-22T15:50:33.740 INFO:tasks.workunit.client.1.smithi062.stdout:5/109: symlink d0/d3/da/l20 0 2024-02-22T15:50:33.740 INFO:tasks.workunit.client.1.smithi062.stdout:8/123: chown d8/d9/f1f 105 1 2024-02-22T15:50:33.743 INFO:tasks.workunit.client.1.smithi062.stdout:2/179: stat d1/d1e 0 2024-02-22T15:50:33.743 INFO:tasks.workunit.client.1.smithi062.stdout:7/194: fdatasync d0/d14/f20 0 2024-02-22T15:50:33.743 INFO:tasks.workunit.client.1.smithi062.stdout:8/124: mknod d8/c23 0 2024-02-22T15:50:33.745 INFO:tasks.workunit.client.1.smithi062.stdout:5/110: read - d0/d3/da/d13/f1b zero size 2024-02-22T15:50:33.746 INFO:tasks.workunit.client.1.smithi062.stdout:8/125: fdatasync d8/f12 0 2024-02-22T15:50:33.746 INFO:tasks.workunit.client.1.smithi062.stdout:3/124: dwrite d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:33.747 INFO:tasks.workunit.client.1.smithi062.stdout:9/120: creat d4/d1c/f27 x:0 0 0 2024-02-22T15:50:33.747 INFO:tasks.workunit.client.0.smithi019.stdout:0/305: creat d4/db/d1b/d63/f70 x:0 0 0 2024-02-22T15:50:33.747 INFO:tasks.workunit.client.1.smithi062.stdout:2/180: mkdir d1/d1e/d20/d32 0 2024-02-22T15:50:33.749 INFO:tasks.workunit.client.1.smithi062.stdout:8/126: write d8/d9/fe [2956778,48157] 0 2024-02-22T15:50:33.750 INFO:tasks.workunit.client.1.smithi062.stdout:7/195: creat d0/d14/d19/d30/f42 x:0 0 0 2024-02-22T15:50:33.750 INFO:tasks.workunit.client.0.smithi019.stdout:2/397: dread d1/d17/d21/d71/f2f [0,4194304] 0 2024-02-22T15:50:33.750 INFO:tasks.workunit.client.1.smithi062.stdout:3/125: rename d5/d16/d21 to d5/d16/d21/d2a 22 2024-02-22T15:50:33.752 INFO:tasks.workunit.client.0.smithi019.stdout:2/398: chown d1/d17/d21/l45 2275 1 2024-02-22T15:50:33.754 INFO:tasks.workunit.client.0.smithi019.stdout:7/520: link f8 d9/d38/faa 0 2024-02-22T15:50:33.755 INFO:tasks.workunit.client.1.smithi062.stdout:2/181: unlink d1/d8/f1c 0 2024-02-22T15:50:33.755 INFO:tasks.workunit.client.0.smithi019.stdout:5/409: getdents d8/d11/d21/d2c 0 2024-02-22T15:50:33.756 INFO:tasks.workunit.client.1.smithi062.stdout:8/127: fdatasync d8/d9/f18 0 2024-02-22T15:50:33.758 INFO:tasks.workunit.client.0.smithi019.stdout:8/390: getdents d9 0 2024-02-22T15:50:33.758 INFO:tasks.workunit.client.0.smithi019.stdout:7/521: stat d9/d38/d41/d45/f70 0 2024-02-22T15:50:33.758 INFO:tasks.workunit.client.1.smithi062.stdout:8/128: write d8/f21 [989844,116624] 0 2024-02-22T15:50:33.758 INFO:tasks.workunit.client.1.smithi062.stdout:7/196: mknod d0/d14/d19/d30/c43 0 2024-02-22T15:50:33.759 INFO:tasks.workunit.client.0.smithi019.stdout:5/410: chown d8/d11/c73 4403697 1 2024-02-22T15:50:33.760 INFO:tasks.workunit.client.1.smithi062.stdout:3/126: creat d5/d16/d21/f2b x:0 0 0 2024-02-22T15:50:33.760 INFO:tasks.workunit.client.1.smithi062.stdout:3/127: stat d5 0 2024-02-22T15:50:33.761 INFO:tasks.workunit.client.0.smithi019.stdout:0/306: dwrite d4/f56 [0,4194304] 0 2024-02-22T15:50:33.761 INFO:tasks.workunit.client.1.smithi062.stdout:2/182: mknod d1/d1e/d20/c33 0 2024-02-22T15:50:33.762 INFO:tasks.workunit.client.1.smithi062.stdout:2/183: readlink l0 0 2024-02-22T15:50:33.762 INFO:tasks.workunit.client.0.smithi019.stdout:5/411: read - d8/d11/f65 zero size 2024-02-22T15:50:33.762 INFO:tasks.workunit.client.0.smithi019.stdout:2/399: creat d1/d17/f90 x:0 0 0 2024-02-22T15:50:33.762 INFO:tasks.workunit.client.0.smithi019.stdout:7/522: dread d9/f34 [0,4194304] 0 2024-02-22T15:50:33.764 INFO:tasks.workunit.client.1.smithi062.stdout:8/129: unlink d8/f17 0 2024-02-22T15:50:33.765 INFO:tasks.workunit.client.0.smithi019.stdout:0/307: creat d4/d5/d52/f71 x:0 0 0 2024-02-22T15:50:33.766 INFO:tasks.workunit.client.1.smithi062.stdout:5/111: getdents d0/d3 0 2024-02-22T15:50:33.766 INFO:tasks.workunit.client.0.smithi019.stdout:8/391: chown d9/d14/d1f/d4c/l71 15174037 1 2024-02-22T15:50:33.767 INFO:tasks.workunit.client.0.smithi019.stdout:7/523: mknod d9/d79/cab 0 2024-02-22T15:50:33.768 INFO:tasks.workunit.client.1.smithi062.stdout:7/197: dread d0/d14/f29 [0,4194304] 0 2024-02-22T15:50:33.768 INFO:tasks.workunit.client.1.smithi062.stdout:3/128: write f3 [90952,31696] 0 2024-02-22T15:50:33.768 INFO:tasks.workunit.client.1.smithi062.stdout:4/103: truncate f2 2367560 0 2024-02-22T15:50:33.770 INFO:tasks.workunit.client.0.smithi019.stdout:0/308: creat d4/d5/d7/f72 x:0 0 0 2024-02-22T15:50:33.771 INFO:tasks.workunit.client.1.smithi062.stdout:8/130: creat d8/f24 x:0 0 0 2024-02-22T15:50:33.772 INFO:tasks.workunit.client.0.smithi019.stdout:0/309: stat d4/d23/f6f 0 2024-02-22T15:50:33.773 INFO:tasks.workunit.client.1.smithi062.stdout:7/198: creat d0/d14/d13/f44 x:0 0 0 2024-02-22T15:50:33.773 INFO:tasks.workunit.client.0.smithi019.stdout:2/400: dwrite d1/d17/d21/d51/f6a [0,4194304] 0 2024-02-22T15:50:33.773 INFO:tasks.workunit.client.0.smithi019.stdout:5/412: creat d8/d11/f87 x:0 0 0 2024-02-22T15:50:33.775 INFO:tasks.workunit.client.0.smithi019.stdout:7/524: symlink d9/d38/lac 0 2024-02-22T15:50:33.776 INFO:tasks.workunit.client.1.smithi062.stdout:4/104: rename d4/l10 to d4/d13/l22 0 2024-02-22T15:50:33.777 INFO:tasks.workunit.client.0.smithi019.stdout:8/392: mkdir d9/d14/d17/d86 0 2024-02-22T15:50:33.777 INFO:tasks.workunit.client.1.smithi062.stdout:2/184: symlink d1/l34 0 2024-02-22T15:50:33.777 INFO:tasks.workunit.client.0.smithi019.stdout:0/310: creat d4/d22/d3d/f73 x:0 0 0 2024-02-22T15:50:33.778 INFO:tasks.workunit.client.0.smithi019.stdout:2/401: symlink d1/d17/d1c/l91 0 2024-02-22T15:50:33.780 INFO:tasks.workunit.client.0.smithi019.stdout:5/413: mknod d8/dd/d19/d3a/d46/c88 0 2024-02-22T15:50:33.781 INFO:tasks.workunit.client.0.smithi019.stdout:0/311: write f1 [257307,17814] 0 2024-02-22T15:50:33.783 INFO:tasks.workunit.client.1.smithi062.stdout:7/199: mkdir d0/d14/d19/d30/d45 0 2024-02-22T15:50:33.784 INFO:tasks.workunit.client.0.smithi019.stdout:5/414: creat d8/f89 x:0 0 0 2024-02-22T15:50:33.784 INFO:tasks.workunit.client.0.smithi019.stdout:2/402: write d1/d17/d21/d51/f72 [2518788,74344] 0 2024-02-22T15:50:33.785 INFO:tasks.workunit.client.0.smithi019.stdout:0/312: mkdir d4/db/d24/d74 0 2024-02-22T15:50:33.785 INFO:tasks.workunit.client.1.smithi062.stdout:4/105: creat d4/d1c/f23 x:0 0 0 2024-02-22T15:50:33.787 INFO:tasks.workunit.client.0.smithi019.stdout:7/525: dwrite d9/d3a/d49/d5d/f9f [0,4194304] 0 2024-02-22T15:50:33.787 INFO:tasks.workunit.client.0.smithi019.stdout:0/313: chown d4/d22/f36 55911 1 2024-02-22T15:50:33.788 INFO:tasks.workunit.client.1.smithi062.stdout:8/131: dwrite d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:33.788 INFO:tasks.workunit.client.1.smithi062.stdout:2/185: rename d1/d8/f10 to d1/d28/f35 0 2024-02-22T15:50:33.788 INFO:tasks.workunit.client.1.smithi062.stdout:3/129: dwrite d5/d1a/f1b [0,4194304] 0 2024-02-22T15:50:33.790 INFO:tasks.workunit.client.0.smithi019.stdout:5/415: creat d8/d11/d21/f8a x:0 0 0 2024-02-22T15:50:33.791 INFO:tasks.workunit.client.1.smithi062.stdout:3/130: readlink l2 0 2024-02-22T15:50:33.792 INFO:tasks.workunit.client.1.smithi062.stdout:7/200: dread d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:33.794 INFO:tasks.workunit.client.0.smithi019.stdout:5/416: write d8/f89 [138053,124625] 0 2024-02-22T15:50:33.795 INFO:tasks.workunit.client.1.smithi062.stdout:2/186: symlink d1/l36 0 2024-02-22T15:50:33.795 INFO:tasks.workunit.client.1.smithi062.stdout:7/201: fdatasync d0/d14/d13/fc 0 2024-02-22T15:50:33.797 INFO:tasks.workunit.client.0.smithi019.stdout:0/314: mkdir d4/db/d1b/d40/d44/d75 0 2024-02-22T15:50:33.797 INFO:tasks.workunit.client.0.smithi019.stdout:2/403: rename d1/d17/d1c/f88 to d1/d17/d4b/f92 0 2024-02-22T15:50:33.797 INFO:tasks.workunit.client.0.smithi019.stdout:2/404: chown d1/d46/f5e 9005 1 2024-02-22T15:50:33.797 INFO:tasks.workunit.client.0.smithi019.stdout:8/393: dread d9/fa [0,4194304] 0 2024-02-22T15:50:33.799 INFO:tasks.workunit.client.0.smithi019.stdout:2/405: mknod d1/d17/d67/c93 0 2024-02-22T15:50:33.801 INFO:tasks.workunit.client.1.smithi062.stdout:8/132: read d8/d9/fe [78260,72626] 0 2024-02-22T15:50:33.801 INFO:tasks.workunit.client.0.smithi019.stdout:2/406: truncate d1/d17/d4b/f92 419127 0 2024-02-22T15:50:33.802 INFO:tasks.workunit.client.0.smithi019.stdout:0/315: mkdir d4/db/d76 0 2024-02-22T15:50:33.803 INFO:tasks.workunit.client.0.smithi019.stdout:5/417: symlink d8/d11/d21/l8b 0 2024-02-22T15:50:33.804 INFO:tasks.workunit.client.0.smithi019.stdout:0/316: write d4/f3e [4243083,73664] 0 2024-02-22T15:50:33.806 INFO:tasks.workunit.client.0.smithi019.stdout:8/394: rename d9/f69 to d9/d14/d1f/d5c/f87 0 2024-02-22T15:50:33.807 INFO:tasks.workunit.client.0.smithi019.stdout:2/407: write d1/d17/d1c/f34 [3210871,103141] 0 2024-02-22T15:50:33.808 INFO:tasks.workunit.client.1.smithi062.stdout:3/131: dread d5/f15 [4194304,4194304] 0 2024-02-22T15:50:33.809 INFO:tasks.workunit.client.0.smithi019.stdout:5/418: creat d8/dd/d19/d3a/d46/d7e/f8c x:0 0 0 2024-02-22T15:50:33.810 INFO:tasks.workunit.client.0.smithi019.stdout:5/419: fdatasync d8/dd/f75 0 2024-02-22T15:50:33.811 INFO:tasks.workunit.client.0.smithi019.stdout:2/408: creat d1/d46/f94 x:0 0 0 2024-02-22T15:50:33.811 INFO:tasks.workunit.client.0.smithi019.stdout:0/317: mkdir d4/db/d76/d77 0 2024-02-22T15:50:33.812 INFO:tasks.workunit.client.1.smithi062.stdout:8/133: dwrite d8/f12 [0,4194304] 0 2024-02-22T15:50:33.812 INFO:tasks.workunit.client.0.smithi019.stdout:5/420: read - d8/dd/f75 zero size 2024-02-22T15:50:33.813 INFO:tasks.workunit.client.1.smithi062.stdout:3/132: symlink d5/d16/l2c 0 2024-02-22T15:50:33.813 INFO:tasks.workunit.client.0.smithi019.stdout:2/409: rmdir d1/d17/d21/d71/d60 39 2024-02-22T15:50:33.814 INFO:tasks.workunit.client.0.smithi019.stdout:5/421: truncate d8/d11/f7d 531509 0 2024-02-22T15:50:33.814 INFO:tasks.workunit.client.0.smithi019.stdout:0/318: mkdir d4/d22/d78 0 2024-02-22T15:50:33.815 INFO:tasks.workunit.client.0.smithi019.stdout:2/410: creat d1/d46/f95 x:0 0 0 2024-02-22T15:50:33.815 INFO:tasks.workunit.client.0.smithi019.stdout:0/319: readlink d4/d5/l2e 0 2024-02-22T15:50:33.817 INFO:tasks.workunit.client.0.smithi019.stdout:0/320: chown d4/d22/d2c/f57 37970 1 2024-02-22T15:50:33.817 INFO:tasks.workunit.client.0.smithi019.stdout:5/422: mknod d8/dd/d19/d3a/d46/c8d 0 2024-02-22T15:50:33.818 INFO:tasks.workunit.client.0.smithi019.stdout:5/423: readlink d8/dd/d19/d77/l82 0 2024-02-22T15:50:33.822 INFO:tasks.workunit.client.0.smithi019.stdout:2/411: rmdir d1/d17/d4b/d70 0 2024-02-22T15:50:33.823 INFO:tasks.workunit.client.1.smithi062.stdout:8/134: dwrite d8/d9/f18 [0,4194304] 0 2024-02-22T15:50:33.823 INFO:tasks.workunit.client.1.smithi062.stdout:3/133: dwrite d5/d1a/f1e [0,4194304] 0 2024-02-22T15:50:33.823 INFO:tasks.workunit.client.0.smithi019.stdout:7/526: dread d9/d3a/f75 [0,4194304] 0 2024-02-22T15:50:33.826 INFO:tasks.workunit.client.1.smithi062.stdout:8/135: rmdir d8/d9 39 2024-02-22T15:50:33.827 INFO:tasks.workunit.client.1.smithi062.stdout:8/136: fsync d8/f24 0 2024-02-22T15:50:33.828 INFO:tasks.workunit.client.0.smithi019.stdout:2/412: creat d1/d17/d4b/f96 x:0 0 0 2024-02-22T15:50:33.828 INFO:tasks.workunit.client.0.smithi019.stdout:5/424: dread d8/dd/f5b [0,4194304] 0 2024-02-22T15:50:33.828 INFO:tasks.workunit.client.0.smithi019.stdout:0/321: dwrite d4/f5c [0,4194304] 0 2024-02-22T15:50:33.830 INFO:tasks.workunit.client.1.smithi062.stdout:3/134: dread d5/d1a/f1e [0,4194304] 0 2024-02-22T15:50:33.831 INFO:tasks.workunit.client.1.smithi062.stdout:3/135: dread - d5/d26/f27 zero size 2024-02-22T15:50:33.835 INFO:tasks.workunit.client.1.smithi062.stdout:8/137: rename d8/d9/cb to d8/d9/c25 0 2024-02-22T15:50:33.839 INFO:tasks.workunit.client.0.smithi019.stdout:7/527: dread d9/f1d [0,4194304] 0 2024-02-22T15:50:33.839 INFO:tasks.workunit.client.0.smithi019.stdout:7/528: read d9/f1d [782337,42306] 0 2024-02-22T15:50:33.839 INFO:tasks.workunit.client.0.smithi019.stdout:2/413: mknod d1/d17/d21/c97 0 2024-02-22T15:50:33.840 INFO:tasks.workunit.client.1.smithi062.stdout:8/138: mkdir d8/d26 0 2024-02-22T15:50:33.840 INFO:tasks.workunit.client.1.smithi062.stdout:8/139: chown c3 103 1 2024-02-22T15:50:33.840 INFO:tasks.workunit.client.1.smithi062.stdout:3/136: getdents d5/d26 0 2024-02-22T15:50:33.841 INFO:tasks.workunit.client.0.smithi019.stdout:7/529: write d9/fa [3566202,106435] 0 2024-02-22T15:50:33.843 INFO:tasks.workunit.client.0.smithi019.stdout:2/414: fsync d1/de/f16 0 2024-02-22T15:50:33.844 INFO:tasks.workunit.client.0.smithi019.stdout:5/425: rename d8/d11/d21 to d8/dd/d8e 0 2024-02-22T15:50:33.844 INFO:tasks.workunit.client.1.smithi062.stdout:3/137: mkdir d5/d26/d2d 0 2024-02-22T15:50:33.846 INFO:tasks.workunit.client.0.smithi019.stdout:2/415: truncate d1/d17/d21/f87 209241 0 2024-02-22T15:50:33.846 INFO:tasks.workunit.client.0.smithi019.stdout:9/401: sync 2024-02-22T15:50:33.847 INFO:tasks.workunit.client.1.smithi062.stdout:3/138: write d5/d1a/f1e [3485577,92350] 0 2024-02-22T15:50:33.849 INFO:tasks.workunit.client.0.smithi019.stdout:7/530: mknod d9/d3a/d49/d5d/d74/d9a/cad 0 2024-02-22T15:50:33.850 INFO:tasks.workunit.client.1.smithi062.stdout:3/139: read d5/f1d [594453,59617] 0 2024-02-22T15:50:33.850 INFO:tasks.workunit.client.1.smithi062.stdout:2/187: dwrite d1/d1e/d20/f24 [0,4194304] 0 2024-02-22T15:50:33.851 INFO:tasks.workunit.client.1.smithi062.stdout:3/140: chown d5/f6 19464 1 2024-02-22T15:50:33.852 INFO:tasks.workunit.client.0.smithi019.stdout:9/402: chown d2/d1f/d2e/d48/d36/f71 0 1 2024-02-22T15:50:33.852 INFO:tasks.workunit.client.1.smithi062.stdout:3/141: write d5/d16/d21/f29 [255548,60085] 0 2024-02-22T15:50:33.854 INFO:tasks.workunit.client.1.smithi062.stdout:2/188: dread d1/f6 [0,4194304] 0 2024-02-22T15:50:33.855 INFO:tasks.workunit.client.1.smithi062.stdout:2/189: chown d1/d8/df/c21 1670 1 2024-02-22T15:50:33.855 INFO:tasks.workunit.client.0.smithi019.stdout:8/395: chown d9/d14/d1f/d4c/l5b 654540 1 2024-02-22T15:50:33.856 INFO:tasks.workunit.client.0.smithi019.stdout:5/426: dwrite d8/dd/d19/d3a/d46/f53 [0,4194304] 0 2024-02-22T15:50:33.860 INFO:tasks.workunit.client.1.smithi062.stdout:2/190: getdents d1/d8/df 0 2024-02-22T15:50:33.861 INFO:tasks.workunit.client.1.smithi062.stdout:2/191: readlink d1/d8/l9 0 2024-02-22T15:50:33.861 INFO:tasks.workunit.client.1.smithi062.stdout:8/140: stat d8/d9/c25 0 2024-02-22T15:50:33.861 INFO:tasks.workunit.client.0.smithi019.stdout:9/403: mknod d2/d1f/d2e/d48/d45/c91 0 2024-02-22T15:50:33.862 INFO:tasks.workunit.client.0.smithi019.stdout:2/416: dwrite d1/d17/d21/d71/f2f [4194304,4194304] 0 2024-02-22T15:50:33.862 INFO:tasks.workunit.client.1.smithi062.stdout:2/192: dread - d1/d1e/f2d zero size 2024-02-22T15:50:33.863 INFO:tasks.workunit.client.0.smithi019.stdout:2/417: fdatasync d1/de/d15/f58 0 2024-02-22T15:50:33.863 INFO:tasks.workunit.client.0.smithi019.stdout:5/427: creat d8/dd/f8f x:0 0 0 2024-02-22T15:50:33.864 INFO:tasks.workunit.client.0.smithi019.stdout:9/404: chown d2/d1f/d2e/d48/d45/d5a/f72 3 1 2024-02-22T15:50:33.866 INFO:tasks.workunit.client.1.smithi062.stdout:2/193: mknod d1/d28/c37 0 2024-02-22T15:50:33.866 INFO:tasks.workunit.client.1.smithi062.stdout:8/141: creat d8/d9/f27 x:0 0 0 2024-02-22T15:50:33.867 INFO:tasks.workunit.client.0.smithi019.stdout:5/428: read - d8/dd/d19/d3a/d46/d7e/f8c zero size 2024-02-22T15:50:33.868 INFO:tasks.workunit.client.0.smithi019.stdout:2/418: truncate d1/d17/d21/d51/f77 266437 0 2024-02-22T15:50:33.869 INFO:tasks.workunit.client.0.smithi019.stdout:8/396: fdatasync d9/d14/d1f/f33 0 2024-02-22T15:50:33.870 INFO:tasks.workunit.client.1.smithi062.stdout:2/194: mkdir d1/d1e/d38 0 2024-02-22T15:50:33.871 INFO:tasks.workunit.client.1.smithi062.stdout:2/195: readlink d1/d8/l9 0 2024-02-22T15:50:33.872 INFO:tasks.workunit.client.0.smithi019.stdout:5/429: mknod d8/dd/d19/c90 0 2024-02-22T15:50:33.872 INFO:tasks.workunit.client.0.smithi019.stdout:2/419: creat d1/d46/f98 x:0 0 0 2024-02-22T15:50:33.874 INFO:tasks.workunit.client.0.smithi019.stdout:8/397: mkdir d9/d88 0 2024-02-22T15:50:33.875 INFO:tasks.workunit.client.1.smithi062.stdout:2/196: mkdir d1/d1e/d20/d39 0 2024-02-22T15:50:33.875 INFO:tasks.workunit.client.1.smithi062.stdout:8/142: dwrite d8/f24 [0,4194304] 0 2024-02-22T15:50:33.875 INFO:tasks.workunit.client.0.smithi019.stdout:2/420: unlink d1/de/d15/f58 0 2024-02-22T15:50:33.877 INFO:tasks.workunit.client.1.smithi062.stdout:8/143: write d8/d9/f18 [4107506,83128] 0 2024-02-22T15:50:33.878 INFO:tasks.workunit.client.0.smithi019.stdout:7/531: dread d9/d26/f32 [0,4194304] 0 2024-02-22T15:50:33.879 INFO:tasks.workunit.client.1.smithi062.stdout:1/165: sync 2024-02-22T15:50:33.880 INFO:tasks.workunit.client.1.smithi062.stdout:2/197: rename d1/d8/l12 to d1/d1e/d20/d32/l3a 0 2024-02-22T15:50:33.881 INFO:tasks.workunit.client.1.smithi062.stdout:2/198: write d1/f2c [858562,57935] 0 2024-02-22T15:50:33.883 INFO:tasks.workunit.client.1.smithi062.stdout:8/144: mkdir d8/d28 0 2024-02-22T15:50:33.884 INFO:tasks.workunit.client.0.smithi019.stdout:7/532: mknod d9/d38/d41/cae 0 2024-02-22T15:50:33.885 INFO:tasks.workunit.client.1.smithi062.stdout:8/145: dread - d8/f22 zero size 2024-02-22T15:50:33.886 INFO:tasks.workunit.client.1.smithi062.stdout:2/199: rename d1/c1d to d1/d1e/d20/c3b 0 2024-02-22T15:50:33.886 INFO:tasks.workunit.client.0.smithi019.stdout:2/421: dwrite d1/d17/d21/f41 [0,4194304] 0 2024-02-22T15:50:33.887 INFO:tasks.workunit.client.1.smithi062.stdout:2/200: read d1/d8/fa [704236,20906] 0 2024-02-22T15:50:33.887 INFO:tasks.workunit.client.1.smithi062.stdout:1/166: getdents d1/d6/d14 0 2024-02-22T15:50:33.889 INFO:tasks.workunit.client.0.smithi019.stdout:8/398: rename d9/d14/d1f/f4e to d9/d14/f89 0 2024-02-22T15:50:33.890 INFO:tasks.workunit.client.1.smithi062.stdout:1/167: unlink d1/c2 0 2024-02-22T15:50:33.891 INFO:tasks.workunit.client.0.smithi019.stdout:2/422: fdatasync d1/f2e 0 2024-02-22T15:50:33.893 INFO:tasks.workunit.client.0.smithi019.stdout:7/533: dread d9/d38/d41/f78 [0,4194304] 0 2024-02-22T15:50:33.893 INFO:tasks.workunit.client.0.smithi019.stdout:0/322: truncate f1 462459 0 2024-02-22T15:50:33.894 INFO:tasks.workunit.client.1.smithi062.stdout:8/146: creat d8/f29 x:0 0 0 2024-02-22T15:50:33.897 INFO:tasks.workunit.client.0.smithi019.stdout:7/534: symlink d9/d3a/d49/d5d/d74/laf 0 2024-02-22T15:50:33.898 INFO:tasks.workunit.client.1.smithi062.stdout:2/201: read d1/d8/df/f31 [2220220,46230] 0 2024-02-22T15:50:33.901 INFO:tasks.workunit.client.0.smithi019.stdout:7/535: chown d9/d26/c35 0 1 2024-02-22T15:50:33.902 INFO:tasks.workunit.client.1.smithi062.stdout:2/202: dread d1/d8/df/f25 [0,4194304] 0 2024-02-22T15:50:33.903 INFO:tasks.workunit.client.1.smithi062.stdout:1/168: getdents d1/d6/de/d21/d24 0 2024-02-22T15:50:33.904 INFO:tasks.workunit.client.0.smithi019.stdout:5/430: truncate d8/dd/d19/d3a/d46/f53 3222911 0 2024-02-22T15:50:33.905 INFO:tasks.workunit.client.0.smithi019.stdout:8/399: dwrite d9/d14/d1f/f41 [4194304,4194304] 0 2024-02-22T15:50:33.905 INFO:tasks.workunit.client.1.smithi062.stdout:8/147: dwrite d8/f21 [0,4194304] 0 2024-02-22T15:50:33.906 INFO:tasks.workunit.client.0.smithi019.stdout:7/536: mkdir d9/d38/d41/d45/db0 0 2024-02-22T15:50:33.906 INFO:tasks.workunit.client.1.smithi062.stdout:1/169: creat d1/d6/f38 x:0 0 0 2024-02-22T15:50:33.907 INFO:tasks.workunit.client.1.smithi062.stdout:8/148: dread - d8/d9/f27 zero size 2024-02-22T15:50:33.908 INFO:tasks.workunit.client.0.smithi019.stdout:5/431: dread - d8/dd/d8e/d2c/f5d zero size 2024-02-22T15:50:33.909 INFO:tasks.workunit.client.0.smithi019.stdout:5/432: truncate d8/f23 866367 0 2024-02-22T15:50:33.910 INFO:tasks.workunit.client.0.smithi019.stdout:9/405: dread d2/f4f [0,4194304] 0 2024-02-22T15:50:33.910 INFO:tasks.workunit.client.0.smithi019.stdout:7/537: rmdir d9/d4c 39 2024-02-22T15:50:33.911 INFO:tasks.workunit.client.0.smithi019.stdout:5/433: dread - d8/dd/d19/d3a/d46/d7e/f8c zero size 2024-02-22T15:50:33.914 INFO:tasks.workunit.client.0.smithi019.stdout:9/406: write d2/d1f/d2e/d48/d36/f83 [875188,114708] 0 2024-02-22T15:50:33.914 INFO:tasks.workunit.client.1.smithi062.stdout:2/203: getdents d1/d1e/d20 0 2024-02-22T15:50:33.914 INFO:tasks.workunit.client.1.smithi062.stdout:1/170: dread d1/d6/de/d9/f25 [0,4194304] 0 2024-02-22T15:50:33.917 INFO:tasks.workunit.client.1.smithi062.stdout:2/204: unlink d1/l36 0 2024-02-22T15:50:33.918 INFO:tasks.workunit.client.0.smithi019.stdout:5/434: creat d8/dd/d19/d77/f91 x:0 0 0 2024-02-22T15:50:33.918 INFO:tasks.workunit.client.1.smithi062.stdout:1/171: symlink d1/d6/d36/d37/l39 0 2024-02-22T15:50:33.919 INFO:tasks.workunit.client.0.smithi019.stdout:0/323: dread d4/d9/f11 [0,4194304] 0 2024-02-22T15:50:33.920 INFO:tasks.workunit.client.1.smithi062.stdout:1/172: write d1/d6/de/d21/f33 [292920,69728] 0 2024-02-22T15:50:33.922 INFO:tasks.workunit.client.0.smithi019.stdout:0/324: truncate d4/d22/d2c/d68/f6a 129298 0 2024-02-22T15:50:33.923 INFO:tasks.workunit.client.0.smithi019.stdout:9/407: creat d2/d1f/d2e/d57/d60/f92 x:0 0 0 2024-02-22T15:50:33.925 INFO:tasks.workunit.client.1.smithi062.stdout:1/173: rename d1/d6/f38 to d1/d6/d7/f3a 0 2024-02-22T15:50:33.925 INFO:tasks.workunit.client.0.smithi019.stdout:0/325: dread d4/d9/f11 [0,4194304] 0 2024-02-22T15:50:33.926 INFO:tasks.workunit.client.0.smithi019.stdout:9/408: mknod d2/d1f/d2e/d48/c93 0 2024-02-22T15:50:33.926 INFO:tasks.workunit.client.1.smithi062.stdout:1/174: mknod d1/d6/de/d9/c3b 0 2024-02-22T15:50:33.927 INFO:tasks.workunit.client.1.smithi062.stdout:1/175: readlink d1/d6/l23 0 2024-02-22T15:50:33.927 INFO:tasks.workunit.client.1.smithi062.stdout:1/176: stat d1/d6/d14 0 2024-02-22T15:50:33.930 INFO:tasks.workunit.client.1.smithi062.stdout:8/149: truncate d8/f24 492687 0 2024-02-22T15:50:33.930 INFO:tasks.workunit.client.0.smithi019.stdout:5/435: dread d8/f23 [0,4194304] 0 2024-02-22T15:50:33.931 INFO:tasks.workunit.client.0.smithi019.stdout:7/538: write d9/d26/f32 [292304,76462] 0 2024-02-22T15:50:33.932 INFO:tasks.workunit.client.0.smithi019.stdout:0/326: dread d4/d32/f46 [0,4194304] 0 2024-02-22T15:50:33.933 INFO:tasks.workunit.client.0.smithi019.stdout:7/539: chown d9/d3a/d49/d5d/d74 6551 1 2024-02-22T15:50:33.933 INFO:tasks.workunit.client.1.smithi062.stdout:2/205: write d1/f16 [243659,108336] 0 2024-02-22T15:50:33.936 INFO:tasks.workunit.client.0.smithi019.stdout:7/540: write d9/d26/f32 [3542601,122653] 0 2024-02-22T15:50:33.938 INFO:tasks.workunit.client.1.smithi062.stdout:8/150: dwrite d8/d9/f27 [0,4194304] 0 2024-02-22T15:50:33.940 INFO:tasks.workunit.client.0.smithi019.stdout:7/541: fsync d9/f24 0 2024-02-22T15:50:33.941 INFO:tasks.workunit.client.0.smithi019.stdout:5/436: link d8/dd/d19/c90 d8/dd/d19/d3a/c92 0 2024-02-22T15:50:33.944 INFO:tasks.workunit.client.1.smithi062.stdout:2/206: dwrite d1/d28/f35 [0,4194304] 0 2024-02-22T15:50:33.944 INFO:tasks.workunit.client.0.smithi019.stdout:5/437: truncate d8/f4e 188263 0 2024-02-22T15:50:33.944 INFO:tasks.workunit.client.0.smithi019.stdout:7/542: write d9/d26/d77/f9b [86417,97902] 0 2024-02-22T15:50:33.947 INFO:tasks.workunit.client.0.smithi019.stdout:8/400: dread d9/d14/f36 [0,4194304] 0 2024-02-22T15:50:33.948 INFO:tasks.workunit.client.1.smithi062.stdout:8/151: dwrite d8/d9/f27 [0,4194304] 0 2024-02-22T15:50:33.950 INFO:tasks.workunit.client.1.smithi062.stdout:8/152: creat d8/d9/f2a x:0 0 0 2024-02-22T15:50:33.951 INFO:tasks.workunit.client.1.smithi062.stdout:8/153: chown d8/f20 2 1 2024-02-22T15:50:33.951 INFO:tasks.workunit.client.1.smithi062.stdout:8/154: dread - d8/f20 zero size 2024-02-22T15:50:33.951 INFO:tasks.workunit.client.1.smithi062.stdout:8/155: dread - d8/f29 zero size 2024-02-22T15:50:33.954 INFO:tasks.workunit.client.0.smithi019.stdout:7/543: unlink d9/d3a/l4f 0 2024-02-22T15:50:33.955 INFO:tasks.workunit.client.0.smithi019.stdout:0/327: dwrite d4/d9/f26 [0,4194304] 0 2024-02-22T15:50:33.955 INFO:tasks.workunit.client.1.smithi062.stdout:8/156: dread d8/f21 [0,4194304] 0 2024-02-22T15:50:33.957 INFO:tasks.workunit.client.0.smithi019.stdout:8/401: read d9/d14/d17/d1c/f24 [2239355,3793] 0 2024-02-22T15:50:33.957 INFO:tasks.workunit.client.0.smithi019.stdout:5/438: unlink d8/dd/d19/d3a/f43 0 2024-02-22T15:50:33.962 INFO:tasks.workunit.client.0.smithi019.stdout:0/328: dread f2 [0,4194304] 0 2024-02-22T15:50:33.962 INFO:tasks.workunit.client.1.smithi062.stdout:8/157: dwrite d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:33.964 INFO:tasks.workunit.client.0.smithi019.stdout:9/409: dread d2/d7/f1b [4194304,4194304] 0 2024-02-22T15:50:33.965 INFO:tasks.workunit.client.0.smithi019.stdout:8/402: fsync d9/d14/f89 0 2024-02-22T15:50:33.967 INFO:tasks.workunit.client.0.smithi019.stdout:9/410: creat d2/d1f/d2e/d57/f94 x:0 0 0 2024-02-22T15:50:33.968 INFO:tasks.workunit.client.0.smithi019.stdout:9/411: chown d2/c1e 658 1 2024-02-22T15:50:33.970 INFO:tasks.workunit.client.0.smithi019.stdout:8/403: dread d9/f32 [0,4194304] 0 2024-02-22T15:50:33.971 INFO:tasks.workunit.client.0.smithi019.stdout:5/439: creat d8/d11/f93 x:0 0 0 2024-02-22T15:50:33.972 INFO:tasks.workunit.client.0.smithi019.stdout:9/412: mkdir d2/d1f/d95 0 2024-02-22T15:50:33.973 INFO:tasks.workunit.client.0.smithi019.stdout:8/404: rmdir d9/d14/d1f 39 2024-02-22T15:50:33.973 INFO:tasks.workunit.client.0.smithi019.stdout:7/544: creat d9/d38/d41/d45/d61/fb1 x:0 0 0 2024-02-22T15:50:33.974 INFO:tasks.workunit.client.0.smithi019.stdout:5/440: mknod d8/d11/d33/c94 0 2024-02-22T15:50:33.975 INFO:tasks.workunit.client.0.smithi019.stdout:9/413: fdatasync d2/d7/f1b 0 2024-02-22T15:50:33.975 INFO:tasks.workunit.client.0.smithi019.stdout:5/441: readlink d8/d11/l1e 0 2024-02-22T15:50:33.979 INFO:tasks.workunit.client.0.smithi019.stdout:5/442: creat d8/dd/d19/d38/d54/f95 x:0 0 0 2024-02-22T15:50:33.980 INFO:tasks.workunit.client.0.smithi019.stdout:8/405: rename d9/d14/d17/d3d/c82 to d9/d14/d6f/d7e/c8a 0 2024-02-22T15:50:33.984 INFO:tasks.workunit.client.0.smithi019.stdout:5/443: write d8/dd/f5b [164902,131024] 0 2024-02-22T15:50:33.985 INFO:tasks.workunit.client.0.smithi019.stdout:8/406: stat d9/d14/d1f/f21 0 2024-02-22T15:50:33.987 INFO:tasks.workunit.client.0.smithi019.stdout:7/545: dwrite d9/d38/d41/f90 [4194304,4194304] 0 2024-02-22T15:50:33.987 INFO:tasks.workunit.client.0.smithi019.stdout:9/414: dwrite d2/fb [0,4194304] 0 2024-02-22T15:50:33.990 INFO:tasks.workunit.client.0.smithi019.stdout:5/444: fdatasync d8/dd/d19/d3a/f5e 0 2024-02-22T15:50:33.991 INFO:tasks.workunit.client.1.smithi062.stdout:8/158: truncate d8/d9/f18 3249550 0 2024-02-22T15:50:33.994 INFO:tasks.workunit.client.1.smithi062.stdout:8/159: symlink d8/d9/l2b 0 2024-02-22T15:50:33.994 INFO:tasks.workunit.client.1.smithi062.stdout:2/207: dread d1/f16 [0,4194304] 0 2024-02-22T15:50:33.994 INFO:tasks.workunit.client.0.smithi019.stdout:7/546: creat d9/d3a/d49/d5d/d74/d9a/fb2 x:0 0 0 2024-02-22T15:50:33.994 INFO:tasks.workunit.client.1.smithi062.stdout:8/160: chown d8/c1d 789 1 2024-02-22T15:50:33.995 INFO:tasks.workunit.client.0.smithi019.stdout:7/547: read - d9/d3a/d49/d5d/d74/f9e zero size 2024-02-22T15:50:33.996 INFO:tasks.workunit.client.0.smithi019.stdout:5/445: symlink d8/dd/d19/d77/l96 0 2024-02-22T15:50:34.000 INFO:tasks.workunit.client.0.smithi019.stdout:9/415: rename d2/d1f/d2e/d48/f4e to d2/d7/dc/d27/d31/f96 0 2024-02-22T15:50:34.003 INFO:tasks.workunit.client.1.smithi062.stdout:8/161: dwrite d8/f21 [0,4194304] 0 2024-02-22T15:50:34.006 INFO:tasks.workunit.client.1.smithi062.stdout:8/162: mkdir d8/d9/d2c 0 2024-02-22T15:50:34.006 INFO:tasks.workunit.client.0.smithi019.stdout:8/407: link d9/d14/d1f/d20/f61 d9/d88/f8b 0 2024-02-22T15:50:34.008 INFO:tasks.workunit.client.1.smithi062.stdout:8/163: write d8/f24 [1235205,100996] 0 2024-02-22T15:50:34.012 INFO:tasks.workunit.client.1.smithi062.stdout:8/164: getdents d8 0 2024-02-22T15:50:34.013 INFO:tasks.workunit.client.0.smithi019.stdout:7/548: symlink d9/d38/d41/d45/d61/lb3 0 2024-02-22T15:50:34.013 INFO:tasks.workunit.client.0.smithi019.stdout:5/446: creat d8/dd/d19/d77/d7a/f97 x:0 0 0 2024-02-22T15:50:34.014 INFO:tasks.workunit.client.0.smithi019.stdout:7/549: readlink d9/d3a/d49/d5d/l7d 0 2024-02-22T15:50:34.017 INFO:tasks.workunit.client.0.smithi019.stdout:7/550: unlink d9/d38/d41/f90 0 2024-02-22T15:50:34.018 INFO:tasks.workunit.client.0.smithi019.stdout:5/447: creat d8/dd/d19/d38/d47/f98 x:0 0 0 2024-02-22T15:50:34.020 INFO:tasks.workunit.client.0.smithi019.stdout:5/448: fsync d8/f48 0 2024-02-22T15:50:34.021 INFO:tasks.workunit.client.0.smithi019.stdout:8/408: link d9/d14/d1f/d20/l27 d9/d14/d17/d3d/d3f/l8c 0 2024-02-22T15:50:34.022 INFO:tasks.workunit.client.0.smithi019.stdout:7/551: symlink d9/d3a/d49/d5d/d74/d9a/lb4 0 2024-02-22T15:50:34.023 INFO:tasks.workunit.client.0.smithi019.stdout:7/552: fdatasync d9/d4c/d5b/f96 0 2024-02-22T15:50:34.025 INFO:tasks.workunit.client.0.smithi019.stdout:8/409: unlink d9/d14/d17/d3d/f78 0 2024-02-22T15:50:34.027 INFO:tasks.workunit.client.0.smithi019.stdout:7/553: write d9/d3a/d49/d5d/d74/d9a/fb2 [642818,93258] 0 2024-02-22T15:50:34.028 INFO:tasks.workunit.client.0.smithi019.stdout:5/449: creat d8/d11/f99 x:0 0 0 2024-02-22T15:50:34.030 INFO:tasks.workunit.client.0.smithi019.stdout:8/410: mkdir d9/d14/d6f/d8d 0 2024-02-22T15:50:34.032 INFO:tasks.workunit.client.0.smithi019.stdout:7/554: mknod d9/d79/cb5 0 2024-02-22T15:50:34.033 INFO:tasks.workunit.client.0.smithi019.stdout:8/411: write d9/d14/d1f/f33 [2947052,115176] 0 2024-02-22T15:50:34.036 INFO:tasks.workunit.client.1.smithi062.stdout:8/165: dwrite d8/d9/f18 [0,4194304] 0 2024-02-22T15:50:34.038 INFO:tasks.workunit.client.1.smithi062.stdout:8/166: write d8/d9/f18 [3501495,44457] 0 2024-02-22T15:50:34.041 INFO:tasks.workunit.client.0.smithi019.stdout:5/450: dwrite d8/d11/d33/f76 [0,4194304] 0 2024-02-22T15:50:34.043 INFO:tasks.workunit.client.1.smithi062.stdout:8/167: dread d8/f12 [0,4194304] 0 2024-02-22T15:50:34.043 INFO:tasks.workunit.client.1.smithi062.stdout:8/168: fsync d8/d9/f16 0 2024-02-22T15:50:34.044 INFO:tasks.workunit.client.1.smithi062.stdout:0/209: sync 2024-02-22T15:50:34.044 INFO:tasks.workunit.client.1.smithi062.stdout:6/244: sync 2024-02-22T15:50:34.045 INFO:tasks.workunit.client.0.smithi019.stdout:9/416: dread d2/f30 [4194304,4194304] 0 2024-02-22T15:50:34.048 INFO:tasks.workunit.client.0.smithi019.stdout:8/412: dread d9/d14/d1f/d6d/f7c [0,4194304] 0 2024-02-22T15:50:34.048 INFO:tasks.workunit.client.0.smithi019.stdout:7/555: dread d9/d3a/d49/d5d/d74/f88 [0,4194304] 0 2024-02-22T15:50:34.051 INFO:tasks.workunit.client.0.smithi019.stdout:7/556: write d9/fa [4691422,96085] 0 2024-02-22T15:50:34.051 INFO:tasks.workunit.client.0.smithi019.stdout:5/451: chown d8/dd/d8e/d2c/f5d 1970 1 2024-02-22T15:50:34.053 INFO:tasks.workunit.client.1.smithi062.stdout:8/169: dwrite d8/f29 [0,4194304] 0 2024-02-22T15:50:34.053 INFO:tasks.workunit.client.0.smithi019.stdout:9/417: getdents d2/d1f/d2e/d48/d36 0 2024-02-22T15:50:34.054 INFO:tasks.workunit.client.0.smithi019.stdout:7/557: fsync d9/d3a/d49/d5d/d74/f88 0 2024-02-22T15:50:34.056 INFO:tasks.workunit.client.0.smithi019.stdout:9/418: mkdir d2/d7/dc/d97 0 2024-02-22T15:50:34.061 INFO:tasks.workunit.client.0.smithi019.stdout:5/452: write d8/f48 [2091497,4990] 0 2024-02-22T15:50:34.063 INFO:tasks.workunit.client.1.smithi062.stdout:8/170: dwrite d8/d9/f1b [0,4194304] 0 2024-02-22T15:50:34.063 INFO:tasks.workunit.client.1.smithi062.stdout:6/245: write d5/f1e [2615144,36986] 0 2024-02-22T15:50:34.064 INFO:tasks.workunit.client.1.smithi062.stdout:6/246: readlink d5/d15/l25 0 2024-02-22T15:50:34.065 INFO:tasks.workunit.client.1.smithi062.stdout:8/171: write d8/f22 [1044418,47519] 0 2024-02-22T15:50:34.066 INFO:tasks.workunit.client.1.smithi062.stdout:8/172: fsync d8/d9/f16 0 2024-02-22T15:50:34.067 INFO:tasks.workunit.client.0.smithi019.stdout:5/453: getdents d8/dd/d8e 0 2024-02-22T15:50:34.067 INFO:tasks.workunit.client.0.smithi019.stdout:7/558: dread d9/d4c/f68 [0,4194304] 0 2024-02-22T15:50:34.068 INFO:tasks.workunit.client.1.smithi062.stdout:6/247: creat d5/df/d32/d47/f50 x:0 0 0 2024-02-22T15:50:34.071 INFO:tasks.workunit.client.0.smithi019.stdout:7/559: mknod d9/d4c/cb6 0 2024-02-22T15:50:34.072 INFO:tasks.workunit.client.1.smithi062.stdout:8/173: getdents d8/d9 0 2024-02-22T15:50:34.074 INFO:tasks.workunit.client.1.smithi062.stdout:6/248: dread d5/dd/f2c [0,4194304] 0 2024-02-22T15:50:34.075 INFO:tasks.workunit.client.1.smithi062.stdout:8/174: creat d8/d9/f2d x:0 0 0 2024-02-22T15:50:34.076 INFO:tasks.workunit.client.1.smithi062.stdout:6/249: read d5/f1e [2433263,9507] 0 2024-02-22T15:50:34.076 INFO:tasks.workunit.client.0.smithi019.stdout:7/560: link d9/d3a/f6b d9/d38/d41/fb7 0 2024-02-22T15:50:34.078 INFO:tasks.workunit.client.0.smithi019.stdout:5/454: dwrite d8/dd/d19/d38/d47/f64 [0,4194304] 0 2024-02-22T15:50:34.078 INFO:tasks.workunit.client.0.smithi019.stdout:8/413: dread f6 [0,4194304] 0 2024-02-22T15:50:34.079 INFO:tasks.workunit.client.1.smithi062.stdout:8/175: rename f0 to d8/d9/f2e 0 2024-02-22T15:50:34.080 INFO:tasks.workunit.client.1.smithi062.stdout:8/176: readlink d8/d9/l15 0 2024-02-22T15:50:34.081 INFO:tasks.workunit.client.0.smithi019.stdout:5/455: readlink d8/dd/d19/d38/d47/l4b 0 2024-02-22T15:50:34.081 INFO:tasks.workunit.client.0.smithi019.stdout:8/414: rmdir d9/d14 39 2024-02-22T15:50:34.081 INFO:tasks.workunit.client.0.smithi019.stdout:7/561: read d9/d3a/d49/d5d/d74/d9a/fb2 [717348,64431] 0 2024-02-22T15:50:34.083 INFO:tasks.workunit.client.1.smithi062.stdout:6/250: unlink d5/d15/l16 0 2024-02-22T15:50:34.084 INFO:tasks.workunit.client.0.smithi019.stdout:8/415: truncate d9/d14/d17/f47 864621 0 2024-02-22T15:50:34.085 INFO:tasks.workunit.client.1.smithi062.stdout:8/177: dread d8/d9/f18 [0,4194304] 0 2024-02-22T15:50:34.086 INFO:tasks.workunit.client.1.smithi062.stdout:6/251: chown d5/df/l4b 7 1 2024-02-22T15:50:34.088 INFO:tasks.workunit.client.1.smithi062.stdout:8/178: creat d8/d9/d2c/f2f x:0 0 0 2024-02-22T15:50:34.088 INFO:tasks.workunit.client.0.smithi019.stdout:5/456: unlink d8/dd/d19/d77/l80 0 2024-02-22T15:50:34.089 INFO:tasks.workunit.client.0.smithi019.stdout:1/478: sync 2024-02-22T15:50:34.089 INFO:tasks.workunit.client.0.smithi019.stdout:6/401: sync 2024-02-22T15:50:34.089 INFO:tasks.workunit.client.0.smithi019.stdout:3/346: sync 2024-02-22T15:50:34.090 INFO:tasks.workunit.client.1.smithi062.stdout:8/179: creat d8/d9/d2c/f30 x:0 0 0 2024-02-22T15:50:34.091 INFO:tasks.workunit.client.0.smithi019.stdout:5/457: readlink d8/dd/d8e/d2c/l42 0 2024-02-22T15:50:34.091 INFO:tasks.workunit.client.0.smithi019.stdout:3/347: stat d9/f6a 0 2024-02-22T15:50:34.093 INFO:tasks.workunit.client.0.smithi019.stdout:8/416: mknod d9/d14/d1f/d20/c8e 0 2024-02-22T15:50:34.093 INFO:tasks.workunit.client.1.smithi062.stdout:6/252: getdents d5/df/d40/d44/d4a/d41 0 2024-02-22T15:50:34.095 INFO:tasks.workunit.client.0.smithi019.stdout:1/479: write d8/fd [6782640,4931] 0 2024-02-22T15:50:34.096 INFO:tasks.workunit.client.0.smithi019.stdout:7/562: dwrite d9/f19 [0,4194304] 0 2024-02-22T15:50:34.097 INFO:tasks.workunit.client.0.smithi019.stdout:8/417: read - d9/d14/d1f/f80 zero size 2024-02-22T15:50:34.098 INFO:tasks.workunit.client.0.smithi019.stdout:5/458: rmdir d8/dd/d19/d3a/d46/d7e 39 2024-02-22T15:50:34.100 INFO:tasks.workunit.client.0.smithi019.stdout:3/348: symlink d9/d26/d3f/d59/d66/l6f 0 2024-02-22T15:50:34.101 INFO:tasks.workunit.client.0.smithi019.stdout:8/418: fsync d9/d14/d17/d3d/d3f/f56 0 2024-02-22T15:50:34.103 INFO:tasks.workunit.client.0.smithi019.stdout:7/563: mknod d9/d26/cb8 0 2024-02-22T15:50:34.106 INFO:tasks.workunit.client.0.smithi019.stdout:3/349: symlink d9/d26/d2d/l70 0 2024-02-22T15:50:34.106 INFO:tasks.workunit.client.0.smithi019.stdout:8/419: write d9/d14/f19 [4459547,89195] 0 2024-02-22T15:50:34.108 INFO:tasks.workunit.client.0.smithi019.stdout:9/419: dread d2/d1f/d2e/d48/d89/f54 [0,4194304] 0 2024-02-22T15:50:34.108 INFO:tasks.workunit.client.0.smithi019.stdout:7/564: fdatasync d9/d38/f4e 0 2024-02-22T15:50:34.111 INFO:tasks.workunit.client.1.smithi062.stdout:8/180: truncate d8/f21 2111247 0 2024-02-22T15:50:34.112 INFO:tasks.workunit.client.0.smithi019.stdout:8/420: read d9/d14/f3e [967038,85615] 0 2024-02-22T15:50:34.112 INFO:tasks.workunit.client.1.smithi062.stdout:8/181: write d8/d9/f16 [990857,29018] 0 2024-02-22T15:50:34.113 INFO:tasks.workunit.client.1.smithi062.stdout:8/182: fdatasync d8/d9/d2c/f2f 0 2024-02-22T15:50:34.115 INFO:tasks.workunit.client.1.smithi062.stdout:8/183: write d8/d9/d2c/f30 [114550,39226] 0 2024-02-22T15:50:34.117 INFO:tasks.workunit.client.0.smithi019.stdout:1/480: dwrite d8/d15/f20 [0,4194304] 0 2024-02-22T15:50:34.119 INFO:tasks.workunit.client.0.smithi019.stdout:6/402: dread d6/d31/f35 [0,4194304] 0 2024-02-22T15:50:34.120 INFO:tasks.workunit.client.0.smithi019.stdout:9/420: dread d2/d1f/d2e/f42 [0,4194304] 0 2024-02-22T15:50:34.122 INFO:tasks.workunit.client.1.smithi062.stdout:8/184: dwrite d8/f24 [0,4194304] 0 2024-02-22T15:50:34.128 INFO:tasks.workunit.client.1.smithi062.stdout:8/185: dread d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:34.188 INFO:tasks.workunit.client.0.smithi019.stdout:4/422: sync 2024-02-22T15:50:34.207 INFO:tasks.workunit.client.1.smithi062.stdout:9/121: sync 2024-02-22T15:50:34.234 INFO:tasks.workunit.client.0.smithi019.stdout:3/350: rmdir d9/d26/d3f/d59 39 2024-02-22T15:50:34.235 INFO:tasks.workunit.client.1.smithi062.stdout:9/122: truncate d4/f13 129059 0 2024-02-22T15:50:34.235 INFO:tasks.workunit.client.0.smithi019.stdout:3/351: readlink d9/d22/d27/l54 0 2024-02-22T15:50:34.236 INFO:tasks.workunit.client.1.smithi062.stdout:9/123: write d4/f14 [2968818,24690] 0 2024-02-22T15:50:34.238 INFO:tasks.workunit.client.0.smithi019.stdout:8/421: unlink d9/d14/d1f/d4c/l5a 0 2024-02-22T15:50:34.238 INFO:tasks.workunit.client.1.smithi062.stdout:9/124: mkdir d4/d28 0 2024-02-22T15:50:34.239 INFO:tasks.workunit.client.0.smithi019.stdout:1/481: stat d8/l7b 0 2024-02-22T15:50:34.248 INFO:tasks.workunit.client.0.smithi019.stdout:3/352: write d9/f2f [327218,33668] 0 2024-02-22T15:50:34.251 INFO:tasks.workunit.client.0.smithi019.stdout:1/482: rename d8/d15/f63 to d8/d15/d34/fac 0 2024-02-22T15:50:34.252 INFO:tasks.workunit.client.0.smithi019.stdout:6/403: write d6/f29 [5230412,121610] 0 2024-02-22T15:50:34.254 INFO:tasks.workunit.client.0.smithi019.stdout:9/421: symlink d2/d1f/d2e/d57/d60/d8c/l98 0 2024-02-22T15:50:34.254 INFO:tasks.workunit.client.0.smithi019.stdout:1/483: write d8/d18/d85/f98 [957288,13968] 0 2024-02-22T15:50:34.256 INFO:tasks.workunit.client.0.smithi019.stdout:4/423: creat d2/db/d14/d23/d33/d66/f8a x:0 0 0 2024-02-22T15:50:34.256 INFO:tasks.workunit.client.0.smithi019.stdout:5/459: link d8/d11/c1d d8/d11/c9a 0 2024-02-22T15:50:34.256 INFO:tasks.workunit.client.1.smithi062.stdout:9/125: dread d4/f8 [0,4194304] 0 2024-02-22T15:50:34.257 INFO:tasks.workunit.client.1.smithi062.stdout:9/126: stat d4/l11 0 2024-02-22T15:50:34.259 INFO:tasks.workunit.client.0.smithi019.stdout:8/422: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:34.259 INFO:tasks.workunit.client.0.smithi019.stdout:9/422: truncate d2/d1f/d2e/f7b 595975 0 2024-02-22T15:50:34.260 INFO:tasks.workunit.client.1.smithi062.stdout:9/127: fdatasync d4/f13 0 2024-02-22T15:50:34.261 INFO:tasks.workunit.client.0.smithi019.stdout:7/565: getdents d9/d26 0 2024-02-22T15:50:34.262 INFO:tasks.workunit.client.0.smithi019.stdout:1/484: unlink d8/d23/d33/l99 0 2024-02-22T15:50:34.262 INFO:tasks.workunit.client.1.smithi062.stdout:9/128: read d4/f13 [90365,111593] 0 2024-02-22T15:50:34.263 INFO:tasks.workunit.client.1.smithi062.stdout:9/129: creat d4/f29 x:0 0 0 2024-02-22T15:50:34.264 INFO:tasks.workunit.client.0.smithi019.stdout:7/566: dread - d9/d3a/d49/d5d/d74/f9e zero size 2024-02-22T15:50:34.264 INFO:tasks.workunit.client.0.smithi019.stdout:3/353: mkdir d9/d11/d71 0 2024-02-22T15:50:34.266 INFO:tasks.workunit.client.0.smithi019.stdout:4/424: chown d2/db/d14/d23/d33/l40 26055385 1 2024-02-22T15:50:34.267 INFO:tasks.workunit.client.0.smithi019.stdout:8/423: creat d9/d14/d1f/d6d/f8f x:0 0 0 2024-02-22T15:50:34.268 INFO:tasks.workunit.client.0.smithi019.stdout:3/354: chown d9/d26/l37 828 1 2024-02-22T15:50:34.268 INFO:tasks.workunit.client.0.smithi019.stdout:9/423: symlink d2/d1f/d2e/d48/d45/d5a/l99 0 2024-02-22T15:50:34.273 INFO:tasks.workunit.client.0.smithi019.stdout:4/425: chown d2/db/d14/d23/d33/d66 519733 1 2024-02-22T15:50:34.273 INFO:tasks.workunit.client.0.smithi019.stdout:1/485: chown d8/d18/d64/d7f/d82/ca9 162590 1 2024-02-22T15:50:34.273 INFO:tasks.workunit.client.0.smithi019.stdout:8/424: creat d9/d88/f90 x:0 0 0 2024-02-22T15:50:34.274 INFO:tasks.workunit.client.0.smithi019.stdout:9/424: write d2/f19 [220587,65643] 0 2024-02-22T15:50:34.274 INFO:tasks.workunit.client.0.smithi019.stdout:3/355: chown d9/d14/c36 44234 1 2024-02-22T15:50:34.277 INFO:tasks.workunit.client.1.smithi062.stdout:9/130: rename d4/fe to d4/f2a 0 2024-02-22T15:50:34.278 INFO:tasks.workunit.client.1.smithi062.stdout:9/131: fdatasync d4/f13 0 2024-02-22T15:50:34.280 INFO:tasks.workunit.client.0.smithi019.stdout:4/426: write d2/db/d14/d23/d33/f7d [539472,119981] 0 2024-02-22T15:50:34.280 INFO:tasks.workunit.client.1.smithi062.stdout:9/132: creat d4/d1c/f2b x:0 0 0 2024-02-22T15:50:34.282 INFO:tasks.workunit.client.0.smithi019.stdout:5/460: dwrite d8/f23 [0,4194304] 0 2024-02-22T15:50:34.283 INFO:tasks.workunit.client.0.smithi019.stdout:7/567: dwrite d9/d38/f52 [0,4194304] 0 2024-02-22T15:50:34.284 INFO:tasks.workunit.client.1.smithi062.stdout:9/133: rename d4/f13 to d4/d1c/f2c 0 2024-02-22T15:50:34.285 INFO:tasks.workunit.client.0.smithi019.stdout:6/404: dread d6/f22 [0,4194304] 0 2024-02-22T15:50:34.285 INFO:tasks.workunit.client.0.smithi019.stdout:4/427: fsync d2/db/f25 0 2024-02-22T15:50:34.285 INFO:tasks.workunit.client.0.smithi019.stdout:3/356: truncate d9/d26/d3f/f6c 838165 0 2024-02-22T15:50:34.285 INFO:tasks.workunit.client.1.smithi062.stdout:9/134: getdents d4/d28 0 2024-02-22T15:50:34.286 INFO:tasks.workunit.client.0.smithi019.stdout:9/425: mknod d2/d7/d52/c9a 0 2024-02-22T15:50:34.286 INFO:tasks.workunit.client.1.smithi062.stdout:9/135: truncate d4/f26 821104 0 2024-02-22T15:50:34.287 INFO:tasks.workunit.client.1.smithi062.stdout:9/136: fdatasync d4/f14 0 2024-02-22T15:50:34.288 INFO:tasks.workunit.client.1.smithi062.stdout:9/137: write d4/d15/f23 [862920,89174] 0 2024-02-22T15:50:34.289 INFO:tasks.workunit.client.0.smithi019.stdout:5/461: creat d8/f9b x:0 0 0 2024-02-22T15:50:34.292 INFO:tasks.workunit.client.0.smithi019.stdout:8/425: write d9/d14/d1f/d6d/f7c [735824,62951] 0 2024-02-22T15:50:34.293 INFO:tasks.workunit.client.0.smithi019.stdout:1/486: truncate d8/f53 1405332 0 2024-02-22T15:50:34.298 INFO:tasks.workunit.client.1.smithi062.stdout:9/138: dwrite d4/d15/f17 [0,4194304] 0 2024-02-22T15:50:34.298 INFO:tasks.workunit.client.0.smithi019.stdout:9/426: read - d2/d1f/d2e/d48/d36/f43 zero size 2024-02-22T15:50:34.299 INFO:tasks.workunit.client.1.smithi062.stdout:9/139: write d4/d1c/f27 [508455,11251] 0 2024-02-22T15:50:34.304 INFO:tasks.workunit.client.0.smithi019.stdout:3/357: mkdir d9/d22/d27/d6d/d72 0 2024-02-22T15:50:34.306 INFO:tasks.workunit.client.0.smithi019.stdout:1/487: readlink d8/d44/d5d/l52 0 2024-02-22T15:50:34.306 INFO:tasks.workunit.client.0.smithi019.stdout:5/462: mknod d8/dd/d19/d3a/c9c 0 2024-02-22T15:50:34.307 INFO:tasks.workunit.client.1.smithi062.stdout:9/140: dwrite d4/fc [0,4194304] 0 2024-02-22T15:50:34.309 INFO:tasks.workunit.client.1.smithi062.stdout:9/141: write d4/d15/f24 [210826,97528] 0 2024-02-22T15:50:34.311 INFO:tasks.workunit.client.0.smithi019.stdout:4/428: chown d2/db/d44/c72 7 1 2024-02-22T15:50:34.311 INFO:tasks.workunit.client.1.smithi062.stdout:9/142: mknod d4/d15/d20/c2d 0 2024-02-22T15:50:34.313 INFO:tasks.workunit.client.0.smithi019.stdout:7/568: dread d9/f21 [0,4194304] 0 2024-02-22T15:50:34.314 INFO:tasks.workunit.client.0.smithi019.stdout:1/488: write d8/d18/d64/f66 [850787,82029] 0 2024-02-22T15:50:34.318 INFO:tasks.workunit.client.0.smithi019.stdout:6/405: dread d6/d4b/f52 [0,4194304] 0 2024-02-22T15:50:34.318 INFO:tasks.workunit.client.0.smithi019.stdout:3/358: unlink d9/d26/d2d/f3a 0 2024-02-22T15:50:34.318 INFO:tasks.workunit.client.0.smithi019.stdout:1/489: stat d8/d18/d64/f66 0 2024-02-22T15:50:34.318 INFO:tasks.workunit.client.0.smithi019.stdout:8/426: dwrite d9/f32 [0,4194304] 0 2024-02-22T15:50:34.319 INFO:tasks.workunit.client.0.smithi019.stdout:9/427: link d2/c3 d2/d1f/d2e/d48/d45/d5a/c9b 0 2024-02-22T15:50:34.320 INFO:tasks.workunit.client.1.smithi062.stdout:9/143: dwrite d4/d1c/f1d [0,4194304] 0 2024-02-22T15:50:34.321 INFO:tasks.workunit.client.0.smithi019.stdout:5/463: unlink d8/dd/d19/d3a/d46/d7e/f83 0 2024-02-22T15:50:34.321 INFO:tasks.workunit.client.0.smithi019.stdout:1/490: readlink d8/d23/d33/l56 0 2024-02-22T15:50:34.323 INFO:tasks.workunit.client.0.smithi019.stdout:4/429: chown d2/db/d14/d23/d33/f7d 10545 1 2024-02-22T15:50:34.324 INFO:tasks.workunit.client.1.smithi062.stdout:9/144: rmdir d4/d15 39 2024-02-22T15:50:34.325 INFO:tasks.workunit.client.0.smithi019.stdout:5/464: fdatasync d8/dd/f8f 0 2024-02-22T15:50:34.326 INFO:tasks.workunit.client.1.smithi062.stdout:5/112: sync 2024-02-22T15:50:34.331 INFO:tasks.workunit.client.1.smithi062.stdout:9/145: getdents d4/d15 0 2024-02-22T15:50:34.331 INFO:tasks.workunit.client.1.smithi062.stdout:4/106: sync 2024-02-22T15:50:34.331 INFO:tasks.workunit.client.0.smithi019.stdout:8/427: creat d9/d14/d17/d3d/f91 x:0 0 0 2024-02-22T15:50:34.333 INFO:tasks.workunit.client.1.smithi062.stdout:9/146: rmdir d4/d15/d20 39 2024-02-22T15:50:34.334 INFO:tasks.workunit.client.0.smithi019.stdout:3/359: symlink d9/d26/d3f/d59/l73 0 2024-02-22T15:50:34.335 INFO:tasks.workunit.client.0.smithi019.stdout:9/428: write d2/d1f/d2e/d57/d60/d8c/f6a [957341,57532] 0 2024-02-22T15:50:34.336 INFO:tasks.workunit.client.0.smithi019.stdout:8/428: write d9/d14/f34 [1887449,77102] 0 2024-02-22T15:50:34.337 INFO:tasks.workunit.client.1.smithi062.stdout:5/113: dwrite d0/f16 [4194304,4194304] 0 2024-02-22T15:50:34.337 INFO:tasks.workunit.client.1.smithi062.stdout:4/107: rename d4/lb to d4/d13/d20/l24 0 2024-02-22T15:50:34.337 INFO:tasks.workunit.client.0.smithi019.stdout:3/360: write f3 [8996480,125900] 0 2024-02-22T15:50:34.338 INFO:tasks.workunit.client.0.smithi019.stdout:6/406: dwrite d6/d31/d37/f60 [0,4194304] 0 2024-02-22T15:50:34.338 INFO:tasks.workunit.client.1.smithi062.stdout:4/108: chown d4/df/c12 60329 1 2024-02-22T15:50:34.338 INFO:tasks.workunit.client.1.smithi062.stdout:9/147: truncate d4/fd 1170845 0 2024-02-22T15:50:34.339 INFO:tasks.workunit.client.1.smithi062.stdout:9/148: chown d4/ca 5042 1 2024-02-22T15:50:34.342 INFO:tasks.workunit.client.1.smithi062.stdout:4/109: creat d4/d13/d20/f25 x:0 0 0 2024-02-22T15:50:34.344 INFO:tasks.workunit.client.1.smithi062.stdout:4/110: creat d4/df/f26 x:0 0 0 2024-02-22T15:50:34.347 INFO:tasks.workunit.client.0.smithi019.stdout:5/465: mknod d8/dd/d19/d38/d47/c9d 0 2024-02-22T15:50:34.348 INFO:tasks.workunit.client.1.smithi062.stdout:4/111: read f2 [446974,123011] 0 2024-02-22T15:50:34.355 INFO:tasks.workunit.client.1.smithi062.stdout:9/149: dread d4/d1c/f1e [0,4194304] 0 2024-02-22T15:50:34.356 INFO:tasks.workunit.client.1.smithi062.stdout:4/112: dwrite d4/df/f26 [0,4194304] 0 2024-02-22T15:50:34.357 INFO:tasks.workunit.client.0.smithi019.stdout:3/361: creat d9/d26/d3f/d59/f74 x:0 0 0 2024-02-22T15:50:34.357 INFO:tasks.workunit.client.1.smithi062.stdout:9/150: chown d4/d15/l1a 2606487 1 2024-02-22T15:50:34.357 INFO:tasks.workunit.client.1.smithi062.stdout:4/113: chown d4/l5 122362670 1 2024-02-22T15:50:34.358 INFO:tasks.workunit.client.1.smithi062.stdout:9/151: write d4/f7 [1505718,63354] 0 2024-02-22T15:50:34.359 INFO:tasks.workunit.client.1.smithi062.stdout:4/114: creat d4/d1c/f27 x:0 0 0 2024-02-22T15:50:34.359 INFO:tasks.workunit.client.1.smithi062.stdout:4/115: readlink l0 0 2024-02-22T15:50:34.360 INFO:tasks.workunit.client.1.smithi062.stdout:9/152: dread - d4/d15/d20/f25 zero size 2024-02-22T15:50:34.361 INFO:tasks.workunit.client.0.smithi019.stdout:5/466: rmdir d8/dd 39 2024-02-22T15:50:34.363 INFO:tasks.workunit.client.0.smithi019.stdout:7/569: dwrite d9/d38/d41/f78 [0,4194304] 0 2024-02-22T15:50:34.368 INFO:tasks.workunit.client.1.smithi062.stdout:4/116: dwrite d4/df/f26 [0,4194304] 0 2024-02-22T15:50:34.368 INFO:tasks.workunit.client.0.smithi019.stdout:6/407: write d6/d3e/f42 [846873,108303] 0 2024-02-22T15:50:34.371 INFO:tasks.workunit.client.1.smithi062.stdout:4/117: write d4/d1c/f27 [915916,103096] 0 2024-02-22T15:50:34.372 INFO:tasks.workunit.client.1.smithi062.stdout:9/153: dwrite d4/d1c/f2b [0,4194304] 0 2024-02-22T15:50:34.374 INFO:tasks.workunit.client.1.smithi062.stdout:4/118: mknod d4/df/c28 0 2024-02-22T15:50:34.378 INFO:tasks.workunit.client.0.smithi019.stdout:4/430: dread d2/f32 [0,4194304] 0 2024-02-22T15:50:34.378 INFO:tasks.workunit.client.0.smithi019.stdout:8/429: dwrite d9/d14/d1f/d5c/f60 [0,4194304] 0 2024-02-22T15:50:34.378 INFO:tasks.workunit.client.0.smithi019.stdout:5/467: chown d8/d11/d33/c49 84 1 2024-02-22T15:50:34.378 INFO:tasks.workunit.client.0.smithi019.stdout:9/429: truncate d2/d7/d14/f82 1997711 0 2024-02-22T15:50:34.379 INFO:tasks.workunit.client.1.smithi062.stdout:4/119: link d4/d13/l22 d4/df/l29 0 2024-02-22T15:50:34.381 INFO:tasks.workunit.client.1.smithi062.stdout:4/120: mkdir d4/d2a 0 2024-02-22T15:50:34.382 INFO:tasks.workunit.client.1.smithi062.stdout:4/121: chown d4/df/f26 2 1 2024-02-22T15:50:34.384 INFO:tasks.workunit.client.1.smithi062.stdout:4/122: dread f2 [0,4194304] 0 2024-02-22T15:50:34.390 INFO:tasks.workunit.client.1.smithi062.stdout:4/123: mknod d4/d1c/c2b 0 2024-02-22T15:50:34.392 INFO:tasks.workunit.client.1.smithi062.stdout:4/124: mkdir d4/d1c/d2c 0 2024-02-22T15:50:34.394 INFO:tasks.workunit.client.1.smithi062.stdout:4/125: mkdir d4/d2a/d2d 0 2024-02-22T15:50:34.397 INFO:tasks.workunit.client.1.smithi062.stdout:4/126: dread d4/df/f26 [0,4194304] 0 2024-02-22T15:50:34.398 INFO:tasks.workunit.client.1.smithi062.stdout:4/127: fsync f2 0 2024-02-22T15:50:34.399 INFO:tasks.workunit.client.1.smithi062.stdout:4/128: read f2 [2168930,70512] 0 2024-02-22T15:50:34.413 INFO:tasks.workunit.client.0.smithi019.stdout:1/491: dwrite d8/d15/f69 [0,4194304] 0 2024-02-22T15:50:34.415 INFO:tasks.workunit.client.0.smithi019.stdout:6/408: creat d6/d58/f76 x:0 0 0 2024-02-22T15:50:34.416 INFO:tasks.workunit.client.0.smithi019.stdout:5/468: fsync d8/d11/f3b 0 2024-02-22T15:50:34.417 INFO:tasks.workunit.client.0.smithi019.stdout:4/431: symlink d2/db/d14/d23/d67/l8b 0 2024-02-22T15:50:34.421 INFO:tasks.workunit.client.0.smithi019.stdout:5/469: write d8/f89 [936848,111157] 0 2024-02-22T15:50:34.421 INFO:tasks.workunit.client.0.smithi019.stdout:1/492: rmdir d8/d18/d64/d7f 39 2024-02-22T15:50:34.423 INFO:tasks.workunit.client.0.smithi019.stdout:7/570: link d9/d26/cb8 d9/d4c/d5b/cb9 0 2024-02-22T15:50:34.424 INFO:tasks.workunit.client.0.smithi019.stdout:8/430: unlink d9/d14/d17/d1c/c42 0 2024-02-22T15:50:34.425 INFO:tasks.workunit.client.0.smithi019.stdout:4/432: mkdir d2/db/d54/d8c 0 2024-02-22T15:50:34.426 INFO:tasks.workunit.client.1.smithi062.stdout:4/129: fsync d4/df/f26 0 2024-02-22T15:50:34.429 INFO:tasks.workunit.client.1.smithi062.stdout:4/130: mkdir d4/d2a/d2d/d2e 0 2024-02-22T15:50:34.430 INFO:tasks.workunit.client.0.smithi019.stdout:4/433: stat d2/db/d44/l60 0 2024-02-22T15:50:34.431 INFO:tasks.workunit.client.0.smithi019.stdout:9/430: dread d2/d1f/d2e/d48/d89/f54 [0,4194304] 0 2024-02-22T15:50:34.431 INFO:tasks.workunit.client.0.smithi019.stdout:5/470: write d8/dd/f67 [4476973,67110] 0 2024-02-22T15:50:34.433 INFO:tasks.workunit.client.0.smithi019.stdout:6/409: dwrite d6/d3e/f4e [0,4194304] 0 2024-02-22T15:50:34.436 INFO:tasks.workunit.client.0.smithi019.stdout:7/571: symlink d9/d38/d41/d45/db0/lba 0 2024-02-22T15:50:34.437 INFO:tasks.workunit.client.0.smithi019.stdout:9/431: mknod d2/d7/d14/c9c 0 2024-02-22T15:50:34.437 INFO:tasks.workunit.client.0.smithi019.stdout:5/471: dread - d8/dd/d19/d77/f91 zero size 2024-02-22T15:50:34.438 INFO:tasks.workunit.client.0.smithi019.stdout:6/410: dread - d6/d31/f3b zero size 2024-02-22T15:50:34.438 INFO:tasks.workunit.client.1.smithi062.stdout:4/131: dwrite f2 [0,4194304] 0 2024-02-22T15:50:34.438 INFO:tasks.workunit.client.1.smithi062.stdout:9/154: dread d4/f14 [0,4194304] 0 2024-02-22T15:50:34.439 INFO:tasks.workunit.client.0.smithi019.stdout:8/431: getdents d9/d14/d17/d3d/d83 0 2024-02-22T15:50:34.439 INFO:tasks.workunit.client.1.smithi062.stdout:7/202: sync 2024-02-22T15:50:34.439 INFO:tasks.workunit.client.1.smithi062.stdout:3/142: sync 2024-02-22T15:50:34.439 INFO:tasks.workunit.client.1.smithi062.stdout:9/155: readlink d4/d15/l1a 0 2024-02-22T15:50:34.440 INFO:tasks.workunit.client.0.smithi019.stdout:5/472: write d8/dd/f5b [469556,117493] 0 2024-02-22T15:50:34.441 INFO:tasks.workunit.client.0.smithi019.stdout:1/493: creat d8/d18/d64/fad x:0 0 0 2024-02-22T15:50:34.442 INFO:tasks.workunit.client.0.smithi019.stdout:6/411: write d6/d31/d43/f46 [739039,74129] 0 2024-02-22T15:50:34.442 INFO:tasks.workunit.client.1.smithi062.stdout:7/203: write d0/d14/d19/d30/f42 [393101,82489] 0 2024-02-22T15:50:34.443 INFO:tasks.workunit.client.0.smithi019.stdout:4/434: mknod d2/c8d 0 2024-02-22T15:50:34.446 INFO:tasks.workunit.client.1.smithi062.stdout:9/156: mknod d4/d28/c2e 0 2024-02-22T15:50:34.448 INFO:tasks.workunit.client.0.smithi019.stdout:9/432: dread d2/d1f/d2e/f42 [0,4194304] 0 2024-02-22T15:50:34.449 INFO:tasks.workunit.client.1.smithi062.stdout:4/132: getdents d4/d13 0 2024-02-22T15:50:34.451 INFO:tasks.workunit.client.0.smithi019.stdout:5/473: unlink d8/d11/d33/c94 0 2024-02-22T15:50:34.452 INFO:tasks.workunit.client.1.smithi062.stdout:9/157: creat d4/f2f x:0 0 0 2024-02-22T15:50:34.452 INFO:tasks.workunit.client.1.smithi062.stdout:4/133: chown l0 5907288 1 2024-02-22T15:50:34.454 INFO:tasks.workunit.client.0.smithi019.stdout:1/494: mknod d8/d2a/d75/cae 0 2024-02-22T15:50:34.455 INFO:tasks.workunit.client.0.smithi019.stdout:4/435: mkdir d2/db/d14/d23/d4b/d8e 0 2024-02-22T15:50:34.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:34 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:34.457 INFO:tasks.workunit.client.0.smithi019.stdout:5/474: mkdir d8/dd/d19/d77/d7a/d9e 0 2024-02-22T15:50:34.458 INFO:tasks.workunit.client.0.smithi019.stdout:9/433: dread d2/d7/dc/d27/d31/f96 [0,4194304] 0 2024-02-22T15:50:34.458 INFO:tasks.workunit.client.1.smithi062.stdout:4/134: write d4/d13/d20/f25 [354142,50952] 0 2024-02-22T15:50:34.458 INFO:tasks.workunit.client.1.smithi062.stdout:9/158: creat d4/d28/f30 x:0 0 0 2024-02-22T15:50:34.458 INFO:tasks.workunit.client.0.smithi019.stdout:6/412: rename d6/l1e to d6/d31/d43/l77 0 2024-02-22T15:50:34.459 INFO:tasks.workunit.client.1.smithi062.stdout:3/143: dwrite d5/f1d [0,4194304] 0 2024-02-22T15:50:34.459 INFO:tasks.workunit.client.0.smithi019.stdout:9/434: fdatasync d2/d1f/d2e/d57/d60/f75 0 2024-02-22T15:50:34.461 INFO:tasks.workunit.client.1.smithi062.stdout:7/204: dwrite d0/d14/d13/f41 [0,4194304] 0 2024-02-22T15:50:34.461 INFO:tasks.workunit.client.0.smithi019.stdout:1/495: write d8/d18/d64/d7f/fa2 [602744,71332] 0 2024-02-22T15:50:34.461 INFO:tasks.workunit.client.0.smithi019.stdout:8/432: link d9/d14/f19 d9/d14/d1f/f92 0 2024-02-22T15:50:34.462 INFO:tasks.workunit.client.0.smithi019.stdout:9/435: dread - d2/d7/dc/d27/d68/f73 zero size 2024-02-22T15:50:34.463 INFO:tasks.workunit.client.1.smithi062.stdout:4/135: chown d4/d1c/c1f 4413 1 2024-02-22T15:50:34.464 INFO:tasks.workunit.client.1.smithi062.stdout:9/159: mkdir d4/d15/d31 0 2024-02-22T15:50:34.466 INFO:tasks.workunit.client.1.smithi062.stdout:7/205: creat d0/d14/d13/f46 x:0 0 0 2024-02-22T15:50:34.468 INFO:tasks.workunit.client.1.smithi062.stdout:4/136: symlink d4/d13/d20/l2f 0 2024-02-22T15:50:34.468 INFO:tasks.workunit.client.0.smithi019.stdout:1/496: unlink d8/d44/f4a 0 2024-02-22T15:50:34.469 INFO:tasks.workunit.client.0.smithi019.stdout:8/433: read d9/d14/d17/f47 [482876,85410] 0 2024-02-22T15:50:34.470 INFO:tasks.workunit.client.0.smithi019.stdout:4/436: dread d2/db/d14/d23/d21/d35/f38 [0,4194304] 0 2024-02-22T15:50:34.471 INFO:tasks.workunit.client.0.smithi019.stdout:7/572: write d9/f23 [3684117,41399] 0 2024-02-22T15:50:34.471 INFO:tasks.workunit.client.0.smithi019.stdout:1/497: truncate d8/d18/d64/f67 506 0 2024-02-22T15:50:34.472 INFO:tasks.workunit.client.1.smithi062.stdout:7/206: getdents d0 0 2024-02-22T15:50:34.473 INFO:tasks.workunit.client.0.smithi019.stdout:6/413: fdatasync d6/f20 0 2024-02-22T15:50:34.474 INFO:tasks.workunit.client.0.smithi019.stdout:9/436: rmdir d2/d1f/d2e/d57 39 2024-02-22T15:50:34.477 INFO:tasks.workunit.client.0.smithi019.stdout:1/498: write d8/d44/f45 [223941,26851] 0 2024-02-22T15:50:34.478 INFO:tasks.workunit.client.1.smithi062.stdout:7/207: creat d0/d14/d28/f47 x:0 0 0 2024-02-22T15:50:34.478 INFO:tasks.workunit.client.1.smithi062.stdout:4/137: dwrite d4/fd [0,4194304] 0 2024-02-22T15:50:34.478 INFO:tasks.workunit.client.0.smithi019.stdout:7/573: symlink d9/d4c/d5b/lbb 0 2024-02-22T15:50:34.479 INFO:tasks.workunit.client.1.smithi062.stdout:9/160: write d4/d1c/f2c [282581,42568] 0 2024-02-22T15:50:34.479 INFO:tasks.workunit.client.0.smithi019.stdout:6/414: chown d6/l53 3296 1 2024-02-22T15:50:34.481 INFO:tasks.workunit.client.1.smithi062.stdout:7/208: write d0/d14/d19/d30/f42 [1309162,19833] 0 2024-02-22T15:50:34.481 INFO:tasks.workunit.client.1.smithi062.stdout:4/138: read - d4/d1c/f23 zero size 2024-02-22T15:50:34.481 INFO:tasks.workunit.client.1.smithi062.stdout:9/161: chown d4/l18 40958 1 2024-02-22T15:50:34.482 INFO:tasks.workunit.client.0.smithi019.stdout:1/499: dread d8/d15/d34/fa3 [0,4194304] 0 2024-02-22T15:50:34.482 INFO:tasks.workunit.client.1.smithi062.stdout:4/139: chown d4/d1c/c1f 12770 1 2024-02-22T15:50:34.484 INFO:tasks.workunit.client.0.smithi019.stdout:7/574: symlink d9/d4c/d5b/lbc 0 2024-02-22T15:50:34.485 INFO:tasks.workunit.client.1.smithi062.stdout:9/162: creat d4/d15/d31/f32 x:0 0 0 2024-02-22T15:50:34.486 INFO:tasks.workunit.client.0.smithi019.stdout:6/415: mkdir d6/d31/d37/d78 0 2024-02-22T15:50:34.487 INFO:tasks.workunit.client.1.smithi062.stdout:9/163: readlink d4/d15/l16 0 2024-02-22T15:50:34.488 INFO:tasks.workunit.client.1.smithi062.stdout:9/164: chown d4/f7 51400575 1 2024-02-22T15:50:34.489 INFO:tasks.workunit.client.0.smithi019.stdout:7/575: truncate d9/d38/d41/d45/d61/fa4 144210 0 2024-02-22T15:50:34.489 INFO:tasks.workunit.client.0.smithi019.stdout:9/437: mkdir d2/d1f/d2e/d57/d60/d8c/d66/d9d 0 2024-02-22T15:50:34.490 INFO:tasks.workunit.client.1.smithi062.stdout:7/209: dwrite d0/d14/f20 [0,4194304] 0 2024-02-22T15:50:34.491 INFO:tasks.workunit.client.1.smithi062.stdout:4/140: dread f2 [0,4194304] 0 2024-02-22T15:50:34.492 INFO:tasks.workunit.client.0.smithi019.stdout:8/434: rename d9/d14/d17/d1c/f24 to d9/d14/d17/f93 0 2024-02-22T15:50:34.493 INFO:tasks.workunit.client.1.smithi062.stdout:9/165: mknod d4/d15/c33 0 2024-02-22T15:50:34.493 INFO:tasks.workunit.client.0.smithi019.stdout:6/416: write d6/f44 [1780144,109516] 0 2024-02-22T15:50:34.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:34 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:34.496 INFO:tasks.workunit.client.1.smithi062.stdout:9/166: truncate d4/d15/f23 1384931 0 2024-02-22T15:50:34.497 INFO:tasks.workunit.client.0.smithi019.stdout:1/500: creat d8/d23/faf x:0 0 0 2024-02-22T15:50:34.499 INFO:tasks.workunit.client.0.smithi019.stdout:9/438: creat d2/d1f/d95/f9e x:0 0 0 2024-02-22T15:50:34.500 INFO:tasks.workunit.client.1.smithi062.stdout:9/167: mknod d4/d15/d20/c34 0 2024-02-22T15:50:34.500 INFO:tasks.workunit.client.1.smithi062.stdout:7/210: dwrite d0/f10 [0,4194304] 0 2024-02-22T15:50:34.501 INFO:tasks.workunit.client.1.smithi062.stdout:7/211: truncate d0/f10 4723861 0 2024-02-22T15:50:34.502 INFO:tasks.workunit.client.0.smithi019.stdout:9/439: write d2/d7/dc/d27/d68/f73 [533438,109380] 0 2024-02-22T15:50:34.505 INFO:tasks.workunit.client.0.smithi019.stdout:1/501: mknod d8/d2a/d75/d81/cb0 0 2024-02-22T15:50:34.506 INFO:tasks.workunit.client.1.smithi062.stdout:7/212: rename d0/db/f34 to d0/d14/d13/f48 0 2024-02-22T15:50:34.508 INFO:tasks.workunit.client.0.smithi019.stdout:4/437: dread d2/db/d14/f27 [0,4194304] 0 2024-02-22T15:50:34.509 INFO:tasks.workunit.client.1.smithi062.stdout:9/168: dwrite d4/f8 [0,4194304] 0 2024-02-22T15:50:34.511 INFO:tasks.workunit.client.1.smithi062.stdout:3/144: dread d5/f6 [0,4194304] 0 2024-02-22T15:50:34.513 INFO:tasks.workunit.client.0.smithi019.stdout:1/502: rename d8/d15/d84/f92 to d8/d2a/da7/fb1 0 2024-02-22T15:50:34.515 INFO:tasks.workunit.client.0.smithi019.stdout:1/503: chown d8/d15/f43 643173 1 2024-02-22T15:50:34.518 INFO:tasks.workunit.client.0.smithi019.stdout:8/435: getdents d9/d14/d1f/d5c 0 2024-02-22T15:50:34.519 INFO:tasks.workunit.client.1.smithi062.stdout:3/145: truncate d5/f6 2638264 0 2024-02-22T15:50:34.519 INFO:tasks.workunit.client.1.smithi062.stdout:9/169: mkdir d4/d35 0 2024-02-22T15:50:34.520 INFO:tasks.workunit.client.1.smithi062.stdout:3/146: mknod d5/d1a/c2e 0 2024-02-22T15:50:34.520 INFO:tasks.workunit.client.1.smithi062.stdout:9/170: readlink d4/d15/l1a 0 2024-02-22T15:50:34.521 INFO:tasks.workunit.client.0.smithi019.stdout:4/438: dread d2/db/d14/d23/d21/f5a [0,4194304] 0 2024-02-22T15:50:34.523 INFO:tasks.workunit.client.1.smithi062.stdout:9/171: chown d4/l11 8975396 1 2024-02-22T15:50:34.523 INFO:tasks.workunit.client.0.smithi019.stdout:8/436: unlink d9/d14/d17/d1c/f1e 0 2024-02-22T15:50:34.523 INFO:tasks.workunit.client.0.smithi019.stdout:4/439: dread d2/db/d14/d23/d21/f5a [0,4194304] 0 2024-02-22T15:50:34.524 INFO:tasks.workunit.client.0.smithi019.stdout:1/504: truncate d8/d15/f17 456981 0 2024-02-22T15:50:34.524 INFO:tasks.workunit.client.1.smithi062.stdout:3/147: rename f4 to d5/d26/d2d/f2f 0 2024-02-22T15:50:34.524 INFO:tasks.workunit.client.1.smithi062.stdout:7/213: dwrite d0/d6/f23 [0,4194304] 0 2024-02-22T15:50:34.525 INFO:tasks.workunit.client.0.smithi019.stdout:1/505: readlink d8/d44/d5d/la1 0 2024-02-22T15:50:34.526 INFO:tasks.workunit.client.1.smithi062.stdout:9/172: creat d4/d35/f36 x:0 0 0 2024-02-22T15:50:34.527 INFO:tasks.workunit.client.0.smithi019.stdout:6/417: write d6/f3c [508777,116811] 0 2024-02-22T15:50:34.527 INFO:tasks.workunit.client.0.smithi019.stdout:4/440: mkdir d2/db/d14/d23/d67/d8f 0 2024-02-22T15:50:34.527 INFO:tasks.workunit.client.0.smithi019.stdout:6/418: chown d6/f22 1980904390 1 2024-02-22T15:50:34.528 INFO:tasks.workunit.client.1.smithi062.stdout:9/173: write d4/d15/d31/f32 [701666,81155] 0 2024-02-22T15:50:34.529 INFO:tasks.workunit.client.1.smithi062.stdout:9/174: dread - d4/d15/d20/f25 zero size 2024-02-22T15:50:34.530 INFO:tasks.workunit.client.0.smithi019.stdout:1/506: creat d8/d15/d5b/fb2 x:0 0 0 2024-02-22T15:50:34.531 INFO:tasks.workunit.client.1.smithi062.stdout:9/175: dread - d4/d15/d20/f25 zero size 2024-02-22T15:50:34.533 INFO:tasks.workunit.client.1.smithi062.stdout:9/176: write d4/d1c/f27 [609959,17006] 0 2024-02-22T15:50:34.533 INFO:tasks.workunit.client.1.smithi062.stdout:7/214: getdents d0/d14/d28 0 2024-02-22T15:50:34.533 INFO:tasks.workunit.client.0.smithi019.stdout:6/419: creat d6/d58/f79 x:0 0 0 2024-02-22T15:50:34.535 INFO:tasks.workunit.client.0.smithi019.stdout:4/441: write d2/db/d14/f87 [594153,37446] 0 2024-02-22T15:50:34.536 INFO:tasks.workunit.client.0.smithi019.stdout:8/437: getdents d9/d88 0 2024-02-22T15:50:34.536 INFO:tasks.workunit.client.0.smithi019.stdout:9/440: dwrite d2/d1f/d2e/d48/d36/f71 [0,4194304] 0 2024-02-22T15:50:34.537 INFO:tasks.workunit.client.1.smithi062.stdout:7/215: link d0/d14/f38 d0/d14/d19/d30/f49 0 2024-02-22T15:50:34.538 INFO:tasks.workunit.client.0.smithi019.stdout:4/442: truncate d2/f32 5750132 0 2024-02-22T15:50:34.542 INFO:tasks.workunit.client.1.smithi062.stdout:9/177: dwrite d4/f2f [0,4194304] 0 2024-02-22T15:50:34.543 INFO:tasks.workunit.client.0.smithi019.stdout:8/438: mkdir d9/d14/d6f/d7e/d94 0 2024-02-22T15:50:34.543 INFO:tasks.workunit.client.0.smithi019.stdout:9/441: mknod d2/d7/c9f 0 2024-02-22T15:50:34.545 INFO:tasks.workunit.client.0.smithi019.stdout:4/443: creat d2/db/d14/d23/d67/d8f/f90 x:0 0 0 2024-02-22T15:50:34.546 INFO:tasks.workunit.client.0.smithi019.stdout:8/439: stat d9/d14/d1f/d4c/c76 0 2024-02-22T15:50:34.547 INFO:tasks.workunit.client.0.smithi019.stdout:4/444: write d2/db/f26 [543259,122787] 0 2024-02-22T15:50:34.548 INFO:tasks.workunit.client.1.smithi062.stdout:7/216: rename d0/db to d0/d6/d2e/d4a 0 2024-02-22T15:50:34.554 INFO:tasks.workunit.client.0.smithi019.stdout:4/445: rename d2/d3f/f6b to d2/db/d14/d23/d33/d66/f91 0 2024-02-22T15:50:34.554 INFO:tasks.workunit.client.1.smithi062.stdout:9/178: getdents d4 0 2024-02-22T15:50:34.555 INFO:tasks.workunit.client.1.smithi062.stdout:9/179: write d4/f26 [501812,45802] 0 2024-02-22T15:50:34.555 INFO:tasks.workunit.client.1.smithi062.stdout:3/148: dread d5/f6 [0,4194304] 0 2024-02-22T15:50:34.556 INFO:tasks.workunit.client.0.smithi019.stdout:2/423: sync 2024-02-22T15:50:34.556 INFO:tasks.workunit.client.0.smithi019.stdout:0/329: sync 2024-02-22T15:50:34.557 INFO:tasks.workunit.client.1.smithi062.stdout:3/149: stat d5/d26/d2d/f2f 0 2024-02-22T15:50:34.559 INFO:tasks.workunit.client.1.smithi062.stdout:3/150: chown d5/d26/d2d/f2f 1 1 2024-02-22T15:50:34.561 INFO:tasks.workunit.client.0.smithi019.stdout:9/442: dwrite d2/d1f/d2e/d57/d60/d8c/f6a [0,4194304] 0 2024-02-22T15:50:34.561 INFO:tasks.workunit.client.1.smithi062.stdout:9/180: dread d4/d1c/f1d [0,4194304] 0 2024-02-22T15:50:34.562 INFO:tasks.workunit.client.0.smithi019.stdout:0/330: mknod d4/db/d76/c79 0 2024-02-22T15:50:34.565 INFO:tasks.workunit.client.0.smithi019.stdout:1/507: dwrite d8/d23/f55 [0,4194304] 0 2024-02-22T15:50:34.567 INFO:tasks.workunit.client.1.smithi062.stdout:7/217: fdatasync d0/d14/d13/f40 0 2024-02-22T15:50:34.571 INFO:tasks.workunit.client.0.smithi019.stdout:9/443: readlink d2/d1f/d2e/d48/d45/l6e 0 2024-02-22T15:50:34.571 INFO:tasks.workunit.client.0.smithi019.stdout:1/508: write d8/fd [2291222,4295] 0 2024-02-22T15:50:34.572 INFO:tasks.workunit.client.1.smithi062.stdout:3/151: dwrite d5/d1a/f25 [0,4194304] 0 2024-02-22T15:50:34.572 INFO:tasks.workunit.client.1.smithi062.stdout:9/181: creat d4/d35/f37 x:0 0 0 2024-02-22T15:50:34.573 INFO:tasks.workunit.client.0.smithi019.stdout:4/446: rename d2/db/d14/d23/d33/d66 to d2/db/d92 0 2024-02-22T15:50:34.573 INFO:tasks.workunit.client.0.smithi019.stdout:2/424: dread d1/d17/d21/d51/f72 [0,4194304] 0 2024-02-22T15:50:34.575 INFO:tasks.workunit.client.0.smithi019.stdout:1/509: write d8/d15/f20 [681063,12827] 0 2024-02-22T15:50:34.577 INFO:tasks.workunit.client.0.smithi019.stdout:0/331: getdents d4/d23 0 2024-02-22T15:50:34.578 INFO:tasks.workunit.client.1.smithi062.stdout:9/182: rename d4/f7 to d4/d28/f38 0 2024-02-22T15:50:34.581 INFO:tasks.workunit.client.0.smithi019.stdout:6/420: dread d6/f57 [0,4194304] 0 2024-02-22T15:50:34.582 INFO:tasks.workunit.client.0.smithi019.stdout:4/447: mknod d2/db/d59/c93 0 2024-02-22T15:50:34.582 INFO:tasks.workunit.client.1.smithi062.stdout:3/152: dwrite d5/d16/f19 [0,4194304] 0 2024-02-22T15:50:34.584 INFO:tasks.workunit.client.1.smithi062.stdout:7/218: truncate d0/d6/d2e/d4a/f2d 303237 0 2024-02-22T15:50:34.584 INFO:tasks.workunit.client.1.smithi062.stdout:3/153: rename d5 to d5/d26/d2d/d30 22 2024-02-22T15:50:34.585 INFO:tasks.workunit.client.1.smithi062.stdout:3/154: stat d5/d26/f27 0 2024-02-22T15:50:34.585 INFO:tasks.workunit.client.1.smithi062.stdout:7/219: dread - d0/d6/d2e/d2f/f3a zero size 2024-02-22T15:50:34.586 INFO:tasks.workunit.client.0.smithi019.stdout:0/332: chown d4/db/d76/c79 1 1 2024-02-22T15:50:34.588 INFO:tasks.workunit.client.0.smithi019.stdout:6/421: symlink d6/l7a 0 2024-02-22T15:50:34.588 INFO:tasks.workunit.client.0.smithi019.stdout:8/440: dwrite d9/d14/d17/f47 [0,4194304] 0 2024-02-22T15:50:34.588 INFO:tasks.workunit.client.0.smithi019.stdout:1/510: getdents d8/d18/d64/d7f/d94 0 2024-02-22T15:50:34.589 INFO:tasks.workunit.client.1.smithi062.stdout:7/220: creat d0/d14/d19/f4b x:0 0 0 2024-02-22T15:50:34.589 INFO:tasks.workunit.client.0.smithi019.stdout:9/444: dread d2/d1f/d2e/d48/d36/f71 [0,4194304] 0 2024-02-22T15:50:34.589 INFO:tasks.workunit.client.0.smithi019.stdout:4/448: mknod d2/db/d14/d23/d21/d42/c94 0 2024-02-22T15:50:34.591 INFO:tasks.workunit.client.1.smithi062.stdout:3/155: link d5/d16/d21/f2b d5/d26/f31 0 2024-02-22T15:50:34.591 INFO:tasks.workunit.client.0.smithi019.stdout:0/333: truncate d4/d22/f36 1111911 0 2024-02-22T15:50:34.591 INFO:tasks.workunit.client.1.smithi062.stdout:3/156: stat d5/lb 0 2024-02-22T15:50:34.593 INFO:tasks.workunit.client.1.smithi062.stdout:7/221: creat d0/d6/d2e/d4a/f4c x:0 0 0 2024-02-22T15:50:34.594 INFO:tasks.workunit.client.1.smithi062.stdout:3/157: mkdir d5/d16/d21/d32 0 2024-02-22T15:50:34.595 INFO:tasks.workunit.client.0.smithi019.stdout:6/422: chown d6 0 1 2024-02-22T15:50:34.596 INFO:tasks.workunit.client.1.smithi062.stdout:7/222: write d0/f10 [1918357,1431] 0 2024-02-22T15:50:34.597 INFO:tasks.workunit.client.1.smithi062.stdout:9/183: truncate d4/f14 2194759 0 2024-02-22T15:50:34.598 INFO:tasks.workunit.client.0.smithi019.stdout:9/445: creat d2/d1f/d2e/d48/d45/d5a/fa0 x:0 0 0 2024-02-22T15:50:34.598 INFO:tasks.workunit.client.1.smithi062.stdout:3/158: creat d5/d1a/f33 x:0 0 0 2024-02-22T15:50:34.599 INFO:tasks.workunit.client.0.smithi019.stdout:0/334: mknod d4/d22/d2c/d68/c7a 0 2024-02-22T15:50:34.600 INFO:tasks.workunit.client.1.smithi062.stdout:7/223: creat d0/d14/d13/f4d x:0 0 0 2024-02-22T15:50:34.601 INFO:tasks.workunit.client.1.smithi062.stdout:3/159: write f3 [1751199,106765] 0 2024-02-22T15:50:34.602 INFO:tasks.workunit.client.0.smithi019.stdout:8/441: read - d9/f64 zero size 2024-02-22T15:50:34.604 INFO:tasks.workunit.client.1.smithi062.stdout:9/184: chown d4/d1c/l1f 13 1 2024-02-22T15:50:34.604 INFO:tasks.workunit.client.0.smithi019.stdout:1/511: getdents d8/d18/d64/d7f/d82 0 2024-02-22T15:50:34.606 INFO:tasks.workunit.client.1.smithi062.stdout:3/160: creat d5/d16/d21/f34 x:0 0 0 2024-02-22T15:50:34.608 INFO:tasks.workunit.client.1.smithi062.stdout:9/185: write d4/f8 [414496,107133] 0 2024-02-22T15:50:34.610 INFO:tasks.workunit.client.1.smithi062.stdout:9/186: read d4/f8 [3287626,17529] 0 2024-02-22T15:50:34.611 INFO:tasks.workunit.client.0.smithi019.stdout:4/449: dwrite d2/fd [0,4194304] 0 2024-02-22T15:50:34.612 INFO:tasks.workunit.client.0.smithi019.stdout:0/335: mknod d4/d32/c7b 0 2024-02-22T15:50:34.612 INFO:tasks.workunit.client.0.smithi019.stdout:6/423: dread d6/f2c [0,4194304] 0 2024-02-22T15:50:34.613 INFO:tasks.workunit.client.1.smithi062.stdout:1/177: sync 2024-02-22T15:50:34.613 INFO:tasks.workunit.client.0.smithi019.stdout:2/425: dread d1/de/f31 [0,4194304] 0 2024-02-22T15:50:34.614 INFO:tasks.workunit.client.0.smithi019.stdout:0/336: symlink d4/d67/l7c 0 2024-02-22T15:50:34.615 INFO:tasks.workunit.client.0.smithi019.stdout:1/512: rename d8/d2a/da7 to d8/d2a/db3 0 2024-02-22T15:50:34.615 INFO:tasks.workunit.client.1.smithi062.stdout:7/224: dwrite d0/d14/f20 [0,4194304] 0 2024-02-22T15:50:34.616 INFO:tasks.workunit.client.0.smithi019.stdout:4/450: chown d2/db/d14/d23/c4c 60693 1 2024-02-22T15:50:34.617 INFO:tasks.workunit.client.0.smithi019.stdout:9/446: dwrite d2/f30 [8388608,4194304] 0 2024-02-22T15:50:34.617 INFO:tasks.workunit.client.0.smithi019.stdout:6/424: rename d6/d31/d37 to d6/d4b/d6d/d7b 0 2024-02-22T15:50:34.618 INFO:tasks.workunit.client.1.smithi062.stdout:1/178: mknod d1/d6/d14/c3c 0 2024-02-22T15:50:34.619 INFO:tasks.workunit.client.1.smithi062.stdout:9/187: truncate d4/d28/f30 16047 0 2024-02-22T15:50:34.621 INFO:tasks.workunit.client.0.smithi019.stdout:6/425: fdatasync d6/f22 0 2024-02-22T15:50:34.621 INFO:tasks.workunit.client.1.smithi062.stdout:1/179: fdatasync d1/d6/d10/f1c 0 2024-02-22T15:50:34.622 INFO:tasks.workunit.client.1.smithi062.stdout:3/161: dwrite d5/f1d [0,4194304] 0 2024-02-22T15:50:34.622 INFO:tasks.workunit.client.0.smithi019.stdout:0/337: chown d4/d22/d3d/l4b 26085 1 2024-02-22T15:50:34.624 INFO:tasks.workunit.client.0.smithi019.stdout:2/426: mknod d1/de/d15/c99 0 2024-02-22T15:50:34.627 INFO:tasks.workunit.client.0.smithi019.stdout:9/447: truncate d2/d7/d14/f82 2078549 0 2024-02-22T15:50:34.629 INFO:tasks.workunit.client.0.smithi019.stdout:0/338: mkdir d4/d23/d7d 0 2024-02-22T15:50:34.632 INFO:tasks.workunit.client.1.smithi062.stdout:3/162: rename d5/d26/d2d/f2f to d5/d26/f35 0 2024-02-22T15:50:34.634 INFO:tasks.workunit.client.0.smithi019.stdout:2/427: mkdir d1/de/d9a 0 2024-02-22T15:50:34.635 INFO:tasks.workunit.client.1.smithi062.stdout:9/188: dwrite d4/f1b [0,4194304] 0 2024-02-22T15:50:34.638 INFO:tasks.workunit.client.1.smithi062.stdout:9/189: write d4/d1c/f2b [2528664,11732] 0 2024-02-22T15:50:34.640 INFO:tasks.workunit.client.0.smithi019.stdout:8/442: write d9/f29 [684236,8689] 0 2024-02-22T15:50:34.642 INFO:tasks.workunit.client.0.smithi019.stdout:9/448: symlink d2/d1f/d2e/d57/d60/d8c/d66/la1 0 2024-02-22T15:50:34.643 INFO:tasks.workunit.client.0.smithi019.stdout:1/513: dwrite d8/d18/d64/f67 [0,4194304] 0 2024-02-22T15:50:34.643 INFO:tasks.workunit.client.0.smithi019.stdout:2/428: creat d1/d17/d4b/f9b x:0 0 0 2024-02-22T15:50:34.649 INFO:tasks.workunit.client.0.smithi019.stdout:1/514: fsync d8/d23/d33/f6a 0 2024-02-22T15:50:34.651 INFO:tasks.workunit.client.0.smithi019.stdout:9/449: creat d2/d1f/d95/fa2 x:0 0 0 2024-02-22T15:50:34.651 INFO:tasks.workunit.client.1.smithi062.stdout:3/163: write d5/f6 [2965346,76608] 0 2024-02-22T15:50:34.652 INFO:tasks.workunit.client.0.smithi019.stdout:2/429: write d1/f14 [511199,11761] 0 2024-02-22T15:50:34.652 INFO:tasks.workunit.client.0.smithi019.stdout:6/426: write d6/d4b/f52 [1190614,77286] 0 2024-02-22T15:50:34.653 INFO:tasks.workunit.client.0.smithi019.stdout:4/451: dwrite d2/db/d92/f8a [0,4194304] 0 2024-02-22T15:50:34.657 INFO:tasks.workunit.client.0.smithi019.stdout:9/450: write d2/d1f/d2e/d48/d45/d5a/fa0 [324880,67244] 0 2024-02-22T15:50:34.661 INFO:tasks.workunit.client.1.smithi062.stdout:3/164: dread d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:34.662 INFO:tasks.workunit.client.0.smithi019.stdout:0/339: dwrite f2 [0,4194304] 0 2024-02-22T15:50:34.662 INFO:tasks.workunit.client.0.smithi019.stdout:9/451: chown d2/d1f/d2e/d57/d60/d8c/d66/l5c 67499 1 2024-02-22T15:50:34.663 INFO:tasks.workunit.client.0.smithi019.stdout:8/443: link d9/d14/d17/d5d/f68 d9/d14/d1f/d6d/f95 0 2024-02-22T15:50:34.664 INFO:tasks.workunit.client.1.smithi062.stdout:3/165: mkdir d5/d16/d21/d36 0 2024-02-22T15:50:34.664 INFO:tasks.workunit.client.0.smithi019.stdout:9/452: fsync d2/d7/dc/f29 0 2024-02-22T15:50:34.665 INFO:tasks.workunit.client.0.smithi019.stdout:2/430: mknod d1/d86/c9c 0 2024-02-22T15:50:34.665 INFO:tasks.workunit.client.0.smithi019.stdout:8/444: readlink d9/d14/d17/d3d/d3f/l4f 0 2024-02-22T15:50:34.666 INFO:tasks.workunit.client.1.smithi062.stdout:3/166: chown d5/d26/d2d 40 1 2024-02-22T15:50:34.666 INFO:tasks.workunit.client.0.smithi019.stdout:2/431: chown d1/d17/d1c/f22 1 1 2024-02-22T15:50:34.667 INFO:tasks.workunit.client.1.smithi062.stdout:3/167: dread - d5/d26/f27 zero size 2024-02-22T15:50:34.672 INFO:tasks.workunit.client.1.smithi062.stdout:1/180: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:34.674 INFO:tasks.workunit.client.0.smithi019.stdout:9/453: rename d2/d1f/d2e/d57/d60/d8c/l78 to d2/d1f/d2e/d57/d60/d8c/d66/la3 0 2024-02-22T15:50:34.674 INFO:tasks.workunit.client.0.smithi019.stdout:0/340: rmdir d4/db/d24/d74 0 2024-02-22T15:50:34.675 INFO:tasks.workunit.client.1.smithi062.stdout:3/168: dread d5/d1a/f1b [0,4194304] 0 2024-02-22T15:50:34.676 INFO:tasks.workunit.client.1.smithi062.stdout:3/169: dread - d5/d26/f27 zero size 2024-02-22T15:50:34.677 INFO:tasks.workunit.client.0.smithi019.stdout:9/454: unlink d2/l35 0 2024-02-22T15:50:34.677 INFO:tasks.workunit.client.1.smithi062.stdout:1/181: getdents d1 0 2024-02-22T15:50:34.678 INFO:tasks.workunit.client.0.smithi019.stdout:6/427: link d6/c30 d6/d31/d43/d55/d67/c7c 0 2024-02-22T15:50:34.679 INFO:tasks.workunit.client.0.smithi019.stdout:0/341: symlink d4/db/d1b/d40/l7e 0 2024-02-22T15:50:34.680 INFO:tasks.workunit.client.1.smithi062.stdout:3/170: rename d5/c18 to d5/d16/d21/d36/c37 0 2024-02-22T15:50:34.681 INFO:tasks.workunit.client.1.smithi062.stdout:3/171: chown f3 4 1 2024-02-22T15:50:34.682 INFO:tasks.workunit.client.1.smithi062.stdout:3/172: write d5/f1d [1819373,49976] 0 2024-02-22T15:50:34.682 INFO:tasks.workunit.client.0.smithi019.stdout:9/455: dread d2/d1f/d2e/d48/d45/f61 [0,4194304] 0 2024-02-22T15:50:34.683 INFO:tasks.workunit.client.0.smithi019.stdout:0/342: mkdir d4/db/d7f 0 2024-02-22T15:50:34.684 INFO:tasks.workunit.client.0.smithi019.stdout:9/456: stat d2/d7/dc/d27/d68/f6b 0 2024-02-22T15:50:34.685 INFO:tasks.workunit.client.0.smithi019.stdout:6/428: creat d6/d4b/d71/f7d x:0 0 0 2024-02-22T15:50:34.685 INFO:tasks.workunit.client.1.smithi062.stdout:3/173: creat d5/d26/f38 x:0 0 0 2024-02-22T15:50:34.686 INFO:tasks.workunit.client.0.smithi019.stdout:0/343: mkdir d4/db/d24/d80 0 2024-02-22T15:50:34.687 INFO:tasks.workunit.client.1.smithi062.stdout:1/182: dwrite d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:34.687 INFO:tasks.workunit.client.0.smithi019.stdout:2/432: rmdir d1 39 2024-02-22T15:50:34.687 INFO:tasks.workunit.client.1.smithi062.stdout:3/174: mknod d5/d1a/c39 0 2024-02-22T15:50:34.690 INFO:tasks.workunit.client.1.smithi062.stdout:3/175: creat d5/d16/d21/f3a x:0 0 0 2024-02-22T15:50:34.694 INFO:tasks.workunit.client.1.smithi062.stdout:1/183: dwrite d1/f2e [0,4194304] 0 2024-02-22T15:50:34.695 INFO:tasks.workunit.client.0.smithi019.stdout:9/457: dwrite d2/f4 [4194304,4194304] 0 2024-02-22T15:50:34.698 INFO:tasks.workunit.client.0.smithi019.stdout:0/344: rename d4/d22/d2c/d68/c7a to d4/db/d76/c81 0 2024-02-22T15:50:34.699 INFO:tasks.workunit.client.1.smithi062.stdout:3/176: dwrite d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:34.700 INFO:tasks.workunit.client.0.smithi019.stdout:2/433: mkdir d1/d46/d9d 0 2024-02-22T15:50:34.703 INFO:tasks.workunit.client.1.smithi062.stdout:3/177: write d5/d1a/f25 [281352,42654] 0 2024-02-22T15:50:34.704 INFO:tasks.workunit.client.0.smithi019.stdout:0/345: write d4/d5/d7/f72 [717945,53319] 0 2024-02-22T15:50:34.704 INFO:tasks.workunit.client.0.smithi019.stdout:9/458: creat d2/d1f/d2e/d48/d36/fa4 x:0 0 0 2024-02-22T15:50:34.707 INFO:tasks.workunit.client.0.smithi019.stdout:8/445: dwrite d9/d14/f3e [0,4194304] 0 2024-02-22T15:50:34.707 INFO:tasks.workunit.client.0.smithi019.stdout:4/452: dread d2/db/d14/d23/d21/f2d [4194304,4194304] 0 2024-02-22T15:50:34.707 INFO:tasks.workunit.client.1.smithi062.stdout:1/184: dwrite d1/d6/d7/f3a [0,4194304] 0 2024-02-22T15:50:34.709 INFO:tasks.workunit.client.1.smithi062.stdout:3/178: write d5/d1a/f25 [3740175,53851] 0 2024-02-22T15:50:34.711 INFO:tasks.workunit.client.0.smithi019.stdout:4/453: chown d2/db/d14/d23/d67/l8b 72415 1 2024-02-22T15:50:34.715 INFO:tasks.workunit.client.1.smithi062.stdout:1/185: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:34.716 INFO:tasks.workunit.client.0.smithi019.stdout:4/454: fdatasync d2/fd 0 2024-02-22T15:50:34.716 INFO:tasks.workunit.client.0.smithi019.stdout:2/434: link d1/d17/f90 d1/d86/f9e 0 2024-02-22T15:50:34.719 INFO:tasks.workunit.client.1.smithi062.stdout:1/186: mknod d1/d6/de/d21/d24/c3d 0 2024-02-22T15:50:34.724 INFO:tasks.workunit.client.0.smithi019.stdout:2/435: write d1/d17/d21/f6f [131512,59203] 0 2024-02-22T15:50:34.726 INFO:tasks.workunit.client.0.smithi019.stdout:4/455: readlink d2/db/d14/d23/d21/d35/l6c 0 2024-02-22T15:50:34.730 INFO:tasks.workunit.client.0.smithi019.stdout:2/436: symlink d1/d46/d9d/l9f 0 2024-02-22T15:50:34.730 INFO:tasks.workunit.client.0.smithi019.stdout:0/346: dwrite d4/d22/f5f [0,4194304] 0 2024-02-22T15:50:34.732 INFO:tasks.workunit.client.0.smithi019.stdout:9/459: dread d2/f4 [4194304,4194304] 0 2024-02-22T15:50:34.733 INFO:tasks.workunit.client.0.smithi019.stdout:0/347: dread - d4/d22/d2c/f57 zero size 2024-02-22T15:50:34.737 INFO:tasks.workunit.client.0.smithi019.stdout:6/429: dread d6/f9 [0,4194304] 0 2024-02-22T15:50:34.738 INFO:tasks.workunit.client.0.smithi019.stdout:9/460: mknod d2/d1f/d95/ca5 0 2024-02-22T15:50:34.742 INFO:tasks.workunit.client.0.smithi019.stdout:9/461: fdatasync d2/f6f 0 2024-02-22T15:50:34.743 INFO:tasks.workunit.client.0.smithi019.stdout:6/430: creat d6/d4b/d6d/d7b/d78/f7e x:0 0 0 2024-02-22T15:50:34.743 INFO:tasks.workunit.client.0.smithi019.stdout:0/348: dread d4/d5/d52/f58 [0,4194304] 0 2024-02-22T15:50:34.745 INFO:tasks.workunit.client.0.smithi019.stdout:2/437: dwrite d1/de/f16 [0,4194304] 0 2024-02-22T15:50:34.750 INFO:tasks.workunit.client.0.smithi019.stdout:4/456: dwrite d2/db/d14/f87 [0,4194304] 0 2024-02-22T15:50:34.751 INFO:tasks.workunit.client.0.smithi019.stdout:9/462: chown d2/d1f/d2e/d48/d45/d5a/c9b 1010171623 1 2024-02-22T15:50:34.753 INFO:tasks.workunit.client.0.smithi019.stdout:0/349: truncate f1 831471 0 2024-02-22T15:50:34.757 INFO:tasks.workunit.client.0.smithi019.stdout:9/463: read d2/d7/dc/f49 [248251,109204] 0 2024-02-22T15:50:34.761 INFO:tasks.workunit.client.1.smithi062.stdout:1/187: write d1/d6/f1a [651078,10352] 0 2024-02-22T15:50:34.764 INFO:tasks.workunit.client.0.smithi019.stdout:9/464: mkdir d2/d1f/d2e/d48/da6 0 2024-02-22T15:50:34.764 INFO:tasks.workunit.client.1.smithi062.stdout:1/188: mknod d1/d6/d7/c3e 0 2024-02-22T15:50:34.766 INFO:tasks.workunit.client.1.smithi062.stdout:1/189: write d1/d6/de/d21/f2f [1080412,70907] 0 2024-02-22T15:50:34.766 INFO:tasks.workunit.client.1.smithi062.stdout:1/190: chown d1/l1e 113375 1 2024-02-22T15:50:34.767 INFO:tasks.workunit.client.0.smithi019.stdout:2/438: dwrite d1/d17/d1c/f22 [0,4194304] 0 2024-02-22T15:50:34.768 INFO:tasks.workunit.client.1.smithi062.stdout:1/191: write d1/d6/f1a [2103611,14994] 0 2024-02-22T15:50:34.768 INFO:tasks.workunit.client.0.smithi019.stdout:4/457: dwrite d2/db/f6a [0,4194304] 0 2024-02-22T15:50:34.769 INFO:tasks.workunit.client.1.smithi062.stdout:1/192: chown d1/d6/d7/c19 14 1 2024-02-22T15:50:34.770 INFO:tasks.workunit.client.1.smithi062.stdout:1/193: mknod d1/c3f 0 2024-02-22T15:50:34.774 INFO:tasks.workunit.client.0.smithi019.stdout:2/439: fdatasync d1/d17/d1c/f34 0 2024-02-22T15:50:34.777 INFO:tasks.workunit.client.0.smithi019.stdout:2/440: write d1/d17/d21/f87 [937100,17794] 0 2024-02-22T15:50:34.777 INFO:tasks.workunit.client.0.smithi019.stdout:4/458: rmdir d2/db/d14/d23/d21/d35/d3d 39 2024-02-22T15:50:34.778 INFO:tasks.workunit.client.0.smithi019.stdout:9/465: symlink d2/d1f/d2e/d48/d36/la7 0 2024-02-22T15:50:34.778 INFO:tasks.workunit.client.1.smithi062.stdout:1/194: getdents d1 0 2024-02-22T15:50:34.782 INFO:tasks.workunit.client.1.smithi062.stdout:2/208: sync 2024-02-22T15:50:34.782 INFO:tasks.workunit.client.1.smithi062.stdout:0/210: sync 2024-02-22T15:50:34.782 INFO:tasks.workunit.client.1.smithi062.stdout:1/195: read d1/d6/de/f17 [3022325,6489] 0 2024-02-22T15:50:34.783 INFO:tasks.workunit.client.0.smithi019.stdout:8/446: dwrite d9/d14/d17/d5d/f68 [0,4194304] 0 2024-02-22T15:50:34.783 INFO:tasks.workunit.client.1.smithi062.stdout:1/196: write d1/d6/d7/f28 [792056,63570] 0 2024-02-22T15:50:34.784 INFO:tasks.workunit.client.1.smithi062.stdout:2/209: readlink d1/d1e/d20/d32/l3a 0 2024-02-22T15:50:34.784 INFO:tasks.workunit.client.1.smithi062.stdout:2/210: readlink d1/lc 0 2024-02-22T15:50:34.785 INFO:tasks.workunit.client.0.smithi019.stdout:9/466: fsync d2/d1f/d2e/d57/d60/d8c/d66/f64 0 2024-02-22T15:50:34.787 INFO:tasks.workunit.client.1.smithi062.stdout:0/211: mknod d1/d3/de/d14/c44 0 2024-02-22T15:50:34.789 INFO:tasks.workunit.client.1.smithi062.stdout:1/197: unlink d1/d6/c11 0 2024-02-22T15:50:34.789 INFO:tasks.workunit.client.1.smithi062.stdout:2/211: fsync d1/d8/df/f31 0 2024-02-22T15:50:34.790 INFO:tasks.workunit.client.1.smithi062.stdout:1/198: fsync d1/d6/f1a 0 2024-02-22T15:50:34.790 INFO:tasks.workunit.client.0.smithi019.stdout:2/441: rename d1/d17/d1c/l91 to d1/de/d15/la0 0 2024-02-22T15:50:34.791 INFO:tasks.workunit.client.0.smithi019.stdout:4/459: creat d2/db/d14/d23/d4b/d8e/f95 x:0 0 0 2024-02-22T15:50:34.792 INFO:tasks.workunit.client.1.smithi062.stdout:2/212: stat d1/c30 0 2024-02-22T15:50:34.793 INFO:tasks.workunit.client.0.smithi019.stdout:0/350: dread f1 [0,4194304] 0 2024-02-22T15:50:34.793 INFO:tasks.workunit.client.0.smithi019.stdout:9/467: mknod d2/d1f/d2e/d48/d89/ca8 0 2024-02-22T15:50:34.794 INFO:tasks.workunit.client.1.smithi062.stdout:0/212: rename d1/d3/de/d14/f17 to d1/f45 0 2024-02-22T15:50:34.796 INFO:tasks.workunit.client.0.smithi019.stdout:6/431: dread d6/f1d [0,4194304] 0 2024-02-22T15:50:34.796 INFO:tasks.workunit.client.0.smithi019.stdout:8/447: dread d9/d14/d1f/d6d/f95 [0,4194304] 0 2024-02-22T15:50:34.796 INFO:tasks.workunit.client.0.smithi019.stdout:2/442: creat d1/d17/d21/d51/fa1 x:0 0 0 2024-02-22T15:50:34.798 INFO:tasks.workunit.client.0.smithi019.stdout:9/468: write d2/d1f/f4b [543126,381] 0 2024-02-22T15:50:34.799 INFO:tasks.workunit.client.1.smithi062.stdout:1/199: rename d1/d6/d7/c3e to d1/d6/d7/d2b/c40 0 2024-02-22T15:50:34.800 INFO:tasks.workunit.client.0.smithi019.stdout:6/432: write d6/d31/d43/f46 [440368,57457] 0 2024-02-22T15:50:34.800 INFO:tasks.workunit.client.1.smithi062.stdout:1/200: write d1/f32 [205895,70698] 0 2024-02-22T15:50:34.801 INFO:tasks.workunit.client.0.smithi019.stdout:4/460: rename d2/db/d14/d23/d21/f5a to d2/db/d54/d8c/f96 0 2024-02-22T15:50:34.802 INFO:tasks.workunit.client.1.smithi062.stdout:0/213: creat d1/d21/d3e/f46 x:0 0 0 2024-02-22T15:50:34.803 INFO:tasks.workunit.client.1.smithi062.stdout:2/213: creat d1/f3c x:0 0 0 2024-02-22T15:50:34.805 INFO:tasks.workunit.client.0.smithi019.stdout:9/469: mknod d2/d1f/d2e/d7d/ca9 0 2024-02-22T15:50:34.806 INFO:tasks.workunit.client.0.smithi019.stdout:8/448: dread d9/fa [0,4194304] 0 2024-02-22T15:50:34.806 INFO:tasks.workunit.client.1.smithi062.stdout:2/214: creat d1/d8/f3d x:0 0 0 2024-02-22T15:50:34.806 INFO:tasks.workunit.client.1.smithi062.stdout:1/201: getdents d1/d6/d14 0 2024-02-22T15:50:34.808 INFO:tasks.workunit.client.1.smithi062.stdout:0/214: rename d1/d3/d4/d1a/f3d to d1/d21/f47 0 2024-02-22T15:50:34.809 INFO:tasks.workunit.client.1.smithi062.stdout:2/215: dread d1/d8/f15 [0,4194304] 0 2024-02-22T15:50:34.810 INFO:tasks.workunit.client.1.smithi062.stdout:2/216: write d1/f16 [2809866,53639] 0 2024-02-22T15:50:34.810 INFO:tasks.workunit.client.1.smithi062.stdout:0/215: write d1/f45 [1643643,20410] 0 2024-02-22T15:50:34.811 INFO:tasks.workunit.client.1.smithi062.stdout:1/202: truncate d1/d6/d10/f1c 2214554 0 2024-02-22T15:50:34.812 INFO:tasks.workunit.client.0.smithi019.stdout:9/470: rename d2/d1f/d2e/d57/d60/d8c/d66/f64 to d2/d7/d52/faa 0 2024-02-22T15:50:34.814 INFO:tasks.workunit.client.0.smithi019.stdout:8/449: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:34.814 INFO:tasks.workunit.client.1.smithi062.stdout:1/203: symlink d1/d6/de/d9/l41 0 2024-02-22T15:50:34.818 INFO:tasks.workunit.client.0.smithi019.stdout:8/450: fdatasync d9/fc 0 2024-02-22T15:50:34.819 INFO:tasks.workunit.client.1.smithi062.stdout:0/216: dread d1/d3/de/d14/f26 [0,4194304] 0 2024-02-22T15:50:34.821 INFO:tasks.workunit.client.0.smithi019.stdout:0/351: dread d4/d32/f46 [0,4194304] 0 2024-02-22T15:50:34.822 INFO:tasks.workunit.client.1.smithi062.stdout:2/217: dwrite d1/f3c [0,4194304] 0 2024-02-22T15:50:34.822 INFO:tasks.workunit.client.1.smithi062.stdout:0/217: readlink d1/d3/l19 0 2024-02-22T15:50:34.824 INFO:tasks.workunit.client.1.smithi062.stdout:1/204: creat d1/d6/de/f42 x:0 0 0 2024-02-22T15:50:34.824 INFO:tasks.workunit.client.0.smithi019.stdout:6/433: dread d6/f10 [0,4194304] 0 2024-02-22T15:50:34.825 INFO:tasks.workunit.client.1.smithi062.stdout:0/218: creat d1/d21/f48 x:0 0 0 2024-02-22T15:50:34.825 INFO:tasks.workunit.client.1.smithi062.stdout:1/205: write d1/f30 [730453,13078] 0 2024-02-22T15:50:34.826 INFO:tasks.workunit.client.1.smithi062.stdout:2/218: getdents d1/d1e/d38 0 2024-02-22T15:50:34.826 INFO:tasks.workunit.client.1.smithi062.stdout:0/219: write d1/d21/f24 [3719905,3360] 0 2024-02-22T15:50:34.828 INFO:tasks.workunit.client.1.smithi062.stdout:2/219: chown d1/d8/df/c1f 46670480 1 2024-02-22T15:50:34.828 INFO:tasks.workunit.client.1.smithi062.stdout:1/206: chown d1/d6/d10/l18 2918 1 2024-02-22T15:50:34.829 INFO:tasks.workunit.client.1.smithi062.stdout:0/220: creat d1/d21/f49 x:0 0 0 2024-02-22T15:50:34.832 INFO:tasks.workunit.client.0.smithi019.stdout:6/434: mkdir d6/d4b/d6d/d7b/d7f 0 2024-02-22T15:50:34.833 INFO:tasks.workunit.client.1.smithi062.stdout:0/221: symlink d1/d3/d4/d1a/d1d/l4a 0 2024-02-22T15:50:34.833 INFO:tasks.workunit.client.1.smithi062.stdout:1/207: dread d1/d6/de/d21/d24/f2a [0,4194304] 0 2024-02-22T15:50:34.834 INFO:tasks.workunit.client.0.smithi019.stdout:0/352: dwrite d4/d22/f5f [4194304,4194304] 0 2024-02-22T15:50:34.835 INFO:tasks.workunit.client.0.smithi019.stdout:8/451: dwrite d9/d14/d17/f93 [0,4194304] 0 2024-02-22T15:50:34.836 INFO:tasks.workunit.client.1.smithi062.stdout:1/208: mknod d1/d6/d10/c43 0 2024-02-22T15:50:34.837 INFO:tasks.workunit.client.0.smithi019.stdout:6/435: write d6/f9 [5716300,65093] 0 2024-02-22T15:50:34.837 INFO:tasks.workunit.client.0.smithi019.stdout:0/353: fdatasync d4/d5/d52/f71 0 2024-02-22T15:50:34.838 INFO:tasks.workunit.client.1.smithi062.stdout:0/222: rename d1/d3/l29 to d1/d13/l4b 0 2024-02-22T15:50:34.841 INFO:tasks.workunit.client.1.smithi062.stdout:2/220: dwrite d1/d1e/f2d [0,4194304] 0 2024-02-22T15:50:34.841 INFO:tasks.workunit.client.0.smithi019.stdout:8/452: fsync d9/fc 0 2024-02-22T15:50:34.844 INFO:tasks.workunit.client.1.smithi062.stdout:2/221: read d1/d1e/d20/f24 [4084952,9749] 0 2024-02-22T15:50:34.844 INFO:tasks.workunit.client.1.smithi062.stdout:0/223: write d1/d3/d4/d1a/f34 [1176407,18320] 0 2024-02-22T15:50:34.846 INFO:tasks.workunit.client.0.smithi019.stdout:9/471: dwrite d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:34.849 INFO:tasks.workunit.client.1.smithi062.stdout:2/222: truncate d1/d8/f3d 454313 0 2024-02-22T15:50:34.851 INFO:tasks.workunit.client.1.smithi062.stdout:2/223: chown d1/cd 50134847 1 2024-02-22T15:50:34.851 INFO:tasks.workunit.client.0.smithi019.stdout:0/354: creat d4/db/d1b/d40/d44/d75/f82 x:0 0 0 2024-02-22T15:50:34.852 INFO:tasks.workunit.client.1.smithi062.stdout:0/224: write d1/d13/f2f [567001,44571] 0 2024-02-22T15:50:34.852 INFO:tasks.workunit.client.1.smithi062.stdout:1/209: dwrite d1/d6/d7/f28 [0,4194304] 0 2024-02-22T15:50:34.857 INFO:tasks.workunit.client.1.smithi062.stdout:1/210: creat d1/f44 x:0 0 0 2024-02-22T15:50:34.858 INFO:tasks.workunit.client.0.smithi019.stdout:6/436: dread d6/d4b/d6d/d7b/f60 [0,4194304] 0 2024-02-22T15:50:34.858 INFO:tasks.workunit.client.0.smithi019.stdout:0/355: symlink d4/db/d76/d77/l83 0 2024-02-22T15:50:34.860 INFO:tasks.workunit.client.1.smithi062.stdout:0/225: creat d1/d3/de/d3f/f4c x:0 0 0 2024-02-22T15:50:34.863 INFO:tasks.workunit.client.0.smithi019.stdout:9/472: unlink d2/d7/dc/l1d 0 2024-02-22T15:50:34.864 INFO:tasks.workunit.client.0.smithi019.stdout:9/473: stat d2/d1f/d2e/d48/d45/d5a/fa0 0 2024-02-22T15:50:34.864 INFO:tasks.workunit.client.1.smithi062.stdout:0/226: dread - d1/d3/de/d3f/f4c zero size 2024-02-22T15:50:34.865 INFO:tasks.workunit.client.1.smithi062.stdout:2/224: dwrite d1/d8/f3d [0,4194304] 0 2024-02-22T15:50:34.866 INFO:tasks.workunit.client.1.smithi062.stdout:0/227: fsync d1/f45 0 2024-02-22T15:50:34.866 INFO:tasks.workunit.client.1.smithi062.stdout:2/225: chown d1/d8/df 2 1 2024-02-22T15:50:34.866 INFO:tasks.workunit.client.0.smithi019.stdout:8/453: truncate d9/ff 745691 0 2024-02-22T15:50:34.867 INFO:tasks.workunit.client.0.smithi019.stdout:0/356: rename d4/d9/l6c to d4/db/l84 0 2024-02-22T15:50:34.868 INFO:tasks.workunit.client.1.smithi062.stdout:6/253: sync 2024-02-22T15:50:34.868 INFO:tasks.workunit.client.0.smithi019.stdout:9/474: rmdir d2/d7/d14 39 2024-02-22T15:50:34.870 INFO:tasks.workunit.client.1.smithi062.stdout:0/228: chown d1/d3/de/d14/l3c 34 1 2024-02-22T15:50:34.871 INFO:tasks.workunit.client.0.smithi019.stdout:8/454: mknod d9/d14/d6f/d8d/c96 0 2024-02-22T15:50:34.871 INFO:tasks.workunit.client.0.smithi019.stdout:6/437: dwrite d6/f2c [0,4194304] 0 2024-02-22T15:50:34.872 INFO:tasks.workunit.client.1.smithi062.stdout:0/229: read - d1/d13/f31 zero size 2024-02-22T15:50:34.872 INFO:tasks.workunit.client.1.smithi062.stdout:8/186: sync 2024-02-22T15:50:34.872 INFO:tasks.workunit.client.1.smithi062.stdout:5/114: sync 2024-02-22T15:50:34.872 INFO:tasks.workunit.client.1.smithi062.stdout:4/141: sync 2024-02-22T15:50:34.874 INFO:tasks.workunit.client.0.smithi019.stdout:0/357: read d4/db/d1b/f2f [3110965,58051] 0 2024-02-22T15:50:34.877 INFO:tasks.workunit.client.1.smithi062.stdout:4/142: mkdir d4/d1c/d30 0 2024-02-22T15:50:34.879 INFO:tasks.workunit.client.0.smithi019.stdout:9/475: creat d2/fab x:0 0 0 2024-02-22T15:50:34.879 INFO:tasks.workunit.client.1.smithi062.stdout:8/187: mknod d8/d26/c31 0 2024-02-22T15:50:34.881 INFO:tasks.workunit.client.0.smithi019.stdout:6/438: dread d6/f2c [0,4194304] 0 2024-02-22T15:50:34.881 INFO:tasks.workunit.client.1.smithi062.stdout:2/226: dwrite d1/d8/df/f14 [0,4194304] 0 2024-02-22T15:50:34.883 INFO:tasks.workunit.client.0.smithi019.stdout:8/455: symlink d9/d14/d17/l97 0 2024-02-22T15:50:34.883 INFO:tasks.workunit.client.1.smithi062.stdout:4/143: chown d4/df/f14 290250037 1 2024-02-22T15:50:34.883 INFO:tasks.workunit.client.1.smithi062.stdout:6/254: dwrite f0 [0,4194304] 0 2024-02-22T15:50:34.885 INFO:tasks.workunit.client.1.smithi062.stdout:8/188: mknod d8/d28/c32 0 2024-02-22T15:50:34.886 INFO:tasks.workunit.client.0.smithi019.stdout:9/476: fdatasync d2/d7/d52/faa 0 2024-02-22T15:50:34.886 INFO:tasks.workunit.client.1.smithi062.stdout:8/189: stat d8/c11 0 2024-02-22T15:50:34.886 INFO:tasks.workunit.client.0.smithi019.stdout:6/439: rename d6/l72 to d6/d3e/l80 0 2024-02-22T15:50:34.889 INFO:tasks.workunit.client.1.smithi062.stdout:8/190: symlink d8/d26/l33 0 2024-02-22T15:50:34.889 INFO:tasks.workunit.client.0.smithi019.stdout:6/440: write d6/f44 [2587803,83965] 0 2024-02-22T15:50:34.890 INFO:tasks.workunit.client.1.smithi062.stdout:8/191: stat d8/cd 0 2024-02-22T15:50:34.891 INFO:tasks.workunit.client.1.smithi062.stdout:2/227: rmdir d1 39 2024-02-22T15:50:34.893 INFO:tasks.workunit.client.1.smithi062.stdout:6/255: mknod d5/df/d32/c51 0 2024-02-22T15:50:34.895 INFO:tasks.workunit.client.0.smithi019.stdout:9/477: fdatasync d2/d1f/d2e/d48/d36/f43 0 2024-02-22T15:50:34.895 INFO:tasks.workunit.client.0.smithi019.stdout:8/456: dwrite d9/d14/d1f/f80 [0,4194304] 0 2024-02-22T15:50:34.901 INFO:tasks.workunit.client.0.smithi019.stdout:8/457: dread d9/f32 [0,4194304] 0 2024-02-22T15:50:34.903 INFO:tasks.workunit.client.1.smithi062.stdout:8/192: dread d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:34.904 INFO:tasks.workunit.client.1.smithi062.stdout:2/228: rename d1/d8/l9 to d1/d1e/d38/l3e 0 2024-02-22T15:50:34.905 INFO:tasks.workunit.client.1.smithi062.stdout:6/256: fdatasync d5/df/d40/d44/d4a/f28 0 2024-02-22T15:50:34.907 INFO:tasks.workunit.client.0.smithi019.stdout:8/458: creat d9/d14/d17/d3d/d3f/d7d/f98 x:0 0 0 2024-02-22T15:50:34.908 INFO:tasks.workunit.client.1.smithi062.stdout:4/144: link d4/df/l16 d4/d13/d20/l31 0 2024-02-22T15:50:34.908 INFO:tasks.workunit.client.1.smithi062.stdout:2/229: mkdir d1/d28/d3f 0 2024-02-22T15:50:34.909 INFO:tasks.workunit.client.0.smithi019.stdout:8/459: mkdir d9/d14/d1f/d5c/d99 0 2024-02-22T15:50:34.910 INFO:tasks.workunit.client.1.smithi062.stdout:8/193: creat d8/d26/f34 x:0 0 0 2024-02-22T15:50:34.911 INFO:tasks.workunit.client.0.smithi019.stdout:8/460: dread d9/fc [0,4194304] 0 2024-02-22T15:50:34.913 INFO:tasks.workunit.client.0.smithi019.stdout:8/461: mkdir d9/d14/d6f/d7e/d94/d9a 0 2024-02-22T15:50:34.914 INFO:tasks.workunit.client.1.smithi062.stdout:4/145: creat d4/d1c/d30/f32 x:0 0 0 2024-02-22T15:50:34.914 INFO:tasks.workunit.client.0.smithi019.stdout:0/358: truncate d4/f19 2730797 0 2024-02-22T15:50:34.915 INFO:tasks.workunit.client.1.smithi062.stdout:8/194: creat d8/d28/f35 x:0 0 0 2024-02-22T15:50:34.916 INFO:tasks.workunit.client.1.smithi062.stdout:9/190: sync 2024-02-22T15:50:34.916 INFO:tasks.workunit.client.1.smithi062.stdout:7/225: sync 2024-02-22T15:50:34.917 INFO:tasks.workunit.client.1.smithi062.stdout:3/179: sync 2024-02-22T15:50:34.917 INFO:tasks.workunit.client.1.smithi062.stdout:1/211: link d1/d6/de/d21/f33 d1/d6/d14/f45 0 2024-02-22T15:50:34.918 INFO:tasks.workunit.client.1.smithi062.stdout:5/115: dwrite d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:34.918 INFO:tasks.workunit.client.0.smithi019.stdout:0/359: creat d4/d23/d7d/f85 x:0 0 0 2024-02-22T15:50:34.920 INFO:tasks.workunit.client.0.smithi019.stdout:6/441: truncate d6/f17 794009 0 2024-02-22T15:50:34.920 INFO:tasks.workunit.client.0.smithi019.stdout:8/462: dread d9/d14/d17/d5d/f63 [0,4194304] 0 2024-02-22T15:50:34.922 INFO:tasks.workunit.client.1.smithi062.stdout:8/195: symlink d8/d26/l36 0 2024-02-22T15:50:34.925 INFO:tasks.workunit.client.1.smithi062.stdout:0/230: truncate d1/d3/d4/f23 536832 0 2024-02-22T15:50:34.926 INFO:tasks.workunit.client.0.smithi019.stdout:6/442: readlink d6/d31/d48/l73 0 2024-02-22T15:50:34.928 INFO:tasks.workunit.client.1.smithi062.stdout:4/146: mknod d4/d2a/d2d/d2e/c33 0 2024-02-22T15:50:34.929 INFO:tasks.workunit.client.0.smithi019.stdout:0/360: dwrite d4/d22/d3d/f4f [0,4194304] 0 2024-02-22T15:50:34.929 INFO:tasks.workunit.client.1.smithi062.stdout:1/212: write d1/d6/d7/f29 [232510,70791] 0 2024-02-22T15:50:34.929 INFO:tasks.workunit.client.1.smithi062.stdout:8/196: write d8/d9/f1b [1687964,20654] 0 2024-02-22T15:50:34.929 INFO:tasks.workunit.client.0.smithi019.stdout:9/478: stat d2/d1f/d2e/d48/d36/fa4 0 2024-02-22T15:50:34.930 INFO:tasks.workunit.client.1.smithi062.stdout:2/230: dwrite d1/f2c [0,4194304] 0 2024-02-22T15:50:34.931 INFO:tasks.workunit.client.1.smithi062.stdout:5/116: write d0/d3/da/fb [3675874,87569] 0 2024-02-22T15:50:34.932 INFO:tasks.workunit.client.1.smithi062.stdout:7/226: dread d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:34.935 INFO:tasks.workunit.client.0.smithi019.stdout:6/443: creat d6/d4b/d6d/f81 x:0 0 0 2024-02-22T15:50:34.935 INFO:tasks.workunit.client.0.smithi019.stdout:9/479: symlink d2/d1f/d2e/d48/lac 0 2024-02-22T15:50:34.935 INFO:tasks.workunit.client.0.smithi019.stdout:6/444: dread - d6/d58/f79 zero size 2024-02-22T15:50:34.936 INFO:tasks.workunit.client.0.smithi019.stdout:0/361: symlink d4/db/d49/d65/l86 0 2024-02-22T15:50:34.937 INFO:tasks.workunit.client.1.smithi062.stdout:9/191: creat d4/d15/d20/f39 x:0 0 0 2024-02-22T15:50:34.937 INFO:tasks.workunit.client.0.smithi019.stdout:0/362: chown d4/db/d49/d65 103 1 2024-02-22T15:50:34.938 INFO:tasks.workunit.client.0.smithi019.stdout:6/445: mkdir d6/d31/d43/d55/d82 0 2024-02-22T15:50:34.941 INFO:tasks.workunit.client.0.smithi019.stdout:9/480: dread d2/f19 [0,4194304] 0 2024-02-22T15:50:34.941 INFO:tasks.workunit.client.0.smithi019.stdout:3/362: sync 2024-02-22T15:50:34.941 INFO:tasks.workunit.client.1.smithi062.stdout:5/117: fdatasync d0/d3/f10 0 2024-02-22T15:50:34.942 INFO:tasks.workunit.client.1.smithi062.stdout:4/147: mkdir d4/d34 0 2024-02-22T15:50:34.947 INFO:tasks.workunit.client.1.smithi062.stdout:5/118: dread d0/d3/da/f18 [0,4194304] 0 2024-02-22T15:50:34.948 INFO:tasks.workunit.client.1.smithi062.stdout:7/227: mknod d0/d14/d19/c4e 0 2024-02-22T15:50:34.949 INFO:tasks.workunit.client.0.smithi019.stdout:8/463: truncate d9/d14/f38 3700470 0 2024-02-22T15:50:34.950 INFO:tasks.workunit.client.1.smithi062.stdout:5/119: chown d0/d3/da/c1c 216794153 1 2024-02-22T15:50:34.951 INFO:tasks.workunit.client.1.smithi062.stdout:3/180: write d5/f8 [428473,1252] 0 2024-02-22T15:50:34.951 INFO:tasks.workunit.client.1.smithi062.stdout:6/257: dwrite d5/dd/f13 [0,4194304] 0 2024-02-22T15:50:34.952 INFO:tasks.workunit.client.0.smithi019.stdout:0/363: dwrite d4/db/d1b/d40/f5d [0,4194304] 0 2024-02-22T15:50:34.954 INFO:tasks.workunit.client.1.smithi062.stdout:0/231: mknod d1/d13/d43/c4d 0 2024-02-22T15:50:34.955 INFO:tasks.workunit.client.0.smithi019.stdout:8/464: write d9/d14/d17/d1c/f48 [8116160,32967] 0 2024-02-22T15:50:34.956 INFO:tasks.workunit.client.0.smithi019.stdout:9/481: symlink d2/d1f/d2e/d48/d36/lad 0 2024-02-22T15:50:34.956 INFO:tasks.workunit.client.0.smithi019.stdout:6/446: dwrite d6/fc [0,4194304] 0 2024-02-22T15:50:34.956 INFO:tasks.workunit.client.1.smithi062.stdout:4/148: creat d4/d2a/d2d/f35 x:0 0 0 2024-02-22T15:50:34.958 INFO:tasks.workunit.client.0.smithi019.stdout:0/364: dread - d4/db/d1b/d40/d44/d75/f82 zero size 2024-02-22T15:50:34.959 INFO:tasks.workunit.client.0.smithi019.stdout:6/447: stat d6/d58/d6a 0 2024-02-22T15:50:34.962 INFO:tasks.workunit.client.1.smithi062.stdout:3/181: mkdir d5/d1a/d3b 0 2024-02-22T15:50:34.962 INFO:tasks.workunit.client.1.smithi062.stdout:0/232: write d1/d3/de/f3a [1225101,93104] 0 2024-02-22T15:50:34.963 INFO:tasks.workunit.client.1.smithi062.stdout:6/258: rename d5/d15/f37 to d5/d15/f52 0 2024-02-22T15:50:34.963 INFO:tasks.workunit.client.0.smithi019.stdout:0/365: creat d4/db/d1b/d63/f87 x:0 0 0 2024-02-22T15:50:34.964 INFO:tasks.workunit.client.1.smithi062.stdout:4/149: symlink d4/d2a/d2d/l36 0 2024-02-22T15:50:34.968 INFO:tasks.workunit.client.1.smithi062.stdout:8/197: dread d8/d9/fe [0,4194304] 0 2024-02-22T15:50:34.973 INFO:tasks.workunit.client.1.smithi062.stdout:8/198: dread d8/f24 [0,4194304] 0 2024-02-22T15:50:34.973 INFO:tasks.workunit.client.1.smithi062.stdout:8/199: readlink d8/d26/l33 0 2024-02-22T15:50:34.973 INFO:tasks.workunit.client.0.smithi019.stdout:9/482: dwrite d2/d7/dc/d27/d68/f73 [0,4194304] 0 2024-02-22T15:50:34.977 INFO:tasks.workunit.client.1.smithi062.stdout:2/231: getdents d1/d1e 0 2024-02-22T15:50:34.977 INFO:tasks.workunit.client.1.smithi062.stdout:1/213: write d1/d6/de/d21/f33 [97939,109573] 0 2024-02-22T15:50:34.978 INFO:tasks.workunit.client.1.smithi062.stdout:3/182: mkdir d5/d1a/d3c 0 2024-02-22T15:50:34.979 INFO:tasks.workunit.client.1.smithi062.stdout:9/192: getdents d4/d1c 0 2024-02-22T15:50:34.979 INFO:tasks.workunit.client.1.smithi062.stdout:4/150: truncate d4/d13/f15 2004760 0 2024-02-22T15:50:34.980 INFO:tasks.workunit.client.0.smithi019.stdout:8/465: dread d9/ff [0,4194304] 0 2024-02-22T15:50:34.980 INFO:tasks.workunit.client.1.smithi062.stdout:0/233: rename d1/d3/de/d14/f18 to d1/d38/f4e 0 2024-02-22T15:50:34.981 INFO:tasks.workunit.client.1.smithi062.stdout:3/183: chown l2 14372888 1 2024-02-22T15:50:34.981 INFO:tasks.workunit.client.1.smithi062.stdout:4/151: readlink d4/d13/d20/l2f 0 2024-02-22T15:50:34.981 INFO:tasks.workunit.client.1.smithi062.stdout:3/184: fdatasync d5/f10 0 2024-02-22T15:50:34.983 INFO:tasks.workunit.client.1.smithi062.stdout:9/193: chown d4/d15/f24 70 1 2024-02-22T15:50:34.983 INFO:tasks.workunit.client.0.smithi019.stdout:0/366: dwrite d4/d22/f36 [0,4194304] 0 2024-02-22T15:50:34.983 INFO:tasks.workunit.client.0.smithi019.stdout:6/448: dwrite d6/f39 [0,4194304] 0 2024-02-22T15:50:34.984 INFO:tasks.workunit.client.1.smithi062.stdout:2/232: creat d1/d1e/d20/f40 x:0 0 0 2024-02-22T15:50:34.985 INFO:tasks.workunit.client.1.smithi062.stdout:1/214: chown d1/l1e 10 1 2024-02-22T15:50:34.986 INFO:tasks.workunit.client.1.smithi062.stdout:2/233: truncate d1/f5 4449739 0 2024-02-22T15:50:34.986 INFO:tasks.workunit.client.1.smithi062.stdout:1/215: fdatasync d1/d6/d7/f28 0 2024-02-22T15:50:34.988 INFO:tasks.workunit.client.0.smithi019.stdout:8/466: creat d9/d14/d6f/d7e/d94/f9b x:0 0 0 2024-02-22T15:50:34.988 INFO:tasks.workunit.client.1.smithi062.stdout:0/234: mknod d1/d13/c4f 0 2024-02-22T15:50:34.989 INFO:tasks.workunit.client.0.smithi019.stdout:0/367: readlink d4/db/l84 0 2024-02-22T15:50:34.989 INFO:tasks.workunit.client.0.smithi019.stdout:6/449: mknod d6/d4b/d6d/c83 0 2024-02-22T15:50:34.990 INFO:tasks.workunit.client.1.smithi062.stdout:3/185: mknod d5/d26/c3d 0 2024-02-22T15:50:34.991 INFO:tasks.workunit.client.0.smithi019.stdout:8/467: unlink d9/d14/d1f/d20/c2f 0 2024-02-22T15:50:34.991 INFO:tasks.workunit.client.1.smithi062.stdout:9/194: symlink d4/d15/d20/l3a 0 2024-02-22T15:50:34.993 INFO:tasks.workunit.client.0.smithi019.stdout:6/450: creat d6/d31/d43/d55/d67/f84 x:0 0 0 2024-02-22T15:50:34.993 INFO:tasks.workunit.client.0.smithi019.stdout:8/468: stat d9/d14/d1f/d20/f61 0 2024-02-22T15:50:34.994 INFO:tasks.workunit.client.0.smithi019.stdout:3/363: dread d9/d16/f42 [0,4194304] 0 2024-02-22T15:50:34.995 INFO:tasks.workunit.client.0.smithi019.stdout:8/469: write d9/d14/f89 [840887,37831] 0 2024-02-22T15:50:34.997 INFO:tasks.workunit.client.1.smithi062.stdout:0/235: mkdir d1/d3/de/d50 0 2024-02-22T15:50:34.997 INFO:tasks.workunit.client.0.smithi019.stdout:6/451: creat d6/d58/d6a/f85 x:0 0 0 2024-02-22T15:50:34.998 INFO:tasks.workunit.client.1.smithi062.stdout:2/234: mknod d1/d1e/d20/d32/c41 0 2024-02-22T15:50:34.998 INFO:tasks.workunit.client.0.smithi019.stdout:8/470: truncate d9/d14/f6a 30034 0 2024-02-22T15:50:34.999 INFO:tasks.workunit.client.0.smithi019.stdout:6/452: readlink d6/l27 0 2024-02-22T15:50:35.001 INFO:tasks.workunit.client.0.smithi019.stdout:8/471: mkdir d9/d14/d6f/d7e/d94/d9c 0 2024-02-22T15:50:35.002 INFO:tasks.workunit.client.1.smithi062.stdout:1/216: dwrite d1/d6/d7/f3a [4194304,4194304] 0 2024-02-22T15:50:35.004 INFO:tasks.workunit.client.1.smithi062.stdout:5/120: write d0/d3/ff [780669,13153] 0 2024-02-22T15:50:35.004 INFO:tasks.workunit.client.0.smithi019.stdout:6/453: creat d6/f86 x:0 0 0 2024-02-22T15:50:35.005 INFO:tasks.workunit.client.1.smithi062.stdout:2/235: creat d1/d8/f42 x:0 0 0 2024-02-22T15:50:35.006 INFO:tasks.workunit.client.1.smithi062.stdout:1/217: read d1/d6/d14/f45 [70074,84194] 0 2024-02-22T15:50:35.006 INFO:tasks.workunit.client.1.smithi062.stdout:6/259: write d5/dd/fe [163057,112977] 0 2024-02-22T15:50:35.007 INFO:tasks.workunit.client.1.smithi062.stdout:5/121: stat d0/d3/da/d13/l15 0 2024-02-22T15:50:35.007 INFO:tasks.workunit.client.0.smithi019.stdout:8/472: rename d9/d14/d1f/d5c/f87 to d9/d14/d17/d1c/f9d 0 2024-02-22T15:50:35.007 INFO:tasks.workunit.client.0.smithi019.stdout:6/454: symlink d6/d58/l87 0 2024-02-22T15:50:35.008 INFO:tasks.workunit.client.1.smithi062.stdout:3/186: mknod d5/c3e 0 2024-02-22T15:50:35.008 INFO:tasks.workunit.client.0.smithi019.stdout:6/455: write d6/f29 [4068439,35876] 0 2024-02-22T15:50:35.009 INFO:tasks.workunit.client.0.smithi019.stdout:8/473: write d9/d14/d6f/d7e/d94/f9b [229738,98276] 0 2024-02-22T15:50:35.009 INFO:tasks.workunit.client.1.smithi062.stdout:3/187: dread - d5/d26/f31 zero size 2024-02-22T15:50:35.010 INFO:tasks.workunit.client.1.smithi062.stdout:6/260: stat d5/df/d32/d47 0 2024-02-22T15:50:35.010 INFO:tasks.workunit.client.1.smithi062.stdout:9/195: rename d4/lf to d4/d15/d31/l3b 0 2024-02-22T15:50:35.011 INFO:tasks.workunit.client.1.smithi062.stdout:0/236: mknod d1/d3/de/d50/c51 0 2024-02-22T15:50:35.014 INFO:tasks.workunit.client.1.smithi062.stdout:5/122: symlink d0/d3/da/d13/l21 0 2024-02-22T15:50:35.016 INFO:tasks.workunit.client.0.smithi019.stdout:6/456: rmdir d6/d4b/d6d/d7b/d78 39 2024-02-22T15:50:35.016 INFO:tasks.workunit.client.0.smithi019.stdout:8/474: creat d9/d14/d17/d3d/d3f/f9e x:0 0 0 2024-02-22T15:50:35.016 INFO:tasks.workunit.client.1.smithi062.stdout:6/261: creat d5/df/d40/f53 x:0 0 0 2024-02-22T15:50:35.017 INFO:tasks.workunit.client.1.smithi062.stdout:3/188: dread d5/d16/f19 [0,4194304] 0 2024-02-22T15:50:35.017 INFO:tasks.workunit.client.1.smithi062.stdout:0/237: mknod d1/d21/d3e/c52 0 2024-02-22T15:50:35.018 INFO:tasks.workunit.client.0.smithi019.stdout:6/457: dread - d6/d31/d48/f51 zero size 2024-02-22T15:50:35.018 INFO:tasks.workunit.client.1.smithi062.stdout:4/152: fsync d4/d13/f15 0 2024-02-22T15:50:35.019 INFO:tasks.workunit.client.0.smithi019.stdout:8/475: creat d9/d14/d17/d3d/f9f x:0 0 0 2024-02-22T15:50:35.019 INFO:tasks.workunit.client.0.smithi019.stdout:8/476: chown d9/d14/d17 1129439 1 2024-02-22T15:50:35.020 INFO:tasks.workunit.client.0.smithi019.stdout:6/458: mkdir d6/d4b/d71/d88 0 2024-02-22T15:50:35.022 INFO:tasks.workunit.client.0.smithi019.stdout:6/459: rmdir d6/d31/d48 39 2024-02-22T15:50:35.022 INFO:tasks.workunit.client.1.smithi062.stdout:5/123: truncate d0/d3/f10 2435136 0 2024-02-22T15:50:35.023 INFO:tasks.workunit.client.1.smithi062.stdout:5/124: readlink d0/d3/da/d13/l21 0 2024-02-22T15:50:35.024 INFO:tasks.workunit.client.1.smithi062.stdout:3/189: mkdir d5/d26/d2d/d3f 0 2024-02-22T15:50:35.025 INFO:tasks.workunit.client.1.smithi062.stdout:4/153: mkdir d4/d1c/d30/d37 0 2024-02-22T15:50:35.025 INFO:tasks.workunit.client.0.smithi019.stdout:8/477: rename d9/d14/d6f/d8d to d9/d14/d17/d1c/da0 0 2024-02-22T15:50:35.028 INFO:tasks.workunit.client.1.smithi062.stdout:5/125: rename d0/c11 to d0/d3/c22 0 2024-02-22T15:50:35.029 INFO:tasks.workunit.client.0.smithi019.stdout:6/460: write d6/f28 [4446287,91649] 0 2024-02-22T15:50:35.029 INFO:tasks.workunit.client.1.smithi062.stdout:8/200: truncate d8/d9/f16 4071409 0 2024-02-22T15:50:35.029 INFO:tasks.workunit.client.1.smithi062.stdout:4/154: dread d4/df/f26 [0,4194304] 0 2024-02-22T15:50:35.031 INFO:tasks.workunit.client.1.smithi062.stdout:4/155: chown d4/d2a/d2d/l36 24 1 2024-02-22T15:50:35.031 INFO:tasks.workunit.client.0.smithi019.stdout:6/461: chown d6/d31/d48 38120082 1 2024-02-22T15:50:35.032 INFO:tasks.workunit.client.1.smithi062.stdout:0/238: link d1/d3/d4/d1a/f34 d1/d13/d43/f53 0 2024-02-22T15:50:35.034 INFO:tasks.workunit.client.0.smithi019.stdout:3/364: truncate d9/d26/d32/f43 2955731 0 2024-02-22T15:50:35.035 INFO:tasks.workunit.client.1.smithi062.stdout:0/239: dread d1/d3/f39 [0,4194304] 0 2024-02-22T15:50:35.035 INFO:tasks.workunit.client.0.smithi019.stdout:6/462: fsync d6/d31/f36 0 2024-02-22T15:50:35.036 INFO:tasks.workunit.client.1.smithi062.stdout:8/201: creat d8/d9/f37 x:0 0 0 2024-02-22T15:50:35.036 INFO:tasks.workunit.client.0.smithi019.stdout:8/478: getdents d9/d14/d17/d3d/d3f/d7d 0 2024-02-22T15:50:35.036 INFO:tasks.workunit.client.1.smithi062.stdout:0/240: dread - d1/d3/de/d3f/f4c zero size 2024-02-22T15:50:35.037 INFO:tasks.workunit.client.1.smithi062.stdout:2/236: rename d1/d1e/d20 to d1/d1e/d43 0 2024-02-22T15:50:35.037 INFO:tasks.workunit.client.1.smithi062.stdout:2/237: stat d1/f5 0 2024-02-22T15:50:35.038 INFO:tasks.workunit.client.1.smithi062.stdout:5/126: truncate d0/d3/da/f18 603401 0 2024-02-22T15:50:35.039 INFO:tasks.workunit.client.1.smithi062.stdout:5/127: readlink d0/d3/da/d13/l21 0 2024-02-22T15:50:35.040 INFO:tasks.workunit.client.1.smithi062.stdout:4/156: rename d4/d2a/d2d/l36 to d4/d1c/d2c/l38 0 2024-02-22T15:50:35.041 INFO:tasks.workunit.client.0.smithi019.stdout:6/463: readlink d6/l24 0 2024-02-22T15:50:35.042 INFO:tasks.workunit.client.1.smithi062.stdout:6/262: link d5/df/d32/c51 d5/dd/d33/c54 0 2024-02-22T15:50:35.043 INFO:tasks.workunit.client.0.smithi019.stdout:8/479: creat d9/d14/d1f/d5c/d99/fa1 x:0 0 0 2024-02-22T15:50:35.043 INFO:tasks.workunit.client.1.smithi062.stdout:6/263: dread - d5/df/d40/d44/d4a/d41/f4d zero size 2024-02-22T15:50:35.044 INFO:tasks.workunit.client.0.smithi019.stdout:8/480: chown d9/d14/d17/f40 2619810 1 2024-02-22T15:50:35.046 INFO:tasks.workunit.client.1.smithi062.stdout:8/202: mknod d8/d26/c38 0 2024-02-22T15:50:35.046 INFO:tasks.workunit.client.0.smithi019.stdout:6/464: creat d6/d4b/d6d/d7b/d7f/f89 x:0 0 0 2024-02-22T15:50:35.047 INFO:tasks.workunit.client.1.smithi062.stdout:0/241: rmdir d1/d3/d4/d1a/d1d/d1f 39 2024-02-22T15:50:35.048 INFO:tasks.workunit.client.1.smithi062.stdout:2/238: creat d1/d1e/d43/d32/f44 x:0 0 0 2024-02-22T15:50:35.050 INFO:tasks.workunit.client.1.smithi062.stdout:5/128: mkdir d0/d3/da/d13/d23 0 2024-02-22T15:50:35.051 INFO:tasks.workunit.client.1.smithi062.stdout:4/157: write d4/df/f21 [1138822,2359] 0 2024-02-22T15:50:35.052 INFO:tasks.workunit.client.0.smithi019.stdout:8/481: dwrite d9/d14/d1f/d4c/f7f [0,4194304] 0 2024-02-22T15:50:35.054 INFO:tasks.workunit.client.1.smithi062.stdout:3/190: getdents d5/d1a 0 2024-02-22T15:50:35.055 INFO:tasks.workunit.client.0.smithi019.stdout:6/465: truncate d6/f22 2336756 0 2024-02-22T15:50:35.055 INFO:tasks.workunit.client.0.smithi019.stdout:8/482: write d9/d14/d17/d3d/f59 [32682,109849] 0 2024-02-22T15:50:35.056 INFO:tasks.workunit.client.0.smithi019.stdout:3/365: dread d9/fb [0,4194304] 0 2024-02-22T15:50:35.059 INFO:tasks.workunit.client.1.smithi062.stdout:8/203: creat d8/f39 x:0 0 0 2024-02-22T15:50:35.061 INFO:tasks.workunit.client.0.smithi019.stdout:3/366: truncate d9/d22/f50 378684 0 2024-02-22T15:50:35.063 INFO:tasks.workunit.client.1.smithi062.stdout:4/158: dwrite d4/df/f21 [0,4194304] 0 2024-02-22T15:50:35.064 INFO:tasks.workunit.client.1.smithi062.stdout:9/196: dread d4/f14 [0,4194304] 0 2024-02-22T15:50:35.067 INFO:tasks.workunit.client.0.smithi019.stdout:6/466: mkdir d6/d31/d43/d55/d67/d8a 0 2024-02-22T15:50:35.067 INFO:tasks.workunit.client.0.smithi019.stdout:8/483: creat d9/d14/d1f/d5c/fa2 x:0 0 0 2024-02-22T15:50:35.068 INFO:tasks.workunit.client.1.smithi062.stdout:6/264: dread d5/df/f10 [0,4194304] 0 2024-02-22T15:50:35.068 INFO:tasks.workunit.client.1.smithi062.stdout:3/191: symlink d5/d1a/l40 0 2024-02-22T15:50:35.068 INFO:tasks.workunit.client.1.smithi062.stdout:4/159: dread - d4/d1c/d30/f32 zero size 2024-02-22T15:50:35.069 INFO:tasks.workunit.client.0.smithi019.stdout:3/367: mknod d9/d22/d27/c75 0 2024-02-22T15:50:35.070 INFO:tasks.workunit.client.1.smithi062.stdout:0/242: mkdir d1/d3/d4/d54 0 2024-02-22T15:50:35.072 INFO:tasks.workunit.client.0.smithi019.stdout:6/467: creat d6/d31/d43/f8b x:0 0 0 2024-02-22T15:50:35.073 INFO:tasks.workunit.client.0.smithi019.stdout:6/468: stat d6/d31/d43/d55/d67/f84 0 2024-02-22T15:50:35.073 INFO:tasks.workunit.client.1.smithi062.stdout:8/204: creat d8/f3a x:0 0 0 2024-02-22T15:50:35.074 INFO:tasks.workunit.client.1.smithi062.stdout:6/265: write d5/df/d32/f4e [994311,36136] 0 2024-02-22T15:50:35.074 INFO:tasks.workunit.client.0.smithi019.stdout:6/469: stat d6/d31/d43/f46 0 2024-02-22T15:50:35.076 INFO:tasks.workunit.client.1.smithi062.stdout:2/239: dwrite d1/d8/df/f25 [0,4194304] 0 2024-02-22T15:50:35.076 INFO:tasks.workunit.client.0.smithi019.stdout:6/470: creat d6/d58/d6a/f8c x:0 0 0 2024-02-22T15:50:35.076 INFO:tasks.workunit.client.1.smithi062.stdout:5/129: creat d0/d3/da/d13/d23/f24 x:0 0 0 2024-02-22T15:50:35.077 INFO:tasks.workunit.client.1.smithi062.stdout:8/205: stat d8/cd 0 2024-02-22T15:50:35.083 INFO:tasks.workunit.client.1.smithi062.stdout:3/192: truncate d5/d16/d21/f29 483243 0 2024-02-22T15:50:35.083 INFO:tasks.workunit.client.1.smithi062.stdout:2/240: chown d1/d1e/d43/c33 1645 1 2024-02-22T15:50:35.083 INFO:tasks.workunit.client.1.smithi062.stdout:3/193: read - d5/d26/f27 zero size 2024-02-22T15:50:35.084 INFO:tasks.workunit.client.1.smithi062.stdout:3/194: stat d5/d16/d21/f3a 0 2024-02-22T15:50:35.086 INFO:tasks.workunit.client.1.smithi062.stdout:4/160: dread d4/df/f21 [0,4194304] 0 2024-02-22T15:50:35.088 INFO:tasks.workunit.client.1.smithi062.stdout:5/130: rename d0/d3/da/d13/l21 to d0/l25 0 2024-02-22T15:50:35.089 INFO:tasks.workunit.client.1.smithi062.stdout:8/206: mknod d8/d9/c3b 0 2024-02-22T15:50:35.090 INFO:tasks.workunit.client.0.smithi019.stdout:6/471: fsync d6/d58/d6a/f8c 0 2024-02-22T15:50:35.090 INFO:tasks.workunit.client.1.smithi062.stdout:8/207: chown d8/d9/f27 31078 1 2024-02-22T15:50:35.093 INFO:tasks.workunit.client.0.smithi019.stdout:6/472: chown d6/d58/d6a/f85 10 1 2024-02-22T15:50:35.094 INFO:tasks.workunit.client.1.smithi062.stdout:8/208: write d8/d9/d2c/f2f [892616,126977] 0 2024-02-22T15:50:35.095 INFO:tasks.workunit.client.1.smithi062.stdout:0/243: creat d1/d3/d4/d1a/d1d/d3b/f55 x:0 0 0 2024-02-22T15:50:35.095 INFO:tasks.workunit.client.0.smithi019.stdout:6/473: mkdir d6/d4b/d8d 0 2024-02-22T15:50:35.096 INFO:tasks.workunit.client.1.smithi062.stdout:4/161: creat d4/f39 x:0 0 0 2024-02-22T15:50:35.097 INFO:tasks.workunit.client.0.smithi019.stdout:6/474: stat d6/d4b/d6d/d7b 0 2024-02-22T15:50:35.097 INFO:tasks.workunit.client.1.smithi062.stdout:9/197: mkdir d4/d15/d3c 0 2024-02-22T15:50:35.098 INFO:tasks.workunit.client.1.smithi062.stdout:2/241: getdents d1/d1e/d43/d39 0 2024-02-22T15:50:35.099 INFO:tasks.workunit.client.0.smithi019.stdout:6/475: read - d6/d31/d43/d55/d67/f84 zero size 2024-02-22T15:50:35.100 INFO:tasks.workunit.client.0.smithi019.stdout:8/484: dread d9/d14/d17/f4a [0,4194304] 0 2024-02-22T15:50:35.101 INFO:tasks.workunit.client.1.smithi062.stdout:8/209: mkdir d8/d9/d3c 0 2024-02-22T15:50:35.101 INFO:tasks.workunit.client.1.smithi062.stdout:8/210: stat d8/f39 0 2024-02-22T15:50:35.102 INFO:tasks.workunit.client.1.smithi062.stdout:0/244: creat d1/d3/d4/d1a/f56 x:0 0 0 2024-02-22T15:50:35.102 INFO:tasks.workunit.client.1.smithi062.stdout:8/211: chown d8/d9/c25 178688 1 2024-02-22T15:50:35.103 INFO:tasks.workunit.client.1.smithi062.stdout:4/162: symlink d4/d2a/d2d/d2e/l3a 0 2024-02-22T15:50:35.103 INFO:tasks.workunit.client.0.smithi019.stdout:6/476: mkdir d6/d31/d69/d8e 0 2024-02-22T15:50:35.104 INFO:tasks.workunit.client.1.smithi062.stdout:4/163: fdatasync d4/d1c/f23 0 2024-02-22T15:50:35.105 INFO:tasks.workunit.client.1.smithi062.stdout:6/266: getdents d5/df/d40/d44/d4a/d41 0 2024-02-22T15:50:35.106 INFO:tasks.workunit.client.0.smithi019.stdout:6/477: write d6/d4b/d6d/d7b/d7f/f89 [1005185,86126] 0 2024-02-22T15:50:35.106 INFO:tasks.workunit.client.1.smithi062.stdout:3/195: creat d5/f41 x:0 0 0 2024-02-22T15:50:35.106 INFO:tasks.workunit.client.0.smithi019.stdout:6/478: read d6/f10 [153034,11315] 0 2024-02-22T15:50:35.108 INFO:tasks.workunit.client.0.smithi019.stdout:6/479: symlink d6/d4b/l8f 0 2024-02-22T15:50:35.109 INFO:tasks.workunit.client.1.smithi062.stdout:2/242: getdents d1/d1e/d43/d39 0 2024-02-22T15:50:35.109 INFO:tasks.workunit.client.1.smithi062.stdout:4/164: mkdir d4/d1c/d30/d3b 0 2024-02-22T15:50:35.110 INFO:tasks.workunit.client.1.smithi062.stdout:9/198: write d4/d28/f38 [4815483,111626] 0 2024-02-22T15:50:35.110 INFO:tasks.workunit.client.1.smithi062.stdout:3/196: mkdir d5/d16/d42 0 2024-02-22T15:50:35.111 INFO:tasks.workunit.client.0.smithi019.stdout:8/485: dwrite f3 [0,4194304] 0 2024-02-22T15:50:35.112 INFO:tasks.workunit.client.0.smithi019.stdout:8/486: chown d9/f29 13149142 1 2024-02-22T15:50:35.112 INFO:tasks.workunit.client.1.smithi062.stdout:8/212: dread d8/f21 [0,4194304] 0 2024-02-22T15:50:35.112 INFO:tasks.workunit.client.1.smithi062.stdout:6/267: creat d5/df/d40/d44/d4a/d41/f55 x:0 0 0 2024-02-22T15:50:35.114 INFO:tasks.workunit.client.1.smithi062.stdout:0/245: truncate d1/f45 28344 0 2024-02-22T15:50:35.115 INFO:tasks.workunit.client.1.smithi062.stdout:8/213: truncate d8/f22 2052962 0 2024-02-22T15:50:35.116 INFO:tasks.workunit.client.1.smithi062.stdout:6/268: read - d5/df/d40/d44/d4a/d41/f55 zero size 2024-02-22T15:50:35.117 INFO:tasks.workunit.client.1.smithi062.stdout:2/243: dread d1/f6 [0,4194304] 0 2024-02-22T15:50:35.119 INFO:tasks.workunit.client.0.smithi019.stdout:8/487: link d9/f10 d9/d14/d6f/d7e/d94/d9c/fa3 0 2024-02-22T15:50:35.119 INFO:tasks.workunit.client.1.smithi062.stdout:0/246: creat d1/d21/d3e/f57 x:0 0 0 2024-02-22T15:50:35.124 INFO:tasks.workunit.client.1.smithi062.stdout:0/247: dread - d1/d21/d3e/f57 zero size 2024-02-22T15:50:35.124 INFO:tasks.workunit.client.1.smithi062.stdout:3/197: dread d5/d1a/f1e [0,4194304] 0 2024-02-22T15:50:35.124 INFO:tasks.workunit.client.1.smithi062.stdout:8/214: creat d8/f3d x:0 0 0 2024-02-22T15:50:35.125 INFO:tasks.workunit.client.1.smithi062.stdout:9/199: getdents d4/d15/d20 0 2024-02-22T15:50:35.127 INFO:tasks.workunit.client.0.smithi019.stdout:8/488: dwrite d9/d14/d17/f4a [0,4194304] 0 2024-02-22T15:50:35.127 INFO:tasks.workunit.client.1.smithi062.stdout:2/244: unlink d1/f3c 0 2024-02-22T15:50:35.128 INFO:tasks.workunit.client.1.smithi062.stdout:4/165: dwrite f2 [0,4194304] 0 2024-02-22T15:50:35.128 INFO:tasks.workunit.client.1.smithi062.stdout:0/248: readlink d1/d3/de/d3f/l40 0 2024-02-22T15:50:35.131 INFO:tasks.workunit.client.1.smithi062.stdout:8/215: rename d8/c19 to d8/d28/c3e 0 2024-02-22T15:50:35.131 INFO:tasks.workunit.client.1.smithi062.stdout:9/200: truncate d4/d28/f30 821981 0 2024-02-22T15:50:35.132 INFO:tasks.workunit.client.0.smithi019.stdout:8/489: read d9/d14/d17/f50 [628175,69530] 0 2024-02-22T15:50:35.133 INFO:tasks.workunit.client.1.smithi062.stdout:2/245: rmdir d1/d1e/d43 39 2024-02-22T15:50:35.134 INFO:tasks.workunit.client.0.smithi019.stdout:8/490: write d9/d14/d1f/f33 [3656693,67805] 0 2024-02-22T15:50:35.135 INFO:tasks.workunit.client.0.smithi019.stdout:6/480: dread d6/f2d [0,4194304] 0 2024-02-22T15:50:35.136 INFO:tasks.workunit.client.1.smithi062.stdout:0/249: write d1/d3/d4/d1a/d1d/d3b/f55 [47795,127655] 0 2024-02-22T15:50:35.137 INFO:tasks.workunit.client.1.smithi062.stdout:6/269: dwrite d5/dd/f13 [0,4194304] 0 2024-02-22T15:50:35.144 INFO:tasks.workunit.client.1.smithi062.stdout:9/201: symlink d4/d15/l3d 0 2024-02-22T15:50:35.145 INFO:tasks.workunit.client.1.smithi062.stdout:0/250: rmdir d1/d3/de/d14 39 2024-02-22T15:50:35.146 INFO:tasks.workunit.client.0.smithi019.stdout:6/481: dwrite d6/f2c [0,4194304] 0 2024-02-22T15:50:35.147 INFO:tasks.workunit.client.1.smithi062.stdout:8/216: getdents d8/d9/d3c 0 2024-02-22T15:50:35.148 INFO:tasks.workunit.client.1.smithi062.stdout:9/202: truncate d4/d35/f36 143711 0 2024-02-22T15:50:35.149 INFO:tasks.workunit.client.1.smithi062.stdout:0/251: chown d1/d3/de/d14/f2a 24 1 2024-02-22T15:50:35.152 INFO:tasks.workunit.client.1.smithi062.stdout:6/270: getdents d5/df/d32/d47 0 2024-02-22T15:50:35.153 INFO:tasks.workunit.client.0.smithi019.stdout:6/482: getdents d6/d31/d43/d55/d82 0 2024-02-22T15:50:35.154 INFO:tasks.workunit.client.1.smithi062.stdout:0/252: mknod d1/d3/de/d14/c58 0 2024-02-22T15:50:35.155 INFO:tasks.workunit.client.1.smithi062.stdout:6/271: dread - d5/df/d40/d44/d4a/d41/f55 zero size 2024-02-22T15:50:35.157 INFO:tasks.workunit.client.1.smithi062.stdout:0/253: symlink d1/d21/l59 0 2024-02-22T15:50:35.158 INFO:tasks.workunit.client.1.smithi062.stdout:6/272: rmdir d5/dd 39 2024-02-22T15:50:35.159 INFO:tasks.workunit.client.0.smithi019.stdout:1/515: sync 2024-02-22T15:50:35.159 INFO:tasks.workunit.client.0.smithi019.stdout:5/475: sync 2024-02-22T15:50:35.159 INFO:tasks.workunit.client.0.smithi019.stdout:7/576: sync 2024-02-22T15:50:35.160 INFO:tasks.workunit.client.0.smithi019.stdout:1/516: write d8/d44/f9f [710438,114205] 0 2024-02-22T15:50:35.160 INFO:tasks.workunit.client.1.smithi062.stdout:8/217: dwrite d8/d9/fe [0,4194304] 0 2024-02-22T15:50:35.164 INFO:tasks.workunit.client.0.smithi019.stdout:5/476: rename d8/d11/c60 to d8/d11/d33/c9f 0 2024-02-22T15:50:35.165 INFO:tasks.workunit.client.1.smithi062.stdout:8/218: dread d8/f21 [0,4194304] 0 2024-02-22T15:50:35.167 INFO:tasks.workunit.client.1.smithi062.stdout:6/273: unlink d5/dd/d33/l42 0 2024-02-22T15:50:35.168 INFO:tasks.workunit.client.1.smithi062.stdout:0/254: rmdir d1/d3/d4/d54 0 2024-02-22T15:50:35.169 INFO:tasks.workunit.client.1.smithi062.stdout:0/255: chown d1/d13/f2f 6442 1 2024-02-22T15:50:35.170 INFO:tasks.workunit.client.0.smithi019.stdout:7/577: dread d9/d26/f29 [0,4194304] 0 2024-02-22T15:50:35.170 INFO:tasks.workunit.client.1.smithi062.stdout:8/219: unlink d8/f3d 0 2024-02-22T15:50:35.172 INFO:tasks.workunit.client.0.smithi019.stdout:7/578: readlink d9/d4c/d5b/lbb 0 2024-02-22T15:50:35.173 INFO:tasks.workunit.client.1.smithi062.stdout:6/274: readlink d5/df/d2a/l3a 0 2024-02-22T15:50:35.174 INFO:tasks.workunit.client.1.smithi062.stdout:8/220: mknod d8/d9/d2c/c3f 0 2024-02-22T15:50:35.175 INFO:tasks.workunit.client.0.smithi019.stdout:5/477: dwrite d8/d11/f87 [0,4194304] 0 2024-02-22T15:50:35.176 INFO:tasks.workunit.client.1.smithi062.stdout:6/275: mkdir d5/d15/d56 0 2024-02-22T15:50:35.177 INFO:tasks.workunit.client.1.smithi062.stdout:8/221: creat d8/f40 x:0 0 0 2024-02-22T15:50:35.178 INFO:tasks.workunit.client.0.smithi019.stdout:1/517: dread d8/d18/d85/f98 [0,4194304] 0 2024-02-22T15:50:35.180 INFO:tasks.workunit.client.1.smithi062.stdout:2/246: dread d1/d1e/f27 [0,4194304] 0 2024-02-22T15:50:35.180 INFO:tasks.workunit.client.0.smithi019.stdout:5/478: mknod d8/ca0 0 2024-02-22T15:50:35.180 INFO:tasks.workunit.client.1.smithi062.stdout:8/222: creat d8/d26/f41 x:0 0 0 2024-02-22T15:50:35.182 INFO:tasks.workunit.client.1.smithi062.stdout:8/223: write d8/f40 [540246,45288] 0 2024-02-22T15:50:35.186 INFO:tasks.workunit.client.1.smithi062.stdout:6/276: link d5/d15/l25 d5/df/d32/d47/l57 0 2024-02-22T15:50:35.186 INFO:tasks.workunit.client.0.smithi019.stdout:1/518: creat d8/fb4 x:0 0 0 2024-02-22T15:50:35.189 INFO:tasks.workunit.client.1.smithi062.stdout:2/247: dread d1/d8/f3d [0,4194304] 0 2024-02-22T15:50:35.190 INFO:tasks.workunit.client.0.smithi019.stdout:6/483: dread d6/f22 [0,4194304] 0 2024-02-22T15:50:35.191 INFO:tasks.workunit.client.1.smithi062.stdout:2/248: write d1/f2c [129062,59016] 0 2024-02-22T15:50:35.192 INFO:tasks.workunit.client.1.smithi062.stdout:2/249: chown d1/f5 1857875 1 2024-02-22T15:50:35.195 INFO:tasks.workunit.client.0.smithi019.stdout:7/579: dwrite d9/d3a/d49/d5d/f72 [0,4194304] 0 2024-02-22T15:50:35.197 INFO:tasks.workunit.client.0.smithi019.stdout:5/479: dwrite d8/dd/f75 [0,4194304] 0 2024-02-22T15:50:35.198 INFO:tasks.workunit.client.1.smithi062.stdout:2/250: symlink d1/d8/df/l45 0 2024-02-22T15:50:35.199 INFO:tasks.workunit.client.1.smithi062.stdout:6/277: creat d5/df/d40/d44/d4a/d41/d43/f58 x:0 0 0 2024-02-22T15:50:35.200 INFO:tasks.workunit.client.1.smithi062.stdout:8/224: dwrite d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:35.201 INFO:tasks.workunit.client.0.smithi019.stdout:7/580: creat d9/d3a/d49/d5d/fbd x:0 0 0 2024-02-22T15:50:35.201 INFO:tasks.workunit.client.0.smithi019.stdout:5/480: write d8/dd/f5b [1076320,45919] 0 2024-02-22T15:50:35.202 INFO:tasks.workunit.client.1.smithi062.stdout:0/256: dread d1/d3/de/f3a [0,4194304] 0 2024-02-22T15:50:35.202 INFO:tasks.workunit.client.1.smithi062.stdout:2/251: chown d1/d1e/d43/d32 50678927 1 2024-02-22T15:50:35.209 INFO:tasks.workunit.client.1.smithi062.stdout:8/225: unlink d8/c1d 0 2024-02-22T15:50:35.212 INFO:tasks.workunit.client.1.smithi062.stdout:8/226: truncate d8/d9/f2a 687520 0 2024-02-22T15:50:35.213 INFO:tasks.workunit.client.1.smithi062.stdout:2/252: creat d1/d28/d3f/f46 x:0 0 0 2024-02-22T15:50:35.214 INFO:tasks.workunit.client.1.smithi062.stdout:2/253: chown d1/d1e/d43/d39 425707 1 2024-02-22T15:50:35.215 INFO:tasks.workunit.client.1.smithi062.stdout:6/278: mkdir d5/df/d2a/d59 0 2024-02-22T15:50:35.215 INFO:tasks.workunit.client.1.smithi062.stdout:2/254: fdatasync d1/d8/df/f25 0 2024-02-22T15:50:35.216 INFO:tasks.workunit.client.1.smithi062.stdout:8/227: mkdir d8/d42 0 2024-02-22T15:50:35.218 INFO:tasks.workunit.client.0.smithi019.stdout:1/519: dwrite d8/d2a/f3f [0,4194304] 0 2024-02-22T15:50:35.218 INFO:tasks.workunit.client.0.smithi019.stdout:6/484: dwrite d6/d4b/d6d/f81 [0,4194304] 0 2024-02-22T15:50:35.219 INFO:tasks.workunit.client.1.smithi062.stdout:6/279: dread - d5/df/d40/d44/d4a/d41/d43/f58 zero size 2024-02-22T15:50:35.219 INFO:tasks.workunit.client.1.smithi062.stdout:8/228: mkdir d8/d42/d43 0 2024-02-22T15:50:35.222 INFO:tasks.workunit.client.1.smithi062.stdout:8/229: mkdir d8/d9/d44 0 2024-02-22T15:50:35.224 INFO:tasks.workunit.client.1.smithi062.stdout:6/280: unlink d5/df/l4b 0 2024-02-22T15:50:35.226 INFO:tasks.workunit.client.1.smithi062.stdout:2/255: rename d1/d1e/d43 to d1/d47 0 2024-02-22T15:50:35.227 INFO:tasks.workunit.client.1.smithi062.stdout:2/256: dread - d1/d8/f1a zero size 2024-02-22T15:50:35.228 INFO:tasks.workunit.client.1.smithi062.stdout:8/230: rmdir d8/d9/d2c 39 2024-02-22T15:50:35.231 INFO:tasks.workunit.client.0.smithi019.stdout:6/485: dread - d6/d31/f3b zero size 2024-02-22T15:50:35.231 INFO:tasks.workunit.client.1.smithi062.stdout:2/257: dread d1/d8/f15 [0,4194304] 0 2024-02-22T15:50:35.237 INFO:tasks.workunit.client.0.smithi019.stdout:1/520: dwrite d8/d15/d34/fac [0,4194304] 0 2024-02-22T15:50:35.238 INFO:tasks.workunit.client.1.smithi062.stdout:2/258: dwrite d1/d28/d3f/f46 [0,4194304] 0 2024-02-22T15:50:35.240 INFO:tasks.workunit.client.1.smithi062.stdout:0/257: dread d1/dd/f36 [0,4194304] 0 2024-02-22T15:50:35.243 INFO:tasks.workunit.client.0.smithi019.stdout:1/521: fsync d8/d15/d34/f39 0 2024-02-22T15:50:35.246 INFO:tasks.workunit.client.1.smithi062.stdout:2/259: dwrite d1/d47/f40 [0,4194304] 0 2024-02-22T15:50:35.248 INFO:tasks.workunit.client.1.smithi062.stdout:2/260: creat d1/d47/f48 x:0 0 0 2024-02-22T15:50:35.249 INFO:tasks.workunit.client.1.smithi062.stdout:2/261: chown d1/d8/df/c2f 1 1 2024-02-22T15:50:35.250 INFO:tasks.workunit.client.1.smithi062.stdout:2/262: symlink d1/d47/d39/l49 0 2024-02-22T15:50:35.251 INFO:tasks.workunit.client.0.smithi019.stdout:5/481: dread d8/dd/d19/d38/d54/f6b [0,4194304] 0 2024-02-22T15:50:35.251 INFO:tasks.workunit.client.1.smithi062.stdout:7/228: sync 2024-02-22T15:50:35.252 INFO:tasks.workunit.client.0.smithi019.stdout:6/486: dwrite d6/d58/f79 [0,4194304] 0 2024-02-22T15:50:35.254 INFO:tasks.workunit.client.1.smithi062.stdout:7/229: chown d0/d14/d19/d30/d37 1 1 2024-02-22T15:50:35.254 INFO:tasks.workunit.client.1.smithi062.stdout:2/263: creat d1/d8/df/f4a x:0 0 0 2024-02-22T15:50:35.256 INFO:tasks.workunit.client.0.smithi019.stdout:7/581: dwrite d9/d3a/f6b [0,4194304] 0 2024-02-22T15:50:35.257 INFO:tasks.workunit.client.1.smithi062.stdout:7/230: chown d0/d14 840861273 1 2024-02-22T15:50:35.258 INFO:tasks.workunit.client.1.smithi062.stdout:2/264: creat d1/d47/d39/f4b x:0 0 0 2024-02-22T15:50:35.259 INFO:tasks.workunit.client.0.smithi019.stdout:7/582: chown d9/d3a/d49/c7a 53257022 1 2024-02-22T15:50:35.259 INFO:tasks.workunit.client.1.smithi062.stdout:2/265: chown d1/cd 15548 1 2024-02-22T15:50:35.259 INFO:tasks.workunit.client.0.smithi019.stdout:6/487: chown d6/d4b/d6d/d7b/l3d 860310 1 2024-02-22T15:50:35.261 INFO:tasks.workunit.client.1.smithi062.stdout:7/231: mkdir d0/d4f 0 2024-02-22T15:50:35.265 INFO:tasks.workunit.client.1.smithi062.stdout:2/266: readlink d1/d1e/d38/l3e 0 2024-02-22T15:50:35.267 INFO:tasks.workunit.client.1.smithi062.stdout:7/232: rmdir d0/d6/d2e/d2f 39 2024-02-22T15:50:35.267 INFO:tasks.workunit.client.0.smithi019.stdout:1/522: mknod d8/d18/cb5 0 2024-02-22T15:50:35.271 INFO:tasks.workunit.client.1.smithi062.stdout:2/267: creat d1/d8/df/f4c x:0 0 0 2024-02-22T15:50:35.271 INFO:tasks.workunit.client.1.smithi062.stdout:6/281: write d5/fb [538811,7943] 0 2024-02-22T15:50:35.272 INFO:tasks.workunit.client.1.smithi062.stdout:2/268: write d1/f5 [2180953,79574] 0 2024-02-22T15:50:35.275 INFO:tasks.workunit.client.0.smithi019.stdout:5/482: symlink d8/d11/la1 0 2024-02-22T15:50:35.277 INFO:tasks.workunit.client.1.smithi062.stdout:6/282: creat d5/dd/f5a x:0 0 0 2024-02-22T15:50:35.279 INFO:tasks.workunit.client.0.smithi019.stdout:5/483: symlink d8/dd/d19/d38/d47/la2 0 2024-02-22T15:50:35.280 INFO:tasks.workunit.client.0.smithi019.stdout:7/583: dwrite d9/d3a/f6b [0,4194304] 0 2024-02-22T15:50:35.282 INFO:tasks.workunit.client.1.smithi062.stdout:2/269: dwrite d1/d47/d32/f44 [0,4194304] 0 2024-02-22T15:50:35.283 INFO:tasks.workunit.client.1.smithi062.stdout:7/233: dwrite d0/f24 [0,4194304] 0 2024-02-22T15:50:35.283 INFO:tasks.workunit.client.1.smithi062.stdout:6/283: creat d5/d15/d56/f5b x:0 0 0 2024-02-22T15:50:35.285 INFO:tasks.workunit.client.0.smithi019.stdout:5/484: creat d8/dd/d19/d38/d47/fa3 x:0 0 0 2024-02-22T15:50:35.287 INFO:tasks.workunit.client.0.smithi019.stdout:6/488: dwrite d6/d31/d43/f5f [0,4194304] 0 2024-02-22T15:50:35.288 INFO:tasks.workunit.client.0.smithi019.stdout:7/584: rename d9/d3a/d49/d5d/l7d to d9/d26/d77/lbe 0 2024-02-22T15:50:35.291 INFO:tasks.workunit.client.0.smithi019.stdout:6/489: fsync d6/d58/d6a/f85 0 2024-02-22T15:50:35.292 INFO:tasks.workunit.client.0.smithi019.stdout:5/485: creat d8/dd/d19/d77/d7a/d9e/fa4 x:0 0 0 2024-02-22T15:50:35.294 INFO:tasks.workunit.client.1.smithi062.stdout:2/270: symlink d1/d8/df/l4d 0 2024-02-22T15:50:35.294 INFO:tasks.workunit.client.1.smithi062.stdout:7/234: dread - d0/d14/f25 zero size 2024-02-22T15:50:35.294 INFO:tasks.workunit.client.1.smithi062.stdout:2/271: rmdir d1/d1e/d38 39 2024-02-22T15:50:35.294 INFO:tasks.workunit.client.1.smithi062.stdout:7/235: creat d0/d6/f50 x:0 0 0 2024-02-22T15:50:35.298 INFO:tasks.workunit.client.1.smithi062.stdout:6/284: creat d5/df/d2a/f5c x:0 0 0 2024-02-22T15:50:35.298 INFO:tasks.workunit.client.0.smithi019.stdout:6/490: truncate d6/ff 255878 0 2024-02-22T15:50:35.299 INFO:tasks.workunit.client.0.smithi019.stdout:7/585: truncate d9/f24 1492307 0 2024-02-22T15:50:35.300 INFO:tasks.workunit.client.1.smithi062.stdout:2/272: dread d1/f2c [0,4194304] 0 2024-02-22T15:50:35.301 INFO:tasks.workunit.client.1.smithi062.stdout:8/231: dread d8/d9/f27 [0,4194304] 0 2024-02-22T15:50:35.302 INFO:tasks.workunit.client.1.smithi062.stdout:0/258: truncate d1/d3/de/f3a 1692890 0 2024-02-22T15:50:35.303 INFO:tasks.workunit.client.1.smithi062.stdout:7/236: symlink d0/d14/d28/l51 0 2024-02-22T15:50:35.303 INFO:tasks.workunit.client.1.smithi062.stdout:6/285: unlink d5/f1f 0 2024-02-22T15:50:35.304 INFO:tasks.workunit.client.1.smithi062.stdout:2/273: dread d1/d8/df/f26 [0,4194304] 0 2024-02-22T15:50:35.305 INFO:tasks.workunit.client.1.smithi062.stdout:8/232: chown d8/d9/f37 9890625 1 2024-02-22T15:50:35.307 INFO:tasks.workunit.client.1.smithi062.stdout:7/237: dread - d0/d14/d28/f47 zero size 2024-02-22T15:50:35.308 INFO:tasks.workunit.client.1.smithi062.stdout:2/274: mknod d1/d47/c4e 0 2024-02-22T15:50:35.309 INFO:tasks.workunit.client.1.smithi062.stdout:7/238: write d0/f24 [4549176,63470] 0 2024-02-22T15:50:35.310 INFO:tasks.workunit.client.1.smithi062.stdout:2/275: chown d1/d28 36487 1 2024-02-22T15:50:35.313 INFO:tasks.workunit.client.1.smithi062.stdout:7/239: dread - d0/d14/f25 zero size 2024-02-22T15:50:35.314 INFO:tasks.workunit.client.0.smithi019.stdout:6/491: dwrite d6/d4b/f5b [0,4194304] 0 2024-02-22T15:50:35.314 INFO:tasks.workunit.client.1.smithi062.stdout:0/259: getdents d1/d3/de/d14 0 2024-02-22T15:50:35.316 INFO:tasks.workunit.client.1.smithi062.stdout:0/260: dread - d1/d21/f47 zero size 2024-02-22T15:50:35.318 INFO:tasks.workunit.client.0.smithi019.stdout:6/492: chown d6/d31/d43/f46 40 1 2024-02-22T15:50:35.318 INFO:tasks.workunit.client.0.smithi019.stdout:1/523: dwrite d8/d2a/f32 [0,4194304] 0 2024-02-22T15:50:35.319 INFO:tasks.workunit.client.1.smithi062.stdout:2/276: write d1/d8/df/f25 [2379311,96172] 0 2024-02-22T15:50:35.320 INFO:tasks.workunit.client.0.smithi019.stdout:6/493: chown d6/d4b/d6d/d7b/l3d 0 1 2024-02-22T15:50:35.322 INFO:tasks.workunit.client.1.smithi062.stdout:6/286: dwrite d5/dd/d33/f48 [0,4194304] 0 2024-02-22T15:50:35.322 INFO:tasks.workunit.client.1.smithi062.stdout:7/240: unlink d0/d14/d19/f4b 0 2024-02-22T15:50:35.324 INFO:tasks.workunit.client.1.smithi062.stdout:7/241: stat d0/d14/c2b 0 2024-02-22T15:50:35.327 INFO:tasks.workunit.client.1.smithi062.stdout:8/233: dwrite d8/d9/d2c/f30 [0,4194304] 0 2024-02-22T15:50:35.330 INFO:tasks.workunit.client.0.smithi019.stdout:5/486: truncate d8/d11/f87 504083 0 2024-02-22T15:50:35.330 INFO:tasks.workunit.client.0.smithi019.stdout:1/524: stat d8/d15/f26 0 2024-02-22T15:50:35.332 INFO:tasks.workunit.client.0.smithi019.stdout:6/494: mkdir d6/d31/d43/d55/d67/d8a/d90 0 2024-02-22T15:50:35.333 INFO:tasks.workunit.client.1.smithi062.stdout:6/287: mknod d5/d15/d56/c5d 0 2024-02-22T15:50:35.337 INFO:tasks.workunit.client.1.smithi062.stdout:0/261: mknod d1/d3/d4/d1e/c5a 0 2024-02-22T15:50:35.337 INFO:tasks.workunit.client.1.smithi062.stdout:6/288: stat d5/dd/f13 0 2024-02-22T15:50:35.338 INFO:tasks.workunit.client.1.smithi062.stdout:8/234: creat d8/d9/d44/f45 x:0 0 0 2024-02-22T15:50:35.339 INFO:tasks.workunit.client.0.smithi019.stdout:1/525: readlink d8/d15/d5b/l6f 0 2024-02-22T15:50:35.340 INFO:tasks.workunit.client.0.smithi019.stdout:5/487: fsync d8/dd/d8e/d2c/f31 0 2024-02-22T15:50:35.341 INFO:tasks.workunit.client.1.smithi062.stdout:7/242: dwrite d0/d6/d2e/d4a/f1f [0,4194304] 0 2024-02-22T15:50:35.344 INFO:tasks.workunit.client.0.smithi019.stdout:7/586: dwrite f6 [0,4194304] 0 2024-02-22T15:50:35.345 INFO:tasks.workunit.client.1.smithi062.stdout:7/243: chown d0/d14/l18 5169 1 2024-02-22T15:50:35.350 INFO:tasks.workunit.client.0.smithi019.stdout:6/495: fdatasync d6/d31/f6b 0 2024-02-22T15:50:35.352 INFO:tasks.workunit.client.1.smithi062.stdout:0/262: dwrite d1/d21/f47 [0,4194304] 0 2024-02-22T15:50:35.353 INFO:tasks.workunit.client.0.smithi019.stdout:5/488: mknod d8/d11/d33/ca5 0 2024-02-22T15:50:35.353 INFO:tasks.workunit.client.0.smithi019.stdout:1/526: rename d8/d15/c13 to d8/d15/d84/cb6 0 2024-02-22T15:50:35.354 INFO:tasks.workunit.client.1.smithi062.stdout:7/244: link d0/d14/d13/fc d0/d6/f52 0 2024-02-22T15:50:35.354 INFO:tasks.workunit.client.1.smithi062.stdout:0/263: stat d1/d13/d43 0 2024-02-22T15:50:35.355 INFO:tasks.workunit.client.1.smithi062.stdout:6/289: dwrite d5/d15/f52 [0,4194304] 0 2024-02-22T15:50:35.357 INFO:tasks.workunit.client.1.smithi062.stdout:7/245: fsync d0/d14/d13/f3e 0 2024-02-22T15:50:35.361 INFO:tasks.workunit.client.0.smithi019.stdout:1/527: symlink d8/d18/lb7 0 2024-02-22T15:50:35.362 INFO:tasks.workunit.client.1.smithi062.stdout:6/290: creat d5/df/d40/d44/d4a/f5e x:0 0 0 2024-02-22T15:50:35.365 INFO:tasks.workunit.client.1.smithi062.stdout:7/246: unlink d0/d14/d19/d30/f49 0 2024-02-22T15:50:35.365 INFO:tasks.workunit.client.1.smithi062.stdout:0/264: rename d1/dd/l32 to d1/d3/l5b 0 2024-02-22T15:50:35.369 INFO:tasks.workunit.client.1.smithi062.stdout:6/291: rename d5/dd/f5a to d5/df/d40/d44/d4a/d41/f5f 0 2024-02-22T15:50:35.369 INFO:tasks.workunit.client.0.smithi019.stdout:5/489: dwrite d8/d11/f93 [0,4194304] 0 2024-02-22T15:50:35.370 INFO:tasks.workunit.client.1.smithi062.stdout:6/292: truncate d5/dd/f20 1330737 0 2024-02-22T15:50:35.371 INFO:tasks.workunit.client.1.smithi062.stdout:0/265: mkdir d1/d3/d4/d5c 0 2024-02-22T15:50:35.373 INFO:tasks.workunit.client.0.smithi019.stdout:1/528: unlink d8/d23/d33/f37 0 2024-02-22T15:50:35.377 INFO:tasks.workunit.client.1.smithi062.stdout:0/266: rename d1/d3/d4/d1a/d1d/d1f to d1/d3/de/d3f/d5d 0 2024-02-22T15:50:35.379 INFO:tasks.workunit.client.1.smithi062.stdout:0/267: write d1/d21/d3e/f46 [873316,4967] 0 2024-02-22T15:50:35.382 INFO:tasks.workunit.client.1.smithi062.stdout:6/293: dwrite d5/dd/d33/f49 [0,4194304] 0 2024-02-22T15:50:35.387 INFO:tasks.workunit.client.1.smithi062.stdout:0/268: mkdir d1/d5e 0 2024-02-22T15:50:35.387 INFO:tasks.workunit.client.1.smithi062.stdout:2/277: truncate d1/d47/d32/f44 3372921 0 2024-02-22T15:50:35.387 INFO:tasks.workunit.client.1.smithi062.stdout:8/235: truncate d8/d9/f27 966901 0 2024-02-22T15:50:35.388 INFO:tasks.workunit.client.1.smithi062.stdout:6/294: symlink d5/d15/d56/l60 0 2024-02-22T15:50:35.388 INFO:tasks.workunit.client.1.smithi062.stdout:2/278: chown d1/d28 299515389 1 2024-02-22T15:50:35.389 INFO:tasks.workunit.client.0.smithi019.stdout:1/529: dwrite d8/d2a/f32 [0,4194304] 0 2024-02-22T15:50:35.389 INFO:tasks.workunit.client.1.smithi062.stdout:0/269: dread - d1/d13/f31 zero size 2024-02-22T15:50:35.391 INFO:tasks.workunit.client.0.smithi019.stdout:1/530: write d8/d18/d64/f66 [180352,68833] 0 2024-02-22T15:50:35.391 INFO:tasks.workunit.client.1.smithi062.stdout:8/236: rmdir d8/d9/d44 39 2024-02-22T15:50:35.392 INFO:tasks.workunit.client.1.smithi062.stdout:8/237: chown d8/f3a 1646228 1 2024-02-22T15:50:35.394 INFO:tasks.workunit.client.1.smithi062.stdout:6/295: creat d5/d15/d56/f61 x:0 0 0 2024-02-22T15:50:35.394 INFO:tasks.workunit.client.0.smithi019.stdout:6/496: dread d6/d3e/f50 [0,4194304] 0 2024-02-22T15:50:35.396 INFO:tasks.workunit.client.1.smithi062.stdout:0/270: rename d1/d21/d3e to d1/d3/d5f 0 2024-02-22T15:50:35.398 INFO:tasks.workunit.client.1.smithi062.stdout:0/271: read - d1/d21/f49 zero size 2024-02-22T15:50:35.398 INFO:tasks.workunit.client.0.smithi019.stdout:5/490: rmdir d8/d11/d33 39 2024-02-22T15:50:35.398 INFO:tasks.workunit.client.0.smithi019.stdout:7/587: write d9/d26/f2b [1708236,90033] 0 2024-02-22T15:50:35.399 INFO:tasks.workunit.client.1.smithi062.stdout:0/272: write d1/d21/f49 [1039156,130772] 0 2024-02-22T15:50:35.399 INFO:tasks.workunit.client.0.smithi019.stdout:6/497: mknod d6/d4b/c91 0 2024-02-22T15:50:35.399 INFO:tasks.workunit.client.1.smithi062.stdout:8/238: rmdir d8/d26 39 2024-02-22T15:50:35.400 INFO:tasks.workunit.client.1.smithi062.stdout:0/273: chown d1/d13/c15 45532294 1 2024-02-22T15:50:35.403 INFO:tasks.workunit.client.1.smithi062.stdout:6/296: symlink d5/df/d40/d44/d4a/l62 0 2024-02-22T15:50:35.405 INFO:tasks.workunit.client.1.smithi062.stdout:2/279: rename d1/d47/f40 to d1/d1e/d38/f4f 0 2024-02-22T15:50:35.407 INFO:tasks.workunit.client.0.smithi019.stdout:7/588: rename d9/d3a/d49/c7b to d9/d38/d41/d45/cbf 0 2024-02-22T15:50:35.407 INFO:tasks.workunit.client.0.smithi019.stdout:5/491: link d8/dd/d8e/l72 d8/dd/d19/d77/la6 0 2024-02-22T15:50:35.408 INFO:tasks.workunit.client.1.smithi062.stdout:2/280: truncate d1/d8/df/f4a 607310 0 2024-02-22T15:50:35.408 INFO:tasks.workunit.client.0.smithi019.stdout:5/492: chown d8/f2d 501 1 2024-02-22T15:50:35.409 INFO:tasks.workunit.client.1.smithi062.stdout:2/281: chown d1/d8/df/f31 1319519 1 2024-02-22T15:50:35.414 INFO:tasks.workunit.client.1.smithi062.stdout:6/297: dwrite d5/dd/d33/f48 [0,4194304] 0 2024-02-22T15:50:35.416 INFO:tasks.workunit.client.0.smithi019.stdout:7/589: creat d9/d38/d41/d45/d61/fc0 x:0 0 0 2024-02-22T15:50:35.417 INFO:tasks.workunit.client.0.smithi019.stdout:5/493: dwrite d8/f89 [0,4194304] 0 2024-02-22T15:50:35.418 INFO:tasks.workunit.client.0.smithi019.stdout:5/494: fsync d8/dd/d8e/f85 0 2024-02-22T15:50:35.423 INFO:tasks.workunit.client.1.smithi062.stdout:0/274: link d1/d13/c15 d1/d5e/c60 0 2024-02-22T15:50:35.425 INFO:tasks.workunit.client.1.smithi062.stdout:8/239: dwrite d8/d26/f34 [0,4194304] 0 2024-02-22T15:50:35.427 INFO:tasks.workunit.client.1.smithi062.stdout:2/282: rename d1/d8/df/f14 to d1/f50 0 2024-02-22T15:50:35.430 INFO:tasks.workunit.client.1.smithi062.stdout:8/240: read d8/f29 [1670385,121470] 0 2024-02-22T15:50:35.432 INFO:tasks.workunit.client.1.smithi062.stdout:0/275: rmdir d1/d3/de/d3f 39 2024-02-22T15:50:35.433 INFO:tasks.workunit.client.1.smithi062.stdout:8/241: write d8/d26/f34 [1721610,90834] 0 2024-02-22T15:50:35.435 INFO:tasks.workunit.client.0.smithi019.stdout:5/495: rename d8/dd/d19/d38/d47/l56 to d8/dd/d19/d3a/la7 0 2024-02-22T15:50:35.435 INFO:tasks.workunit.client.1.smithi062.stdout:6/298: symlink d5/df/d2a/d59/l63 0 2024-02-22T15:50:35.437 INFO:tasks.workunit.client.0.smithi019.stdout:7/590: link d9/d38/d41/d45/cbf d9/d26/cc1 0 2024-02-22T15:50:35.438 INFO:tasks.workunit.client.0.smithi019.stdout:6/498: dread d6/d31/d43/f56 [0,4194304] 0 2024-02-22T15:50:35.438 INFO:tasks.workunit.client.1.smithi062.stdout:2/283: rename d1/d47/f24 to d1/d28/d3f/f51 0 2024-02-22T15:50:35.439 INFO:tasks.workunit.client.0.smithi019.stdout:7/591: write d9/d4c/d5b/f65 [4589702,80544] 0 2024-02-22T15:50:35.440 INFO:tasks.workunit.client.0.smithi019.stdout:6/499: chown d6/d4b/f5b 85 1 2024-02-22T15:50:35.444 INFO:tasks.workunit.client.0.smithi019.stdout:2/443: sync 2024-02-22T15:50:35.444 INFO:tasks.workunit.client.0.smithi019.stdout:4/461: sync 2024-02-22T15:50:35.445 INFO:tasks.workunit.client.1.smithi062.stdout:0/276: dwrite d1/d13/f2f [0,4194304] 0 2024-02-22T15:50:35.445 INFO:tasks.workunit.client.0.smithi019.stdout:4/462: chown d2/db/d54/c5d 56812444 1 2024-02-22T15:50:35.445 INFO:tasks.workunit.client.0.smithi019.stdout:7/592: rename d9/d38/d41/d45/d61/fa4 to d9/d26/d77/fc2 0 2024-02-22T15:50:35.446 INFO:tasks.workunit.client.1.smithi062.stdout:2/284: truncate d1/d1e/f2d 4851614 0 2024-02-22T15:50:35.446 INFO:tasks.workunit.client.1.smithi062.stdout:6/299: dwrite d5/df/d32/d47/f50 [0,4194304] 0 2024-02-22T15:50:35.448 INFO:tasks.workunit.client.0.smithi019.stdout:4/463: fsync d2/db/d14/d23/d21/d35/f64 0 2024-02-22T15:50:35.449 INFO:tasks.workunit.client.0.smithi019.stdout:7/593: write d9/d3a/d49/d5d/fbd [739381,78708] 0 2024-02-22T15:50:35.450 INFO:tasks.workunit.client.0.smithi019.stdout:6/500: creat d6/d4b/d6d/d7b/f92 x:0 0 0 2024-02-22T15:50:35.450 INFO:tasks.workunit.client.0.smithi019.stdout:2/444: read d1/f18 [2527794,72701] 0 2024-02-22T15:50:35.451 INFO:tasks.workunit.client.1.smithi062.stdout:0/277: mknod d1/c61 0 2024-02-22T15:50:35.456 INFO:tasks.workunit.client.0.smithi019.stdout:4/464: rename d2/d3f/d48/l71 to d2/d3f/d48/l97 0 2024-02-22T15:50:35.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:35 smithi062 ceph-mon[48130]: pgmap v33: 65 pgs: 65 active+clean; 870 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 9.6 MiB/s rd, 81 MiB/s wr, 202 op/s 2024-02-22T15:50:35.458 INFO:tasks.workunit.client.0.smithi019.stdout:6/501: mknod d6/d31/d43/d55/d67/d8a/c93 0 2024-02-22T15:50:35.459 INFO:tasks.workunit.client.0.smithi019.stdout:5/496: dread d8/f28 [0,4194304] 0 2024-02-22T15:50:35.462 INFO:tasks.workunit.client.0.smithi019.stdout:6/502: read d6/d3e/f42 [801585,25275] 0 2024-02-22T15:50:35.463 INFO:tasks.workunit.client.0.smithi019.stdout:2/445: dwrite d1/f1f [0,4194304] 0 2024-02-22T15:50:35.464 INFO:tasks.workunit.client.0.smithi019.stdout:4/465: getdents d2/db/d14/d23/d21/d35 0 2024-02-22T15:50:35.464 INFO:tasks.workunit.client.0.smithi019.stdout:6/503: write d6/d31/f36 [4746794,58684] 0 2024-02-22T15:50:35.465 INFO:tasks.workunit.client.0.smithi019.stdout:5/497: creat d8/d11/fa8 x:0 0 0 2024-02-22T15:50:35.466 INFO:tasks.workunit.client.0.smithi019.stdout:4/466: readlink d2/db/d14/l24 0 2024-02-22T15:50:35.467 INFO:tasks.workunit.client.0.smithi019.stdout:6/504: chown d6/f20 8231450 1 2024-02-22T15:50:35.472 INFO:tasks.workunit.client.0.smithi019.stdout:4/467: write d2/db/d92/f8a [2723556,18031] 0 2024-02-22T15:50:35.476 INFO:tasks.workunit.client.0.smithi019.stdout:4/468: creat d2/f98 x:0 0 0 2024-02-22T15:50:35.477 INFO:tasks.workunit.client.0.smithi019.stdout:5/498: link d8/dd/d19/f29 d8/d11/d33/fa9 0 2024-02-22T15:50:35.480 INFO:tasks.workunit.client.0.smithi019.stdout:6/505: dwrite d6/d3e/f42 [0,4194304] 0 2024-02-22T15:50:35.480 INFO:tasks.workunit.client.0.smithi019.stdout:4/469: mknod d2/db/d14/d23/d4b/d8e/c99 0 2024-02-22T15:50:35.482 INFO:tasks.workunit.client.0.smithi019.stdout:5/499: symlink d8/dd/d8e/d2c/laa 0 2024-02-22T15:50:35.482 INFO:tasks.workunit.client.0.smithi019.stdout:2/446: dread d1/f14 [0,4194304] 0 2024-02-22T15:50:35.484 INFO:tasks.workunit.client.0.smithi019.stdout:6/506: creat d6/d4b/d6d/d7b/d7f/f94 x:0 0 0 2024-02-22T15:50:35.484 INFO:tasks.workunit.client.1.smithi062.stdout:1/218: sync 2024-02-22T15:50:35.484 INFO:tasks.workunit.client.0.smithi019.stdout:4/470: symlink d2/db/d14/d23/d21/d42/l9a 0 2024-02-22T15:50:35.485 INFO:tasks.workunit.client.0.smithi019.stdout:5/500: rmdir d8/dd/d19/d77/d7a 39 2024-02-22T15:50:35.488 INFO:tasks.workunit.client.1.smithi062.stdout:6/300: dwrite d5/df/d32/d47/f50 [0,4194304] 0 2024-02-22T15:50:35.489 INFO:tasks.workunit.client.0.smithi019.stdout:6/507: creat d6/d4b/d8d/f95 x:0 0 0 2024-02-22T15:50:35.489 INFO:tasks.workunit.client.1.smithi062.stdout:0/278: readlink d1/d13/l4b 0 2024-02-22T15:50:35.492 INFO:tasks.workunit.client.0.smithi019.stdout:7/594: truncate d9/d38/d41/fb7 40410 0 2024-02-22T15:50:35.492 INFO:tasks.workunit.client.0.smithi019.stdout:2/447: dwrite d1/f2e [0,4194304] 0 2024-02-22T15:50:35.495 INFO:tasks.workunit.client.0.smithi019.stdout:6/508: creat d6/d4b/d8d/f96 x:0 0 0 2024-02-22T15:50:35.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:35 smithi019 ceph-mon[67418]: pgmap v33: 65 pgs: 65 active+clean; 870 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 9.6 MiB/s rd, 81 MiB/s wr, 202 op/s 2024-02-22T15:50:35.495 INFO:tasks.workunit.client.1.smithi062.stdout:6/301: chown d5/df/f26 7 1 2024-02-22T15:50:35.496 INFO:tasks.workunit.client.1.smithi062.stdout:2/285: fdatasync d1/d47/d32/f44 0 2024-02-22T15:50:35.496 INFO:tasks.workunit.client.0.smithi019.stdout:4/471: dread d2/db/d14/d23/d21/f29 [0,4194304] 0 2024-02-22T15:50:35.497 INFO:tasks.workunit.client.1.smithi062.stdout:0/279: creat d1/d3/de/d3f/f62 x:0 0 0 2024-02-22T15:50:35.499 INFO:tasks.workunit.client.1.smithi062.stdout:2/286: read - d1/d47/f48 zero size 2024-02-22T15:50:35.499 INFO:tasks.workunit.client.0.smithi019.stdout:5/501: creat d8/dd/d19/d3a/fab x:0 0 0 2024-02-22T15:50:35.500 INFO:tasks.workunit.client.1.smithi062.stdout:2/287: chown d1/d1e/d38/l3e 1939506 1 2024-02-22T15:50:35.500 INFO:tasks.workunit.client.1.smithi062.stdout:0/280: creat d1/dd/f63 x:0 0 0 2024-02-22T15:50:35.501 INFO:tasks.workunit.client.1.smithi062.stdout:6/302: dread d5/dd/f13 [0,4194304] 0 2024-02-22T15:50:35.502 INFO:tasks.workunit.client.0.smithi019.stdout:5/502: truncate d8/f89 5099008 0 2024-02-22T15:50:35.502 INFO:tasks.workunit.client.0.smithi019.stdout:2/448: creat d1/d17/d1c/d7f/fa2 x:0 0 0 2024-02-22T15:50:35.503 INFO:tasks.workunit.client.1.smithi062.stdout:6/303: dread - d5/dd/d33/f46 zero size 2024-02-22T15:50:35.503 INFO:tasks.workunit.client.1.smithi062.stdout:0/281: write d1/d3/d5f/f46 [113088,86908] 0 2024-02-22T15:50:35.504 INFO:tasks.workunit.client.1.smithi062.stdout:2/288: mkdir d1/d47/d32/d52 0 2024-02-22T15:50:35.504 INFO:tasks.workunit.client.1.smithi062.stdout:6/304: chown d5/f38 616 1 2024-02-22T15:50:35.504 INFO:tasks.workunit.client.1.smithi062.stdout:0/282: readlink d1/d3/de/l11 0 2024-02-22T15:50:35.505 INFO:tasks.workunit.client.0.smithi019.stdout:4/472: creat d2/db/d14/d23/d21/f9b x:0 0 0 2024-02-22T15:50:35.505 INFO:tasks.workunit.client.1.smithi062.stdout:0/283: chown d1/d3/de/d14 451 1 2024-02-22T15:50:35.506 INFO:tasks.workunit.client.1.smithi062.stdout:6/305: fsync d5/df/d32/f3d 0 2024-02-22T15:50:35.507 INFO:tasks.workunit.client.0.smithi019.stdout:4/473: fsync d2/db/d44/f55 0 2024-02-22T15:50:35.508 INFO:tasks.workunit.client.0.smithi019.stdout:5/503: mknod d8/dd/d19/d77/d7a/cac 0 2024-02-22T15:50:35.508 INFO:tasks.workunit.client.0.smithi019.stdout:6/509: dwrite d6/fc [4194304,4194304] 0 2024-02-22T15:50:35.509 INFO:tasks.workunit.client.1.smithi062.stdout:0/284: dread d1/d3/f39 [0,4194304] 0 2024-02-22T15:50:35.509 INFO:tasks.workunit.client.1.smithi062.stdout:6/306: read - d5/df/d40/d44/d4a/d41/d43/f58 zero size 2024-02-22T15:50:35.510 INFO:tasks.workunit.client.0.smithi019.stdout:6/510: chown d6/d31/d43 47 1 2024-02-22T15:50:35.510 INFO:tasks.workunit.client.0.smithi019.stdout:2/449: rmdir d1/d17/d21/d51 39 2024-02-22T15:50:35.512 INFO:tasks.workunit.client.1.smithi062.stdout:6/307: chown d5/df/d32/f34 2 1 2024-02-22T15:50:35.513 INFO:tasks.workunit.client.1.smithi062.stdout:0/285: fdatasync d1/d3/d4/d1a/f34 0 2024-02-22T15:50:35.514 INFO:tasks.workunit.client.1.smithi062.stdout:2/289: dwrite d1/f50 [4194304,4194304] 0 2024-02-22T15:50:35.515 INFO:tasks.workunit.client.0.smithi019.stdout:5/504: fdatasync d8/d11/f3e 0 2024-02-22T15:50:35.515 INFO:tasks.workunit.client.1.smithi062.stdout:0/286: chown d1/dd/c25 152967 1 2024-02-22T15:50:35.516 INFO:tasks.workunit.client.0.smithi019.stdout:4/474: mknod d2/db/d14/d23/d21/d35/d85/c9c 0 2024-02-22T15:50:35.518 INFO:tasks.workunit.client.0.smithi019.stdout:6/511: mkdir d6/d4b/d6d/d7b/d97 0 2024-02-22T15:50:35.521 INFO:tasks.workunit.client.0.smithi019.stdout:2/450: rename d1/d17/d21/d71/l48 to d1/d17/d21/d71/d60/la3 0 2024-02-22T15:50:35.521 INFO:tasks.workunit.client.1.smithi062.stdout:2/290: rename d1/f6 to d1/d28/f53 0 2024-02-22T15:50:35.521 INFO:tasks.workunit.client.1.smithi062.stdout:6/308: symlink d5/df/d40/d44/d4a/l64 0 2024-02-22T15:50:35.524 INFO:tasks.workunit.client.0.smithi019.stdout:5/505: read d8/dd/d19/d38/d47/f68 [1557050,123896] 0 2024-02-22T15:50:35.525 INFO:tasks.workunit.client.0.smithi019.stdout:2/451: mkdir d1/da4 0 2024-02-22T15:50:35.525 INFO:tasks.workunit.client.1.smithi062.stdout:0/287: creat d1/d3/de/d3f/f64 x:0 0 0 2024-02-22T15:50:35.525 INFO:tasks.workunit.client.1.smithi062.stdout:0/288: stat d1/d3/d4/l33 0 2024-02-22T15:50:35.525 INFO:tasks.workunit.client.0.smithi019.stdout:5/506: chown d8/d11/f99 574334037 1 2024-02-22T15:50:35.526 INFO:tasks.workunit.client.1.smithi062.stdout:2/291: rename d1/d28/d3f/f51 to d1/d1e/d38/f54 0 2024-02-22T15:50:35.526 INFO:tasks.workunit.client.0.smithi019.stdout:5/507: readlink d8/dd/d19/d3a/d46/l6d 0 2024-02-22T15:50:35.527 INFO:tasks.workunit.client.0.smithi019.stdout:2/452: write d1/d17/d21/d51/f7c [724694,47178] 0 2024-02-22T15:50:35.528 INFO:tasks.workunit.client.1.smithi062.stdout:6/309: creat d5/d15/d56/f65 x:0 0 0 2024-02-22T15:50:35.529 INFO:tasks.workunit.client.1.smithi062.stdout:0/289: creat d1/d13/f65 x:0 0 0 2024-02-22T15:50:35.530 INFO:tasks.workunit.client.1.smithi062.stdout:1/219: write d1/d6/de/d21/d24/f2a [1341178,36645] 0 2024-02-22T15:50:35.530 INFO:tasks.workunit.client.0.smithi019.stdout:5/508: creat d8/dd/d19/d38/d54/fad x:0 0 0 2024-02-22T15:50:35.531 INFO:tasks.workunit.client.0.smithi019.stdout:5/509: chown d8/dd/c16 9 1 2024-02-22T15:50:35.531 INFO:tasks.workunit.client.1.smithi062.stdout:0/290: chown d1/dd/f36 49286891 1 2024-02-22T15:50:35.532 INFO:tasks.workunit.client.0.smithi019.stdout:5/510: write d8/d11/d33/f74 [810850,30296] 0 2024-02-22T15:50:35.532 INFO:tasks.workunit.client.1.smithi062.stdout:6/310: mkdir d5/dd/d33/d66 0 2024-02-22T15:50:35.533 INFO:tasks.workunit.client.0.smithi019.stdout:5/511: chown d8/dd/d19/d38/d47/l4b 3087 1 2024-02-22T15:50:35.534 INFO:tasks.workunit.client.0.smithi019.stdout:5/512: fdatasync d8/dd/d19/d77/d7a/f97 0 2024-02-22T15:50:35.534 INFO:tasks.workunit.client.1.smithi062.stdout:1/220: dread - d1/f27 zero size 2024-02-22T15:50:35.534 INFO:tasks.workunit.client.0.smithi019.stdout:5/513: fdatasync d8/d11/f3b 0 2024-02-22T15:50:35.535 INFO:tasks.workunit.client.0.smithi019.stdout:5/514: chown d8/ca0 0 1 2024-02-22T15:50:35.536 INFO:tasks.workunit.client.1.smithi062.stdout:2/292: dwrite d1/d8/df/f25 [0,4194304] 0 2024-02-22T15:50:35.536 INFO:tasks.workunit.client.1.smithi062.stdout:6/311: symlink d5/df/l67 0 2024-02-22T15:50:35.537 INFO:tasks.workunit.client.0.smithi019.stdout:5/515: rmdir d8/dd/d19/d77/d7a/d9e 39 2024-02-22T15:50:35.542 INFO:tasks.workunit.client.1.smithi062.stdout:2/293: mkdir d1/d1e/d38/d55 0 2024-02-22T15:50:35.543 INFO:tasks.workunit.client.0.smithi019.stdout:5/516: link d8/d11/l62 d8/dd/lae 0 2024-02-22T15:50:35.543 INFO:tasks.workunit.client.1.smithi062.stdout:6/312: symlink d5/df/d32/l68 0 2024-02-22T15:50:35.544 INFO:tasks.workunit.client.1.smithi062.stdout:0/291: link d1/d3/l19 d1/l66 0 2024-02-22T15:50:35.546 INFO:tasks.workunit.client.0.smithi019.stdout:7/595: read d9/d3a/f6b [11940,119110] 0 2024-02-22T15:50:35.549 INFO:tasks.workunit.client.1.smithi062.stdout:1/221: dwrite d1/d6/de/f31 [0,4194304] 0 2024-02-22T15:50:35.549 INFO:tasks.workunit.client.0.smithi019.stdout:5/517: symlink d8/dd/d19/d3a/d46/d7e/laf 0 2024-02-22T15:50:35.549 INFO:tasks.workunit.client.1.smithi062.stdout:6/313: mknod d5/dd/d33/c69 0 2024-02-22T15:50:35.554 INFO:tasks.workunit.client.0.smithi019.stdout:6/512: dread d6/d31/d48/f4a [0,4194304] 0 2024-02-22T15:50:35.554 INFO:tasks.workunit.client.0.smithi019.stdout:4/475: truncate d2/db/f6a 1252399 0 2024-02-22T15:50:35.554 INFO:tasks.workunit.client.1.smithi062.stdout:1/222: symlink d1/d6/de/l46 0 2024-02-22T15:50:35.555 INFO:tasks.workunit.client.0.smithi019.stdout:6/513: chown d6/d4b/l8f 31616 1 2024-02-22T15:50:35.556 INFO:tasks.workunit.client.1.smithi062.stdout:2/294: dread d1/d1e/d38/f4f [0,4194304] 0 2024-02-22T15:50:35.556 INFO:tasks.workunit.client.1.smithi062.stdout:6/314: getdents d5/d15 0 2024-02-22T15:50:35.557 INFO:tasks.workunit.client.1.smithi062.stdout:1/223: fsync d1/d6/de/f42 0 2024-02-22T15:50:35.557 INFO:tasks.workunit.client.1.smithi062.stdout:0/292: dwrite d1/dd/f63 [0,4194304] 0 2024-02-22T15:50:35.557 INFO:tasks.workunit.client.0.smithi019.stdout:2/453: write d1/d17/f90 [820263,100883] 0 2024-02-22T15:50:35.558 INFO:tasks.workunit.client.0.smithi019.stdout:4/476: fdatasync d2/db/d14/d23/d4b/f86 0 2024-02-22T15:50:35.559 INFO:tasks.workunit.client.0.smithi019.stdout:7/596: link d9/d3a/d49/d5d/d74/d9a/lb4 d9/d38/d41/d45/lc3 0 2024-02-22T15:50:35.560 INFO:tasks.workunit.client.1.smithi062.stdout:0/293: truncate d1/d21/f49 1923004 0 2024-02-22T15:50:35.560 INFO:tasks.workunit.client.1.smithi062.stdout:6/315: dread - d5/df/d40/f53 zero size 2024-02-22T15:50:35.561 INFO:tasks.workunit.client.0.smithi019.stdout:2/454: write d1/d86/f9e [1125166,92114] 0 2024-02-22T15:50:35.562 INFO:tasks.workunit.client.0.smithi019.stdout:7/597: write d9/d4c/f66 [1157389,42982] 0 2024-02-22T15:50:35.563 INFO:tasks.workunit.client.0.smithi019.stdout:2/455: stat d1/d17/d21/d71/d60 0 2024-02-22T15:50:35.565 INFO:tasks.workunit.client.0.smithi019.stdout:7/598: read d9/d3a/f75 [1388413,93509] 0 2024-02-22T15:50:35.565 INFO:tasks.workunit.client.1.smithi062.stdout:1/224: rename d1/d6/de/d21 to d1/d6/d7/d2b/d47 0 2024-02-22T15:50:35.567 INFO:tasks.workunit.client.0.smithi019.stdout:2/456: creat d1/d17/d21/fa5 x:0 0 0 2024-02-22T15:50:35.569 INFO:tasks.workunit.client.1.smithi062.stdout:0/294: unlink d1/d13/c28 0 2024-02-22T15:50:35.569 INFO:tasks.workunit.client.1.smithi062.stdout:2/295: fdatasync d1/d8/fa 0 2024-02-22T15:50:35.569 INFO:tasks.workunit.client.0.smithi019.stdout:5/518: write d8/d11/d33/fa9 [4019112,22040] 0 2024-02-22T15:50:35.570 INFO:tasks.workunit.client.1.smithi062.stdout:6/316: creat d5/d15/f6a x:0 0 0 2024-02-22T15:50:35.572 INFO:tasks.workunit.client.0.smithi019.stdout:2/457: mknod d1/d17/d21/d71/ca6 0 2024-02-22T15:50:35.575 INFO:tasks.workunit.client.0.smithi019.stdout:2/458: symlink d1/d86/la7 0 2024-02-22T15:50:35.576 INFO:tasks.workunit.client.0.smithi019.stdout:7/599: dwrite d9/d3a/d49/d5d/f72 [0,4194304] 0 2024-02-22T15:50:35.577 INFO:tasks.workunit.client.1.smithi062.stdout:1/225: rename d1/d6/de/d9/c3b to d1/d6/de/c48 0 2024-02-22T15:50:35.579 INFO:tasks.workunit.client.1.smithi062.stdout:1/226: write d1/d6/d7/f28 [3789810,124021] 0 2024-02-22T15:50:35.580 INFO:tasks.workunit.client.0.smithi019.stdout:2/459: creat d1/de/d15/fa8 x:0 0 0 2024-02-22T15:50:35.582 INFO:tasks.workunit.client.0.smithi019.stdout:7/600: mkdir d9/d26/dc4 0 2024-02-22T15:50:35.583 INFO:tasks.workunit.client.1.smithi062.stdout:1/227: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:35.587 INFO:tasks.workunit.client.0.smithi019.stdout:6/514: dread d6/f44 [0,4194304] 0 2024-02-22T15:50:35.588 INFO:tasks.workunit.client.0.smithi019.stdout:7/601: link d9/c44 d9/d3a/d49/d5d/d74/d9a/cc5 0 2024-02-22T15:50:35.590 INFO:tasks.workunit.client.0.smithi019.stdout:6/515: mkdir d6/d98 0 2024-02-22T15:50:35.592 INFO:tasks.workunit.client.0.smithi019.stdout:6/516: mknod d6/d4b/d8d/c99 0 2024-02-22T15:50:35.594 INFO:tasks.workunit.client.0.smithi019.stdout:7/602: getdents d9/d26/d77 0 2024-02-22T15:50:35.595 INFO:tasks.workunit.client.0.smithi019.stdout:6/517: creat d6/d31/d43/d55/d67/f9a x:0 0 0 2024-02-22T15:50:35.596 INFO:tasks.workunit.client.0.smithi019.stdout:7/603: truncate d9/d26/f87 643155 0 2024-02-22T15:50:35.596 INFO:tasks.workunit.client.0.smithi019.stdout:2/460: truncate d1/f2e 593097 0 2024-02-22T15:50:35.597 INFO:tasks.workunit.client.0.smithi019.stdout:6/518: chown d6/f28 11046069 1 2024-02-22T15:50:35.598 INFO:tasks.workunit.client.0.smithi019.stdout:2/461: mknod d1/d17/d21/d51/ca9 0 2024-02-22T15:50:35.600 INFO:tasks.workunit.client.0.smithi019.stdout:7/604: rmdir d9/d3a/d49/d5d/d74 39 2024-02-22T15:50:35.602 INFO:tasks.workunit.client.0.smithi019.stdout:7/605: mknod d9/d3a/cc6 0 2024-02-22T15:50:35.608 INFO:tasks.workunit.client.0.smithi019.stdout:2/462: rename d1/d17/l49 to d1/laa 0 2024-02-22T15:50:35.609 INFO:tasks.workunit.client.0.smithi019.stdout:7/606: truncate d9/d3a/f60 4160294 0 2024-02-22T15:50:35.609 INFO:tasks.workunit.client.0.smithi019.stdout:7/607: chown d9/f23 243 1 2024-02-22T15:50:35.612 INFO:tasks.workunit.client.0.smithi019.stdout:7/608: creat d9/d26/dc4/fc7 x:0 0 0 2024-02-22T15:50:35.615 INFO:tasks.workunit.client.1.smithi062.stdout:3/198: sync 2024-02-22T15:50:35.615 INFO:tasks.workunit.client.1.smithi062.stdout:5/131: sync 2024-02-22T15:50:35.616 INFO:tasks.workunit.client.0.smithi019.stdout:2/463: dwrite d1/de/f1e [8388608,4194304] 0 2024-02-22T15:50:35.619 INFO:tasks.workunit.client.0.smithi019.stdout:7/609: getdents d9/d79 0 2024-02-22T15:50:35.619 INFO:tasks.workunit.client.1.smithi062.stdout:5/132: creat d0/f26 x:0 0 0 2024-02-22T15:50:35.620 INFO:tasks.workunit.client.1.smithi062.stdout:6/317: getdents d5/df/d40/d44/d4a 0 2024-02-22T15:50:35.621 INFO:tasks.workunit.client.0.smithi019.stdout:2/464: creat d1/d46/fab x:0 0 0 2024-02-22T15:50:35.621 INFO:tasks.workunit.client.1.smithi062.stdout:5/133: chown d0/f16 2308177 1 2024-02-22T15:50:35.622 INFO:tasks.workunit.client.0.smithi019.stdout:2/465: fsync d1/d17/d4b/f79 0 2024-02-22T15:50:35.624 INFO:tasks.workunit.client.0.smithi019.stdout:2/466: dread - d1/d17/d1c/d66/f68 zero size 2024-02-22T15:50:35.624 INFO:tasks.workunit.client.1.smithi062.stdout:5/134: write d0/d3/f14 [1162247,109690] 0 2024-02-22T15:50:35.625 INFO:tasks.workunit.client.1.smithi062.stdout:3/199: rename d5/d1a/c1c to d5/d1a/d3b/c43 0 2024-02-22T15:50:35.626 INFO:tasks.workunit.client.1.smithi062.stdout:3/200: readlink l2 0 2024-02-22T15:50:35.627 INFO:tasks.workunit.client.0.smithi019.stdout:7/610: dwrite f6 [0,4194304] 0 2024-02-22T15:50:35.627 INFO:tasks.workunit.client.0.smithi019.stdout:2/467: mknod d1/cac 0 2024-02-22T15:50:35.627 INFO:tasks.workunit.client.1.smithi062.stdout:6/318: creat d5/df/d40/d44/f6b x:0 0 0 2024-02-22T15:50:35.627 INFO:tasks.workunit.client.1.smithi062.stdout:5/135: fsync d0/d3/da/f1d 0 2024-02-22T15:50:35.631 INFO:tasks.workunit.client.1.smithi062.stdout:5/136: creat d0/d3/da/f27 x:0 0 0 2024-02-22T15:50:35.632 INFO:tasks.workunit.client.0.smithi019.stdout:7/611: creat d9/d3a/d49/d5d/d74/fc8 x:0 0 0 2024-02-22T15:50:35.636 INFO:tasks.workunit.client.0.smithi019.stdout:2/468: mknod d1/d17/d21/d71/cad 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.1.smithi062.stdout:5/137: truncate d0/d3/da/f27 583294 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.1.smithi062.stdout:6/319: creat d5/f6c x:0 0 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.0.smithi019.stdout:7/612: mkdir d9/d4c/dc9 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.0.smithi019.stdout:7/613: write d9/d26/f87 [1510889,104230] 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.0.smithi019.stdout:7/614: readlink d9/d38/d41/d45/l94 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.0.smithi019.stdout:2/469: link d1/d46/f95 d1/d17/fae 0 2024-02-22T15:50:35.637 INFO:tasks.workunit.client.0.smithi019.stdout:7/615: write d9/d38/d41/d45/f8c [13458,42240] 0 2024-02-22T15:50:35.638 INFO:tasks.workunit.client.1.smithi062.stdout:2/296: write d1/d8/fa [1841635,32502] 0 2024-02-22T15:50:35.638 INFO:tasks.workunit.client.0.smithi019.stdout:2/470: truncate d1/f14 1934032 0 2024-02-22T15:50:35.639 INFO:tasks.workunit.client.1.smithi062.stdout:6/320: mknod d5/df/d32/c6d 0 2024-02-22T15:50:35.640 INFO:tasks.workunit.client.1.smithi062.stdout:0/295: truncate d1/d21/f49 1177727 0 2024-02-22T15:50:35.640 INFO:tasks.workunit.client.0.smithi019.stdout:2/471: chown d1/d17/d21/d71/f2f 4149384 1 2024-02-22T15:50:35.640 INFO:tasks.workunit.client.0.smithi019.stdout:5/519: dread d8/d11/f3b [0,4194304] 0 2024-02-22T15:50:35.641 INFO:tasks.workunit.client.0.smithi019.stdout:7/616: rename d9/f4d to d9/d4c/d5b/fca 0 2024-02-22T15:50:35.641 INFO:tasks.workunit.client.1.smithi062.stdout:2/297: truncate d1/d8/f42 216018 0 2024-02-22T15:50:35.641 INFO:tasks.workunit.client.1.smithi062.stdout:1/228: truncate d1/d6/d7/f28 2541106 0 2024-02-22T15:50:35.641 INFO:tasks.workunit.client.1.smithi062.stdout:6/321: chown d5/df/f26 3780 1 2024-02-22T15:50:35.642 INFO:tasks.workunit.client.1.smithi062.stdout:2/298: read - d1/d8/f1a zero size 2024-02-22T15:50:35.642 INFO:tasks.workunit.client.0.smithi019.stdout:2/472: mkdir d1/d17/d21/daf 0 2024-02-22T15:50:35.643 INFO:tasks.workunit.client.0.smithi019.stdout:7/617: symlink d9/d4c/lcb 0 2024-02-22T15:50:35.643 INFO:tasks.workunit.client.1.smithi062.stdout:0/296: rmdir d1/d3/d4 39 2024-02-22T15:50:35.643 INFO:tasks.workunit.client.1.smithi062.stdout:1/229: chown d1/d6/d7/c13 24378 1 2024-02-22T15:50:35.644 INFO:tasks.workunit.client.0.smithi019.stdout:2/473: symlink d1/da4/lb0 0 2024-02-22T15:50:35.644 INFO:tasks.workunit.client.0.smithi019.stdout:7/618: readlink d9/d38/d41/d45/db0/lba 0 2024-02-22T15:50:35.645 INFO:tasks.workunit.client.0.smithi019.stdout:7/619: rmdir d9/d79 39 2024-02-22T15:50:35.645 INFO:tasks.workunit.client.1.smithi062.stdout:2/299: creat d1/d28/f56 x:0 0 0 2024-02-22T15:50:35.646 INFO:tasks.workunit.client.1.smithi062.stdout:0/297: rmdir d1/d3/de/d14 39 2024-02-22T15:50:35.647 INFO:tasks.workunit.client.0.smithi019.stdout:7/620: rmdir d9/d38/d41/d45/d61/d6a 39 2024-02-22T15:50:35.649 INFO:tasks.workunit.client.1.smithi062.stdout:1/230: mkdir d1/d6/d7/d2b/d49 0 2024-02-22T15:50:35.649 INFO:tasks.workunit.client.0.smithi019.stdout:7/621: getdents d9/d3a/d49/d5d 0 2024-02-22T15:50:35.650 INFO:tasks.workunit.client.1.smithi062.stdout:2/300: mknod d1/d28/d3f/c57 0 2024-02-22T15:50:35.651 INFO:tasks.workunit.client.1.smithi062.stdout:2/301: chown d1/d47/d39 384 1 2024-02-22T15:50:35.651 INFO:tasks.workunit.client.1.smithi062.stdout:6/322: dwrite d5/dd/f13 [0,4194304] 0 2024-02-22T15:50:35.652 INFO:tasks.workunit.client.0.smithi019.stdout:7/622: truncate d9/f16 623877 0 2024-02-22T15:50:35.653 INFO:tasks.workunit.client.1.smithi062.stdout:0/298: rename d1/d38/f4e to d1/d38/f67 0 2024-02-22T15:50:35.653 INFO:tasks.workunit.client.1.smithi062.stdout:6/323: write d5/d15/f6a [701625,122100] 0 2024-02-22T15:50:35.657 INFO:tasks.workunit.client.1.smithi062.stdout:6/324: write d5/dd/d33/f49 [3717102,102414] 0 2024-02-22T15:50:35.662 INFO:tasks.workunit.client.1.smithi062.stdout:6/325: chown d5/f1e 1 1 2024-02-22T15:50:35.663 INFO:tasks.workunit.client.1.smithi062.stdout:0/299: mkdir d1/d3/d4/d5c/d68 0 2024-02-22T15:50:35.664 INFO:tasks.workunit.client.1.smithi062.stdout:6/326: write d5/d15/d56/f5b [108874,45119] 0 2024-02-22T15:50:35.665 INFO:tasks.workunit.client.1.smithi062.stdout:0/300: chown d1/d13/f41 6980044 1 2024-02-22T15:50:35.666 INFO:tasks.workunit.client.1.smithi062.stdout:6/327: chown d5/df/d40/d44/d4a/l64 4 1 2024-02-22T15:50:35.667 INFO:tasks.workunit.client.0.smithi019.stdout:9/483: sync 2024-02-22T15:50:35.670 INFO:tasks.workunit.client.1.smithi062.stdout:0/301: write d1/d3/de/d14/f26 [741975,60417] 0 2024-02-22T15:50:35.670 INFO:tasks.workunit.client.1.smithi062.stdout:6/328: dread - d5/df/d40/d44/d4a/d41/f4d zero size 2024-02-22T15:50:35.674 INFO:tasks.workunit.client.1.smithi062.stdout:1/231: dwrite d1/d6/d10/f1c [0,4194304] 0 2024-02-22T15:50:35.674 INFO:tasks.workunit.client.1.smithi062.stdout:6/329: stat d5/d15/d56/l60 0 2024-02-22T15:50:35.676 INFO:tasks.workunit.client.1.smithi062.stdout:0/302: write d1/d3/de/d3f/f64 [151030,21063] 0 2024-02-22T15:50:35.676 INFO:tasks.workunit.client.1.smithi062.stdout:1/232: stat d1/ca 0 2024-02-22T15:50:35.679 INFO:tasks.workunit.client.1.smithi062.stdout:2/302: dwrite d1/d1e/d38/f54 [0,4194304] 0 2024-02-22T15:50:35.679 INFO:tasks.workunit.client.1.smithi062.stdout:0/303: dread - d1/d13/f41 zero size 2024-02-22T15:50:35.679 INFO:tasks.workunit.client.0.smithi019.stdout:9/484: dwrite d2/d1f/d2e/d57/d60/f75 [0,4194304] 0 2024-02-22T15:50:35.680 INFO:tasks.workunit.client.1.smithi062.stdout:6/330: truncate d5/df/d40/d44/d4a/f2d 4005376 0 2024-02-22T15:50:35.680 INFO:tasks.workunit.client.1.smithi062.stdout:1/233: chown d1/f30 1 1 2024-02-22T15:50:35.681 INFO:tasks.workunit.client.1.smithi062.stdout:0/304: write d1/dd/f63 [71634,84536] 0 2024-02-22T15:50:35.685 INFO:tasks.workunit.client.1.smithi062.stdout:2/303: mknod d1/d47/d32/d52/c58 0 2024-02-22T15:50:35.687 INFO:tasks.workunit.client.1.smithi062.stdout:6/331: mkdir d5/d6e 0 2024-02-22T15:50:35.688 INFO:tasks.workunit.client.1.smithi062.stdout:0/305: mknod d1/d3/de/d14/c69 0 2024-02-22T15:50:35.689 INFO:tasks.workunit.client.0.smithi019.stdout:9/485: symlink d2/d1f/d2e/lae 0 2024-02-22T15:50:35.695 INFO:tasks.workunit.client.1.smithi062.stdout:1/234: dwrite d1/d6/de/f42 [0,4194304] 0 2024-02-22T15:50:35.695 INFO:tasks.workunit.client.0.smithi019.stdout:7/623: dwrite d9/f25 [0,4194304] 0 2024-02-22T15:50:35.697 INFO:tasks.workunit.client.0.smithi019.stdout:7/624: dread - d9/d4c/d5b/f96 zero size 2024-02-22T15:50:35.697 INFO:tasks.workunit.client.1.smithi062.stdout:6/332: symlink d5/df/d40/d44/d4a/d41/d43/l6f 0 2024-02-22T15:50:35.698 INFO:tasks.workunit.client.0.smithi019.stdout:9/486: dwrite d2/d1f/f4b [0,4194304] 0 2024-02-22T15:50:35.699 INFO:tasks.workunit.client.1.smithi062.stdout:1/235: mkdir d1/d6/d7/d4a 0 2024-02-22T15:50:35.700 INFO:tasks.workunit.client.1.smithi062.stdout:2/304: link d1/d28/f53 d1/d28/d3f/f59 0 2024-02-22T15:50:35.701 INFO:tasks.workunit.client.1.smithi062.stdout:1/236: write d1/d6/de/f42 [1266323,14692] 0 2024-02-22T15:50:35.703 INFO:tasks.workunit.client.1.smithi062.stdout:6/333: getdents d5/df/d40/d44 0 2024-02-22T15:50:35.703 INFO:tasks.workunit.client.1.smithi062.stdout:2/305: truncate d1/d47/d39/f4b 962752 0 2024-02-22T15:50:35.708 INFO:tasks.workunit.client.1.smithi062.stdout:2/306: mkdir d1/d8/d5a 0 2024-02-22T15:50:35.708 INFO:tasks.workunit.client.1.smithi062.stdout:1/237: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:35.709 INFO:tasks.workunit.client.1.smithi062.stdout:2/307: fdatasync d1/d1e/f2d 0 2024-02-22T15:50:35.713 INFO:tasks.workunit.client.0.smithi019.stdout:7/625: dwrite d9/f1b [4194304,4194304] 0 2024-02-22T15:50:35.714 INFO:tasks.workunit.client.0.smithi019.stdout:9/487: dwrite d2/d1f/d2e/f42 [4194304,4194304] 0 2024-02-22T15:50:35.715 INFO:tasks.workunit.client.1.smithi062.stdout:1/238: dread d1/d6/d10/f1c [0,4194304] 0 2024-02-22T15:50:35.716 INFO:tasks.workunit.client.1.smithi062.stdout:1/239: fsync d1/f30 0 2024-02-22T15:50:35.716 INFO:tasks.workunit.client.1.smithi062.stdout:2/308: mknod d1/d1e/d38/d55/c5b 0 2024-02-22T15:50:35.719 INFO:tasks.workunit.client.0.smithi019.stdout:7/626: mknod d9/d3a/ccc 0 2024-02-22T15:50:35.719 INFO:tasks.workunit.client.0.smithi019.stdout:9/488: mkdir d2/d1f/d2e/d48/d45/d5a/daf 0 2024-02-22T15:50:35.719 INFO:tasks.workunit.client.0.smithi019.stdout:7/627: chown d9/d38/d41/d45/d61/fb1 7425 1 2024-02-22T15:50:35.720 INFO:tasks.workunit.client.1.smithi062.stdout:0/306: creat d1/d3/de/d14/f6a x:0 0 0 2024-02-22T15:50:35.720 INFO:tasks.workunit.client.0.smithi019.stdout:7/628: read d9/f25 [1449654,80560] 0 2024-02-22T15:50:35.721 INFO:tasks.workunit.client.1.smithi062.stdout:2/309: creat d1/d28/f5c x:0 0 0 2024-02-22T15:50:35.722 INFO:tasks.workunit.client.0.smithi019.stdout:9/489: creat d2/d1f/d95/fb0 x:0 0 0 2024-02-22T15:50:35.722 INFO:tasks.workunit.client.1.smithi062.stdout:1/240: creat d1/d6/d10/f4b x:0 0 0 2024-02-22T15:50:35.723 INFO:tasks.workunit.client.1.smithi062.stdout:1/241: readlink d1/d6/d36/d37/l39 0 2024-02-22T15:50:35.724 INFO:tasks.workunit.client.0.smithi019.stdout:7/629: symlink d9/d38/d41/d45/d61/lcd 0 2024-02-22T15:50:35.724 INFO:tasks.workunit.client.1.smithi062.stdout:0/307: creat d1/dd/f6b x:0 0 0 2024-02-22T15:50:35.724 INFO:tasks.workunit.client.0.smithi019.stdout:9/490: truncate d2/d7/d52/faa 1020835 0 2024-02-22T15:50:35.725 INFO:tasks.workunit.client.0.smithi019.stdout:7/630: mknod d9/d26/cce 0 2024-02-22T15:50:35.726 INFO:tasks.workunit.client.1.smithi062.stdout:0/308: fdatasync d1/d13/f2f 0 2024-02-22T15:50:35.727 INFO:tasks.workunit.client.1.smithi062.stdout:6/334: dread d5/d15/f36 [0,4194304] 0 2024-02-22T15:50:35.730 INFO:tasks.workunit.client.1.smithi062.stdout:0/309: creat d1/dd/f6c x:0 0 0 2024-02-22T15:50:35.731 INFO:tasks.workunit.client.1.smithi062.stdout:6/335: unlink d5/df/f10 0 2024-02-22T15:50:35.732 INFO:tasks.workunit.client.0.smithi019.stdout:9/491: dwrite d2/d1f/d2e/d57/d60/f75 [0,4194304] 0 2024-02-22T15:50:35.732 INFO:tasks.workunit.client.1.smithi062.stdout:0/310: truncate d1/d3/de/d14/f6a 84125 0 2024-02-22T15:50:35.734 INFO:tasks.workunit.client.1.smithi062.stdout:6/336: read d5/df/f30 [1779219,23276] 0 2024-02-22T15:50:35.736 INFO:tasks.workunit.client.1.smithi062.stdout:6/337: truncate d5/d15/d56/f5b 360687 0 2024-02-22T15:50:35.737 INFO:tasks.workunit.client.1.smithi062.stdout:0/311: dread d1/d13/f2f [0,4194304] 0 2024-02-22T15:50:35.740 INFO:tasks.workunit.client.1.smithi062.stdout:6/338: creat d5/df/d2a/f70 x:0 0 0 2024-02-22T15:50:35.740 INFO:tasks.workunit.client.1.smithi062.stdout:0/312: truncate d1/d13/d43/f53 515814 0 2024-02-22T15:50:35.747 INFO:tasks.workunit.client.1.smithi062.stdout:0/313: rename d1/d3/d4/d1a/d1d/c42 to d1/d3/d4/d1a/d1d/d3b/c6d 0 2024-02-22T15:50:35.747 INFO:tasks.workunit.client.1.smithi062.stdout:6/339: dwrite d5/dd/f20 [0,4194304] 0 2024-02-22T15:50:35.751 INFO:tasks.workunit.client.1.smithi062.stdout:6/340: mkdir d5/dd/d33/d71 0 2024-02-22T15:50:35.752 INFO:tasks.workunit.client.1.smithi062.stdout:0/314: mknod d1/d3/d4/d1a/c6e 0 2024-02-22T15:50:35.753 INFO:tasks.workunit.client.1.smithi062.stdout:6/341: truncate f2 320468 0 2024-02-22T15:50:35.754 INFO:tasks.workunit.client.1.smithi062.stdout:6/342: stat d5/f6 0 2024-02-22T15:50:35.755 INFO:tasks.workunit.client.1.smithi062.stdout:1/242: write d1/d6/de/f17 [188984,72701] 0 2024-02-22T15:50:35.756 INFO:tasks.workunit.client.1.smithi062.stdout:0/315: write d1/d3/de/d14/f26 [212932,124501] 0 2024-02-22T15:50:35.760 INFO:tasks.workunit.client.0.smithi019.stdout:9/492: dread d2/d7/dc/d27/d68/f6b [0,4194304] 0 2024-02-22T15:50:35.763 INFO:tasks.workunit.client.0.smithi019.stdout:9/493: dread d2/d1f/d2e/d48/d45/f61 [0,4194304] 0 2024-02-22T15:50:35.764 INFO:tasks.workunit.client.0.smithi019.stdout:9/494: read d2/d1f/d2e/f42 [2154342,85606] 0 2024-02-22T15:50:35.767 INFO:tasks.workunit.client.1.smithi062.stdout:2/310: dwrite d1/d28/f53 [0,4194304] 0 2024-02-22T15:50:35.767 INFO:tasks.workunit.client.0.smithi019.stdout:9/495: creat d2/d7/dc/d27/d87/fb1 x:0 0 0 2024-02-22T15:50:35.770 INFO:tasks.workunit.client.1.smithi062.stdout:6/343: dwrite d5/f1a [0,4194304] 0 2024-02-22T15:50:35.772 INFO:tasks.workunit.client.0.smithi019.stdout:9/496: dread d2/d1f/f4b [0,4194304] 0 2024-02-22T15:50:35.777 INFO:tasks.workunit.client.0.smithi019.stdout:9/497: getdents d2/d1f/d2e/d48/d45/d5a/daf 0 2024-02-22T15:50:35.778 INFO:tasks.workunit.client.1.smithi062.stdout:2/311: dwrite d1/d28/f5c [0,4194304] 0 2024-02-22T15:50:35.780 INFO:tasks.workunit.client.1.smithi062.stdout:0/316: dread d1/f12 [0,4194304] 0 2024-02-22T15:50:35.780 INFO:tasks.workunit.client.1.smithi062.stdout:6/344: link d5/df/d40/d44/d4a/f28 d5/df/d40/d44/d4a/d41/f72 0 2024-02-22T15:50:35.781 INFO:tasks.workunit.client.0.smithi019.stdout:9/498: fsync d2/d1f/d2e/d48/d36/fa4 0 2024-02-22T15:50:35.781 INFO:tasks.workunit.client.1.smithi062.stdout:6/345: chown d5/d15/d56 111 1 2024-02-22T15:50:35.782 INFO:tasks.workunit.client.0.smithi019.stdout:9/499: chown d2/d7/dc/d97 1472072 1 2024-02-22T15:50:35.783 INFO:tasks.workunit.client.1.smithi062.stdout:4/166: sync 2024-02-22T15:50:35.783 INFO:tasks.workunit.client.1.smithi062.stdout:9/203: sync 2024-02-22T15:50:35.783 INFO:tasks.workunit.client.1.smithi062.stdout:2/312: write d1/d1e/d38/f4f [2416769,42136] 0 2024-02-22T15:50:35.783 INFO:tasks.workunit.client.1.smithi062.stdout:6/346: chown d5/df/d2a 13311383 1 2024-02-22T15:50:35.784 INFO:tasks.workunit.client.0.smithi019.stdout:9/500: creat d2/d7/dc/d27/d31/d79/fb2 x:0 0 0 2024-02-22T15:50:35.785 INFO:tasks.workunit.client.1.smithi062.stdout:0/317: getdents d1/d3 0 2024-02-22T15:50:35.787 INFO:tasks.workunit.client.1.smithi062.stdout:4/167: truncate d4/d1c/d30/f32 460630 0 2024-02-22T15:50:35.788 INFO:tasks.workunit.client.0.smithi019.stdout:9/501: mknod d2/d1f/d2e/d48/d36/cb3 0 2024-02-22T15:50:35.789 INFO:tasks.workunit.client.1.smithi062.stdout:2/313: fdatasync d1/d47/f48 0 2024-02-22T15:50:35.790 INFO:tasks.workunit.client.1.smithi062.stdout:0/318: rename d1/d3/l20 to d1/d3/d5f/l6f 0 2024-02-22T15:50:35.790 INFO:tasks.workunit.client.1.smithi062.stdout:6/347: write d5/df/d2a/f3b [404558,112543] 0 2024-02-22T15:50:35.790 INFO:tasks.workunit.client.1.smithi062.stdout:4/168: mknod d4/d1c/c3c 0 2024-02-22T15:50:35.792 INFO:tasks.workunit.client.1.smithi062.stdout:9/204: read d4/d1c/f2c [199582,9338] 0 2024-02-22T15:50:35.795 INFO:tasks.workunit.client.1.smithi062.stdout:0/319: write d1/dd/f36 [1309364,107525] 0 2024-02-22T15:50:35.795 INFO:tasks.workunit.client.1.smithi062.stdout:4/169: unlink d4/df/f21 0 2024-02-22T15:50:35.799 INFO:tasks.workunit.client.1.smithi062.stdout:6/348: write d5/df/f21 [811693,30627] 0 2024-02-22T15:50:35.799 INFO:tasks.workunit.client.1.smithi062.stdout:4/170: chown d4/d1c/d30/d3b 2225 1 2024-02-22T15:50:35.799 INFO:tasks.workunit.client.1.smithi062.stdout:9/205: rmdir d4/d15/d20 39 2024-02-22T15:50:35.799 INFO:tasks.workunit.client.1.smithi062.stdout:6/349: stat d5/d15/d56 0 2024-02-22T15:50:35.802 INFO:tasks.workunit.client.1.smithi062.stdout:4/171: dread - d4/d2a/d2d/f35 zero size 2024-02-22T15:50:35.802 INFO:tasks.workunit.client.1.smithi062.stdout:2/314: dwrite d1/d47/f2b [4194304,4194304] 0 2024-02-22T15:50:35.804 INFO:tasks.workunit.client.1.smithi062.stdout:6/350: write d5/f1e [4521552,102319] 0 2024-02-22T15:50:35.810 INFO:tasks.workunit.client.1.smithi062.stdout:4/172: creat d4/d1c/f3d x:0 0 0 2024-02-22T15:50:35.810 INFO:tasks.workunit.client.1.smithi062.stdout:9/206: dread d4/f26 [0,4194304] 0 2024-02-22T15:50:35.811 INFO:tasks.workunit.client.1.smithi062.stdout:6/351: write d5/df/d40/d44/d4a/d41/f55 [449609,94309] 0 2024-02-22T15:50:35.811 INFO:tasks.workunit.client.1.smithi062.stdout:2/315: mkdir d1/d47/d39/d5d 0 2024-02-22T15:50:35.812 INFO:tasks.workunit.client.1.smithi062.stdout:2/316: write d1/f16 [566923,48370] 0 2024-02-22T15:50:35.813 INFO:tasks.workunit.client.1.smithi062.stdout:0/320: dread - d1/d3/d4/d1a/f56 zero size 2024-02-22T15:50:35.813 INFO:tasks.workunit.client.1.smithi062.stdout:6/352: stat d5/d15/d56/c5d 0 2024-02-22T15:50:35.814 INFO:tasks.workunit.client.1.smithi062.stdout:6/353: chown d5/fb 813 1 2024-02-22T15:50:35.815 INFO:tasks.workunit.client.1.smithi062.stdout:4/173: rename d4/d13/f15 to d4/d34/f3e 0 2024-02-22T15:50:35.815 INFO:tasks.workunit.client.1.smithi062.stdout:6/354: write d5/d15/d56/f5b [685790,60310] 0 2024-02-22T15:50:35.816 INFO:tasks.workunit.client.1.smithi062.stdout:6/355: stat d5/dd 0 2024-02-22T15:50:35.817 INFO:tasks.workunit.client.1.smithi062.stdout:9/207: unlink d4/d15/c33 0 2024-02-22T15:50:35.819 INFO:tasks.workunit.client.1.smithi062.stdout:2/317: creat d1/d1e/d38/d55/f5e x:0 0 0 2024-02-22T15:50:35.895 INFO:tasks.workunit.client.0.smithi019.stdout:3/368: sync 2024-02-22T15:50:35.895 INFO:tasks.workunit.client.0.smithi019.stdout:0/368: sync 2024-02-22T15:50:35.896 INFO:tasks.workunit.client.0.smithi019.stdout:0/369: chown d4/d5/d7/f72 2766575 1 2024-02-22T15:50:35.896 INFO:tasks.workunit.client.1.smithi062.stdout:0/321: write d1/d3/de/d14/f26 [728735,72228] 0 2024-02-22T15:50:35.896 INFO:tasks.workunit.client.1.smithi062.stdout:4/174: creat d4/f3f x:0 0 0 2024-02-22T15:50:35.896 INFO:tasks.workunit.client.1.smithi062.stdout:2/318: write d1/d8/f42 [18175,30883] 0 2024-02-22T15:50:35.896 INFO:tasks.workunit.client.1.smithi062.stdout:9/208: rmdir d4/d1c 39 2024-02-22T15:50:35.897 INFO:tasks.workunit.client.0.smithi019.stdout:3/369: write d9/d11/fd [466125,66416] 0 2024-02-22T15:50:35.897 INFO:tasks.workunit.client.0.smithi019.stdout:0/370: write d4/db/d1b/f2f [1276277,83889] 0 2024-02-22T15:50:35.899 INFO:tasks.workunit.client.0.smithi019.stdout:0/371: dread - d4/db/d1b/d63/f87 zero size 2024-02-22T15:50:35.900 INFO:tasks.workunit.client.0.smithi019.stdout:0/372: readlink d4/d5/dc/l66 0 2024-02-22T15:50:35.901 INFO:tasks.workunit.client.1.smithi062.stdout:0/322: creat d1/d3/d4/d5c/f70 x:0 0 0 2024-02-22T15:50:35.902 INFO:tasks.workunit.client.0.smithi019.stdout:0/373: write d4/db/d1b/f2f [2721,39634] 0 2024-02-22T15:50:35.902 INFO:tasks.workunit.client.1.smithi062.stdout:4/175: creat d4/d1c/d30/d3b/f40 x:0 0 0 2024-02-22T15:50:35.904 INFO:tasks.workunit.client.0.smithi019.stdout:0/374: write d4/d9/f13 [769423,105779] 0 2024-02-22T15:50:35.905 INFO:tasks.workunit.client.1.smithi062.stdout:9/209: write d4/d15/f17 [2326580,107191] 0 2024-02-22T15:50:35.905 INFO:tasks.workunit.client.0.smithi019.stdout:3/370: dwrite d9/fb [4194304,4194304] 0 2024-02-22T15:50:35.908 INFO:tasks.workunit.client.1.smithi062.stdout:0/323: mknod d1/d3/d4/d5c/d68/c71 0 2024-02-22T15:50:35.908 INFO:tasks.workunit.client.0.smithi019.stdout:0/375: creat d4/d23/d7d/f88 x:0 0 0 2024-02-22T15:50:35.908 INFO:tasks.workunit.client.1.smithi062.stdout:2/319: dwrite d1/d8/f42 [0,4194304] 0 2024-02-22T15:50:35.909 INFO:tasks.workunit.client.1.smithi062.stdout:9/210: write f2 [2798569,21918] 0 2024-02-22T15:50:35.909 INFO:tasks.workunit.client.0.smithi019.stdout:3/371: mkdir d9/d26/d76 0 2024-02-22T15:50:35.910 INFO:tasks.workunit.client.0.smithi019.stdout:3/372: truncate d9/f2f 655469 0 2024-02-22T15:50:35.914 INFO:tasks.workunit.client.0.smithi019.stdout:0/376: link d4/d5/d52/f58 d4/db/d1b/d40/f89 0 2024-02-22T15:50:35.915 INFO:tasks.workunit.client.0.smithi019.stdout:0/377: chown d4/d9/f13 7 1 2024-02-22T15:50:35.918 INFO:tasks.workunit.client.1.smithi062.stdout:9/211: dread d4/d15/f24 [0,4194304] 0 2024-02-22T15:50:35.919 INFO:tasks.workunit.client.0.smithi019.stdout:0/378: mknod d4/d32/c8a 0 2024-02-22T15:50:35.919 INFO:tasks.workunit.client.1.smithi062.stdout:4/176: unlink d4/d34/f3e 0 2024-02-22T15:50:35.920 INFO:tasks.workunit.client.0.smithi019.stdout:3/373: dwrite d9/d26/d3f/d59/f74 [0,4194304] 0 2024-02-22T15:50:35.922 INFO:tasks.workunit.client.1.smithi062.stdout:0/324: rename d1/d3/f6 to d1/d3/d4/d1a/f72 0 2024-02-22T15:50:35.924 INFO:tasks.workunit.client.0.smithi019.stdout:3/374: creat d9/d26/d76/f77 x:0 0 0 2024-02-22T15:50:35.927 INFO:tasks.workunit.client.0.smithi019.stdout:0/379: dwrite d4/db/f43 [0,4194304] 0 2024-02-22T15:50:35.928 INFO:tasks.workunit.client.1.smithi062.stdout:0/325: mknod d1/dd/c73 0 2024-02-22T15:50:35.929 INFO:tasks.workunit.client.0.smithi019.stdout:0/380: chown d4/d22/f5f 1094 1 2024-02-22T15:50:35.929 INFO:tasks.workunit.client.0.smithi019.stdout:3/375: rmdir d9/d22/d58/d5d 0 2024-02-22T15:50:35.930 INFO:tasks.workunit.client.1.smithi062.stdout:9/212: rename d4/d1c/f2b to d4/d15/d3c/f3e 0 2024-02-22T15:50:35.930 INFO:tasks.workunit.client.1.smithi062.stdout:4/177: dwrite d4/d13/d20/f25 [0,4194304] 0 2024-02-22T15:50:35.932 INFO:tasks.workunit.client.1.smithi062.stdout:9/213: chown d4/d28/f38 139 1 2024-02-22T15:50:35.934 INFO:tasks.workunit.client.1.smithi062.stdout:4/178: unlink d4/d1c/c2b 0 2024-02-22T15:50:35.938 INFO:tasks.workunit.client.1.smithi062.stdout:4/179: unlink d4/d1c/d30/d3b/f40 0 2024-02-22T15:50:35.939 INFO:tasks.workunit.client.0.smithi019.stdout:0/381: dwrite d4/d22/f36 [0,4194304] 0 2024-02-22T15:50:35.942 INFO:tasks.workunit.client.0.smithi019.stdout:0/382: rmdir d4/db 39 2024-02-22T15:50:35.942 INFO:tasks.workunit.client.1.smithi062.stdout:4/180: creat d4/d13/d20/f41 x:0 0 0 2024-02-22T15:50:35.943 INFO:tasks.workunit.client.1.smithi062.stdout:4/181: dread - d4/f39 zero size 2024-02-22T15:50:35.944 INFO:tasks.workunit.client.0.smithi019.stdout:0/383: mknod d4/d5/dc/c8b 0 2024-02-22T15:50:35.949 INFO:tasks.workunit.client.1.smithi062.stdout:7/247: sync 2024-02-22T15:50:35.949 INFO:tasks.workunit.client.1.smithi062.stdout:8/242: sync 2024-02-22T15:50:35.949 INFO:tasks.workunit.client.1.smithi062.stdout:9/214: dwrite d4/d15/d20/f39 [0,4194304] 0 2024-02-22T15:50:35.950 INFO:tasks.workunit.client.1.smithi062.stdout:4/182: dread d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:35.950 INFO:tasks.workunit.client.0.smithi019.stdout:0/384: dwrite d4/d5/f39 [0,4194304] 0 2024-02-22T15:50:35.952 INFO:tasks.workunit.client.0.smithi019.stdout:0/385: dread - d4/d22/d2c/f57 zero size 2024-02-22T15:50:35.956 INFO:tasks.workunit.client.1.smithi062.stdout:2/320: write d1/d8/df/f26 [657267,2506] 0 2024-02-22T15:50:35.957 INFO:tasks.workunit.client.0.smithi019.stdout:0/386: unlink d4/d22/d3d/c5e 0 2024-02-22T15:50:35.958 INFO:tasks.workunit.client.1.smithi062.stdout:0/326: truncate d1/d3/d4/d1a/f56 870337 0 2024-02-22T15:50:35.960 INFO:tasks.workunit.client.0.smithi019.stdout:0/387: mkdir d4/db/d1b/d40/d8c 0 2024-02-22T15:50:35.960 INFO:tasks.workunit.client.1.smithi062.stdout:8/243: mknod d8/d42/d43/c46 0 2024-02-22T15:50:35.962 INFO:tasks.workunit.client.1.smithi062.stdout:7/248: creat d0/d14/d19/d30/d37/f53 x:0 0 0 2024-02-22T15:50:35.962 INFO:tasks.workunit.client.0.smithi019.stdout:0/388: write d4/db/d1b/d40/d44/d75/f82 [585153,51815] 0 2024-02-22T15:50:35.962 INFO:tasks.workunit.client.1.smithi062.stdout:0/327: chown d1/d3/de/c2b 1242 1 2024-02-22T15:50:35.964 INFO:tasks.workunit.client.0.smithi019.stdout:3/376: dread d9/d26/f35 [0,4194304] 0 2024-02-22T15:50:35.965 INFO:tasks.workunit.client.0.smithi019.stdout:3/377: stat d9/d26/d2d/d53 0 2024-02-22T15:50:35.966 INFO:tasks.workunit.client.1.smithi062.stdout:2/321: rename d1/d28/f53 to d1/d1e/d38/f5f 0 2024-02-22T15:50:35.966 INFO:tasks.workunit.client.1.smithi062.stdout:4/183: link d4/d1c/l1d d4/d1c/d30/d37/l42 0 2024-02-22T15:50:35.969 INFO:tasks.workunit.client.1.smithi062.stdout:8/244: rename d8/d9/d2c/f2f to d8/d26/f47 0 2024-02-22T15:50:35.972 INFO:tasks.workunit.client.1.smithi062.stdout:8/245: chown l5 3 1 2024-02-22T15:50:35.975 INFO:tasks.workunit.client.1.smithi062.stdout:0/328: rename d1/d3/d4/d5c to d1/d3/d4/d1e/d74 0 2024-02-22T15:50:35.977 INFO:tasks.workunit.client.1.smithi062.stdout:4/184: dwrite d4/d13/d20/f25 [0,4194304] 0 2024-02-22T15:50:35.978 INFO:tasks.workunit.client.1.smithi062.stdout:2/322: getdents d1 0 2024-02-22T15:50:35.983 INFO:tasks.workunit.client.0.smithi019.stdout:0/389: dread d4/d22/f5f [4194304,4194304] 0 2024-02-22T15:50:35.985 INFO:tasks.workunit.client.1.smithi062.stdout:9/215: dread d4/d28/f38 [0,4194304] 0 2024-02-22T15:50:35.985 INFO:tasks.workunit.client.1.smithi062.stdout:2/323: dread d1/d28/f5c [0,4194304] 0 2024-02-22T15:50:35.986 INFO:tasks.workunit.client.0.smithi019.stdout:0/390: rename d4/d32/c7b to d4/db/d76/c8d 0 2024-02-22T15:50:35.988 INFO:tasks.workunit.client.0.smithi019.stdout:0/391: dread - d4/db/d1b/d63/f70 zero size 2024-02-22T15:50:35.988 INFO:tasks.workunit.client.0.smithi019.stdout:3/378: dread f4 [8388608,4194304] 0 2024-02-22T15:50:35.989 INFO:tasks.workunit.client.1.smithi062.stdout:4/185: rename d4/df/f26 to d4/d13/f43 0 2024-02-22T15:50:35.989 INFO:tasks.workunit.client.1.smithi062.stdout:0/329: dread d1/d13/d43/f53 [0,4194304] 0 2024-02-22T15:50:35.990 INFO:tasks.workunit.client.1.smithi062.stdout:0/330: chown d1/d3/de/d3f/d5d 135657 1 2024-02-22T15:50:35.991 INFO:tasks.workunit.client.0.smithi019.stdout:3/379: mknod d9/d26/d2d/c78 0 2024-02-22T15:50:35.992 INFO:tasks.workunit.client.1.smithi062.stdout:8/246: rename d8/d9/d3c to d8/d9/d48 0 2024-02-22T15:50:35.993 INFO:tasks.workunit.client.0.smithi019.stdout:3/380: truncate d9/d26/d32/f2c 613445 0 2024-02-22T15:50:35.994 INFO:tasks.workunit.client.1.smithi062.stdout:2/324: symlink d1/d47/d32/l60 0 2024-02-22T15:50:35.995 INFO:tasks.workunit.client.1.smithi062.stdout:4/186: mknod d4/d1c/d30/c44 0 2024-02-22T15:50:35.995 INFO:tasks.workunit.client.0.smithi019.stdout:0/392: dwrite d4/d22/d2c/f57 [0,4194304] 0 2024-02-22T15:50:35.996 INFO:tasks.workunit.client.0.smithi019.stdout:3/381: symlink d9/d22/d58/l79 0 2024-02-22T15:50:35.996 INFO:tasks.workunit.client.1.smithi062.stdout:9/216: unlink d4/ca 0 2024-02-22T15:50:35.997 INFO:tasks.workunit.client.1.smithi062.stdout:0/331: rmdir d1/d3/de/d14 39 2024-02-22T15:50:35.998 INFO:tasks.workunit.client.0.smithi019.stdout:3/382: read f3 [2198852,126415] 0 2024-02-22T15:50:36.000 INFO:tasks.workunit.client.1.smithi062.stdout:8/247: dwrite d8/d26/f34 [0,4194304] 0 2024-02-22T15:50:36.002 INFO:tasks.workunit.client.1.smithi062.stdout:4/187: creat d4/f45 x:0 0 0 2024-02-22T15:50:36.003 INFO:tasks.workunit.client.0.smithi019.stdout:3/383: creat d9/d22/d27/f7a x:0 0 0 2024-02-22T15:50:36.004 INFO:tasks.workunit.client.1.smithi062.stdout:8/248: mkdir d8/d9/d2c/d49 0 2024-02-22T15:50:36.004 INFO:tasks.workunit.client.1.smithi062.stdout:9/217: write d4/d35/f37 [209628,75912] 0 2024-02-22T15:50:36.004 INFO:tasks.workunit.client.0.smithi019.stdout:3/384: read d9/d26/f35 [3088755,59578] 0 2024-02-22T15:50:36.005 INFO:tasks.workunit.client.1.smithi062.stdout:0/332: symlink d1/d3/d4/l75 0 2024-02-22T15:50:36.006 INFO:tasks.workunit.client.1.smithi062.stdout:8/249: chown d8/c14 147973073 1 2024-02-22T15:50:36.007 INFO:tasks.workunit.client.1.smithi062.stdout:8/250: chown d8/d9/f2a 1490731 1 2024-02-22T15:50:36.007 INFO:tasks.workunit.client.0.smithi019.stdout:3/385: symlink d9/d22/d27/d6d/l7b 0 2024-02-22T15:50:36.008 INFO:tasks.workunit.client.0.smithi019.stdout:0/393: write d4/d23/f28 [2126116,29404] 0 2024-02-22T15:50:36.009 INFO:tasks.workunit.client.0.smithi019.stdout:3/386: dread - d9/d26/d3f/d59/d66/f69 zero size 2024-02-22T15:50:36.009 INFO:tasks.workunit.client.1.smithi062.stdout:4/188: mkdir d4/d2a/d2d/d46 0 2024-02-22T15:50:36.009 INFO:tasks.workunit.client.1.smithi062.stdout:4/189: write d4/f45 [585560,93566] 0 2024-02-22T15:50:36.012 INFO:tasks.workunit.client.1.smithi062.stdout:4/190: dread d4/d1c/d30/f32 [0,4194304] 0 2024-02-22T15:50:36.015 INFO:tasks.workunit.client.1.smithi062.stdout:8/251: dread - d8/d9/d44/f45 zero size 2024-02-22T15:50:36.016 INFO:tasks.workunit.client.0.smithi019.stdout:3/387: dwrite d9/d26/d2d/d39/f5f [0,4194304] 0 2024-02-22T15:50:36.022 INFO:tasks.workunit.client.1.smithi062.stdout:4/191: rename d4/df/c19 to d4/d2a/c47 0 2024-02-22T15:50:36.024 INFO:tasks.workunit.client.1.smithi062.stdout:4/192: creat d4/d34/f48 x:0 0 0 2024-02-22T15:50:36.026 INFO:tasks.workunit.client.1.smithi062.stdout:4/193: chown d4/df/c28 1355 1 2024-02-22T15:50:36.026 INFO:tasks.workunit.client.1.smithi062.stdout:2/325: dwrite d1/d47/d32/f44 [0,4194304] 0 2024-02-22T15:50:36.027 INFO:tasks.workunit.client.0.smithi019.stdout:3/388: write d9/d26/d2d/d39/f5f [4927119,87670] 0 2024-02-22T15:50:36.029 INFO:tasks.workunit.client.0.smithi019.stdout:3/389: creat d9/d22/d49/f7c x:0 0 0 2024-02-22T15:50:36.030 INFO:tasks.workunit.client.1.smithi062.stdout:2/326: creat d1/d8/df/f61 x:0 0 0 2024-02-22T15:50:36.031 INFO:tasks.workunit.client.0.smithi019.stdout:3/390: creat d9/d65/f7d x:0 0 0 2024-02-22T15:50:36.033 INFO:tasks.workunit.client.0.smithi019.stdout:3/391: write d9/d26/d32/f33 [8233105,91666] 0 2024-02-22T15:50:36.033 INFO:tasks.workunit.client.1.smithi062.stdout:2/327: rmdir d1/d47/d32/d52 39 2024-02-22T15:50:36.034 INFO:tasks.workunit.client.1.smithi062.stdout:9/218: write d4/f14 [3100500,3960] 0 2024-02-22T15:50:36.035 INFO:tasks.workunit.client.0.smithi019.stdout:3/392: symlink d9/d26/d32/l7e 0 2024-02-22T15:50:36.038 INFO:tasks.workunit.client.1.smithi062.stdout:2/328: rmdir d1/d8/d5a 0 2024-02-22T15:50:36.038 INFO:tasks.workunit.client.0.smithi019.stdout:3/393: creat d9/d26/d3f/d59/d66/f7f x:0 0 0 2024-02-22T15:50:36.040 INFO:tasks.workunit.client.0.smithi019.stdout:3/394: mkdir d9/d26/d2d/d39/d80 0 2024-02-22T15:50:36.041 INFO:tasks.workunit.client.1.smithi062.stdout:2/329: mkdir d1/d8/d62 0 2024-02-22T15:50:36.044 INFO:tasks.workunit.client.1.smithi062.stdout:0/333: dwrite d1/f45 [0,4194304] 0 2024-02-22T15:50:36.045 INFO:tasks.workunit.client.1.smithi062.stdout:8/252: dread d8/d9/f18 [0,4194304] 0 2024-02-22T15:50:36.046 INFO:tasks.workunit.client.1.smithi062.stdout:2/330: rmdir d1/d28/d3f 39 2024-02-22T15:50:36.052 INFO:tasks.workunit.client.1.smithi062.stdout:9/219: truncate d4/d28/f30 197965 0 2024-02-22T15:50:36.052 INFO:tasks.workunit.client.1.smithi062.stdout:0/334: mkdir d1/d3/d4/d1a/d76 0 2024-02-22T15:50:36.056 INFO:tasks.workunit.client.1.smithi062.stdout:4/194: dread d4/df/f14 [0,4194304] 0 2024-02-22T15:50:36.057 INFO:tasks.workunit.client.1.smithi062.stdout:9/220: unlink d4/d15/d31/f32 0 2024-02-22T15:50:36.058 INFO:tasks.workunit.client.1.smithi062.stdout:8/253: dwrite d8/d9/fe [0,4194304] 0 2024-02-22T15:50:36.060 INFO:tasks.workunit.client.1.smithi062.stdout:2/331: dwrite d1/d1e/f2d [0,4194304] 0 2024-02-22T15:50:36.061 INFO:tasks.workunit.client.1.smithi062.stdout:8/254: write d8/f3a [834940,53835] 0 2024-02-22T15:50:36.061 INFO:tasks.workunit.client.1.smithi062.stdout:0/335: mknod d1/d3/d4/d1a/d1d/c77 0 2024-02-22T15:50:36.061 INFO:tasks.workunit.client.1.smithi062.stdout:9/221: truncate d4/f29 344261 0 2024-02-22T15:50:36.062 INFO:tasks.workunit.client.1.smithi062.stdout:2/332: rmdir d1/d1e/d38 39 2024-02-22T15:50:36.062 INFO:tasks.workunit.client.1.smithi062.stdout:3/201: sync 2024-02-22T15:50:36.066 INFO:tasks.workunit.client.1.smithi062.stdout:2/333: write d1/f5 [5266734,37891] 0 2024-02-22T15:50:36.068 INFO:tasks.workunit.client.1.smithi062.stdout:4/195: getdents d4/d2a/d2d 0 2024-02-22T15:50:36.071 INFO:tasks.workunit.client.1.smithi062.stdout:4/196: chown d4/d1c/d30/d37 46932559 1 2024-02-22T15:50:36.075 INFO:tasks.workunit.client.1.smithi062.stdout:3/202: creat d5/d16/f44 x:0 0 0 2024-02-22T15:50:36.078 INFO:tasks.workunit.client.1.smithi062.stdout:0/336: unlink d1/d3/de/d14/f6a 0 2024-02-22T15:50:36.081 INFO:tasks.workunit.client.1.smithi062.stdout:2/334: creat d1/d47/d39/d5d/f63 x:0 0 0 2024-02-22T15:50:36.082 INFO:tasks.workunit.client.1.smithi062.stdout:8/255: dwrite d8/d9/f1b [0,4194304] 0 2024-02-22T15:50:36.084 INFO:tasks.workunit.client.1.smithi062.stdout:3/203: symlink d5/l45 0 2024-02-22T15:50:36.085 INFO:tasks.workunit.client.0.smithi019.stdout:1/531: sync 2024-02-22T15:50:36.086 INFO:tasks.workunit.client.0.smithi019.stdout:8/491: sync 2024-02-22T15:50:36.087 INFO:tasks.workunit.client.1.smithi062.stdout:4/197: getdents d4/d13/d20 0 2024-02-22T15:50:36.087 INFO:tasks.workunit.client.1.smithi062.stdout:2/335: write d1/d8/df/f26 [996901,33687] 0 2024-02-22T15:50:36.090 INFO:tasks.workunit.client.1.smithi062.stdout:2/336: chown d1 11195146 1 2024-02-22T15:50:36.090 INFO:tasks.workunit.client.1.smithi062.stdout:4/198: unlink d4/l1a 0 2024-02-22T15:50:36.090 INFO:tasks.workunit.client.0.smithi019.stdout:1/532: unlink d8/d15/d84/cb6 0 2024-02-22T15:50:36.091 INFO:tasks.workunit.client.1.smithi062.stdout:0/337: getdents d1/d3/d4/d1a/d1d/d3b 0 2024-02-22T15:50:36.093 INFO:tasks.workunit.client.1.smithi062.stdout:9/222: dread d4/f2f [0,4194304] 0 2024-02-22T15:50:36.094 INFO:tasks.workunit.client.0.smithi019.stdout:1/533: rename d8/d18/d64/d7f/fa2 to d8/d18/fb8 0 2024-02-22T15:50:36.095 INFO:tasks.workunit.client.1.smithi062.stdout:2/337: symlink d1/d47/d39/l64 0 2024-02-22T15:50:36.095 INFO:tasks.workunit.client.0.smithi019.stdout:1/534: dread - d8/d18/d64/fad zero size 2024-02-22T15:50:36.095 INFO:tasks.workunit.client.1.smithi062.stdout:8/256: dread d8/d26/f47 [0,4194304] 0 2024-02-22T15:50:36.096 INFO:tasks.workunit.client.0.smithi019.stdout:1/535: chown d8/d18/lb7 8 1 2024-02-22T15:50:36.098 INFO:tasks.workunit.client.0.smithi019.stdout:1/536: unlink d8/d2a/f3a 0 2024-02-22T15:50:36.099 INFO:tasks.workunit.client.1.smithi062.stdout:0/338: symlink d1/d3/de/d3f/d2c/l78 0 2024-02-22T15:50:36.100 INFO:tasks.workunit.client.0.smithi019.stdout:1/537: rename d8/d18/l4f to d8/d18/d72/lb9 0 2024-02-22T15:50:36.101 INFO:tasks.workunit.client.0.smithi019.stdout:1/538: read d8/d2a/f3f [772953,53602] 0 2024-02-22T15:50:36.102 INFO:tasks.workunit.client.1.smithi062.stdout:2/338: mkdir d1/d28/d3f/d65 0 2024-02-22T15:50:36.103 INFO:tasks.workunit.client.1.smithi062.stdout:9/223: write d4/f1b [4631507,104863] 0 2024-02-22T15:50:36.106 INFO:tasks.workunit.client.1.smithi062.stdout:0/339: truncate d1/dd/f6b 738964 0 2024-02-22T15:50:36.107 INFO:tasks.workunit.client.1.smithi062.stdout:3/204: dwrite d5/f10 [0,4194304] 0 2024-02-22T15:50:36.108 INFO:tasks.workunit.client.1.smithi062.stdout:9/224: stat d4/l11 0 2024-02-22T15:50:36.108 INFO:tasks.workunit.client.0.smithi019.stdout:1/539: dwrite d8/d15/f20 [4194304,4194304] 0 2024-02-22T15:50:36.109 INFO:tasks.workunit.client.1.smithi062.stdout:3/205: readlink d5/lb 0 2024-02-22T15:50:36.109 INFO:tasks.workunit.client.1.smithi062.stdout:4/199: dwrite d4/d34/f48 [0,4194304] 0 2024-02-22T15:50:36.112 INFO:tasks.workunit.client.1.smithi062.stdout:3/206: chown d5/c14 615 1 2024-02-22T15:50:36.114 INFO:tasks.workunit.client.1.smithi062.stdout:3/207: chown d5/d16/d21 22 1 2024-02-22T15:50:36.114 INFO:tasks.workunit.client.1.smithi062.stdout:8/257: dread d8/f22 [0,4194304] 0 2024-02-22T15:50:36.124 INFO:tasks.workunit.client.0.smithi019.stdout:1/540: truncate d8/d18/d85/f8d 2899087 0 2024-02-22T15:50:36.125 INFO:tasks.workunit.client.0.smithi019.stdout:1/541: chown d8/d23/l2f 62 1 2024-02-22T15:50:36.125 INFO:tasks.workunit.client.1.smithi062.stdout:0/340: chown d1/d3/d4/d1a/f34 5693 1 2024-02-22T15:50:36.126 INFO:tasks.workunit.client.1.smithi062.stdout:4/200: mkdir d4/d1c/d30/d3b/d49 0 2024-02-22T15:50:36.127 INFO:tasks.workunit.client.1.smithi062.stdout:8/258: dwrite d8/d9/d2c/f30 [0,4194304] 0 2024-02-22T15:50:36.127 INFO:tasks.workunit.client.1.smithi062.stdout:3/208: dwrite f3 [0,4194304] 0 2024-02-22T15:50:36.127 INFO:tasks.workunit.client.1.smithi062.stdout:2/339: rename d1/d8/df/f26 to d1/d1e/f66 0 2024-02-22T15:50:36.128 INFO:tasks.workunit.client.0.smithi019.stdout:1/542: read d8/d15/f20 [1382631,88097] 0 2024-02-22T15:50:36.129 INFO:tasks.workunit.client.0.smithi019.stdout:1/543: stat d8/d18/l28 0 2024-02-22T15:50:36.131 INFO:tasks.workunit.client.0.smithi019.stdout:1/544: creat d8/d15/fba x:0 0 0 2024-02-22T15:50:36.138 INFO:tasks.workunit.client.1.smithi062.stdout:8/259: chown d8/d26/f34 120578026 1 2024-02-22T15:50:36.139 INFO:tasks.workunit.client.1.smithi062.stdout:9/225: rename d4/d15/d20 to d4/d35/d3f 0 2024-02-22T15:50:36.139 INFO:tasks.workunit.client.1.smithi062.stdout:2/340: write d1/d47/f48 [305053,21194] 0 2024-02-22T15:50:36.140 INFO:tasks.workunit.client.0.smithi019.stdout:1/545: dwrite d8/d18/d64/fad [0,4194304] 0 2024-02-22T15:50:36.141 INFO:tasks.workunit.client.1.smithi062.stdout:3/209: write d5/d26/f35 [3556783,96449] 0 2024-02-22T15:50:36.142 INFO:tasks.workunit.client.1.smithi062.stdout:9/226: symlink d4/d15/d31/l40 0 2024-02-22T15:50:36.143 INFO:tasks.workunit.client.1.smithi062.stdout:8/260: fdatasync d8/d9/f37 0 2024-02-22T15:50:36.144 INFO:tasks.workunit.client.0.smithi019.stdout:1/546: symlink d8/d2a/d75/d81/d8e/lbb 0 2024-02-22T15:50:36.148 INFO:tasks.workunit.client.1.smithi062.stdout:8/261: mknod d8/d42/c4a 0 2024-02-22T15:50:36.148 INFO:tasks.workunit.client.0.smithi019.stdout:1/547: rename d8/d18/d64/d7f/d82 to d8/d15/d5b/dbc 0 2024-02-22T15:50:36.149 INFO:tasks.workunit.client.1.smithi062.stdout:9/227: read d4/d1c/f1e [176539,27683] 0 2024-02-22T15:50:36.150 INFO:tasks.workunit.client.0.smithi019.stdout:1/548: creat d8/d18/d85/fbd x:0 0 0 2024-02-22T15:50:36.151 INFO:tasks.workunit.client.1.smithi062.stdout:2/341: fdatasync d1/d1e/d38/f54 0 2024-02-22T15:50:36.152 INFO:tasks.workunit.client.1.smithi062.stdout:3/210: getdents d5/d16/d21/d36 0 2024-02-22T15:50:36.155 INFO:tasks.workunit.client.0.smithi019.stdout:1/549: link d8/f53 d8/d15/d34/d9e/fbe 0 2024-02-22T15:50:36.156 INFO:tasks.workunit.client.1.smithi062.stdout:9/228: symlink d4/l41 0 2024-02-22T15:50:36.159 INFO:tasks.workunit.client.0.smithi019.stdout:1/550: read d8/d18/f27 [3176184,57102] 0 2024-02-22T15:50:36.161 INFO:tasks.workunit.client.1.smithi062.stdout:8/262: getdents d8/d9 0 2024-02-22T15:50:36.162 INFO:tasks.workunit.client.0.smithi019.stdout:1/551: creat d8/d2a/fbf x:0 0 0 2024-02-22T15:50:36.164 INFO:tasks.workunit.client.0.smithi019.stdout:1/552: mknod d8/d18/d64/cc0 0 2024-02-22T15:50:36.164 INFO:tasks.workunit.client.1.smithi062.stdout:9/229: write d4/d35/d3f/f25 [153238,26666] 0 2024-02-22T15:50:36.165 INFO:tasks.workunit.client.1.smithi062.stdout:2/342: truncate d1/d28/f35 612114 0 2024-02-22T15:50:36.167 INFO:tasks.workunit.client.1.smithi062.stdout:8/263: truncate d8/d9/f1f 5022788 0 2024-02-22T15:50:36.167 INFO:tasks.workunit.client.1.smithi062.stdout:3/211: dwrite d5/d16/d21/f29 [0,4194304] 0 2024-02-22T15:50:36.169 INFO:tasks.workunit.client.1.smithi062.stdout:3/212: readlink d5/d16/l2c 0 2024-02-22T15:50:36.169 INFO:tasks.workunit.client.1.smithi062.stdout:4/201: dread d4/f11 [0,4194304] 0 2024-02-22T15:50:36.173 INFO:tasks.workunit.client.1.smithi062.stdout:4/202: dread - d4/d1c/f23 zero size 2024-02-22T15:50:36.175 INFO:tasks.workunit.client.1.smithi062.stdout:8/264: unlink d8/d9/f2d 0 2024-02-22T15:50:36.179 INFO:tasks.workunit.client.0.smithi019.stdout:1/553: dread d8/d23/f55 [0,4194304] 0 2024-02-22T15:50:36.179 INFO:tasks.workunit.client.1.smithi062.stdout:9/230: dwrite d4/d28/f38 [0,4194304] 0 2024-02-22T15:50:36.179 INFO:tasks.workunit.client.1.smithi062.stdout:4/203: mknod d4/d1c/d30/d3b/c4a 0 2024-02-22T15:50:36.179 INFO:tasks.workunit.client.1.smithi062.stdout:9/231: readlink d4/d15/d31/l40 0 2024-02-22T15:50:36.179 INFO:tasks.workunit.client.1.smithi062.stdout:2/343: link d1/d47/c33 d1/d8/df/c67 0 2024-02-22T15:50:36.180 INFO:tasks.workunit.client.1.smithi062.stdout:9/232: readlink d4/d15/l16 0 2024-02-22T15:50:36.181 INFO:tasks.workunit.client.0.smithi019.stdout:1/554: rename d8/d15/l1d to d8/d2a/d75/lc1 0 2024-02-22T15:50:36.185 INFO:tasks.workunit.client.1.smithi062.stdout:4/204: mkdir d4/d1c/d30/d3b/d49/d4b 0 2024-02-22T15:50:36.185 INFO:tasks.workunit.client.1.smithi062.stdout:9/233: dread d4/d35/d3f/f25 [0,4194304] 0 2024-02-22T15:50:36.186 INFO:tasks.workunit.client.1.smithi062.stdout:9/234: fdatasync d4/d35/d3f/f39 0 2024-02-22T15:50:36.188 INFO:tasks.workunit.client.1.smithi062.stdout:4/205: mknod d4/d1c/d2c/c4c 0 2024-02-22T15:50:36.189 INFO:tasks.workunit.client.1.smithi062.stdout:2/344: creat d1/d8/d62/f68 x:0 0 0 2024-02-22T15:50:36.190 INFO:tasks.workunit.client.1.smithi062.stdout:4/206: readlink d4/d13/d20/l24 0 2024-02-22T15:50:36.191 INFO:tasks.workunit.client.0.smithi019.stdout:1/555: write d8/d15/d34/d9e/fbe [270010,55672] 0 2024-02-22T15:50:36.194 INFO:tasks.workunit.client.1.smithi062.stdout:8/265: rmdir d8 39 2024-02-22T15:50:36.196 INFO:tasks.workunit.client.1.smithi062.stdout:2/345: mknod d1/d47/d32/c69 0 2024-02-22T15:50:36.196 INFO:tasks.workunit.client.1.smithi062.stdout:3/213: dwrite d5/d26/f31 [0,4194304] 0 2024-02-22T15:50:36.201 INFO:tasks.workunit.client.1.smithi062.stdout:2/346: rename d1/d8/df/c1f to d1/d8/d62/c6a 0 2024-02-22T15:50:36.207 INFO:tasks.workunit.client.1.smithi062.stdout:2/347: dread d1/d47/d39/f4b [0,4194304] 0 2024-02-22T15:50:36.208 INFO:tasks.workunit.client.1.smithi062.stdout:9/235: dread f2 [0,4194304] 0 2024-02-22T15:50:36.208 INFO:tasks.workunit.client.1.smithi062.stdout:8/266: readlink d8/d9/l1c 0 2024-02-22T15:50:36.210 INFO:tasks.workunit.client.1.smithi062.stdout:3/214: creat d5/f46 x:0 0 0 2024-02-22T15:50:36.210 INFO:tasks.workunit.client.1.smithi062.stdout:9/236: chown d4/d35/d3f/c34 2651372 1 2024-02-22T15:50:36.211 INFO:tasks.workunit.client.1.smithi062.stdout:4/207: getdents d4/d1c/d30 0 2024-02-22T15:50:36.211 INFO:tasks.workunit.client.1.smithi062.stdout:3/215: readlink d5/d16/l28 0 2024-02-22T15:50:36.211 INFO:tasks.workunit.client.1.smithi062.stdout:9/237: fdatasync d4/f14 0 2024-02-22T15:50:36.212 INFO:tasks.workunit.client.1.smithi062.stdout:2/348: symlink d1/l6b 0 2024-02-22T15:50:36.213 INFO:tasks.workunit.client.1.smithi062.stdout:4/208: write d4/f39 [99877,48864] 0 2024-02-22T15:50:36.213 INFO:tasks.workunit.client.1.smithi062.stdout:3/216: truncate d5/d26/f27 690238 0 2024-02-22T15:50:36.216 INFO:tasks.workunit.client.1.smithi062.stdout:8/267: write d8/d9/fe [3846070,1066] 0 2024-02-22T15:50:36.216 INFO:tasks.workunit.client.1.smithi062.stdout:4/209: stat d4/d13/l22 0 2024-02-22T15:50:36.217 INFO:tasks.workunit.client.0.smithi019.stdout:6/519: sync 2024-02-22T15:50:36.217 INFO:tasks.workunit.client.0.smithi019.stdout:4/477: sync 2024-02-22T15:50:36.218 INFO:tasks.workunit.client.0.smithi019.stdout:5/520: sync 2024-02-22T15:50:36.218 INFO:tasks.workunit.client.0.smithi019.stdout:2/474: sync 2024-02-22T15:50:36.218 INFO:tasks.workunit.client.1.smithi062.stdout:3/217: truncate d5/d16/d21/f34 1022159 0 2024-02-22T15:50:36.218 INFO:tasks.workunit.client.1.smithi062.stdout:9/238: link d4/d35/d3f/f25 d4/d28/f42 0 2024-02-22T15:50:36.219 INFO:tasks.workunit.client.0.smithi019.stdout:6/520: fdatasync d6/d3e/f42 0 2024-02-22T15:50:36.219 INFO:tasks.workunit.client.1.smithi062.stdout:3/218: fsync d5/d16/d21/f34 0 2024-02-22T15:50:36.220 INFO:tasks.workunit.client.0.smithi019.stdout:1/556: dread d8/d18/f47 [4194304,4194304] 0 2024-02-22T15:50:36.220 INFO:tasks.workunit.client.1.smithi062.stdout:4/210: creat d4/d2a/f4d x:0 0 0 2024-02-22T15:50:36.221 INFO:tasks.workunit.client.0.smithi019.stdout:4/478: fsync d2/db/d14/d23/d21/d35/f64 0 2024-02-22T15:50:36.222 INFO:tasks.workunit.client.1.smithi062.stdout:8/268: unlink d8/d9/l1c 0 2024-02-22T15:50:36.222 INFO:tasks.workunit.client.0.smithi019.stdout:5/521: mkdir d8/dd/d19/d38/d54/db0 0 2024-02-22T15:50:36.224 INFO:tasks.workunit.client.0.smithi019.stdout:1/557: unlink d8/d18/d64/l7c 0 2024-02-22T15:50:36.224 INFO:tasks.workunit.client.0.smithi019.stdout:4/479: read d2/d3f/d48/f5b [851809,22143] 0 2024-02-22T15:50:36.224 INFO:tasks.workunit.client.1.smithi062.stdout:4/211: dread d4/d34/f48 [0,4194304] 0 2024-02-22T15:50:36.225 INFO:tasks.workunit.client.1.smithi062.stdout:3/219: creat d5/d26/d2d/f47 x:0 0 0 2024-02-22T15:50:36.226 INFO:tasks.workunit.client.0.smithi019.stdout:1/558: write d8/f88 [912728,84006] 0 2024-02-22T15:50:36.226 INFO:tasks.workunit.client.0.smithi019.stdout:5/522: mkdir d8/db1 0 2024-02-22T15:50:36.229 INFO:tasks.workunit.client.0.smithi019.stdout:2/475: dwrite d1/d17/d21/f41 [0,4194304] 0 2024-02-22T15:50:36.229 INFO:tasks.workunit.client.1.smithi062.stdout:8/269: dread d8/d9/f27 [0,4194304] 0 2024-02-22T15:50:36.230 INFO:tasks.workunit.client.0.smithi019.stdout:4/480: write d2/db/d14/d23/d21/f9b [311980,125119] 0 2024-02-22T15:50:36.230 INFO:tasks.workunit.client.0.smithi019.stdout:1/559: chown d8/d15/f26 27079 1 2024-02-22T15:50:36.231 INFO:tasks.workunit.client.0.smithi019.stdout:5/523: read - d8/dd/d19/d38/d47/f98 zero size 2024-02-22T15:50:36.231 INFO:tasks.workunit.client.1.smithi062.stdout:3/220: unlink d5/d16/d21/f3a 0 2024-02-22T15:50:36.233 INFO:tasks.workunit.client.0.smithi019.stdout:2/476: write d1/d17/d21/d51/fa1 [42826,131032] 0 2024-02-22T15:50:36.237 INFO:tasks.workunit.client.0.smithi019.stdout:1/560: rename d8/d2a/d75/cae to d8/d2a/d75/cc2 0 2024-02-22T15:50:36.239 INFO:tasks.workunit.client.0.smithi019.stdout:7/631: sync 2024-02-22T15:50:36.239 INFO:tasks.workunit.client.0.smithi019.stdout:9/502: sync 2024-02-22T15:50:36.239 INFO:tasks.workunit.client.0.smithi019.stdout:8/492: sync 2024-02-22T15:50:36.239 INFO:tasks.workunit.client.0.smithi019.stdout:0/394: sync 2024-02-22T15:50:36.239 INFO:tasks.workunit.client.0.smithi019.stdout:3/395: sync 2024-02-22T15:50:36.240 INFO:tasks.workunit.client.0.smithi019.stdout:5/524: creat d8/dd/d19/d38/d54/fb2 x:0 0 0 2024-02-22T15:50:36.240 INFO:tasks.workunit.client.0.smithi019.stdout:2/477: truncate d1/d17/d21/d51/f7c 863029 0 2024-02-22T15:50:36.240 INFO:tasks.workunit.client.1.smithi062.stdout:8/270: symlink d8/d28/l4b 0 2024-02-22T15:50:36.240 INFO:tasks.workunit.client.1.smithi062.stdout:2/349: rmdir d1/d47/d32/d52 39 2024-02-22T15:50:36.242 INFO:tasks.workunit.client.0.smithi019.stdout:5/525: read d8/d11/f3b [672603,129751] 0 2024-02-22T15:50:36.245 INFO:tasks.workunit.client.1.smithi062.stdout:3/221: dwrite d5/d16/d21/f29 [0,4194304] 0 2024-02-22T15:50:36.246 INFO:tasks.workunit.client.1.smithi062.stdout:8/271: creat d8/d28/f4c x:0 0 0 2024-02-22T15:50:36.247 INFO:tasks.workunit.client.0.smithi019.stdout:4/481: dwrite d2/db/d92/f91 [0,4194304] 0 2024-02-22T15:50:36.247 INFO:tasks.workunit.client.1.smithi062.stdout:3/222: creat d5/d16/f48 x:0 0 0 2024-02-22T15:50:36.248 INFO:tasks.workunit.client.0.smithi019.stdout:2/478: fdatasync d1/d17/d21/f87 0 2024-02-22T15:50:36.248 INFO:tasks.workunit.client.0.smithi019.stdout:9/503: truncate d2/d7/f44 4257806 0 2024-02-22T15:50:36.248 INFO:tasks.workunit.client.1.smithi062.stdout:3/223: readlink d5/d16/l2c 0 2024-02-22T15:50:36.249 INFO:tasks.workunit.client.0.smithi019.stdout:0/395: write d4/db/f43 [1541195,15458] 0 2024-02-22T15:50:36.251 INFO:tasks.workunit.client.1.smithi062.stdout:8/272: mkdir d8/d9/d44/d4d 0 2024-02-22T15:50:36.251 INFO:tasks.workunit.client.0.smithi019.stdout:3/396: readlink d9/l68 0 2024-02-22T15:50:36.253 INFO:tasks.workunit.client.0.smithi019.stdout:7/632: mknod d9/d38/d41/d45/db0/ccf 0 2024-02-22T15:50:36.254 INFO:tasks.workunit.client.1.smithi062.stdout:3/224: mknod d5/d26/d2d/c49 0 2024-02-22T15:50:36.255 INFO:tasks.workunit.client.0.smithi019.stdout:7/633: read d9/f25 [4177368,2200] 0 2024-02-22T15:50:36.256 INFO:tasks.workunit.client.0.smithi019.stdout:2/479: write d1/d17/d1c/d66/f83 [1005485,104271] 0 2024-02-22T15:50:36.257 INFO:tasks.workunit.client.1.smithi062.stdout:8/273: rename d8/d42/d43 to d8/d28/d4e 0 2024-02-22T15:50:36.257 INFO:tasks.workunit.client.1.smithi062.stdout:4/212: dread d4/fd [0,4194304] 0 2024-02-22T15:50:36.258 INFO:tasks.workunit.client.0.smithi019.stdout:7/634: chown d9/d26/d77/f8e 80 1 2024-02-22T15:50:36.259 INFO:tasks.workunit.client.0.smithi019.stdout:0/396: read f1 [246063,65659] 0 2024-02-22T15:50:36.260 INFO:tasks.workunit.client.0.smithi019.stdout:4/482: mknod d2/db/d14/d23/d21/c9d 0 2024-02-22T15:50:36.260 INFO:tasks.workunit.client.1.smithi062.stdout:3/225: fdatasync d5/f15 0 2024-02-22T15:50:36.262 INFO:tasks.workunit.client.1.smithi062.stdout:4/213: symlink d4/df/l4e 0 2024-02-22T15:50:36.263 INFO:tasks.workunit.client.1.smithi062.stdout:3/226: getdents d5/d26/d2d/d3f 0 2024-02-22T15:50:36.264 INFO:tasks.workunit.client.1.smithi062.stdout:5/138: sync 2024-02-22T15:50:36.264 INFO:tasks.workunit.client.1.smithi062.stdout:1/243: sync 2024-02-22T15:50:36.264 INFO:tasks.workunit.client.0.smithi019.stdout:1/561: dread d8/d23/f55 [0,4194304] 0 2024-02-22T15:50:36.265 INFO:tasks.workunit.client.0.smithi019.stdout:9/504: creat d2/d7/dc/d27/fb4 x:0 0 0 2024-02-22T15:50:36.266 INFO:tasks.workunit.client.1.smithi062.stdout:4/214: rename d4/d1c/c1f to d4/d13/d20/c4f 0 2024-02-22T15:50:36.269 INFO:tasks.workunit.client.1.smithi062.stdout:4/215: write d4/f3f [434916,113034] 0 2024-02-22T15:50:36.269 INFO:tasks.workunit.client.1.smithi062.stdout:1/244: mknod d1/d6/d36/d37/c4c 0 2024-02-22T15:50:36.270 INFO:tasks.workunit.client.1.smithi062.stdout:3/227: dread d5/f10 [0,4194304] 0 2024-02-22T15:50:36.270 INFO:tasks.workunit.client.1.smithi062.stdout:1/245: stat d1/d6/d14/c3c 0 2024-02-22T15:50:36.273 INFO:tasks.workunit.client.1.smithi062.stdout:3/228: getdents d5/d26/d2d/d3f 0 2024-02-22T15:50:36.274 INFO:tasks.workunit.client.0.smithi019.stdout:6/521: dread d6/d4b/d6d/d7b/f60 [0,4194304] 0 2024-02-22T15:50:36.277 INFO:tasks.workunit.client.1.smithi062.stdout:3/229: write d5/d16/f48 [782282,119568] 0 2024-02-22T15:50:36.277 INFO:tasks.workunit.client.0.smithi019.stdout:9/505: mkdir d2/d1f/db5 0 2024-02-22T15:50:36.278 INFO:tasks.workunit.client.0.smithi019.stdout:2/480: dwrite d1/de/f4e [0,4194304] 0 2024-02-22T15:50:36.280 INFO:tasks.workunit.client.0.smithi019.stdout:0/397: dread d4/d9/f13 [0,4194304] 0 2024-02-22T15:50:36.282 INFO:tasks.workunit.client.1.smithi062.stdout:4/216: dwrite d4/fd [0,4194304] 0 2024-02-22T15:50:36.283 INFO:tasks.workunit.client.1.smithi062.stdout:3/230: write d5/d16/f19 [3450460,115080] 0 2024-02-22T15:50:36.283 INFO:tasks.workunit.client.1.smithi062.stdout:1/246: dwrite d1/d6/d10/f4b [0,4194304] 0 2024-02-22T15:50:36.287 INFO:tasks.workunit.client.1.smithi062.stdout:5/139: dread d0/d3/f14 [0,4194304] 0 2024-02-22T15:50:36.288 INFO:tasks.workunit.client.0.smithi019.stdout:1/562: dwrite d8/d2a/f78 [0,4194304] 0 2024-02-22T15:50:36.290 INFO:tasks.workunit.client.1.smithi062.stdout:4/217: fsync d4/d1c/f27 0 2024-02-22T15:50:36.290 INFO:tasks.workunit.client.0.smithi019.stdout:1/563: chown d8/d44/f9f 3609 1 2024-02-22T15:50:36.290 INFO:tasks.workunit.client.0.smithi019.stdout:7/635: truncate d9/f1b 570740 0 2024-02-22T15:50:36.292 INFO:tasks.workunit.client.1.smithi062.stdout:5/140: symlink d0/l28 0 2024-02-22T15:50:36.294 INFO:tasks.workunit.client.1.smithi062.stdout:1/247: mknod d1/d6/de/c4d 0 2024-02-22T15:50:36.297 INFO:tasks.workunit.client.1.smithi062.stdout:4/218: creat d4/d1c/d30/d3b/d49/f50 x:0 0 0 2024-02-22T15:50:36.298 INFO:tasks.workunit.client.1.smithi062.stdout:3/231: dwrite d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:36.298 INFO:tasks.workunit.client.0.smithi019.stdout:6/522: symlink d6/d4b/d6d/l9b 0 2024-02-22T15:50:36.299 INFO:tasks.workunit.client.1.smithi062.stdout:1/248: mknod d1/d6/d7/d2b/d47/d24/c4e 0 2024-02-22T15:50:36.300 INFO:tasks.workunit.client.1.smithi062.stdout:4/219: creat d4/f51 x:0 0 0 2024-02-22T15:50:36.302 INFO:tasks.workunit.client.0.smithi019.stdout:5/526: dread d8/dd/d19/d3a/d46/f7b [0,4194304] 0 2024-02-22T15:50:36.302 INFO:tasks.workunit.client.0.smithi019.stdout:4/483: dwrite d2/db/d14/d23/f65 [0,4194304] 0 2024-02-22T15:50:36.302 INFO:tasks.workunit.client.1.smithi062.stdout:3/232: getdents d5/d16/d21/d36 0 2024-02-22T15:50:36.303 INFO:tasks.workunit.client.1.smithi062.stdout:3/233: chown d5/d16/d21/d36/c37 62244 1 2024-02-22T15:50:36.303 INFO:tasks.workunit.client.1.smithi062.stdout:4/220: write d4/d1c/d30/d3b/d49/f50 [937890,50851] 0 2024-02-22T15:50:36.304 INFO:tasks.workunit.client.0.smithi019.stdout:2/481: creat d1/d17/fb1 x:0 0 0 2024-02-22T15:50:36.304 INFO:tasks.workunit.client.0.smithi019.stdout:1/564: rename d8/d23/d33/l56 to d8/d2a/d75/lc3 0 2024-02-22T15:50:36.307 INFO:tasks.workunit.client.1.smithi062.stdout:4/221: creat d4/d1c/d30/d3b/d49/d4b/f52 x:0 0 0 2024-02-22T15:50:36.308 INFO:tasks.workunit.client.0.smithi019.stdout:9/506: mkdir d2/d1f/d2e/d48/d45/d5a/daf/db6 0 2024-02-22T15:50:36.310 INFO:tasks.workunit.client.0.smithi019.stdout:0/398: creat d4/f8e x:0 0 0 2024-02-22T15:50:36.310 INFO:tasks.workunit.client.0.smithi019.stdout:7/636: creat d9/d38/d41/d45/d61/d6a/fd0 x:0 0 0 2024-02-22T15:50:36.311 INFO:tasks.workunit.client.0.smithi019.stdout:1/565: dread - d8/d23/faf zero size 2024-02-22T15:50:36.312 INFO:tasks.workunit.client.1.smithi062.stdout:3/234: dwrite d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:36.312 INFO:tasks.workunit.client.1.smithi062.stdout:4/222: dread d4/f3f [0,4194304] 0 2024-02-22T15:50:36.314 INFO:tasks.workunit.client.1.smithi062.stdout:3/235: stat d5/d16/d21/f2b 0 2024-02-22T15:50:36.314 INFO:tasks.workunit.client.1.smithi062.stdout:4/223: stat d4/d1c 0 2024-02-22T15:50:36.314 INFO:tasks.workunit.client.0.smithi019.stdout:0/399: stat d4 0 2024-02-22T15:50:36.315 INFO:tasks.workunit.client.0.smithi019.stdout:2/482: dread d1/d17/d4b/f4a [0,4194304] 0 2024-02-22T15:50:36.315 INFO:tasks.workunit.client.0.smithi019.stdout:4/484: write d2/db/d54/d8c/f96 [1247403,51318] 0 2024-02-22T15:50:36.315 INFO:tasks.workunit.client.0.smithi019.stdout:5/527: dread d8/dd/d19/d3a/f5e [0,4194304] 0 2024-02-22T15:50:36.317 INFO:tasks.workunit.client.0.smithi019.stdout:4/485: chown d2/f98 760661383 1 2024-02-22T15:50:36.317 INFO:tasks.workunit.client.1.smithi062.stdout:4/224: creat d4/f53 x:0 0 0 2024-02-22T15:50:36.320 INFO:tasks.workunit.client.1.smithi062.stdout:4/225: symlink d4/l54 0 2024-02-22T15:50:36.320 INFO:tasks.workunit.client.0.smithi019.stdout:2/483: readlink d1/d86/la7 0 2024-02-22T15:50:36.321 INFO:tasks.workunit.client.0.smithi019.stdout:1/566: creat d8/d44/d5d/fc4 x:0 0 0 2024-02-22T15:50:36.322 INFO:tasks.workunit.client.1.smithi062.stdout:3/236: mkdir d5/d16/d21/d36/d4a 0 2024-02-22T15:50:36.322 INFO:tasks.workunit.client.1.smithi062.stdout:4/226: truncate d4/d2a/d2d/f35 445960 0 2024-02-22T15:50:36.323 INFO:tasks.workunit.client.0.smithi019.stdout:7/637: unlink d9/d38/d41/d45/d61/lb3 0 2024-02-22T15:50:36.324 INFO:tasks.workunit.client.1.smithi062.stdout:3/237: truncate d5/d1a/f25 4556982 0 2024-02-22T15:50:36.325 INFO:tasks.workunit.client.1.smithi062.stdout:4/227: write d4/f11 [2033979,30341] 0 2024-02-22T15:50:36.326 INFO:tasks.workunit.client.1.smithi062.stdout:3/238: symlink d5/d16/d21/d36/l4b 0 2024-02-22T15:50:36.327 INFO:tasks.workunit.client.1.smithi062.stdout:4/228: readlink d4/d1c/d2c/l38 0 2024-02-22T15:50:36.328 INFO:tasks.workunit.client.0.smithi019.stdout:1/567: stat d8/d15/l3d 0 2024-02-22T15:50:36.329 INFO:tasks.workunit.client.1.smithi062.stdout:4/229: write d4/f11 [2424439,39319] 0 2024-02-22T15:50:36.329 INFO:tasks.workunit.client.0.smithi019.stdout:9/507: dwrite d2/d1f/d2e/d48/d45/d5a/f72 [0,4194304] 0 2024-02-22T15:50:36.330 INFO:tasks.workunit.client.0.smithi019.stdout:6/523: mknod d6/d31/c9c 0 2024-02-22T15:50:36.331 INFO:tasks.workunit.client.0.smithi019.stdout:7/638: symlink d9/d4c/d5b/ld1 0 2024-02-22T15:50:36.332 INFO:tasks.workunit.client.0.smithi019.stdout:0/400: rename d4/d5/l2b to d4/db/d24/l8f 0 2024-02-22T15:50:36.335 INFO:tasks.workunit.client.0.smithi019.stdout:6/524: dread - d6/d58/d6a/f8c zero size 2024-02-22T15:50:36.336 INFO:tasks.workunit.client.0.smithi019.stdout:0/401: write d4/f3e [4578934,92303] 0 2024-02-22T15:50:36.336 INFO:tasks.workunit.client.0.smithi019.stdout:4/486: mkdir d2/db/d14/d23/d9e 0 2024-02-22T15:50:36.337 INFO:tasks.workunit.client.0.smithi019.stdout:2/484: truncate d1/d17/d1c/f4f 264959 0 2024-02-22T15:50:36.338 INFO:tasks.workunit.client.1.smithi062.stdout:4/230: dwrite d4/d13/d20/f41 [0,4194304] 0 2024-02-22T15:50:36.342 INFO:tasks.workunit.client.0.smithi019.stdout:5/528: rmdir d8/dd/d19/d86 0 2024-02-22T15:50:36.343 INFO:tasks.workunit.client.0.smithi019.stdout:9/508: chown d2/d1f/d2e/d48/l46 0 1 2024-02-22T15:50:36.343 INFO:tasks.workunit.client.0.smithi019.stdout:2/485: write d1/f1f [3563904,8799] 0 2024-02-22T15:50:36.343 INFO:tasks.workunit.client.0.smithi019.stdout:1/568: dwrite d8/d15/fba [0,4194304] 0 2024-02-22T15:50:36.345 INFO:tasks.workunit.client.0.smithi019.stdout:0/402: unlink d4/d23/f28 0 2024-02-22T15:50:36.353 INFO:tasks.workunit.client.0.smithi019.stdout:7/639: symlink d9/d4c/dc9/ld2 0 2024-02-22T15:50:36.353 INFO:tasks.workunit.client.0.smithi019.stdout:4/487: read d2/db/d14/d23/f36 [2548858,97069] 0 2024-02-22T15:50:36.353 INFO:tasks.workunit.client.0.smithi019.stdout:6/525: symlink d6/d31/d43/d55/d67/d8a/l9d 0 2024-02-22T15:50:36.354 INFO:tasks.workunit.client.0.smithi019.stdout:2/486: unlink d1/d17/d1c/f22 0 2024-02-22T15:50:36.357 INFO:tasks.workunit.client.0.smithi019.stdout:1/569: symlink d8/d23/lc5 0 2024-02-22T15:50:36.357 INFO:tasks.workunit.client.0.smithi019.stdout:9/509: creat d2/d7/dc/fb7 x:0 0 0 2024-02-22T15:50:36.358 INFO:tasks.workunit.client.0.smithi019.stdout:4/488: unlink d2/f98 0 2024-02-22T15:50:36.360 INFO:tasks.workunit.client.1.smithi062.stdout:4/231: dwrite d4/d1c/d30/d3b/d49/f50 [0,4194304] 0 2024-02-22T15:50:36.360 INFO:tasks.workunit.client.0.smithi019.stdout:6/526: symlink d6/d31/d69/l9e 0 2024-02-22T15:50:36.363 INFO:tasks.workunit.client.0.smithi019.stdout:2/487: unlink d1/de/f31 0 2024-02-22T15:50:36.366 INFO:tasks.workunit.client.0.smithi019.stdout:1/570: write d8/d23/f55 [51691,21333] 0 2024-02-22T15:50:36.367 INFO:tasks.workunit.client.1.smithi062.stdout:4/232: mknod d4/d34/c55 0 2024-02-22T15:50:36.368 INFO:tasks.workunit.client.1.smithi062.stdout:1/249: rmdir d1/d6/d7 39 2024-02-22T15:50:36.369 INFO:tasks.workunit.client.1.smithi062.stdout:3/239: truncate d5/f1d 3487884 0 2024-02-22T15:50:36.369 INFO:tasks.workunit.client.0.smithi019.stdout:4/489: mknod d2/db/d14/d23/d4b/d8e/c9f 0 2024-02-22T15:50:36.370 INFO:tasks.workunit.client.1.smithi062.stdout:4/233: dread d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:36.370 INFO:tasks.workunit.client.1.smithi062.stdout:1/250: unlink d1/d6/d10/l1b 0 2024-02-22T15:50:36.371 INFO:tasks.workunit.client.1.smithi062.stdout:3/240: write d5/d16/f19 [2849463,122238] 0 2024-02-22T15:50:36.372 INFO:tasks.workunit.client.1.smithi062.stdout:3/241: chown d5/l7 1172 1 2024-02-22T15:50:36.373 INFO:tasks.workunit.client.1.smithi062.stdout:1/251: chown d1/d6/d7/f29 323961940 1 2024-02-22T15:50:36.373 INFO:tasks.workunit.client.1.smithi062.stdout:3/242: chown d5/cf 42 1 2024-02-22T15:50:36.373 INFO:tasks.workunit.client.0.smithi019.stdout:2/488: dread d1/d17/f90 [0,4194304] 0 2024-02-22T15:50:36.374 INFO:tasks.workunit.client.1.smithi062.stdout:3/243: stat d5/d26/f27 0 2024-02-22T15:50:36.374 INFO:tasks.workunit.client.1.smithi062.stdout:4/234: creat d4/d1c/d2c/f56 x:0 0 0 2024-02-22T15:50:36.374 INFO:tasks.workunit.client.0.smithi019.stdout:9/510: symlink d2/d1f/d2e/d57/d60/d8c/d66/d9d/lb8 0 2024-02-22T15:50:36.376 INFO:tasks.workunit.client.0.smithi019.stdout:9/511: write d2/f6 [7320115,49738] 0 2024-02-22T15:50:36.376 INFO:tasks.workunit.client.1.smithi062.stdout:5/141: dread d0/f9 [0,4194304] 0 2024-02-22T15:50:36.377 INFO:tasks.workunit.client.1.smithi062.stdout:1/252: unlink d1/d6/d7/d2b/d47/d24/c4e 0 2024-02-22T15:50:36.378 INFO:tasks.workunit.client.0.smithi019.stdout:6/527: symlink d6/l9f 0 2024-02-22T15:50:36.378 INFO:tasks.workunit.client.1.smithi062.stdout:3/244: creat d5/d16/d21/d32/f4c x:0 0 0 2024-02-22T15:50:36.379 INFO:tasks.workunit.client.0.smithi019.stdout:4/490: symlink d2/db/d14/d23/d33/la0 0 2024-02-22T15:50:36.380 INFO:tasks.workunit.client.1.smithi062.stdout:4/235: read d4/d13/f43 [210620,105740] 0 2024-02-22T15:50:36.388 INFO:tasks.workunit.client.1.smithi062.stdout:1/253: dwrite d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:36.389 INFO:tasks.workunit.client.0.smithi019.stdout:7/640: rename d9/d4c/cb6 to d9/d38/cd3 0 2024-02-22T15:50:36.389 INFO:tasks.workunit.client.1.smithi062.stdout:4/236: rmdir d4/d34 39 2024-02-22T15:50:36.389 INFO:tasks.workunit.client.0.smithi019.stdout:4/491: stat d2/db/d14/d23/d4b/d8e/c9f 0 2024-02-22T15:50:36.390 INFO:tasks.workunit.client.1.smithi062.stdout:3/245: rmdir d5/d16/d42 0 2024-02-22T15:50:36.391 INFO:tasks.workunit.client.0.smithi019.stdout:9/512: unlink d2/d7/dc/d27/d31/f96 0 2024-02-22T15:50:36.392 INFO:tasks.workunit.client.1.smithi062.stdout:3/246: chown d5/d16/l28 1 1 2024-02-22T15:50:36.392 INFO:tasks.workunit.client.1.smithi062.stdout:1/254: write d1/f2e [3445789,129270] 0 2024-02-22T15:50:36.393 INFO:tasks.workunit.client.0.smithi019.stdout:6/528: mknod d6/d31/d43/d55/d67/ca0 0 2024-02-22T15:50:36.396 INFO:tasks.workunit.client.1.smithi062.stdout:1/255: readlink d1/d6/de/d9/l41 0 2024-02-22T15:50:36.396 INFO:tasks.workunit.client.1.smithi062.stdout:5/142: truncate d0/f7 309067 0 2024-02-22T15:50:36.397 INFO:tasks.workunit.client.0.smithi019.stdout:5/529: write f5 [7123558,81754] 0 2024-02-22T15:50:36.398 INFO:tasks.workunit.client.1.smithi062.stdout:1/256: chown d1/d6/de/c4d 0 1 2024-02-22T15:50:36.403 INFO:tasks.workunit.client.1.smithi062.stdout:1/257: write d1/d6/d7/f3a [1520675,116288] 0 2024-02-22T15:50:36.403 INFO:tasks.workunit.client.1.smithi062.stdout:4/237: dwrite d4/d13/d20/f41 [4194304,4194304] 0 2024-02-22T15:50:36.405 INFO:tasks.workunit.client.1.smithi062.stdout:1/258: readlink d1/d6/l23 0 2024-02-22T15:50:36.407 INFO:tasks.workunit.client.1.smithi062.stdout:4/238: mknod d4/d1c/d2c/c57 0 2024-02-22T15:50:36.409 INFO:tasks.workunit.client.1.smithi062.stdout:3/247: dread d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:36.411 INFO:tasks.workunit.client.1.smithi062.stdout:3/248: readlink d5/l45 0 2024-02-22T15:50:36.412 INFO:tasks.workunit.client.1.smithi062.stdout:4/239: mknod d4/d2a/c58 0 2024-02-22T15:50:36.412 INFO:tasks.workunit.client.1.smithi062.stdout:1/259: symlink d1/d6/d7/l4f 0 2024-02-22T15:50:36.413 INFO:tasks.workunit.client.1.smithi062.stdout:5/143: dwrite d0/d3/da/d13/d23/f24 [0,4194304] 0 2024-02-22T15:50:36.413 INFO:tasks.workunit.client.1.smithi062.stdout:3/249: rename d5 to d5/d26/d2d/d3f/d4d 22 2024-02-22T15:50:36.417 INFO:tasks.workunit.client.1.smithi062.stdout:4/240: unlink d4/d1c/d30/d3b/d49/d4b/f52 0 2024-02-22T15:50:36.417 INFO:tasks.workunit.client.1.smithi062.stdout:3/250: dread d5/d16/d21/f34 [0,4194304] 0 2024-02-22T15:50:36.421 INFO:tasks.workunit.client.1.smithi062.stdout:1/260: link d1/d6/d10/f4b d1/d6/d7/f50 0 2024-02-22T15:50:36.421 INFO:tasks.workunit.client.1.smithi062.stdout:1/261: chown d1/l1e 1 1 2024-02-22T15:50:36.423 INFO:tasks.workunit.client.1.smithi062.stdout:5/144: link d0/l25 d0/d3/da/d13/l29 0 2024-02-22T15:50:36.429 INFO:tasks.workunit.client.1.smithi062.stdout:3/251: getdents d5/d16 0 2024-02-22T15:50:36.430 INFO:tasks.workunit.client.1.smithi062.stdout:4/241: getdents d4/d34 0 2024-02-22T15:50:36.432 INFO:tasks.workunit.client.1.smithi062.stdout:1/262: getdents d1/d6/d7/d2b/d47 0 2024-02-22T15:50:36.432 INFO:tasks.workunit.client.1.smithi062.stdout:4/242: read - d4/d1c/f23 zero size 2024-02-22T15:50:36.433 INFO:tasks.workunit.client.0.smithi019.stdout:7/641: creat d9/d38/d41/d45/d92/fd4 x:0 0 0 2024-02-22T15:50:36.434 INFO:tasks.workunit.client.1.smithi062.stdout:5/145: dwrite d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:36.434 INFO:tasks.workunit.client.0.smithi019.stdout:1/571: dwrite d8/d15/f24 [0,4194304] 0 2024-02-22T15:50:36.435 INFO:tasks.workunit.client.1.smithi062.stdout:4/243: read d4/d34/f48 [3622294,65543] 0 2024-02-22T15:50:36.436 INFO:tasks.workunit.client.1.smithi062.stdout:3/252: rename d5/f15 to d5/d16/d21/f4e 0 2024-02-22T15:50:36.437 INFO:tasks.workunit.client.0.smithi019.stdout:5/530: unlink d8/dd/d8e/f85 0 2024-02-22T15:50:36.437 INFO:tasks.workunit.client.1.smithi062.stdout:5/146: fdatasync d0/d3/da/fb 0 2024-02-22T15:50:36.438 INFO:tasks.workunit.client.1.smithi062.stdout:4/244: chown d4/df/l1e 18 1 2024-02-22T15:50:36.438 INFO:tasks.workunit.client.1.smithi062.stdout:1/263: creat d1/d6/d7/d2b/d49/f51 x:0 0 0 2024-02-22T15:50:36.440 INFO:tasks.workunit.client.1.smithi062.stdout:3/253: fdatasync d5/d1a/f1e 0 2024-02-22T15:50:36.440 INFO:tasks.workunit.client.1.smithi062.stdout:4/245: write d4/d1c/d2c/f56 [1001766,10117] 0 2024-02-22T15:50:36.442 INFO:tasks.workunit.client.0.smithi019.stdout:4/492: link d2/db/d14/d23/d21/f29 d2/db/d14/d23/d4b/fa1 0 2024-02-22T15:50:36.442 INFO:tasks.workunit.client.0.smithi019.stdout:9/513: symlink d2/d1f/d2e/d48/d45/d5a/daf/db6/lb9 0 2024-02-22T15:50:36.445 INFO:tasks.workunit.client.0.smithi019.stdout:5/531: chown d8/c20 2088533 1 2024-02-22T15:50:36.446 INFO:tasks.workunit.client.0.smithi019.stdout:5/532: write d8/dd/d8e/d2c/f31 [518124,93512] 0 2024-02-22T15:50:36.447 INFO:tasks.workunit.client.1.smithi062.stdout:3/254: dread f3 [0,4194304] 0 2024-02-22T15:50:36.447 INFO:tasks.workunit.client.0.smithi019.stdout:5/533: fdatasync d8/f9b 0 2024-02-22T15:50:36.452 INFO:tasks.workunit.client.1.smithi062.stdout:1/264: unlink d1/d6/d14/c2c 0 2024-02-22T15:50:36.453 INFO:tasks.workunit.client.0.smithi019.stdout:7/642: mknod d9/d79/cd5 0 2024-02-22T15:50:36.453 INFO:tasks.workunit.client.1.smithi062.stdout:1/265: readlink d1/d6/l23 0 2024-02-22T15:50:36.454 INFO:tasks.workunit.client.1.smithi062.stdout:4/246: creat d4/f59 x:0 0 0 2024-02-22T15:50:36.455 INFO:tasks.workunit.client.0.smithi019.stdout:6/529: getdents d6/d4b/d6d/d7b/d7f 0 2024-02-22T15:50:36.455 INFO:tasks.workunit.client.1.smithi062.stdout:5/147: getdents d0/d3/da/d13 0 2024-02-22T15:50:36.457 INFO:tasks.workunit.client.0.smithi019.stdout:5/534: dwrite d8/dd/d19/d3a/fab [0,4194304] 0 2024-02-22T15:50:36.457 INFO:tasks.workunit.client.0.smithi019.stdout:4/493: link d2/db/d14/f87 d2/db/d44/fa2 0 2024-02-22T15:50:36.458 INFO:tasks.workunit.client.1.smithi062.stdout:3/255: dwrite d5/d16/d21/d32/f4c [0,4194304] 0 2024-02-22T15:50:36.458 INFO:tasks.workunit.client.1.smithi062.stdout:4/247: write d4/f3f [1088351,43938] 0 2024-02-22T15:50:36.461 INFO:tasks.workunit.client.1.smithi062.stdout:4/248: write d4/d1c/f27 [957653,87364] 0 2024-02-22T15:50:36.461 INFO:tasks.workunit.client.1.smithi062.stdout:1/266: rename d1/d6/de/d9/f25 to d1/d6/d7/d2b/d47/f52 0 2024-02-22T15:50:36.464 INFO:tasks.workunit.client.0.smithi019.stdout:5/535: mkdir d8/dd/d8e/db3 0 2024-02-22T15:50:36.466 INFO:tasks.workunit.client.1.smithi062.stdout:4/249: write d4/df/f14 [3349090,15664] 0 2024-02-22T15:50:36.467 INFO:tasks.workunit.client.1.smithi062.stdout:3/256: creat d5/d16/d21/d36/d4a/f4f x:0 0 0 2024-02-22T15:50:36.468 INFO:tasks.workunit.client.1.smithi062.stdout:4/250: chown d4/d1c/d30/c44 6801 1 2024-02-22T15:50:36.470 INFO:tasks.workunit.client.1.smithi062.stdout:6/356: sync 2024-02-22T15:50:36.470 INFO:tasks.workunit.client.1.smithi062.stdout:7/249: sync 2024-02-22T15:50:36.472 INFO:tasks.workunit.client.0.smithi019.stdout:5/536: mknod d8/dd/d19/d38/cb4 0 2024-02-22T15:50:36.472 INFO:tasks.workunit.client.0.smithi019.stdout:4/494: creat d2/db/fa3 x:0 0 0 2024-02-22T15:50:36.474 INFO:tasks.workunit.client.0.smithi019.stdout:6/530: getdents d6/d31/d43/d55 0 2024-02-22T15:50:36.474 INFO:tasks.workunit.client.1.smithi062.stdout:4/251: symlink d4/l5a 0 2024-02-22T15:50:36.474 INFO:tasks.workunit.client.0.smithi019.stdout:4/495: write d2/db/d14/f18 [3772774,47331] 0 2024-02-22T15:50:36.475 INFO:tasks.workunit.client.1.smithi062.stdout:1/267: link d1/d6/d7/d2b/c2d d1/d6/d7/c53 0 2024-02-22T15:50:36.475 INFO:tasks.workunit.client.0.smithi019.stdout:5/537: chown d8/dd/c41 115402 1 2024-02-22T15:50:36.476 INFO:tasks.workunit.client.0.smithi019.stdout:5/538: fdatasync d8/dd/d8e/d2c/f31 0 2024-02-22T15:50:36.476 INFO:tasks.workunit.client.1.smithi062.stdout:5/148: truncate d0/d3/ff 1990805 0 2024-02-22T15:50:36.477 INFO:tasks.workunit.client.0.smithi019.stdout:6/531: unlink d6/d4b/d6d/d7b/l6c 0 2024-02-22T15:50:36.479 INFO:tasks.workunit.client.1.smithi062.stdout:3/257: dwrite d5/d16/f48 [0,4194304] 0 2024-02-22T15:50:36.480 INFO:tasks.workunit.client.1.smithi062.stdout:6/357: creat d5/f73 x:0 0 0 2024-02-22T15:50:36.480 INFO:tasks.workunit.client.0.smithi019.stdout:9/514: write d2/d1f/d2e/f42 [9397645,24405] 0 2024-02-22T15:50:36.481 INFO:tasks.workunit.client.0.smithi019.stdout:1/572: truncate d8/d15/f1b 6346295 0 2024-02-22T15:50:36.483 INFO:tasks.workunit.client.1.smithi062.stdout:1/268: creat d1/d6/d7/f54 x:0 0 0 2024-02-22T15:50:36.484 INFO:tasks.workunit.client.0.smithi019.stdout:4/496: link d2/db/d44/f55 d2/db/d54/d8c/fa4 0 2024-02-22T15:50:36.484 INFO:tasks.workunit.client.0.smithi019.stdout:7/643: dread d9/d26/f2b [0,4194304] 0 2024-02-22T15:50:36.487 INFO:tasks.workunit.client.0.smithi019.stdout:9/515: creat d2/d1f/d2e/d57/d60/d8c/d66/fba x:0 0 0 2024-02-22T15:50:36.489 INFO:tasks.workunit.client.0.smithi019.stdout:9/516: chown d2/d7/dc/d27/d31 145 1 2024-02-22T15:50:36.489 INFO:tasks.workunit.client.1.smithi062.stdout:1/269: rmdir d1/d6/de/d9 39 2024-02-22T15:50:36.491 INFO:tasks.workunit.client.0.smithi019.stdout:7/644: mkdir d9/d3a/d49/d5d/dd6 0 2024-02-22T15:50:36.491 INFO:tasks.workunit.client.0.smithi019.stdout:9/517: write d2/d1f/d2e/f42 [8280691,48616] 0 2024-02-22T15:50:36.491 INFO:tasks.workunit.client.0.smithi019.stdout:4/497: truncate d2/db/d14/d23/d67/d8f/f90 98120 0 2024-02-22T15:50:36.492 INFO:tasks.workunit.client.0.smithi019.stdout:5/539: getdents d8 0 2024-02-22T15:50:36.493 INFO:tasks.workunit.client.1.smithi062.stdout:5/149: dread d0/d3/da/f18 [0,4194304] 0 2024-02-22T15:50:36.493 INFO:tasks.workunit.client.0.smithi019.stdout:6/532: getdents d6/d3e 0 2024-02-22T15:50:36.496 INFO:tasks.workunit.client.0.smithi019.stdout:4/498: chown d2/db/d54/c88 49276787 1 2024-02-22T15:50:36.497 INFO:tasks.workunit.client.0.smithi019.stdout:5/540: mknod d8/dd/d8e/d2c/cb5 0 2024-02-22T15:50:36.497 INFO:tasks.workunit.client.0.smithi019.stdout:7/645: write d9/d26/dc4/fc7 [414585,119988] 0 2024-02-22T15:50:36.498 INFO:tasks.workunit.client.1.smithi062.stdout:7/250: dwrite d0/d14/d19/d30/d37/f53 [0,4194304] 0 2024-02-22T15:50:36.498 INFO:tasks.workunit.client.0.smithi019.stdout:5/541: fdatasync d8/f9b 0 2024-02-22T15:50:36.498 INFO:tasks.workunit.client.1.smithi062.stdout:4/252: dwrite d4/df/f14 [4194304,4194304] 0 2024-02-22T15:50:36.500 INFO:tasks.workunit.client.0.smithi019.stdout:9/518: creat d2/d1f/d2e/d48/d45/d5a/daf/db6/fbb x:0 0 0 2024-02-22T15:50:36.503 INFO:tasks.workunit.client.1.smithi062.stdout:9/239: sync 2024-02-22T15:50:36.503 INFO:tasks.workunit.client.1.smithi062.stdout:0/341: sync 2024-02-22T15:50:36.503 INFO:tasks.workunit.client.1.smithi062.stdout:2/350: sync 2024-02-22T15:50:36.503 INFO:tasks.workunit.client.1.smithi062.stdout:8/274: sync 2024-02-22T15:50:36.506 INFO:tasks.workunit.client.0.smithi019.stdout:6/533: rename d6/c30 to d6/d4b/d6d/d7b/d7f/ca1 0 2024-02-22T15:50:36.507 INFO:tasks.workunit.client.0.smithi019.stdout:5/542: rmdir d8/dd/d19 39 2024-02-22T15:50:36.508 INFO:tasks.workunit.client.1.smithi062.stdout:0/342: chown d1/d3/de/d3f/f64 1538 1 2024-02-22T15:50:36.510 INFO:tasks.workunit.client.0.smithi019.stdout:4/499: mknod d2/db/d14/d23/d9e/ca5 0 2024-02-22T15:50:36.510 INFO:tasks.workunit.client.1.smithi062.stdout:1/270: link d1/d6/l23 d1/d6/d7/d2b/d47/l55 0 2024-02-22T15:50:36.511 INFO:tasks.workunit.client.0.smithi019.stdout:4/500: fsync d2/db/d14/d23/d21/d35/f64 0 2024-02-22T15:50:36.513 INFO:tasks.workunit.client.1.smithi062.stdout:3/258: dwrite d5/f6 [0,4194304] 0 2024-02-22T15:50:36.514 INFO:tasks.workunit.client.1.smithi062.stdout:0/343: creat d1/d3/d4/d1e/f79 x:0 0 0 2024-02-22T15:50:36.515 INFO:tasks.workunit.client.1.smithi062.stdout:4/253: mknod d4/c5b 0 2024-02-22T15:50:36.515 INFO:tasks.workunit.client.0.smithi019.stdout:9/519: rename d2/d7/d14 to d2/d7/dc/d27/dbc 0 2024-02-22T15:50:36.516 INFO:tasks.workunit.client.1.smithi062.stdout:6/358: dwrite d5/df/d32/d47/f50 [0,4194304] 0 2024-02-22T15:50:36.517 INFO:tasks.workunit.client.1.smithi062.stdout:9/240: creat d4/d15/d3c/f43 x:0 0 0 2024-02-22T15:50:36.517 INFO:tasks.workunit.client.1.smithi062.stdout:3/259: chown d5/d1a 151003018 1 2024-02-22T15:50:36.518 INFO:tasks.workunit.client.1.smithi062.stdout:0/344: stat d1/d3/de/l11 0 2024-02-22T15:50:36.525 INFO:tasks.workunit.client.0.smithi019.stdout:4/501: getdents d2/db/d59 0 2024-02-22T15:50:36.526 INFO:tasks.workunit.client.1.smithi062.stdout:9/241: stat d4/l11 0 2024-02-22T15:50:36.526 INFO:tasks.workunit.client.1.smithi062.stdout:6/359: truncate d5/df/d32/f35 725530 0 2024-02-22T15:50:36.527 INFO:tasks.workunit.client.0.smithi019.stdout:7/646: read d9/d3a/d49/d5d/f9f [2731590,76507] 0 2024-02-22T15:50:36.527 INFO:tasks.workunit.client.1.smithi062.stdout:0/345: dread - d1/d3/de/d3f/f4c zero size 2024-02-22T15:50:36.528 INFO:tasks.workunit.client.0.smithi019.stdout:5/543: write d8/d11/f1c [3106891,7664] 0 2024-02-22T15:50:36.529 INFO:tasks.workunit.client.1.smithi062.stdout:8/275: dread d8/f40 [0,4194304] 0 2024-02-22T15:50:36.530 INFO:tasks.workunit.client.1.smithi062.stdout:9/242: chown d4/d15/d31/l40 2875 1 2024-02-22T15:50:36.530 INFO:tasks.workunit.client.1.smithi062.stdout:2/351: creat d1/d28/d3f/d65/f6c x:0 0 0 2024-02-22T15:50:36.531 INFO:tasks.workunit.client.0.smithi019.stdout:9/520: dwrite d2/d1f/d2e/d48/d45/d5a/daf/db6/fbb [0,4194304] 0 2024-02-22T15:50:36.531 INFO:tasks.workunit.client.0.smithi019.stdout:4/502: truncate d2/db/d44/fa2 1207649 0 2024-02-22T15:50:36.532 INFO:tasks.workunit.client.0.smithi019.stdout:9/521: fdatasync d2/d1f/d2e/d57/d60/d8c/f6a 0 2024-02-22T15:50:36.533 INFO:tasks.workunit.client.0.smithi019.stdout:5/544: dread - d8/dd/d19/d38/d47/fa3 zero size 2024-02-22T15:50:36.535 INFO:tasks.workunit.client.0.smithi019.stdout:5/545: dread - d8/dd/d19/d38/d54/fad zero size 2024-02-22T15:50:36.535 INFO:tasks.workunit.client.1.smithi062.stdout:7/251: dwrite d0/d14/d13/f41 [4194304,4194304] 0 2024-02-22T15:50:36.536 INFO:tasks.workunit.client.0.smithi019.stdout:7/647: symlink d9/d3a/d49/d5d/d8f/ld7 0 2024-02-22T15:50:36.537 INFO:tasks.workunit.client.1.smithi062.stdout:3/260: symlink d5/d26/d2d/d3f/l50 0 2024-02-22T15:50:36.537 INFO:tasks.workunit.client.1.smithi062.stdout:9/243: fdatasync d4/d35/d3f/f39 0 2024-02-22T15:50:36.537 INFO:tasks.workunit.client.0.smithi019.stdout:6/534: truncate d6/f2a 481599 0 2024-02-22T15:50:36.538 INFO:tasks.workunit.client.1.smithi062.stdout:6/360: mknod d5/df/d2a/c74 0 2024-02-22T15:50:36.539 INFO:tasks.workunit.client.0.smithi019.stdout:7/648: stat d9/d3a/d49/d5d/d8f/ld7 0 2024-02-22T15:50:36.539 INFO:tasks.workunit.client.0.smithi019.stdout:6/535: dread - d6/f86 zero size 2024-02-22T15:50:36.540 INFO:tasks.workunit.client.1.smithi062.stdout:7/252: dread - d0/d6/d2e/d4a/f4c zero size 2024-02-22T15:50:36.543 INFO:tasks.workunit.client.1.smithi062.stdout:1/271: write d1/d6/d7/f50 [2299809,9430] 0 2024-02-22T15:50:36.543 INFO:tasks.workunit.client.0.smithi019.stdout:6/536: read - d6/d4b/d6d/d7b/d78/f7e zero size 2024-02-22T15:50:36.544 INFO:tasks.workunit.client.1.smithi062.stdout:2/352: write d1/d8/f3d [1482526,3201] 0 2024-02-22T15:50:36.544 INFO:tasks.workunit.client.0.smithi019.stdout:9/522: dwrite d2/d1f/d2e/d57/d60/f75 [0,4194304] 0 2024-02-22T15:50:36.546 INFO:tasks.workunit.client.1.smithi062.stdout:2/353: write d1/d8/f42 [741395,103535] 0 2024-02-22T15:50:36.548 INFO:tasks.workunit.client.1.smithi062.stdout:2/354: dread - d1/d8/df/f61 zero size 2024-02-22T15:50:36.549 INFO:tasks.workunit.client.1.smithi062.stdout:3/261: creat d5/d16/d21/d32/f51 x:0 0 0 2024-02-22T15:50:36.550 INFO:tasks.workunit.client.0.smithi019.stdout:7/649: rename d9/ce to d9/d38/d41/d45/d61/d6a/cd8 0 2024-02-22T15:50:36.551 INFO:tasks.workunit.client.1.smithi062.stdout:1/272: mkdir d1/d6/d7/d2b/d56 0 2024-02-22T15:50:36.553 INFO:tasks.workunit.client.0.smithi019.stdout:6/537: fsync d6/ff 0 2024-02-22T15:50:36.553 INFO:tasks.workunit.client.1.smithi062.stdout:1/273: readlink d1/d6/d10/l1f 0 2024-02-22T15:50:36.556 INFO:tasks.workunit.client.1.smithi062.stdout:0/346: rename d1/d3/de/d14/l3c to d1/d3/d4/d1a/d76/l7a 0 2024-02-22T15:50:36.557 INFO:tasks.workunit.client.0.smithi019.stdout:6/538: fsync d6/d4b/d6d/d7b/f60 0 2024-02-22T15:50:36.558 INFO:tasks.workunit.client.1.smithi062.stdout:8/276: symlink d8/d9/d44/d4d/l4f 0 2024-02-22T15:50:36.559 INFO:tasks.workunit.client.0.smithi019.stdout:7/650: mkdir d9/d26/dc4/dd9 0 2024-02-22T15:50:36.559 INFO:tasks.workunit.client.0.smithi019.stdout:9/523: dwrite d2/f6 [8388608,4194304] 0 2024-02-22T15:50:36.560 INFO:tasks.workunit.client.1.smithi062.stdout:2/355: dwrite d1/d28/d3f/f46 [0,4194304] 0 2024-02-22T15:50:36.560 INFO:tasks.workunit.client.1.smithi062.stdout:4/254: truncate d4/d1c/f27 328652 0 2024-02-22T15:50:36.562 INFO:tasks.workunit.client.1.smithi062.stdout:5/150: dread d0/d3/ff [0,4194304] 0 2024-02-22T15:50:36.563 INFO:tasks.workunit.client.1.smithi062.stdout:3/262: symlink d5/d1a/d3b/l52 0 2024-02-22T15:50:36.564 INFO:tasks.workunit.client.0.smithi019.stdout:6/539: truncate d6/d31/d43/d55/d67/f9a 160055 0 2024-02-22T15:50:36.564 INFO:tasks.workunit.client.0.smithi019.stdout:7/651: chown d9/d3a/d49/d5d/d74/laf 84579 1 2024-02-22T15:50:36.565 INFO:tasks.workunit.client.1.smithi062.stdout:0/347: creat d1/d38/f7b x:0 0 0 2024-02-22T15:50:36.568 INFO:tasks.workunit.client.0.smithi019.stdout:6/540: stat d6/d4b/d8d/f96 0 2024-02-22T15:50:36.568 INFO:tasks.workunit.client.0.smithi019.stdout:5/546: dwrite d8/dd/d19/d38/d47/f68 [0,4194304] 0 2024-02-22T15:50:36.568 INFO:tasks.workunit.client.1.smithi062.stdout:8/277: chown d8/d9/f1f 15 1 2024-02-22T15:50:36.569 INFO:tasks.workunit.client.1.smithi062.stdout:6/361: getdents d5/dd/d33 0 2024-02-22T15:50:36.571 INFO:tasks.workunit.client.0.smithi019.stdout:7/652: creat d9/d26/dc4/fda x:0 0 0 2024-02-22T15:50:36.575 INFO:tasks.workunit.client.0.smithi019.stdout:9/524: dwrite d2/d1f/d95/fb0 [0,4194304] 0 2024-02-22T15:50:36.575 INFO:tasks.workunit.client.0.smithi019.stdout:7/653: fsync d9/d38/d41/f48 0 2024-02-22T15:50:36.577 INFO:tasks.workunit.client.0.smithi019.stdout:6/541: unlink d6/d31/d43/d55/d67/d8a/c93 0 2024-02-22T15:50:36.578 INFO:tasks.workunit.client.0.smithi019.stdout:5/547: fsync d8/dd/d19/d3a/d46/f53 0 2024-02-22T15:50:36.579 INFO:tasks.workunit.client.0.smithi019.stdout:7/654: fdatasync d9/d3a/f6b 0 2024-02-22T15:50:36.583 INFO:tasks.workunit.client.0.smithi019.stdout:6/542: rename d6/d31/d43/d55/c68 to d6/d98/ca2 0 2024-02-22T15:50:36.584 INFO:tasks.workunit.client.0.smithi019.stdout:5/548: truncate d8/dd/d8e/d2c/f63 94598 0 2024-02-22T15:50:36.585 INFO:tasks.workunit.client.0.smithi019.stdout:6/543: chown d6/d3e/c4f 15075524 1 2024-02-22T15:50:36.590 INFO:tasks.workunit.client.1.smithi062.stdout:9/244: truncate d4/f2f 948259 0 2024-02-22T15:50:36.590 INFO:tasks.workunit.client.1.smithi062.stdout:8/278: fdatasync d8/d9/d2c/f30 0 2024-02-22T15:50:36.591 INFO:tasks.workunit.client.1.smithi062.stdout:4/255: dwrite d4/f45 [0,4194304] 0 2024-02-22T15:50:36.591 INFO:tasks.workunit.client.0.smithi019.stdout:9/525: truncate d2/d1f/d2e/d48/d36/f43 167370 0 2024-02-22T15:50:36.592 INFO:tasks.workunit.client.1.smithi062.stdout:0/348: fsync d1/d3/de/f3a 0 2024-02-22T15:50:36.593 INFO:tasks.workunit.client.0.smithi019.stdout:5/549: rename d8/dd/d19/d38/f5a to d8/d11/fb6 0 2024-02-22T15:50:36.594 INFO:tasks.workunit.client.0.smithi019.stdout:6/544: getdents d6/d31/d43/d55/d82 0 2024-02-22T15:50:36.595 INFO:tasks.workunit.client.1.smithi062.stdout:7/253: dread d0/f24 [0,4194304] 0 2024-02-22T15:50:36.595 INFO:tasks.workunit.client.1.smithi062.stdout:3/263: dwrite d5/d26/f31 [0,4194304] 0 2024-02-22T15:50:36.596 INFO:tasks.workunit.client.0.smithi019.stdout:9/526: write d2/d1f/d2e/d48/d36/fa4 [751694,99193] 0 2024-02-22T15:50:36.596 INFO:tasks.workunit.client.0.smithi019.stdout:5/550: creat d8/db1/fb7 x:0 0 0 2024-02-22T15:50:36.596 INFO:tasks.workunit.client.1.smithi062.stdout:2/356: dread d1/d1e/d38/f5f [0,4194304] 0 2024-02-22T15:50:36.597 INFO:tasks.workunit.client.0.smithi019.stdout:6/545: mkdir d6/d4b/d6d/da3 0 2024-02-22T15:50:36.599 INFO:tasks.workunit.client.1.smithi062.stdout:6/362: getdents d5/d15/d56 0 2024-02-22T15:50:36.599 INFO:tasks.workunit.client.1.smithi062.stdout:8/279: write d8/d9/f2a [984779,21286] 0 2024-02-22T15:50:36.603 INFO:tasks.workunit.client.1.smithi062.stdout:7/254: write d0/d14/d19/d30/d37/f53 [1686894,88424] 0 2024-02-22T15:50:36.605 INFO:tasks.workunit.client.0.smithi019.stdout:6/546: symlink d6/d31/la4 0 2024-02-22T15:50:36.607 INFO:tasks.workunit.client.0.smithi019.stdout:9/527: dwrite d2/d1f/d95/fb0 [0,4194304] 0 2024-02-22T15:50:36.607 INFO:tasks.workunit.client.0.smithi019.stdout:5/551: dread d8/dd/d19/f81 [0,4194304] 0 2024-02-22T15:50:36.609 INFO:tasks.workunit.client.1.smithi062.stdout:7/255: readlink d0/d14/d19/d30/d37/l39 0 2024-02-22T15:50:36.611 INFO:tasks.workunit.client.1.smithi062.stdout:4/256: dwrite d4/f59 [0,4194304] 0 2024-02-22T15:50:36.612 INFO:tasks.workunit.client.1.smithi062.stdout:4/257: fsync d4/fd 0 2024-02-22T15:50:36.615 INFO:tasks.workunit.client.0.smithi019.stdout:9/528: creat d2/d1f/d2e/d48/d45/d5a/daf/fbd x:0 0 0 2024-02-22T15:50:36.615 INFO:tasks.workunit.client.0.smithi019.stdout:9/529: fdatasync d2/f6 0 2024-02-22T15:50:36.616 INFO:tasks.workunit.client.0.smithi019.stdout:5/552: creat d8/d11/fb8 x:0 0 0 2024-02-22T15:50:36.616 INFO:tasks.workunit.client.1.smithi062.stdout:2/357: creat d1/d47/d39/d5d/f6d x:0 0 0 2024-02-22T15:50:36.617 INFO:tasks.workunit.client.1.smithi062.stdout:8/280: dread d8/d9/d2c/f30 [0,4194304] 0 2024-02-22T15:50:36.617 INFO:tasks.workunit.client.1.smithi062.stdout:2/358: fsync d1/d1e/d38/f4f 0 2024-02-22T15:50:36.619 INFO:tasks.workunit.client.1.smithi062.stdout:6/363: rename d5/d15/l3c to d5/dd/d33/l75 0 2024-02-22T15:50:36.619 INFO:tasks.workunit.client.1.smithi062.stdout:7/256: creat d0/d6/f54 x:0 0 0 2024-02-22T15:50:36.620 INFO:tasks.workunit.client.1.smithi062.stdout:4/258: fdatasync d4/d34/f48 0 2024-02-22T15:50:36.620 INFO:tasks.workunit.client.0.smithi019.stdout:5/553: dread - d8/dd/d19/d38/d47/f98 zero size 2024-02-22T15:50:36.621 INFO:tasks.workunit.client.1.smithi062.stdout:7/257: dread - d0/d6/f50 zero size 2024-02-22T15:50:36.621 INFO:tasks.workunit.client.1.smithi062.stdout:3/264: link d5/d1a/f1b d5/d1a/d3c/f53 0 2024-02-22T15:50:36.623 INFO:tasks.workunit.client.1.smithi062.stdout:6/364: chown d5/dd/f2c 7 1 2024-02-22T15:50:36.623 INFO:tasks.workunit.client.1.smithi062.stdout:4/259: read - d4/d1c/f3d zero size 2024-02-22T15:50:36.624 INFO:tasks.workunit.client.0.smithi019.stdout:5/554: read - d8/dd/d8e/d2c/f5d zero size 2024-02-22T15:50:36.624 INFO:tasks.workunit.client.0.smithi019.stdout:7/655: dwrite d9/d4c/d5b/fca [0,4194304] 0 2024-02-22T15:50:36.625 INFO:tasks.workunit.client.1.smithi062.stdout:7/258: dread - d0/d14/d13/f48 zero size 2024-02-22T15:50:36.625 INFO:tasks.workunit.client.1.smithi062.stdout:3/265: read - d5/d16/d21/d32/f51 zero size 2024-02-22T15:50:36.625 INFO:tasks.workunit.client.1.smithi062.stdout:4/260: truncate d4/f51 343729 0 2024-02-22T15:50:36.626 INFO:tasks.workunit.client.1.smithi062.stdout:8/281: rename d8/d28/c3e to d8/d26/c50 0 2024-02-22T15:50:36.628 INFO:tasks.workunit.client.0.smithi019.stdout:5/555: rmdir d8/dd/d19/d77/d7a 39 2024-02-22T15:50:36.629 INFO:tasks.workunit.client.1.smithi062.stdout:3/266: write d5/d26/f27 [515894,10105] 0 2024-02-22T15:50:36.629 INFO:tasks.workunit.client.0.smithi019.stdout:9/530: dwrite d2/d1f/d2e/d48/d36/fa4 [0,4194304] 0 2024-02-22T15:50:36.629 INFO:tasks.workunit.client.0.smithi019.stdout:7/656: chown d9/d38/d41/d45/d61/d6a/cd8 496 1 2024-02-22T15:50:36.630 INFO:tasks.workunit.client.0.smithi019.stdout:7/657: readlink d9/d4c/d5b/lbc 0 2024-02-22T15:50:36.631 INFO:tasks.workunit.client.0.smithi019.stdout:5/556: creat d8/dd/d19/d38/fb9 x:0 0 0 2024-02-22T15:50:36.633 INFO:tasks.workunit.client.0.smithi019.stdout:7/658: getdents d9/d4c/d5b 0 2024-02-22T15:50:36.634 INFO:tasks.workunit.client.1.smithi062.stdout:3/267: dread - d5/d16/f44 zero size 2024-02-22T15:50:36.634 INFO:tasks.workunit.client.1.smithi062.stdout:3/268: dread - d5/f41 zero size 2024-02-22T15:50:36.634 INFO:tasks.workunit.client.1.smithi062.stdout:3/269: chown d5/f6 2535 1 2024-02-22T15:50:36.636 INFO:tasks.workunit.client.0.smithi019.stdout:8/493: sync 2024-02-22T15:50:36.636 INFO:tasks.workunit.client.0.smithi019.stdout:7/659: read - d9/d26/d77/f8e zero size 2024-02-22T15:50:36.638 INFO:tasks.workunit.client.1.smithi062.stdout:3/270: creat d5/d1a/d3c/f54 x:0 0 0 2024-02-22T15:50:36.639 INFO:tasks.workunit.client.0.smithi019.stdout:9/531: dwrite d2/d7/f1b [4194304,4194304] 0 2024-02-22T15:50:36.642 INFO:tasks.workunit.client.1.smithi062.stdout:4/261: dwrite d4/f53 [0,4194304] 0 2024-02-22T15:50:36.642 INFO:tasks.workunit.client.1.smithi062.stdout:3/271: rmdir d5/d26 39 2024-02-22T15:50:36.645 INFO:tasks.workunit.client.0.smithi019.stdout:8/494: symlink d9/d14/d17/la4 0 2024-02-22T15:50:36.646 INFO:tasks.workunit.client.0.smithi019.stdout:7/660: symlink d9/d3a/d49/d5d/d74/d9a/ldb 0 2024-02-22T15:50:36.646 INFO:tasks.workunit.client.1.smithi062.stdout:4/262: unlink d4/d13/d20/f41 0 2024-02-22T15:50:36.647 INFO:tasks.workunit.client.0.smithi019.stdout:5/557: dwrite d8/d11/f1c [0,4194304] 0 2024-02-22T15:50:36.648 INFO:tasks.workunit.client.0.smithi019.stdout:7/661: readlink d9/d3a/d49/l89 0 2024-02-22T15:50:36.653 INFO:tasks.workunit.client.0.smithi019.stdout:5/558: fdatasync d8/dd/d19/d3a/fab 0 2024-02-22T15:50:36.654 INFO:tasks.workunit.client.1.smithi062.stdout:3/272: dwrite d5/f41 [0,4194304] 0 2024-02-22T15:50:36.655 INFO:tasks.workunit.client.1.smithi062.stdout:7/259: read d0/d14/d19/f1a [334937,122073] 0 2024-02-22T15:50:36.656 INFO:tasks.workunit.client.0.smithi019.stdout:5/559: readlink d8/dd/d19/l2a 0 2024-02-22T15:50:36.656 INFO:tasks.workunit.client.0.smithi019.stdout:7/662: creat d9/d4c/dc9/fdc x:0 0 0 2024-02-22T15:50:36.659 INFO:tasks.workunit.client.0.smithi019.stdout:7/663: truncate d9/d26/d77/f9b 401520 0 2024-02-22T15:50:36.660 INFO:tasks.workunit.client.0.smithi019.stdout:5/560: symlink d8/dd/d19/lba 0 2024-02-22T15:50:36.663 INFO:tasks.workunit.client.0.smithi019.stdout:8/495: link d9/d14/d1f/d5c/d99/fa1 d9/d14/d1f/fa5 0 2024-02-22T15:50:36.664 INFO:tasks.workunit.client.1.smithi062.stdout:4/263: unlink d4/d13/d20/c4f 0 2024-02-22T15:50:36.664 INFO:tasks.workunit.client.1.smithi062.stdout:6/365: rename d5/d15 to d5/df/d40/d44/d76 0 2024-02-22T15:50:36.667 INFO:tasks.workunit.client.1.smithi062.stdout:2/359: write d1/d8/f15 [505054,27342] 0 2024-02-22T15:50:36.667 INFO:tasks.workunit.client.1.smithi062.stdout:4/264: rename d4/d1c/d2c/c57 to d4/d1c/d30/c5c 0 2024-02-22T15:50:36.667 INFO:tasks.workunit.client.0.smithi019.stdout:7/664: read d9/d3a/d49/d5d/d74/f88 [2074859,41910] 0 2024-02-22T15:50:36.668 INFO:tasks.workunit.client.0.smithi019.stdout:5/561: creat d8/d11/fbb x:0 0 0 2024-02-22T15:50:36.669 INFO:tasks.workunit.client.0.smithi019.stdout:5/562: stat d8/d11/c9a 0 2024-02-22T15:50:36.671 INFO:tasks.workunit.client.0.smithi019.stdout:7/665: chown d9/d38/d41/d45/d92/fd4 1 1 2024-02-22T15:50:36.672 INFO:tasks.workunit.client.1.smithi062.stdout:7/260: write d0/f10 [3445283,19846] 0 2024-02-22T15:50:36.673 INFO:tasks.workunit.client.0.smithi019.stdout:8/496: dwrite d9/d14/d1f/f33 [4194304,4194304] 0 2024-02-22T15:50:36.674 INFO:tasks.workunit.client.1.smithi062.stdout:4/265: rmdir d4/d2a/d2d/d2e 39 2024-02-22T15:50:36.677 INFO:tasks.workunit.client.1.smithi062.stdout:0/349: dwrite d1/d3/de/ff [0,4194304] 0 2024-02-22T15:50:36.678 INFO:tasks.workunit.client.1.smithi062.stdout:4/266: stat d4/f39 0 2024-02-22T15:50:36.678 INFO:tasks.workunit.client.1.smithi062.stdout:3/273: dwrite d5/d16/f19 [0,4194304] 0 2024-02-22T15:50:36.679 INFO:tasks.workunit.client.1.smithi062.stdout:6/366: creat d5/df/d40/d44/d76/d56/f77 x:0 0 0 2024-02-22T15:50:36.679 INFO:tasks.workunit.client.0.smithi019.stdout:5/563: truncate d8/dd/d19/d3a/d46/f61 3012602 0 2024-02-22T15:50:36.680 INFO:tasks.workunit.client.0.smithi019.stdout:5/564: fsync d8/d11/fa8 0 2024-02-22T15:50:36.680 INFO:tasks.workunit.client.1.smithi062.stdout:7/261: fdatasync d0/d14/d13/f46 0 2024-02-22T15:50:36.681 INFO:tasks.workunit.client.0.smithi019.stdout:5/565: chown d8/c9 2332 1 2024-02-22T15:50:36.681 INFO:tasks.workunit.client.0.smithi019.stdout:7/666: getdents d9/d38/d41/d45/d92 0 2024-02-22T15:50:36.682 INFO:tasks.workunit.client.0.smithi019.stdout:7/667: chown d9/d79/cd5 1130234202 1 2024-02-22T15:50:36.684 INFO:tasks.workunit.client.1.smithi062.stdout:0/350: unlink d1/d38/f67 0 2024-02-22T15:50:36.686 INFO:tasks.workunit.client.1.smithi062.stdout:6/367: fsync d5/df/d32/d47/f50 0 2024-02-22T15:50:36.688 INFO:tasks.workunit.client.1.smithi062.stdout:8/282: dwrite d8/f22 [0,4194304] 0 2024-02-22T15:50:36.689 INFO:tasks.workunit.client.1.smithi062.stdout:0/351: write d1/d3/de/d14/f26 [2577785,63404] 0 2024-02-22T15:50:36.690 INFO:tasks.workunit.client.0.smithi019.stdout:7/668: creat d9/d38/d41/d45/d61/d6a/fdd x:0 0 0 2024-02-22T15:50:36.691 INFO:tasks.workunit.client.0.smithi019.stdout:7/669: readlink d9/d3a/d49/d5d/d74/laf 0 2024-02-22T15:50:36.696 INFO:tasks.workunit.client.0.smithi019.stdout:7/670: link d9/d38/d41/d45/f70 d9/d38/d41/d45/db0/fde 0 2024-02-22T15:50:36.696 INFO:tasks.workunit.client.1.smithi062.stdout:8/283: dread - d8/d9/d44/f45 zero size 2024-02-22T15:50:36.697 INFO:tasks.workunit.client.0.smithi019.stdout:7/671: mkdir d9/d4c/ddf 0 2024-02-22T15:50:36.699 INFO:tasks.workunit.client.1.smithi062.stdout:2/360: dwrite d1/d8/f1a [0,4194304] 0 2024-02-22T15:50:36.700 INFO:tasks.workunit.client.0.smithi019.stdout:7/672: rename d9/d3a/d49/d5d/d74/d9a/cad to d9/d79/ce0 0 2024-02-22T15:50:36.701 INFO:tasks.workunit.client.1.smithi062.stdout:6/368: rename d5/df/d2a to d5/df/d40/d44/d78 0 2024-02-22T15:50:36.703 INFO:tasks.workunit.client.1.smithi062.stdout:6/369: stat d5/df/d40/d44/d4a/f1c 0 2024-02-22T15:50:36.703 INFO:tasks.workunit.client.1.smithi062.stdout:3/274: dread d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:36.704 INFO:tasks.workunit.client.1.smithi062.stdout:0/352: symlink d1/d3/l7c 0 2024-02-22T15:50:36.705 INFO:tasks.workunit.client.1.smithi062.stdout:9/245: write d4/f2f [1302895,62876] 0 2024-02-22T15:50:36.705 INFO:tasks.workunit.client.1.smithi062.stdout:6/370: fdatasync d5/dd/f2c 0 2024-02-22T15:50:36.708 INFO:tasks.workunit.client.1.smithi062.stdout:8/284: getdents d8/d42 0 2024-02-22T15:50:36.709 INFO:tasks.workunit.client.0.smithi019.stdout:7/673: dread d9/d3a/d49/d5d/d74/d9a/fb2 [0,4194304] 0 2024-02-22T15:50:36.710 INFO:tasks.workunit.client.1.smithi062.stdout:8/285: readlink d8/d28/l4b 0 2024-02-22T15:50:36.711 INFO:tasks.workunit.client.0.smithi019.stdout:7/674: write d9/d4c/dc9/fdc [546009,17242] 0 2024-02-22T15:50:36.711 INFO:tasks.workunit.client.1.smithi062.stdout:0/353: symlink d1/d3/d4/l7d 0 2024-02-22T15:50:36.713 INFO:tasks.workunit.client.0.smithi019.stdout:5/566: dread d8/f48 [0,4194304] 0 2024-02-22T15:50:36.714 INFO:tasks.workunit.client.1.smithi062.stdout:6/371: creat d5/dd/d33/d66/f79 x:0 0 0 2024-02-22T15:50:36.714 INFO:tasks.workunit.client.1.smithi062.stdout:8/286: fsync d8/f40 0 2024-02-22T15:50:36.715 INFO:tasks.workunit.client.0.smithi019.stdout:5/567: chown d8/dd/d19/d3a/d46/d7e 1752 1 2024-02-22T15:50:36.717 INFO:tasks.workunit.client.1.smithi062.stdout:0/354: rename d1/d13/d43/c4d to d1/d3/d4/d1a/d1d/d3b/c7e 0 2024-02-22T15:50:36.719 INFO:tasks.workunit.client.1.smithi062.stdout:6/372: getdents d5/df/d40 0 2024-02-22T15:50:36.720 INFO:tasks.workunit.client.0.smithi019.stdout:7/675: dwrite d9/d3a/d49/d5d/d8f/fa3 [0,4194304] 0 2024-02-22T15:50:36.721 INFO:tasks.workunit.client.1.smithi062.stdout:0/355: symlink d1/dd/l7f 0 2024-02-22T15:50:36.722 INFO:tasks.workunit.client.1.smithi062.stdout:9/246: dread d4/f2a [0,4194304] 0 2024-02-22T15:50:36.723 INFO:tasks.workunit.client.0.smithi019.stdout:5/568: dread d8/f48 [0,4194304] 0 2024-02-22T15:50:36.723 INFO:tasks.workunit.client.0.smithi019.stdout:5/569: read - d8/d11/f99 zero size 2024-02-22T15:50:36.724 INFO:tasks.workunit.client.0.smithi019.stdout:7/676: mknod d9/d4c/d5b/ce1 0 2024-02-22T15:50:36.729 INFO:tasks.workunit.client.0.smithi019.stdout:5/570: rename d8/dd/d19/d77/c69 to d8/dd/d8e/cbc 0 2024-02-22T15:50:36.732 INFO:tasks.workunit.client.1.smithi062.stdout:0/356: creat d1/d3/de/d3f/f80 x:0 0 0 2024-02-22T15:50:36.733 INFO:tasks.workunit.client.1.smithi062.stdout:7/262: dread d0/d14/f20 [0,4194304] 0 2024-02-22T15:50:36.734 INFO:tasks.workunit.client.1.smithi062.stdout:7/263: stat d0/d14/f20 0 2024-02-22T15:50:36.736 INFO:tasks.workunit.client.0.smithi019.stdout:7/677: dwrite d9/f1d [0,4194304] 0 2024-02-22T15:50:36.736 INFO:tasks.workunit.client.0.smithi019.stdout:5/571: creat d8/dd/d19/d38/d54/db0/fbd x:0 0 0 2024-02-22T15:50:36.738 INFO:tasks.workunit.client.0.smithi019.stdout:8/497: dwrite d9/f32 [0,4194304] 0 2024-02-22T15:50:36.739 INFO:tasks.workunit.client.0.smithi019.stdout:8/498: fdatasync f6 0 2024-02-22T15:50:36.739 INFO:tasks.workunit.client.1.smithi062.stdout:0/357: dwrite d1/dd/f63 [0,4194304] 0 2024-02-22T15:50:36.742 INFO:tasks.workunit.client.1.smithi062.stdout:4/267: truncate d4/d2a/d2d/f35 144980 0 2024-02-22T15:50:36.744 INFO:tasks.workunit.client.1.smithi062.stdout:3/275: truncate d5/f6 3866568 0 2024-02-22T15:50:36.746 INFO:tasks.workunit.client.0.smithi019.stdout:5/572: dwrite d8/dd/d19/d38/fb9 [0,4194304] 0 2024-02-22T15:50:36.747 INFO:tasks.workunit.client.0.smithi019.stdout:8/499: rename d9/d14/d17/f40 to d9/d14/d17/d3d/d83/fa6 0 2024-02-22T15:50:36.748 INFO:tasks.workunit.client.1.smithi062.stdout:4/268: write d4/d1c/f3d [372224,1490] 0 2024-02-22T15:50:36.749 INFO:tasks.workunit.client.0.smithi019.stdout:5/573: rmdir d8/dd/d19/d3a 39 2024-02-22T15:50:36.753 INFO:tasks.workunit.client.0.smithi019.stdout:5/574: dread - d8/dd/d19/d38/d54/fad zero size 2024-02-22T15:50:36.753 INFO:tasks.workunit.client.1.smithi062.stdout:9/247: write d4/d28/f30 [2746,80327] 0 2024-02-22T15:50:36.754 INFO:tasks.workunit.client.1.smithi062.stdout:3/276: mkdir d5/d16/d21/d55 0 2024-02-22T15:50:36.758 INFO:tasks.workunit.client.0.smithi019.stdout:8/500: symlink d9/d14/d17/d3d/d83/la7 0 2024-02-22T15:50:36.759 INFO:tasks.workunit.client.1.smithi062.stdout:4/269: chown d4/d2a/d2d/d2e/c33 1259 1 2024-02-22T15:50:36.759 INFO:tasks.workunit.client.0.smithi019.stdout:8/501: chown d9/d14/d17/d86 28685278 1 2024-02-22T15:50:36.759 INFO:tasks.workunit.client.0.smithi019.stdout:5/575: mkdir d8/dd/d19/d3a/d46/d7e/dbe 0 2024-02-22T15:50:36.760 INFO:tasks.workunit.client.1.smithi062.stdout:7/264: dwrite d0/d6/f23 [0,4194304] 0 2024-02-22T15:50:36.765 INFO:tasks.workunit.client.1.smithi062.stdout:6/373: dread d5/f1e [0,4194304] 0 2024-02-22T15:50:36.767 INFO:tasks.workunit.client.1.smithi062.stdout:7/265: chown d0/d14/f38 1859099361 1 2024-02-22T15:50:36.768 INFO:tasks.workunit.client.0.smithi019.stdout:5/576: dwrite d8/dd/d19/d38/d54/db0/fbd [0,4194304] 0 2024-02-22T15:50:36.771 INFO:tasks.workunit.client.1.smithi062.stdout:4/270: unlink d4/d1c/d30/d3b/d49/f50 0 2024-02-22T15:50:36.771 INFO:tasks.workunit.client.0.smithi019.stdout:7/678: dwrite d9/d38/d41/d45/db0/fde [0,4194304] 0 2024-02-22T15:50:36.773 INFO:tasks.workunit.client.0.smithi019.stdout:5/577: write d8/dd/d19/d38/d47/f68 [2772266,27541] 0 2024-02-22T15:50:36.774 INFO:tasks.workunit.client.1.smithi062.stdout:0/358: dwrite d1/d3/d4/d1a/f56 [0,4194304] 0 2024-02-22T15:50:36.778 INFO:tasks.workunit.client.1.smithi062.stdout:0/359: write d1/f45 [1655623,36173] 0 2024-02-22T15:50:36.780 INFO:tasks.workunit.client.1.smithi062.stdout:6/374: unlink d5/df/d40/d44/d4a/l24 0 2024-02-22T15:50:36.780 INFO:tasks.workunit.client.1.smithi062.stdout:3/277: dread d5/d16/d21/d32/f4c [0,4194304] 0 2024-02-22T15:50:36.780 INFO:tasks.workunit.client.0.smithi019.stdout:5/578: link d8/dd/d19/d77/l79 d8/d11/lbf 0 2024-02-22T15:50:36.782 INFO:tasks.workunit.client.0.smithi019.stdout:5/579: symlink d8/dd/d19/d38/lc0 0 2024-02-22T15:50:36.783 INFO:tasks.workunit.client.1.smithi062.stdout:3/278: mkdir d5/d16/d56 0 2024-02-22T15:50:36.784 INFO:tasks.workunit.client.1.smithi062.stdout:9/248: dwrite f2 [0,4194304] 0 2024-02-22T15:50:36.793 INFO:tasks.workunit.client.1.smithi062.stdout:9/249: dread d4/d28/f42 [0,4194304] 0 2024-02-22T15:50:36.793 INFO:tasks.workunit.client.1.smithi062.stdout:9/250: chown d4/d28 126 1 2024-02-22T15:50:36.793 INFO:tasks.workunit.client.1.smithi062.stdout:3/279: unlink d5/f46 0 2024-02-22T15:50:36.793 INFO:tasks.workunit.client.1.smithi062.stdout:0/360: rmdir d1/d3/de/d3f/d2c 39 2024-02-22T15:50:36.796 INFO:tasks.workunit.client.0.smithi019.stdout:8/502: write d9/d14/f53 [1525856,84650] 0 2024-02-22T15:50:36.796 INFO:tasks.workunit.client.1.smithi062.stdout:6/375: dread d5/df/d32/d47/f50 [0,4194304] 0 2024-02-22T15:50:36.797 INFO:tasks.workunit.client.0.smithi019.stdout:8/503: rename d9/d14 to d9/d14/d17/d3d/d3f/da8 22 2024-02-22T15:50:36.799 INFO:tasks.workunit.client.0.smithi019.stdout:8/504: mknod d9/d88/ca9 0 2024-02-22T15:50:36.799 INFO:tasks.workunit.client.1.smithi062.stdout:9/251: creat d4/d15/f44 x:0 0 0 2024-02-22T15:50:36.799 INFO:tasks.workunit.client.1.smithi062.stdout:4/271: dread d4/d1c/f3d [0,4194304] 0 2024-02-22T15:50:36.804 INFO:tasks.workunit.client.1.smithi062.stdout:6/376: rmdir d5/df/d32 39 2024-02-22T15:50:36.806 INFO:tasks.workunit.client.1.smithi062.stdout:9/252: rename d4/d35/d3f/c34 to d4/d15/d3c/c45 0 2024-02-22T15:50:36.807 INFO:tasks.workunit.client.1.smithi062.stdout:3/280: dwrite d5/d1a/f1e [0,4194304] 0 2024-02-22T15:50:36.810 INFO:tasks.workunit.client.1.smithi062.stdout:3/281: readlink d5/d16/l2c 0 2024-02-22T15:50:36.812 INFO:tasks.workunit.client.1.smithi062.stdout:4/272: getdents d4 0 2024-02-22T15:50:36.812 INFO:tasks.workunit.client.1.smithi062.stdout:6/377: chown d5/df/d40/d44/d78/f3b 206406 1 2024-02-22T15:50:36.817 INFO:tasks.workunit.client.1.smithi062.stdout:3/282: write d5/d1a/f1e [718510,3691] 0 2024-02-22T15:50:36.818 INFO:tasks.workunit.client.1.smithi062.stdout:0/361: dwrite d1/d3/d4/d1a/d1d/d3b/f55 [0,4194304] 0 2024-02-22T15:50:36.819 INFO:tasks.workunit.client.0.smithi019.stdout:5/580: dread d8/fe [0,4194304] 0 2024-02-22T15:50:36.820 INFO:tasks.workunit.client.1.smithi062.stdout:4/273: symlink d4/d2a/l5d 0 2024-02-22T15:50:36.823 INFO:tasks.workunit.client.0.smithi019.stdout:8/505: write d9/d14/f38 [3002709,86831] 0 2024-02-22T15:50:36.823 INFO:tasks.workunit.client.0.smithi019.stdout:2/489: sync 2024-02-22T15:50:36.823 INFO:tasks.workunit.client.0.smithi019.stdout:0/403: sync 2024-02-22T15:50:36.823 INFO:tasks.workunit.client.0.smithi019.stdout:3/397: sync 2024-02-22T15:50:36.824 INFO:tasks.workunit.client.1.smithi062.stdout:3/283: symlink d5/d1a/d3c/l57 0 2024-02-22T15:50:36.825 INFO:tasks.workunit.client.1.smithi062.stdout:9/253: dread d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:36.825 INFO:tasks.workunit.client.1.smithi062.stdout:6/378: getdents d5/dd/d33/d71 0 2024-02-22T15:50:36.826 INFO:tasks.workunit.client.1.smithi062.stdout:7/266: dread d0/d14/f2c [0,4194304] 0 2024-02-22T15:50:36.830 INFO:tasks.workunit.client.0.smithi019.stdout:3/398: unlink d9/d26/d32/f2c 0 2024-02-22T15:50:36.832 INFO:tasks.workunit.client.0.smithi019.stdout:7/679: dwrite d9/d3a/f6b [0,4194304] 0 2024-02-22T15:50:36.833 INFO:tasks.workunit.client.1.smithi062.stdout:4/274: dwrite d4/f11 [4194304,4194304] 0 2024-02-22T15:50:36.833 INFO:tasks.workunit.client.0.smithi019.stdout:5/581: dwrite d8/dd/d19/d3a/fab [0,4194304] 0 2024-02-22T15:50:36.835 INFO:tasks.workunit.client.1.smithi062.stdout:7/267: mkdir d0/d6/d2e/d55 0 2024-02-22T15:50:36.835 INFO:tasks.workunit.client.0.smithi019.stdout:8/506: mkdir d9/d14/d6f/d7e/d94/daa 0 2024-02-22T15:50:36.837 INFO:tasks.workunit.client.1.smithi062.stdout:3/284: rename d5/d16/c24 to d5/d1a/c58 0 2024-02-22T15:50:36.838 INFO:tasks.workunit.client.0.smithi019.stdout:0/404: creat d4/d5/dc/d41/f90 x:0 0 0 2024-02-22T15:50:36.838 INFO:tasks.workunit.client.0.smithi019.stdout:2/490: creat d1/d17/fb2 x:0 0 0 2024-02-22T15:50:36.838 INFO:tasks.workunit.client.1.smithi062.stdout:1/274: sync 2024-02-22T15:50:36.840 INFO:tasks.workunit.client.1.smithi062.stdout:0/362: creat d1/d3/de/d14/f81 x:0 0 0 2024-02-22T15:50:36.841 INFO:tasks.workunit.client.1.smithi062.stdout:6/379: creat d5/df/d32/d47/f7a x:0 0 0 2024-02-22T15:50:36.843 INFO:tasks.workunit.client.1.smithi062.stdout:4/275: mkdir d4/d2a/d5e 0 2024-02-22T15:50:36.846 INFO:tasks.workunit.client.1.smithi062.stdout:7/268: write d0/d6/d2e/d2f/f3a [11501,76939] 0 2024-02-22T15:50:36.846 INFO:tasks.workunit.client.1.smithi062.stdout:0/363: mknod d1/d3/d4/d1a/d1d/d3b/c82 0 2024-02-22T15:50:36.849 INFO:tasks.workunit.client.1.smithi062.stdout:4/276: symlink d4/d1c/d30/l5f 0 2024-02-22T15:50:36.849 INFO:tasks.workunit.client.1.smithi062.stdout:3/285: dread d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:36.852 INFO:tasks.workunit.client.0.smithi019.stdout:5/582: creat d8/d11/d33/fc1 x:0 0 0 2024-02-22T15:50:36.852 INFO:tasks.workunit.client.0.smithi019.stdout:7/680: dwrite d9/d38/d41/fb7 [0,4194304] 0 2024-02-22T15:50:36.855 INFO:tasks.workunit.client.0.smithi019.stdout:0/405: unlink d4/f19 0 2024-02-22T15:50:36.856 INFO:tasks.workunit.client.1.smithi062.stdout:6/380: symlink d5/df/d40/d44/d78/l7b 0 2024-02-22T15:50:36.856 INFO:tasks.workunit.client.1.smithi062.stdout:1/275: getdents d1/d6/d7/d2b/d47 0 2024-02-22T15:50:36.856 INFO:tasks.workunit.client.1.smithi062.stdout:3/286: rmdir d5/d1a/d3b 39 2024-02-22T15:50:36.857 INFO:tasks.workunit.client.0.smithi019.stdout:3/399: dwrite d9/d16/f42 [0,4194304] 0 2024-02-22T15:50:36.859 INFO:tasks.workunit.client.0.smithi019.stdout:2/491: dwrite d1/d17/d21/d51/fa1 [0,4194304] 0 2024-02-22T15:50:36.859 INFO:tasks.workunit.client.0.smithi019.stdout:8/507: link d9/d14/d1f/d6d/c74 d9/d14/d6f/cab 0 2024-02-22T15:50:36.860 INFO:tasks.workunit.client.1.smithi062.stdout:9/254: dread d4/d35/d3f/f39 [0,4194304] 0 2024-02-22T15:50:36.861 INFO:tasks.workunit.client.1.smithi062.stdout:6/381: creat d5/df/d40/d44/f7c x:0 0 0 2024-02-22T15:50:36.861 INFO:tasks.workunit.client.0.smithi019.stdout:5/583: mknod d8/dd/d19/d38/d54/db0/cc2 0 2024-02-22T15:50:36.862 INFO:tasks.workunit.client.1.smithi062.stdout:7/269: link d0/d14/d19/c4e d0/d6/d2e/c56 0 2024-02-22T15:50:36.862 INFO:tasks.workunit.client.0.smithi019.stdout:3/400: write d9/d26/d3f/d59/f74 [4326949,60858] 0 2024-02-22T15:50:36.862 INFO:tasks.workunit.client.0.smithi019.stdout:0/406: symlink d4/db/d1b/d63/l91 0 2024-02-22T15:50:36.864 INFO:tasks.workunit.client.1.smithi062.stdout:4/277: dwrite d4/f53 [0,4194304] 0 2024-02-22T15:50:36.865 INFO:tasks.workunit.client.1.smithi062.stdout:3/287: dread - d5/d26/f38 zero size 2024-02-22T15:50:36.865 INFO:tasks.workunit.client.1.smithi062.stdout:9/255: fdatasync d4/d35/f37 0 2024-02-22T15:50:36.866 INFO:tasks.workunit.client.0.smithi019.stdout:7/681: creat d9/fe2 x:0 0 0 2024-02-22T15:50:36.870 INFO:tasks.workunit.client.0.smithi019.stdout:2/492: read d1/f2e [525635,32005] 0 2024-02-22T15:50:36.872 INFO:tasks.workunit.client.0.smithi019.stdout:7/682: mkdir d9/d3a/d49/d5d/d74/d9a/de3 0 2024-02-22T15:50:36.873 INFO:tasks.workunit.client.1.smithi062.stdout:1/276: creat d1/d6/d7/d2b/d49/f57 x:0 0 0 2024-02-22T15:50:36.873 INFO:tasks.workunit.client.1.smithi062.stdout:4/278: write d4/f59 [2379066,124868] 0 2024-02-22T15:50:36.873 INFO:tasks.workunit.client.1.smithi062.stdout:3/288: rmdir d5/d1a 39 2024-02-22T15:50:36.874 INFO:tasks.workunit.client.1.smithi062.stdout:6/382: chown d5/df/f17 256146735 1 2024-02-22T15:50:36.875 INFO:tasks.workunit.client.0.smithi019.stdout:3/401: creat d9/d22/d27/d6d/d72/f81 x:0 0 0 2024-02-22T15:50:36.876 INFO:tasks.workunit.client.0.smithi019.stdout:8/508: creat d9/d14/fac x:0 0 0 2024-02-22T15:50:36.877 INFO:tasks.workunit.client.1.smithi062.stdout:7/270: rmdir d0/d6/d2e 39 2024-02-22T15:50:36.877 INFO:tasks.workunit.client.1.smithi062.stdout:6/383: stat d5/df 0 2024-02-22T15:50:36.878 INFO:tasks.workunit.client.1.smithi062.stdout:1/277: stat d1/d6/d36/d37/l39 0 2024-02-22T15:50:36.881 INFO:tasks.workunit.client.1.smithi062.stdout:7/271: dread - d0/d14/d13/f4d zero size 2024-02-22T15:50:36.882 INFO:tasks.workunit.client.0.smithi019.stdout:0/407: rename d4/d23/f6f to d4/f92 0 2024-02-22T15:50:36.883 INFO:tasks.workunit.client.1.smithi062.stdout:3/289: mknod d5/d16/d21/d36/d4a/c59 0 2024-02-22T15:50:36.884 INFO:tasks.workunit.client.0.smithi019.stdout:3/402: creat d9/d22/d27/d6d/f82 x:0 0 0 2024-02-22T15:50:36.885 INFO:tasks.workunit.client.0.smithi019.stdout:7/683: dwrite d9/d4c/f66 [0,4194304] 0 2024-02-22T15:50:36.886 INFO:tasks.workunit.client.0.smithi019.stdout:3/403: readlink d9/d26/d32/l48 0 2024-02-22T15:50:36.887 INFO:tasks.workunit.client.1.smithi062.stdout:4/279: symlink d4/df/l60 0 2024-02-22T15:50:36.887 INFO:tasks.workunit.client.1.smithi062.stdout:1/278: mknod d1/d6/d7/c58 0 2024-02-22T15:50:36.888 INFO:tasks.workunit.client.1.smithi062.stdout:9/256: dwrite d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:36.889 INFO:tasks.workunit.client.1.smithi062.stdout:6/384: mknod d5/d6e/c7d 0 2024-02-22T15:50:36.889 INFO:tasks.workunit.client.0.smithi019.stdout:0/408: read d4/d22/f5f [1321939,46112] 0 2024-02-22T15:50:36.890 INFO:tasks.workunit.client.0.smithi019.stdout:0/409: dread - d4/f92 zero size 2024-02-22T15:50:36.892 INFO:tasks.workunit.client.0.smithi019.stdout:7/684: symlink d9/d38/d41/d45/d61/d6a/le4 0 2024-02-22T15:50:36.893 INFO:tasks.workunit.client.0.smithi019.stdout:8/509: link d9/d14/d1f/d4c/l4d d9/d14/d17/d3d/d83/lad 0 2024-02-22T15:50:36.893 INFO:tasks.workunit.client.0.smithi019.stdout:3/404: dread d9/d26/f35 [0,4194304] 0 2024-02-22T15:50:36.893 INFO:tasks.workunit.client.0.smithi019.stdout:7/685: readlink d9/d4c/lcb 0 2024-02-22T15:50:36.894 INFO:tasks.workunit.client.1.smithi062.stdout:3/290: rmdir d5/d1a/d3c 39 2024-02-22T15:50:36.894 INFO:tasks.workunit.client.0.smithi019.stdout:0/410: mknod d4/db/d1b/d40/d8c/c93 0 2024-02-22T15:50:36.897 INFO:tasks.workunit.client.0.smithi019.stdout:8/510: truncate f6 4593403 0 2024-02-22T15:50:36.897 INFO:tasks.workunit.client.1.smithi062.stdout:7/272: creat d0/d6/d2e/d2f/f57 x:0 0 0 2024-02-22T15:50:36.899 INFO:tasks.workunit.client.0.smithi019.stdout:8/511: rename d9/d14/d6f to d9/d14/d6f/dae 22 2024-02-22T15:50:36.899 INFO:tasks.workunit.client.1.smithi062.stdout:6/385: fdatasync d5/df/d40/d44/d4a/f1c 0 2024-02-22T15:50:36.900 INFO:tasks.workunit.client.1.smithi062.stdout:0/364: dread d1/f12 [0,4194304] 0 2024-02-22T15:50:36.901 INFO:tasks.workunit.client.1.smithi062.stdout:1/279: mknod d1/d6/d7/d2b/d56/c59 0 2024-02-22T15:50:36.903 INFO:tasks.workunit.client.0.smithi019.stdout:8/512: write d9/d88/f8b [1145514,69520] 0 2024-02-22T15:50:36.904 INFO:tasks.workunit.client.0.smithi019.stdout:0/411: dwrite d4/f5c [0,4194304] 0 2024-02-22T15:50:36.905 INFO:tasks.workunit.client.1.smithi062.stdout:3/291: rmdir d5/d1a/d3b 39 2024-02-22T15:50:36.905 INFO:tasks.workunit.client.0.smithi019.stdout:3/405: rmdir d9/d26/d2d/d39/d80 0 2024-02-22T15:50:36.905 INFO:tasks.workunit.client.0.smithi019.stdout:2/493: dread d1/d17/d21/d71/f2f [0,4194304] 0 2024-02-22T15:50:36.906 INFO:tasks.workunit.client.0.smithi019.stdout:3/406: fsync d9/d11/f4d 0 2024-02-22T15:50:36.907 INFO:tasks.workunit.client.1.smithi062.stdout:7/273: mkdir d0/d6/d2e/d58 0 2024-02-22T15:50:36.907 INFO:tasks.workunit.client.1.smithi062.stdout:6/386: chown d5/df/d40/d44/d4a/d41/f55 766567129 1 2024-02-22T15:50:36.907 INFO:tasks.workunit.client.0.smithi019.stdout:8/513: write d9/d14/d17/d1c/f48 [3189569,90246] 0 2024-02-22T15:50:36.908 INFO:tasks.workunit.client.1.smithi062.stdout:9/257: dwrite d4/f1b [4194304,4194304] 0 2024-02-22T15:50:36.910 INFO:tasks.workunit.client.0.smithi019.stdout:0/412: unlink d4/d5/c29 0 2024-02-22T15:50:36.911 INFO:tasks.workunit.client.1.smithi062.stdout:3/292: mkdir d5/d16/d21/d36/d5a 0 2024-02-22T15:50:36.911 INFO:tasks.workunit.client.0.smithi019.stdout:2/494: creat d1/de/d15/fb3 x:0 0 0 2024-02-22T15:50:36.912 INFO:tasks.workunit.client.0.smithi019.stdout:8/514: creat d9/d14/d1f/d5c/d99/faf x:0 0 0 2024-02-22T15:50:36.913 INFO:tasks.workunit.client.0.smithi019.stdout:0/413: chown d4/db/d24/l8f 18963 1 2024-02-22T15:50:36.915 INFO:tasks.workunit.client.0.smithi019.stdout:2/495: mkdir d1/db4 0 2024-02-22T15:50:36.916 INFO:tasks.workunit.client.1.smithi062.stdout:4/280: dwrite d4/d13/f43 [0,4194304] 0 2024-02-22T15:50:36.917 INFO:tasks.workunit.client.0.smithi019.stdout:0/414: symlink d4/d5/dc/l94 0 2024-02-22T15:50:36.918 INFO:tasks.workunit.client.0.smithi019.stdout:8/515: truncate d9/d14/d17/d1c/f51 5046095 0 2024-02-22T15:50:36.921 INFO:tasks.workunit.client.1.smithi062.stdout:9/258: rename d4/d35/d3f/l3a to d4/d35/d3f/l46 0 2024-02-22T15:50:36.923 INFO:tasks.workunit.client.1.smithi062.stdout:6/387: getdents d5/df/d40/d44/d4a 0 2024-02-22T15:50:36.924 INFO:tasks.workunit.client.0.smithi019.stdout:2/496: write d1/f14 [2800292,37539] 0 2024-02-22T15:50:36.925 INFO:tasks.workunit.client.1.smithi062.stdout:4/281: symlink d4/d1c/d30/d3b/d49/l61 0 2024-02-22T15:50:36.927 INFO:tasks.workunit.client.1.smithi062.stdout:9/259: read d4/d35/f36 [111544,103200] 0 2024-02-22T15:50:36.927 INFO:tasks.workunit.client.0.smithi019.stdout:3/407: getdents d9/d22/d27/d6d 0 2024-02-22T15:50:36.928 INFO:tasks.workunit.client.1.smithi062.stdout:3/293: dread d5/d16/d21/f29 [0,4194304] 0 2024-02-22T15:50:36.929 INFO:tasks.workunit.client.0.smithi019.stdout:8/516: unlink d9/d14/d17/d3d/f59 0 2024-02-22T15:50:36.930 INFO:tasks.workunit.client.1.smithi062.stdout:6/388: creat d5/df/d40/d44/d78/d59/f7e x:0 0 0 2024-02-22T15:50:36.931 INFO:tasks.workunit.client.1.smithi062.stdout:3/294: truncate d5/d1a/f33 119449 0 2024-02-22T15:50:36.932 INFO:tasks.workunit.client.0.smithi019.stdout:0/415: dwrite d4/db/d1b/f1c [0,4194304] 0 2024-02-22T15:50:36.933 INFO:tasks.workunit.client.0.smithi019.stdout:2/497: creat d1/d46/d9d/fb5 x:0 0 0 2024-02-22T15:50:36.933 INFO:tasks.workunit.client.1.smithi062.stdout:4/282: truncate d4/d1c/f27 812932 0 2024-02-22T15:50:36.936 INFO:tasks.workunit.client.0.smithi019.stdout:7/686: write d9/f34 [736886,102224] 0 2024-02-22T15:50:36.936 INFO:tasks.workunit.client.1.smithi062.stdout:3/295: truncate d5/d1a/d3c/f54 38449 0 2024-02-22T15:50:36.938 INFO:tasks.workunit.client.1.smithi062.stdout:4/283: chown d4/d1c/f27 0 1 2024-02-22T15:50:36.938 INFO:tasks.workunit.client.0.smithi019.stdout:3/408: mknod d9/c83 0 2024-02-22T15:50:36.939 INFO:tasks.workunit.client.1.smithi062.stdout:6/389: getdents d5/dd/d33 0 2024-02-22T15:50:36.943 INFO:tasks.workunit.client.0.smithi019.stdout:8/517: symlink d9/d14/d6f/d7e/d94/d9a/lb0 0 2024-02-22T15:50:36.943 INFO:tasks.workunit.client.0.smithi019.stdout:3/409: chown d9/d22 28743 1 2024-02-22T15:50:36.943 INFO:tasks.workunit.client.0.smithi019.stdout:0/416: rmdir d4/db/d1b/d40/d44 39 2024-02-22T15:50:36.946 INFO:tasks.workunit.client.0.smithi019.stdout:7/687: symlink d9/d3a/d49/d5d/d74/le5 0 2024-02-22T15:50:36.946 INFO:tasks.workunit.client.0.smithi019.stdout:2/498: dwrite d1/d17/d1c/f34 [0,4194304] 0 2024-02-22T15:50:36.946 INFO:tasks.workunit.client.1.smithi062.stdout:4/284: dread d4/f59 [0,4194304] 0 2024-02-22T15:50:36.948 INFO:tasks.workunit.client.1.smithi062.stdout:1/280: truncate d1/d6/d7/f50 2376882 0 2024-02-22T15:50:36.948 INFO:tasks.workunit.client.0.smithi019.stdout:3/410: mknod d9/d26/d2d/d53/c84 0 2024-02-22T15:50:36.949 INFO:tasks.workunit.client.0.smithi019.stdout:8/518: write d9/d14/d17/f50 [2246955,72017] 0 2024-02-22T15:50:36.952 INFO:tasks.workunit.client.0.smithi019.stdout:3/411: write d9/d26/d2d/d39/f5f [1790265,43210] 0 2024-02-22T15:50:36.954 INFO:tasks.workunit.client.0.smithi019.stdout:0/417: mkdir d4/d22/d78/d95 0 2024-02-22T15:50:36.955 INFO:tasks.workunit.client.1.smithi062.stdout:0/365: dwrite d1/d13/d43/f53 [0,4194304] 0 2024-02-22T15:50:36.955 INFO:tasks.workunit.client.0.smithi019.stdout:2/499: mknod d1/d17/d21/daf/cb6 0 2024-02-22T15:50:36.955 INFO:tasks.workunit.client.1.smithi062.stdout:9/260: rename d4/d35/d3f/l46 to d4/d28/l47 0 2024-02-22T15:50:36.956 INFO:tasks.workunit.client.0.smithi019.stdout:8/519: mknod d9/d14/d17/d1c/da0/cb1 0 2024-02-22T15:50:36.958 INFO:tasks.workunit.client.1.smithi062.stdout:6/390: dwrite d5/df/d32/d47/f50 [0,4194304] 0 2024-02-22T15:50:36.958 INFO:tasks.workunit.client.0.smithi019.stdout:7/688: rename d9/l2f to d9/d38/d41/d45/d61/d6a/le6 0 2024-02-22T15:50:36.959 INFO:tasks.workunit.client.1.smithi062.stdout:1/281: symlink d1/d6/d36/l5a 0 2024-02-22T15:50:36.960 INFO:tasks.workunit.client.0.smithi019.stdout:8/520: write d9/d14/f34 [1548192,100751] 0 2024-02-22T15:50:36.960 INFO:tasks.workunit.client.0.smithi019.stdout:3/412: creat d9/d26/d3f/d59/d66/f85 x:0 0 0 2024-02-22T15:50:36.961 INFO:tasks.workunit.client.1.smithi062.stdout:5/151: sync 2024-02-22T15:50:36.961 INFO:tasks.workunit.client.0.smithi019.stdout:7/689: rmdir d9/d79 39 2024-02-22T15:50:36.962 INFO:tasks.workunit.client.1.smithi062.stdout:6/391: fsync d5/dd/d33/f46 0 2024-02-22T15:50:36.964 INFO:tasks.workunit.client.1.smithi062.stdout:3/296: rename d5/d1a/c39 to d5/d16/d56/c5b 0 2024-02-22T15:50:36.964 INFO:tasks.workunit.client.0.smithi019.stdout:0/418: dread d4/d22/d2c/f57 [0,4194304] 0 2024-02-22T15:50:36.964 INFO:tasks.workunit.client.1.smithi062.stdout:0/366: dread - d1/d3/d4/d1e/d74/f70 zero size 2024-02-22T15:50:36.966 INFO:tasks.workunit.client.1.smithi062.stdout:1/282: write d1/f30 [145174,118049] 0 2024-02-22T15:50:36.966 INFO:tasks.workunit.client.1.smithi062.stdout:5/152: readlink d0/l25 0 2024-02-22T15:50:36.966 INFO:tasks.workunit.client.1.smithi062.stdout:9/261: unlink d4/d15/d3c/f43 0 2024-02-22T15:50:36.973 INFO:tasks.workunit.client.1.smithi062.stdout:0/367: rmdir d1/d3/de/d14 39 2024-02-22T15:50:36.973 INFO:tasks.workunit.client.1.smithi062.stdout:4/285: dwrite d4/d1c/d2c/f56 [0,4194304] 0 2024-02-22T15:50:36.977 INFO:tasks.workunit.client.0.smithi019.stdout:0/419: dwrite d4/d22/d2c/f57 [0,4194304] 0 2024-02-22T15:50:36.977 INFO:tasks.workunit.client.1.smithi062.stdout:1/283: unlink d1/d6/d7/c58 0 2024-02-22T15:50:36.977 INFO:tasks.workunit.client.1.smithi062.stdout:7/274: dread d0/f10 [0,4194304] 0 2024-02-22T15:50:36.978 INFO:tasks.workunit.client.1.smithi062.stdout:0/368: stat d1/d3/d4/d1a 0 2024-02-22T15:50:36.978 INFO:tasks.workunit.client.0.smithi019.stdout:0/420: readlink d4/d67/l7c 0 2024-02-22T15:50:36.979 INFO:tasks.workunit.client.1.smithi062.stdout:6/392: creat d5/df/d40/d44/d78/d59/f7f x:0 0 0 2024-02-22T15:50:36.979 INFO:tasks.workunit.client.1.smithi062.stdout:2/361: sync 2024-02-22T15:50:36.979 INFO:tasks.workunit.client.1.smithi062.stdout:8/287: sync 2024-02-22T15:50:36.981 INFO:tasks.workunit.client.1.smithi062.stdout:9/262: truncate d4/d35/f36 391954 0 2024-02-22T15:50:36.984 INFO:tasks.workunit.client.1.smithi062.stdout:2/362: stat d1/d28 0 2024-02-22T15:50:36.986 INFO:tasks.workunit.client.1.smithi062.stdout:1/284: creat d1/d6/d7/d2b/f5b x:0 0 0 2024-02-22T15:50:36.988 INFO:tasks.workunit.client.0.smithi019.stdout:0/421: dwrite d4/d5/d7/f72 [0,4194304] 0 2024-02-22T15:50:36.988 INFO:tasks.workunit.client.1.smithi062.stdout:9/263: mkdir d4/d1c/d48 0 2024-02-22T15:50:36.990 INFO:tasks.workunit.client.1.smithi062.stdout:0/369: read d1/d3/de/d14/f26 [1730597,19601] 0 2024-02-22T15:50:36.996 INFO:tasks.workunit.client.0.smithi019.stdout:0/422: dwrite d4/f3e [4194304,4194304] 0 2024-02-22T15:50:36.996 INFO:tasks.workunit.client.1.smithi062.stdout:8/288: fsync d8/f29 0 2024-02-22T15:50:36.996 INFO:tasks.workunit.client.1.smithi062.stdout:7/275: dread d0/d14/d13/fc [4194304,4194304] 0 2024-02-22T15:50:36.996 INFO:tasks.workunit.client.1.smithi062.stdout:6/393: mknod d5/df/d40/d44/d4a/c80 0 2024-02-22T15:50:36.996 INFO:tasks.workunit.client.1.smithi062.stdout:5/153: dwrite d0/d3/da/fb [4194304,4194304] 0 2024-02-22T15:50:36.996 INFO:tasks.workunit.client.1.smithi062.stdout:2/363: fsync d1/d1e/d38/f5f 0 2024-02-22T15:50:36.997 INFO:tasks.workunit.client.0.smithi019.stdout:9/532: sync 2024-02-22T15:50:36.998 INFO:tasks.workunit.client.0.smithi019.stdout:6/547: sync 2024-02-22T15:50:36.998 INFO:tasks.workunit.client.0.smithi019.stdout:1/573: sync 2024-02-22T15:50:36.999 INFO:tasks.workunit.client.0.smithi019.stdout:4/503: sync 2024-02-22T15:50:36.999 INFO:tasks.workunit.client.0.smithi019.stdout:0/423: mknod d4/d5/c96 0 2024-02-22T15:50:36.999 INFO:tasks.workunit.client.1.smithi062.stdout:8/289: chown d8/d9/d2c 1 1 2024-02-22T15:50:37.000 INFO:tasks.workunit.client.1.smithi062.stdout:5/154: fdatasync d0/d3/da/d13/d23/f24 0 2024-02-22T15:50:37.001 INFO:tasks.workunit.client.1.smithi062.stdout:1/285: creat d1/d6/d10/f5c x:0 0 0 2024-02-22T15:50:37.002 INFO:tasks.workunit.client.0.smithi019.stdout:9/533: symlink d2/d7/dc/d27/dbc/lbe 0 2024-02-22T15:50:37.002 INFO:tasks.workunit.client.1.smithi062.stdout:0/370: mkdir d1/d21/d83 0 2024-02-22T15:50:37.005 INFO:tasks.workunit.client.1.smithi062.stdout:9/264: unlink d4/f2f 0 2024-02-22T15:50:37.005 INFO:tasks.workunit.client.0.smithi019.stdout:4/504: symlink d2/db/d59/la6 0 2024-02-22T15:50:37.006 INFO:tasks.workunit.client.1.smithi062.stdout:9/265: stat d4 0 2024-02-22T15:50:37.007 INFO:tasks.workunit.client.0.smithi019.stdout:1/574: unlink d8/d15/f1b 0 2024-02-22T15:50:37.008 INFO:tasks.workunit.client.1.smithi062.stdout:4/286: dwrite d4/f45 [0,4194304] 0 2024-02-22T15:50:37.010 INFO:tasks.workunit.client.0.smithi019.stdout:1/575: readlink d8/d44/d5d/l52 0 2024-02-22T15:50:37.011 INFO:tasks.workunit.client.0.smithi019.stdout:6/548: link d6/d4b/d6d/d7b/d78/f7e d6/d98/fa5 0 2024-02-22T15:50:37.014 INFO:tasks.workunit.client.0.smithi019.stdout:8/521: dwrite d9/d14/f52 [0,4194304] 0 2024-02-22T15:50:37.015 INFO:tasks.workunit.client.0.smithi019.stdout:9/534: link d2/d1f/d2e/d48/d89/c39 d2/d1f/d2e/d48/d45/cbf 0 2024-02-22T15:50:37.015 INFO:tasks.workunit.client.0.smithi019.stdout:4/505: fdatasync d2/db/f6a 0 2024-02-22T15:50:37.015 INFO:tasks.workunit.client.1.smithi062.stdout:0/371: write d1/d3/d4/d1a/f56 [2366840,123701] 0 2024-02-22T15:50:37.018 INFO:tasks.workunit.client.1.smithi062.stdout:8/290: mkdir d8/d9/d44/d4d/d51 0 2024-02-22T15:50:37.018 INFO:tasks.workunit.client.1.smithi062.stdout:5/155: chown d0/d3/f10 1611748345 1 2024-02-22T15:50:37.019 INFO:tasks.workunit.client.0.smithi019.stdout:2/500: truncate d1/d17/d21/d51/f72 1923672 0 2024-02-22T15:50:37.022 INFO:tasks.workunit.client.1.smithi062.stdout:4/287: write d4/d1c/f23 [641619,13029] 0 2024-02-22T15:50:37.022 INFO:tasks.workunit.client.1.smithi062.stdout:0/372: stat d1/d38/f7b 0 2024-02-22T15:50:37.022 INFO:tasks.workunit.client.1.smithi062.stdout:2/364: chown d1/cd 0 1 2024-02-22T15:50:37.022 INFO:tasks.workunit.client.1.smithi062.stdout:6/394: dwrite d5/dd/d33/f49 [0,4194304] 0 2024-02-22T15:50:37.024 INFO:tasks.workunit.client.0.smithi019.stdout:9/535: unlink d2/d1f/d2e/f7b 0 2024-02-22T15:50:37.028 INFO:tasks.workunit.client.0.smithi019.stdout:3/413: dread d9/d22/d27/f46 [4194304,4194304] 0 2024-02-22T15:50:37.030 INFO:tasks.workunit.client.0.smithi019.stdout:1/576: dwrite d8/d44/d5d/fc4 [0,4194304] 0 2024-02-22T15:50:37.030 INFO:tasks.workunit.client.0.smithi019.stdout:4/506: fdatasync d2/db/d14/f87 0 2024-02-22T15:50:37.031 INFO:tasks.workunit.client.1.smithi062.stdout:8/291: stat d8/d28/f35 0 2024-02-22T15:50:37.031 INFO:tasks.workunit.client.1.smithi062.stdout:4/288: chown d4/d34/c55 10 1 2024-02-22T15:50:37.031 INFO:tasks.workunit.client.1.smithi062.stdout:3/297: rename d5/d16/d56/c5b to d5/d16/d21/d36/c5c 0 2024-02-22T15:50:37.032 INFO:tasks.workunit.client.1.smithi062.stdout:9/266: creat d4/d35/f49 x:0 0 0 2024-02-22T15:50:37.032 INFO:tasks.workunit.client.0.smithi019.stdout:6/549: read d6/f57 [69599,115208] 0 2024-02-22T15:50:37.033 INFO:tasks.workunit.client.1.smithi062.stdout:0/373: chown d1/d3/de/d14/c69 125340088 1 2024-02-22T15:50:37.036 INFO:tasks.workunit.client.1.smithi062.stdout:7/276: getdents d0/d6/d2e/d58 0 2024-02-22T15:50:37.037 INFO:tasks.workunit.client.0.smithi019.stdout:9/536: mkdir d2/d7/d52/dc0 0 2024-02-22T15:50:37.039 INFO:tasks.workunit.client.1.smithi062.stdout:1/286: dwrite d1/d6/d10/f5c [0,4194304] 0 2024-02-22T15:50:37.040 INFO:tasks.workunit.client.1.smithi062.stdout:4/289: write d4/f11 [3930110,22329] 0 2024-02-22T15:50:37.041 INFO:tasks.workunit.client.0.smithi019.stdout:4/507: truncate d2/db/d14/d23/d33/f7d 1367936 0 2024-02-22T15:50:37.042 INFO:tasks.workunit.client.0.smithi019.stdout:9/537: dread - d2/d1f/d2e/d57/f94 zero size 2024-02-22T15:50:37.044 INFO:tasks.workunit.client.1.smithi062.stdout:7/277: read - d0/d14/f25 zero size 2024-02-22T15:50:37.045 INFO:tasks.workunit.client.1.smithi062.stdout:9/267: write d4/d15/f24 [72098,60137] 0 2024-02-22T15:50:37.046 INFO:tasks.workunit.client.0.smithi019.stdout:4/508: fdatasync d2/db/f26 0 2024-02-22T15:50:37.047 INFO:tasks.workunit.client.1.smithi062.stdout:0/374: chown d1/d3/d4/c8 3003 1 2024-02-22T15:50:37.049 INFO:tasks.workunit.client.1.smithi062.stdout:6/395: dread d5/df/d40/d44/d4a/d41/f72 [0,4194304] 0 2024-02-22T15:50:37.049 INFO:tasks.workunit.client.1.smithi062.stdout:8/292: write d8/d9/f2e [6635007,42922] 0 2024-02-22T15:50:37.050 INFO:tasks.workunit.client.0.smithi019.stdout:1/577: dwrite d8/d2a/fbf [0,4194304] 0 2024-02-22T15:50:37.051 INFO:tasks.workunit.client.1.smithi062.stdout:7/278: unlink d0/d14/d13/f44 0 2024-02-22T15:50:37.052 INFO:tasks.workunit.client.0.smithi019.stdout:6/550: dwrite d6/f28 [0,4194304] 0 2024-02-22T15:50:37.053 INFO:tasks.workunit.client.0.smithi019.stdout:9/538: mkdir d2/d1f/d2e/d57/d60/dc1 0 2024-02-22T15:50:37.053 INFO:tasks.workunit.client.0.smithi019.stdout:4/509: write d2/db/f25 [561778,82974] 0 2024-02-22T15:50:37.054 INFO:tasks.workunit.client.0.smithi019.stdout:3/414: dread d9/d26/d2d/d53/f56 [0,4194304] 0 2024-02-22T15:50:37.056 INFO:tasks.workunit.client.1.smithi062.stdout:2/365: dwrite d1/d47/d39/d5d/f6d [0,4194304] 0 2024-02-22T15:50:37.056 INFO:tasks.workunit.client.1.smithi062.stdout:6/396: write d5/df/f21 [979096,49860] 0 2024-02-22T15:50:37.057 INFO:tasks.workunit.client.0.smithi019.stdout:1/578: write d8/d23/d33/f93 [1003683,81960] 0 2024-02-22T15:50:37.060 INFO:tasks.workunit.client.0.smithi019.stdout:1/579: read d8/d23/f55 [755107,51094] 0 2024-02-22T15:50:37.061 INFO:tasks.workunit.client.1.smithi062.stdout:8/293: truncate d8/d9/f2a 1728205 0 2024-02-22T15:50:37.062 INFO:tasks.workunit.client.1.smithi062.stdout:3/298: rename d5/d16/l23 to d5/d1a/l5d 0 2024-02-22T15:50:37.063 INFO:tasks.workunit.client.0.smithi019.stdout:3/415: creat d9/d14/f86 x:0 0 0 2024-02-22T15:50:37.065 INFO:tasks.workunit.client.1.smithi062.stdout:0/375: fdatasync d1/d21/f49 0 2024-02-22T15:50:37.065 INFO:tasks.workunit.client.1.smithi062.stdout:7/279: creat d0/d4f/f59 x:0 0 0 2024-02-22T15:50:37.068 INFO:tasks.workunit.client.0.smithi019.stdout:6/551: rename d6/d4b/f5b to d6/d31/d69/fa6 0 2024-02-22T15:50:37.069 INFO:tasks.workunit.client.1.smithi062.stdout:2/366: fsync d1/d28/f5c 0 2024-02-22T15:50:37.071 INFO:tasks.workunit.client.0.smithi019.stdout:0/424: dread f2 [0,4194304] 0 2024-02-22T15:50:37.072 INFO:tasks.workunit.client.1.smithi062.stdout:0/376: stat d1/d3/de/d3f/l40 0 2024-02-22T15:50:37.073 INFO:tasks.workunit.client.0.smithi019.stdout:3/416: creat d9/d26/d3f/d59/d66/f87 x:0 0 0 2024-02-22T15:50:37.074 INFO:tasks.workunit.client.0.smithi019.stdout:9/539: dread d2/d1f/d2e/d48/d45/d5a/f63 [0,4194304] 0 2024-02-22T15:50:37.074 INFO:tasks.workunit.client.1.smithi062.stdout:9/268: dwrite d4/d15/f44 [0,4194304] 0 2024-02-22T15:50:37.074 INFO:tasks.workunit.client.1.smithi062.stdout:8/294: dread d8/f40 [0,4194304] 0 2024-02-22T15:50:37.075 INFO:tasks.workunit.client.0.smithi019.stdout:1/580: dwrite d8/f53 [0,4194304] 0 2024-02-22T15:50:37.075 INFO:tasks.workunit.client.1.smithi062.stdout:6/397: symlink d5/dd/l81 0 2024-02-22T15:50:37.078 INFO:tasks.workunit.client.0.smithi019.stdout:0/425: mknod d4/d23/d7d/c97 0 2024-02-22T15:50:37.080 INFO:tasks.workunit.client.1.smithi062.stdout:2/367: rmdir d1/d8 39 2024-02-22T15:50:37.080 INFO:tasks.workunit.client.1.smithi062.stdout:6/398: write d5/dd/d33/f49 [5200243,94517] 0 2024-02-22T15:50:37.081 INFO:tasks.workunit.client.0.smithi019.stdout:4/510: creat d2/db/d14/d23/fa7 x:0 0 0 2024-02-22T15:50:37.083 INFO:tasks.workunit.client.1.smithi062.stdout:4/290: rename d4/d1c/d30/c5c to d4/d2a/c62 0 2024-02-22T15:50:37.085 INFO:tasks.workunit.client.1.smithi062.stdout:7/280: write d0/d14/d19/f1a [787506,96023] 0 2024-02-22T15:50:37.087 INFO:tasks.workunit.client.1.smithi062.stdout:1/287: dwrite d1/d6/d7/f28 [0,4194304] 0 2024-02-22T15:50:37.089 INFO:tasks.workunit.client.0.smithi019.stdout:1/581: symlink d8/d2a/d75/da0/lc6 0 2024-02-22T15:50:37.090 INFO:tasks.workunit.client.0.smithi019.stdout:8/522: truncate d9/d88/f8b 389766 0 2024-02-22T15:50:37.092 INFO:tasks.workunit.client.0.smithi019.stdout:4/511: truncate d2/db/f25 1441568 0 2024-02-22T15:50:37.094 INFO:tasks.workunit.client.0.smithi019.stdout:0/426: dwrite d4/d9/f26 [0,4194304] 0 2024-02-22T15:50:37.096 INFO:tasks.workunit.client.0.smithi019.stdout:9/540: dwrite d2/d7/dc/d27/d31/d79/fb2 [0,4194304] 0 2024-02-22T15:50:37.096 INFO:tasks.workunit.client.1.smithi062.stdout:3/299: dwrite d5/d1a/d3c/f54 [0,4194304] 0 2024-02-22T15:50:37.096 INFO:tasks.workunit.client.1.smithi062.stdout:0/377: fdatasync d1/d21/f49 0 2024-02-22T15:50:37.102 INFO:tasks.workunit.client.1.smithi062.stdout:5/156: write d0/f16 [158726,103663] 0 2024-02-22T15:50:37.103 INFO:tasks.workunit.client.1.smithi062.stdout:0/378: chown d1/d21/f49 230151022 1 2024-02-22T15:50:37.107 INFO:tasks.workunit.client.0.smithi019.stdout:1/582: mknod d8/d15/d5b/cc7 0 2024-02-22T15:50:37.108 INFO:tasks.workunit.client.0.smithi019.stdout:8/523: creat d9/d14/d6f/d7e/d94/d9c/fb2 x:0 0 0 2024-02-22T15:50:37.110 INFO:tasks.workunit.client.1.smithi062.stdout:6/399: truncate d5/df/d32/d47/f7a 773847 0 2024-02-22T15:50:37.113 INFO:tasks.workunit.client.0.smithi019.stdout:4/512: truncate d2/d3f/f51 963649 0 2024-02-22T15:50:37.114 INFO:tasks.workunit.client.0.smithi019.stdout:6/552: dwrite d6/d98/fa5 [0,4194304] 0 2024-02-22T15:50:37.115 INFO:tasks.workunit.client.1.smithi062.stdout:7/281: stat d0/d6/d2e/c56 0 2024-02-22T15:50:37.117 INFO:tasks.workunit.client.0.smithi019.stdout:9/541: rename d2/d7/dc/d27/dbc/c9c to d2/d1f/d2e/d57/cc2 0 2024-02-22T15:50:37.117 INFO:tasks.workunit.client.1.smithi062.stdout:5/157: write d0/f9 [1605759,105734] 0 2024-02-22T15:50:37.118 INFO:tasks.workunit.client.1.smithi062.stdout:1/288: fsync d1/d6/d7/f50 0 2024-02-22T15:50:37.121 INFO:tasks.workunit.client.1.smithi062.stdout:8/295: dwrite d8/f22 [0,4194304] 0 2024-02-22T15:50:37.121 INFO:tasks.workunit.client.0.smithi019.stdout:1/583: dread - d8/d2a/f5e zero size 2024-02-22T15:50:37.122 INFO:tasks.workunit.client.0.smithi019.stdout:1/584: chown d8/d44 436027 1 2024-02-22T15:50:37.123 INFO:tasks.workunit.client.1.smithi062.stdout:7/282: mkdir d0/d6/d2e/d4a/d5a 0 2024-02-22T15:50:37.125 INFO:tasks.workunit.client.0.smithi019.stdout:4/513: creat d2/d3f/d48/d76/fa8 x:0 0 0 2024-02-22T15:50:37.126 INFO:tasks.workunit.client.1.smithi062.stdout:3/300: dwrite d5/d1a/f25 [0,4194304] 0 2024-02-22T15:50:37.127 INFO:tasks.workunit.client.0.smithi019.stdout:0/427: dwrite d4/db/d1b/d63/f87 [0,4194304] 0 2024-02-22T15:50:37.127 INFO:tasks.workunit.client.0.smithi019.stdout:8/524: creat d9/d14/fb3 x:0 0 0 2024-02-22T15:50:37.128 INFO:tasks.workunit.client.0.smithi019.stdout:1/585: dread - d8/d2a/db3/fb1 zero size 2024-02-22T15:50:37.129 INFO:tasks.workunit.client.1.smithi062.stdout:2/368: dread d1/d8/f1a [0,4194304] 0 2024-02-22T15:50:37.130 INFO:tasks.workunit.client.0.smithi019.stdout:0/428: dread - d4/d22/d3d/f73 zero size 2024-02-22T15:50:37.130 INFO:tasks.workunit.client.1.smithi062.stdout:5/158: truncate d0/d3/da/d13/f1b 696040 0 2024-02-22T15:50:37.133 INFO:tasks.workunit.client.0.smithi019.stdout:6/553: dwrite d6/f57 [0,4194304] 0 2024-02-22T15:50:37.137 INFO:tasks.workunit.client.0.smithi019.stdout:6/554: read - d6/d4b/d71/f7d zero size 2024-02-22T15:50:37.138 INFO:tasks.workunit.client.1.smithi062.stdout:6/400: dwrite d5/df/d40/d44/d78/d59/f7e [0,4194304] 0 2024-02-22T15:50:37.139 INFO:tasks.workunit.client.1.smithi062.stdout:6/401: read - d5/f6 zero size 2024-02-22T15:50:37.139 INFO:tasks.workunit.client.0.smithi019.stdout:8/525: symlink d9/d14/d17/d1c/da0/lb4 0 2024-02-22T15:50:37.139 INFO:tasks.workunit.client.1.smithi062.stdout:1/289: rename d1/d6/d7/f3a to d1/d6/d36/f5d 0 2024-02-22T15:50:37.141 INFO:tasks.workunit.client.0.smithi019.stdout:0/429: chown d4/db/d1b/d40/d44/l6e 24667868 1 2024-02-22T15:50:37.142 INFO:tasks.workunit.client.0.smithi019.stdout:4/514: rename d2/db/d14/d23/d67/d8f/f90 to d2/db/d14/d23/d21/d42/fa9 0 2024-02-22T15:50:37.143 INFO:tasks.workunit.client.0.smithi019.stdout:3/417: dread d9/d14/f3c [0,4194304] 0 2024-02-22T15:50:37.144 INFO:tasks.workunit.client.0.smithi019.stdout:6/555: rename d6/d4b to d6/d4b/d6d/da3/da7 22 2024-02-22T15:50:37.145 INFO:tasks.workunit.client.0.smithi019.stdout:8/526: mkdir d9/d14/d1f/d6d/db5 0 2024-02-22T15:50:37.145 INFO:tasks.workunit.client.0.smithi019.stdout:0/430: write d4/d9/f26 [819774,38491] 0 2024-02-22T15:50:37.147 INFO:tasks.workunit.client.0.smithi019.stdout:1/586: dwrite d8/d18/d64/f66 [0,4194304] 0 2024-02-22T15:50:37.148 INFO:tasks.workunit.client.1.smithi062.stdout:2/369: creat d1/d8/f6e x:0 0 0 2024-02-22T15:50:37.149 INFO:tasks.workunit.client.1.smithi062.stdout:8/296: symlink d8/d42/l52 0 2024-02-22T15:50:37.151 INFO:tasks.workunit.client.0.smithi019.stdout:3/418: mknod d9/d26/d76/c88 0 2024-02-22T15:50:37.151 INFO:tasks.workunit.client.0.smithi019.stdout:0/431: read d4/d9/f26 [3688805,55749] 0 2024-02-22T15:50:37.154 INFO:tasks.workunit.client.0.smithi019.stdout:1/587: creat d8/d15/d34/fc8 x:0 0 0 2024-02-22T15:50:37.154 INFO:tasks.workunit.client.1.smithi062.stdout:2/370: write d1/d8/df/f31 [3770372,35132] 0 2024-02-22T15:50:37.154 INFO:tasks.workunit.client.1.smithi062.stdout:7/283: mkdir d0/d6/d5b 0 2024-02-22T15:50:37.156 INFO:tasks.workunit.client.1.smithi062.stdout:3/301: fsync d5/d1a/d3c/f54 0 2024-02-22T15:50:37.156 INFO:tasks.workunit.client.1.smithi062.stdout:9/269: dwrite d4/d35/f36 [0,4194304] 0 2024-02-22T15:50:37.158 INFO:tasks.workunit.client.1.smithi062.stdout:4/291: dwrite d4/d1c/d30/f32 [0,4194304] 0 2024-02-22T15:50:37.160 INFO:tasks.workunit.client.0.smithi019.stdout:4/515: read d2/d3f/d48/f3e [143797,71988] 0 2024-02-22T15:50:37.163 INFO:tasks.workunit.client.1.smithi062.stdout:3/302: fdatasync d5/d1a/f33 0 2024-02-22T15:50:37.163 INFO:tasks.workunit.client.1.smithi062.stdout:1/290: symlink d1/d6/de/l5e 0 2024-02-22T15:50:37.163 INFO:tasks.workunit.client.0.smithi019.stdout:0/432: dwrite d4/db/d1b/d63/f87 [0,4194304] 0 2024-02-22T15:50:37.164 INFO:tasks.workunit.client.0.smithi019.stdout:4/516: stat d2/db/d14/d23/d21/d35/l6c 0 2024-02-22T15:50:37.167 INFO:tasks.workunit.client.1.smithi062.stdout:7/284: truncate d0/d14/d13/f46 34928 0 2024-02-22T15:50:37.168 INFO:tasks.workunit.client.0.smithi019.stdout:3/419: dread d9/d26/d32/f33 [4194304,4194304] 0 2024-02-22T15:50:37.169 INFO:tasks.workunit.client.1.smithi062.stdout:7/285: fsync d0/d14/d13/f46 0 2024-02-22T15:50:37.169 INFO:tasks.workunit.client.1.smithi062.stdout:3/303: write d5/d26/f38 [398428,75161] 0 2024-02-22T15:50:37.172 INFO:tasks.workunit.client.1.smithi062.stdout:3/304: read d5/d16/d21/f34 [837279,29698] 0 2024-02-22T15:50:37.173 INFO:tasks.workunit.client.0.smithi019.stdout:3/420: readlink d9/d26/d2d/d39/l51 0 2024-02-22T15:50:37.173 INFO:tasks.workunit.client.0.smithi019.stdout:4/517: chown d2/db/d14/d23/d21/c50 8215 1 2024-02-22T15:50:37.176 INFO:tasks.workunit.client.1.smithi062.stdout:8/297: rename d8/d9/f1b to d8/f53 0 2024-02-22T15:50:37.177 INFO:tasks.workunit.client.1.smithi062.stdout:1/291: mknod d1/d6/c5f 0 2024-02-22T15:50:37.179 INFO:tasks.workunit.client.1.smithi062.stdout:1/292: fsync d1/d6/d7/f28 0 2024-02-22T15:50:37.180 INFO:tasks.workunit.client.0.smithi019.stdout:4/518: mknod d2/db/d14/d23/caa 0 2024-02-22T15:50:37.180 INFO:tasks.workunit.client.0.smithi019.stdout:0/433: dwrite d4/d23/f4a [0,4194304] 0 2024-02-22T15:50:37.183 INFO:tasks.workunit.client.1.smithi062.stdout:4/292: rename d4/d1c/d2c/l38 to d4/df/l63 0 2024-02-22T15:50:37.184 INFO:tasks.workunit.client.1.smithi062.stdout:5/159: dread d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:37.185 INFO:tasks.workunit.client.1.smithi062.stdout:2/371: getdents d1/d8/d62 0 2024-02-22T15:50:37.187 INFO:tasks.workunit.client.1.smithi062.stdout:0/379: dwrite d1/d3/de/f3a [4194304,4194304] 0 2024-02-22T15:50:37.187 INFO:tasks.workunit.client.1.smithi062.stdout:7/286: creat d0/d6/d2e/d58/f5c x:0 0 0 2024-02-22T15:50:37.188 INFO:tasks.workunit.client.1.smithi062.stdout:5/160: stat d0/d3/da/f27 0 2024-02-22T15:50:37.189 INFO:tasks.workunit.client.0.smithi019.stdout:0/434: mkdir d4/d22/d2c/d68/d98 0 2024-02-22T15:50:37.189 INFO:tasks.workunit.client.0.smithi019.stdout:3/421: dwrite d9/d26/d32/f33 [8388608,4194304] 0 2024-02-22T15:50:37.190 INFO:tasks.workunit.client.1.smithi062.stdout:3/305: mkdir d5/d16/d21/d55/d5e 0 2024-02-22T15:50:37.190 INFO:tasks.workunit.client.1.smithi062.stdout:0/380: dread - d1/dd/f6c zero size 2024-02-22T15:50:37.190 INFO:tasks.workunit.client.1.smithi062.stdout:8/298: mkdir d8/d9/d44/d4d/d54 0 2024-02-22T15:50:37.191 INFO:tasks.workunit.client.0.smithi019.stdout:3/422: fsync d9/d22/d27/f7a 0 2024-02-22T15:50:37.193 INFO:tasks.workunit.client.0.smithi019.stdout:0/435: mkdir d4/d5/dc/d99 0 2024-02-22T15:50:37.197 INFO:tasks.workunit.client.1.smithi062.stdout:9/270: rename d4/f2a to d4/d15/d31/f4a 0 2024-02-22T15:50:37.197 INFO:tasks.workunit.client.1.smithi062.stdout:0/381: read d1/f45 [635351,45238] 0 2024-02-22T15:50:37.197 INFO:tasks.workunit.client.0.smithi019.stdout:0/436: creat d4/d22/d2c/f9a x:0 0 0 2024-02-22T15:50:37.200 INFO:tasks.workunit.client.0.smithi019.stdout:3/423: dwrite d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:37.201 INFO:tasks.workunit.client.0.smithi019.stdout:0/437: mknod d4/d22/c9b 0 2024-02-22T15:50:37.205 INFO:tasks.workunit.client.0.smithi019.stdout:8/527: write d9/d14/d1f/f21 [1547002,102632] 0 2024-02-22T15:50:37.206 INFO:tasks.workunit.client.1.smithi062.stdout:6/402: truncate d5/df/d40/d44/d78/f31 1055105 0 2024-02-22T15:50:37.207 INFO:tasks.workunit.client.0.smithi019.stdout:0/438: rename d4/d22/d3d/f50 to d4/d32/f9c 0 2024-02-22T15:50:37.209 INFO:tasks.workunit.client.0.smithi019.stdout:6/556: truncate d6/d31/f32 3360789 0 2024-02-22T15:50:37.209 INFO:tasks.workunit.client.1.smithi062.stdout:8/299: readlink d8/d9/l15 0 2024-02-22T15:50:37.209 INFO:tasks.workunit.client.1.smithi062.stdout:1/293: mknod d1/d6/de/c60 0 2024-02-22T15:50:37.211 INFO:tasks.workunit.client.0.smithi019.stdout:0/439: mknod d4/d5/dc/d41/c9d 0 2024-02-22T15:50:37.212 INFO:tasks.workunit.client.0.smithi019.stdout:6/557: fdatasync d6/f2d 0 2024-02-22T15:50:37.213 INFO:tasks.workunit.client.0.smithi019.stdout:0/440: fdatasync d4/f3e 0 2024-02-22T15:50:37.215 INFO:tasks.workunit.client.1.smithi062.stdout:2/372: read d1/d47/f2b [5415358,59932] 0 2024-02-22T15:50:37.215 INFO:tasks.workunit.client.0.smithi019.stdout:8/528: link d9/d14/d17/f28 d9/d14/d6f/d7e/d94/daa/fb6 0 2024-02-22T15:50:37.216 INFO:tasks.workunit.client.0.smithi019.stdout:6/558: fdatasync d6/f1d 0 2024-02-22T15:50:37.217 INFO:tasks.workunit.client.0.smithi019.stdout:8/529: fsync d9/f64 0 2024-02-22T15:50:37.218 INFO:tasks.workunit.client.0.smithi019.stdout:1/588: dread d8/d15/d34/f71 [0,4194304] 0 2024-02-22T15:50:37.218 INFO:tasks.workunit.client.1.smithi062.stdout:5/161: dread d0/d3/da/d13/f1b [0,4194304] 0 2024-02-22T15:50:37.219 INFO:tasks.workunit.client.1.smithi062.stdout:4/293: mkdir d4/d2a/d2d/d64 0 2024-02-22T15:50:37.219 INFO:tasks.workunit.client.0.smithi019.stdout:4/519: write d2/d3f/d48/f3e [297181,123940] 0 2024-02-22T15:50:37.220 INFO:tasks.workunit.client.1.smithi062.stdout:6/403: dwrite d5/df/d40/d44/d4a/f5e [0,4194304] 0 2024-02-22T15:50:37.220 INFO:tasks.workunit.client.0.smithi019.stdout:6/559: creat d6/d4b/d6d/fa8 x:0 0 0 2024-02-22T15:50:37.221 INFO:tasks.workunit.client.1.smithi062.stdout:0/382: dwrite d1/d3/de/d3f/f4c [0,4194304] 0 2024-02-22T15:50:37.222 INFO:tasks.workunit.client.0.smithi019.stdout:1/589: chown d8/f14 31892515 1 2024-02-22T15:50:37.225 INFO:tasks.workunit.client.0.smithi019.stdout:8/530: dread d9/d88/f8b [0,4194304] 0 2024-02-22T15:50:37.225 INFO:tasks.workunit.client.0.smithi019.stdout:6/560: rename d6/d31/d43/d55 to d6/d4b/da9 0 2024-02-22T15:50:37.226 INFO:tasks.workunit.client.0.smithi019.stdout:8/531: truncate d9/d14/d1f/d5c/d99/fa1 826933 0 2024-02-22T15:50:37.227 INFO:tasks.workunit.client.0.smithi019.stdout:1/590: mkdir d8/d44/dc9 0 2024-02-22T15:50:37.228 INFO:tasks.workunit.client.0.smithi019.stdout:8/532: write d9/d14/d6f/d7e/d94/f9b [549454,72704] 0 2024-02-22T15:50:37.229 INFO:tasks.workunit.client.1.smithi062.stdout:8/300: read d8/f3a [143657,87618] 0 2024-02-22T15:50:37.229 INFO:tasks.workunit.client.1.smithi062.stdout:3/306: truncate d5/d16/d21/f34 67063 0 2024-02-22T15:50:37.229 INFO:tasks.workunit.client.1.smithi062.stdout:2/373: rmdir d1/d8/df 39 2024-02-22T15:50:37.229 INFO:tasks.workunit.client.1.smithi062.stdout:0/383: write d1/d38/f7b [146746,1637] 0 2024-02-22T15:50:37.231 INFO:tasks.workunit.client.0.smithi019.stdout:1/591: symlink d8/d2a/lca 0 2024-02-22T15:50:37.232 INFO:tasks.workunit.client.0.smithi019.stdout:1/592: chown d8/d18 444823 1 2024-02-22T15:50:37.233 INFO:tasks.workunit.client.1.smithi062.stdout:5/162: fdatasync d0/d3/da/fb 0 2024-02-22T15:50:37.233 INFO:tasks.workunit.client.1.smithi062.stdout:0/384: write d1/d3/de/f3a [3283727,19636] 0 2024-02-22T15:50:37.233 INFO:tasks.workunit.client.1.smithi062.stdout:9/271: symlink d4/d1c/d48/l4b 0 2024-02-22T15:50:37.233 INFO:tasks.workunit.client.1.smithi062.stdout:6/404: write d5/df/d40/d44/d78/f70 [571014,11322] 0 2024-02-22T15:50:37.233 INFO:tasks.workunit.client.0.smithi019.stdout:8/533: creat d9/d14/d6f/d7e/d94/d9a/fb7 x:0 0 0 2024-02-22T15:50:37.234 INFO:tasks.workunit.client.0.smithi019.stdout:1/593: rmdir d8/d18/d72 39 2024-02-22T15:50:37.235 INFO:tasks.workunit.client.1.smithi062.stdout:0/385: dread - d1/d13/f31 zero size 2024-02-22T15:50:37.236 INFO:tasks.workunit.client.0.smithi019.stdout:6/561: dwrite d6/d31/f3b [0,4194304] 0 2024-02-22T15:50:37.238 INFO:tasks.workunit.client.1.smithi062.stdout:1/294: creat d1/d6/f61 x:0 0 0 2024-02-22T15:50:37.240 INFO:tasks.workunit.client.1.smithi062.stdout:7/287: getdents d0/d6/d2e/d4a 0 2024-02-22T15:50:37.241 INFO:tasks.workunit.client.1.smithi062.stdout:5/163: creat d0/d3/da/d13/f2a x:0 0 0 2024-02-22T15:50:37.241 INFO:tasks.workunit.client.1.smithi062.stdout:8/301: mkdir d8/d9/d44/d4d/d51/d55 0 2024-02-22T15:50:37.241 INFO:tasks.workunit.client.1.smithi062.stdout:6/405: dread - d5/f6c zero size 2024-02-22T15:50:37.242 INFO:tasks.workunit.client.0.smithi019.stdout:8/534: rename d9/d14/d1f/d5c/d99 to d9/d14/d6f/d7e/d94/db8 0 2024-02-22T15:50:37.242 INFO:tasks.workunit.client.1.smithi062.stdout:3/307: creat d5/d16/d21/d55/d5e/f5f x:0 0 0 2024-02-22T15:50:37.243 INFO:tasks.workunit.client.1.smithi062.stdout:2/374: rmdir d1/d28 39 2024-02-22T15:50:37.247 INFO:tasks.workunit.client.1.smithi062.stdout:8/302: fdatasync d8/d9/f27 0 2024-02-22T15:50:37.248 INFO:tasks.workunit.client.0.smithi019.stdout:6/562: mknod d6/d31/d43/caa 0 2024-02-22T15:50:37.248 INFO:tasks.workunit.client.0.smithi019.stdout:3/424: write d9/d26/d2d/f4b [130005,108147] 0 2024-02-22T15:50:37.248 INFO:tasks.workunit.client.1.smithi062.stdout:1/295: write d1/d6/de/d9/fd [133353,80471] 0 2024-02-22T15:50:37.249 INFO:tasks.workunit.client.1.smithi062.stdout:4/294: dwrite d4/f39 [0,4194304] 0 2024-02-22T15:50:37.249 INFO:tasks.workunit.client.1.smithi062.stdout:6/406: chown d5/df/d40/d44/d78/l7b 1989366209 1 2024-02-22T15:50:37.252 INFO:tasks.workunit.client.1.smithi062.stdout:5/164: mknod d0/d3/c2b 0 2024-02-22T15:50:37.254 INFO:tasks.workunit.client.1.smithi062.stdout:6/407: write d5/fb [675694,74703] 0 2024-02-22T15:50:37.255 INFO:tasks.workunit.client.0.smithi019.stdout:1/594: rmdir d8/d44/dc9 0 2024-02-22T15:50:37.257 INFO:tasks.workunit.client.1.smithi062.stdout:3/308: dread d5/d1a/f33 [0,4194304] 0 2024-02-22T15:50:37.259 INFO:tasks.workunit.client.0.smithi019.stdout:6/563: creat d6/d31/d69/d8e/fab x:0 0 0 2024-02-22T15:50:37.259 INFO:tasks.workunit.client.1.smithi062.stdout:8/303: chown d8/d9/f1f 573638 1 2024-02-22T15:50:37.261 INFO:tasks.workunit.client.1.smithi062.stdout:2/375: unlink d1/d28/d3f/d65/f6c 0 2024-02-22T15:50:37.261 INFO:tasks.workunit.client.0.smithi019.stdout:3/425: dwrite d9/d22/d27/f7a [0,4194304] 0 2024-02-22T15:50:37.262 INFO:tasks.workunit.client.0.smithi019.stdout:3/426: chown f6 6 1 2024-02-22T15:50:37.263 INFO:tasks.workunit.client.1.smithi062.stdout:1/296: read d1/d6/d10/f1c [1678998,84599] 0 2024-02-22T15:50:37.263 INFO:tasks.workunit.client.1.smithi062.stdout:4/295: creat d4/d1c/d30/f65 x:0 0 0 2024-02-22T15:50:37.263 INFO:tasks.workunit.client.1.smithi062.stdout:9/272: read d4/d35/d3f/f25 [40345,47051] 0 2024-02-22T15:50:37.264 INFO:tasks.workunit.client.1.smithi062.stdout:5/165: write d0/f9 [8801134,76431] 0 2024-02-22T15:50:37.266 INFO:tasks.workunit.client.1.smithi062.stdout:8/304: creat d8/d9/d44/f56 x:0 0 0 2024-02-22T15:50:37.267 INFO:tasks.workunit.client.0.smithi019.stdout:6/564: chown d6/c1f 6452 1 2024-02-22T15:50:37.267 INFO:tasks.workunit.client.0.smithi019.stdout:4/520: write d2/db/d14/d23/d21/d35/f38 [2857423,130700] 0 2024-02-22T15:50:37.268 INFO:tasks.workunit.client.1.smithi062.stdout:5/166: chown d0/d3/da/d13/f1b 11021 1 2024-02-22T15:50:37.270 INFO:tasks.workunit.client.0.smithi019.stdout:3/427: creat d9/d65/f89 x:0 0 0 2024-02-22T15:50:37.270 INFO:tasks.workunit.client.1.smithi062.stdout:0/386: rename d1/d3/de/d14 to d1/d3/de/d3f/d2c/d84 0 2024-02-22T15:50:37.271 INFO:tasks.workunit.client.0.smithi019.stdout:1/595: creat d8/d18/fcb x:0 0 0 2024-02-22T15:50:37.276 INFO:tasks.workunit.client.0.smithi019.stdout:3/428: truncate f6 3678057 0 2024-02-22T15:50:37.277 INFO:tasks.workunit.client.0.smithi019.stdout:0/441: dwrite d4/db/d1b/d40/f89 [0,4194304] 0 2024-02-22T15:50:37.277 INFO:tasks.workunit.client.1.smithi062.stdout:4/296: mkdir d4/d66 0 2024-02-22T15:50:37.279 INFO:tasks.workunit.client.0.smithi019.stdout:3/429: write d9/d26/d3f/d59/d66/f7f [866108,100590] 0 2024-02-22T15:50:37.279 INFO:tasks.workunit.client.0.smithi019.stdout:1/596: chown d8/c10 129 1 2024-02-22T15:50:37.279 INFO:tasks.workunit.client.0.smithi019.stdout:6/565: link d6/d31/d48/l73 d6/d4b/d71/d88/lac 0 2024-02-22T15:50:37.280 INFO:tasks.workunit.client.1.smithi062.stdout:9/273: dwrite d4/d28/f38 [4194304,4194304] 0 2024-02-22T15:50:37.280 INFO:tasks.workunit.client.1.smithi062.stdout:5/167: write d0/d3/da/d13/d23/f24 [1372520,54204] 0 2024-02-22T15:50:37.280 INFO:tasks.workunit.client.0.smithi019.stdout:3/430: fsync d9/d22/d27/d6d/f82 0 2024-02-22T15:50:37.281 INFO:tasks.workunit.client.0.smithi019.stdout:1/597: fsync d8/d44/d5d/fc4 0 2024-02-22T15:50:37.283 INFO:tasks.workunit.client.1.smithi062.stdout:8/305: dwrite d8/d9/fe [0,4194304] 0 2024-02-22T15:50:37.284 INFO:tasks.workunit.client.1.smithi062.stdout:0/387: read d1/d3/de/d3f/d5d/f37 [3587827,7759] 0 2024-02-22T15:50:37.285 INFO:tasks.workunit.client.0.smithi019.stdout:4/521: dread d2/db/d14/d23/d33/f7d [0,4194304] 0 2024-02-22T15:50:37.286 INFO:tasks.workunit.client.1.smithi062.stdout:1/297: unlink d1/d6/d7/d2b/d47/f33 0 2024-02-22T15:50:37.286 INFO:tasks.workunit.client.0.smithi019.stdout:8/535: truncate d9/d14/d1f/d4c/f7f 533752 0 2024-02-22T15:50:37.287 INFO:tasks.workunit.client.0.smithi019.stdout:4/522: dread - d2/db/d14/d61/f73 zero size 2024-02-22T15:50:37.288 INFO:tasks.workunit.client.1.smithi062.stdout:0/388: write d1/d3/de/d3f/f4c [1052343,16238] 0 2024-02-22T15:50:37.288 INFO:tasks.workunit.client.1.smithi062.stdout:6/408: rename d5/dd/fe to d5/df/d40/d44/f82 0 2024-02-22T15:50:37.289 INFO:tasks.workunit.client.0.smithi019.stdout:6/566: mknod d6/d4b/da9/cad 0 2024-02-22T15:50:37.290 INFO:tasks.workunit.client.0.smithi019.stdout:1/598: mknod d8/d44/ccc 0 2024-02-22T15:50:37.291 INFO:tasks.workunit.client.1.smithi062.stdout:1/298: truncate d1/d6/d7/d2b/d49/f51 468732 0 2024-02-22T15:50:37.291 INFO:tasks.workunit.client.1.smithi062.stdout:2/376: mknod d1/d8/df/c6f 0 2024-02-22T15:50:37.292 INFO:tasks.workunit.client.1.smithi062.stdout:6/409: read - d5/df/d40/d44/d4a/d41/f4d zero size 2024-02-22T15:50:37.292 INFO:tasks.workunit.client.1.smithi062.stdout:2/377: chown d1/d8/df/l45 1001379 1 2024-02-22T15:50:37.293 INFO:tasks.workunit.client.0.smithi019.stdout:6/567: chown d6/f28 3792 1 2024-02-22T15:50:37.294 INFO:tasks.workunit.client.0.smithi019.stdout:4/523: truncate d2/db/f25 2467465 0 2024-02-22T15:50:37.297 INFO:tasks.workunit.client.1.smithi062.stdout:9/274: chown d4/l18 572 1 2024-02-22T15:50:37.297 INFO:tasks.workunit.client.1.smithi062.stdout:9/275: stat d4/d28/c2e 0 2024-02-22T15:50:37.299 INFO:tasks.workunit.client.1.smithi062.stdout:6/410: dread d5/df/d40/d44/d78/f70 [0,4194304] 0 2024-02-22T15:50:37.299 INFO:tasks.workunit.client.1.smithi062.stdout:7/288: getdents d0/d6/d2e/d58 0 2024-02-22T15:50:37.300 INFO:tasks.workunit.client.0.smithi019.stdout:1/599: stat d8/d18/d72/lb9 0 2024-02-22T15:50:37.300 INFO:tasks.workunit.client.0.smithi019.stdout:6/568: write d6/d4b/d6d/d7b/f92 [377371,73259] 0 2024-02-22T15:50:37.301 INFO:tasks.workunit.client.1.smithi062.stdout:6/411: chown d5/df/d40/d44/d4a/d41/d43 2046 1 2024-02-22T15:50:37.302 INFO:tasks.workunit.client.1.smithi062.stdout:7/289: dread d0/d14/d13/f2a [0,4194304] 0 2024-02-22T15:50:37.304 INFO:tasks.workunit.client.0.smithi019.stdout:0/442: dwrite d4/f92 [0,4194304] 0 2024-02-22T15:50:37.307 INFO:tasks.workunit.client.0.smithi019.stdout:6/569: symlink d6/d4b/d71/lae 0 2024-02-22T15:50:37.311 INFO:tasks.workunit.client.0.smithi019.stdout:8/536: dwrite d9/d14/d1f/d6d/f8f [0,4194304] 0 2024-02-22T15:50:37.312 INFO:tasks.workunit.client.1.smithi062.stdout:1/299: fdatasync d1/f32 0 2024-02-22T15:50:37.312 INFO:tasks.workunit.client.1.smithi062.stdout:0/389: mkdir d1/d21/d85 0 2024-02-22T15:50:37.312 INFO:tasks.workunit.client.1.smithi062.stdout:3/309: getdents d5/d26 0 2024-02-22T15:50:37.314 INFO:tasks.workunit.client.0.smithi019.stdout:4/524: dread d2/db/d44/f55 [0,4194304] 0 2024-02-22T15:50:37.314 INFO:tasks.workunit.client.0.smithi019.stdout:0/443: dread d4/d23/f4a [0,4194304] 0 2024-02-22T15:50:37.317 INFO:tasks.workunit.client.1.smithi062.stdout:8/306: creat d8/d9/d48/f57 x:0 0 0 2024-02-22T15:50:37.318 INFO:tasks.workunit.client.0.smithi019.stdout:4/525: stat d2/db/d44/l5c 0 2024-02-22T15:50:37.320 INFO:tasks.workunit.client.1.smithi062.stdout:8/307: dread - d8/f20 zero size 2024-02-22T15:50:37.320 INFO:tasks.workunit.client.0.smithi019.stdout:8/537: truncate d9/d14/d1f/fa5 1277199 0 2024-02-22T15:50:37.321 INFO:tasks.workunit.client.0.smithi019.stdout:8/538: stat f5 0 2024-02-22T15:50:37.321 INFO:tasks.workunit.client.1.smithi062.stdout:6/412: symlink d5/df/d40/d44/d78/d59/l83 0 2024-02-22T15:50:37.323 INFO:tasks.workunit.client.1.smithi062.stdout:3/310: dwrite d5/d26/f31 [4194304,4194304] 0 2024-02-22T15:50:37.323 INFO:tasks.workunit.client.1.smithi062.stdout:7/290: mkdir d0/d6/d2e/d58/d5d 0 2024-02-22T15:50:37.323 INFO:tasks.workunit.client.0.smithi019.stdout:0/444: rmdir d4/d23/d7d 39 2024-02-22T15:50:37.327 INFO:tasks.workunit.client.0.smithi019.stdout:0/445: mknod d4/d67/c9e 0 2024-02-22T15:50:37.327 INFO:tasks.workunit.client.1.smithi062.stdout:0/390: symlink d1/d21/l86 0 2024-02-22T15:50:37.329 INFO:tasks.workunit.client.1.smithi062.stdout:2/378: mknod d1/c70 0 2024-02-22T15:50:37.334 INFO:tasks.workunit.client.0.smithi019.stdout:0/446: rmdir d4/d22 39 2024-02-22T15:50:37.335 INFO:tasks.workunit.client.1.smithi062.stdout:0/391: dwrite d1/d3/de/d3f/f62 [0,4194304] 0 2024-02-22T15:50:37.336 INFO:tasks.workunit.client.0.smithi019.stdout:8/539: dwrite d9/d14/d17/f70 [0,4194304] 0 2024-02-22T15:50:37.339 INFO:tasks.workunit.client.0.smithi019.stdout:4/526: write d2/db/d14/d23/f4d [341512,58481] 0 2024-02-22T15:50:37.339 INFO:tasks.workunit.client.0.smithi019.stdout:8/540: chown d9/d14/f19 1924 1 2024-02-22T15:50:37.340 INFO:tasks.workunit.client.0.smithi019.stdout:6/570: dread d6/f10 [0,4194304] 0 2024-02-22T15:50:37.342 INFO:tasks.workunit.client.0.smithi019.stdout:0/447: rename d4/d5/lf to d4/db/d1b/d40/d44/d75/l9f 0 2024-02-22T15:50:37.342 INFO:tasks.workunit.client.0.smithi019.stdout:6/571: write d6/f2c [2530669,131022] 0 2024-02-22T15:50:37.348 INFO:tasks.workunit.client.0.smithi019.stdout:4/527: write d2/db/d14/f18 [1145054,54657] 0 2024-02-22T15:50:37.353 INFO:tasks.workunit.client.0.smithi019.stdout:8/541: dread d9/d14/d1f/f21 [0,4194304] 0 2024-02-22T15:50:37.357 INFO:tasks.workunit.client.0.smithi019.stdout:8/542: symlink d9/d14/d1f/d5c/lb9 0 2024-02-22T15:50:37.362 INFO:tasks.workunit.client.0.smithi019.stdout:0/448: dwrite d4/d22/d2c/d68/f6a [0,4194304] 0 2024-02-22T15:50:37.363 INFO:tasks.workunit.client.0.smithi019.stdout:8/543: write d9/d14/d1f/f21 [172320,22213] 0 2024-02-22T15:50:37.365 INFO:tasks.workunit.client.0.smithi019.stdout:1/600: dread d8/d2a/f3f [0,4194304] 0 2024-02-22T15:50:37.366 INFO:tasks.workunit.client.0.smithi019.stdout:4/528: write d2/d3f/d48/f5b [190336,98647] 0 2024-02-22T15:50:37.367 INFO:tasks.workunit.client.0.smithi019.stdout:0/449: write d4/d32/f9c [1183347,72764] 0 2024-02-22T15:50:37.371 INFO:tasks.workunit.client.0.smithi019.stdout:0/450: write d4/f56 [4408690,101590] 0 2024-02-22T15:50:37.374 INFO:tasks.workunit.client.0.smithi019.stdout:1/601: symlink d8/d15/d5b/dbc/lcd 0 2024-02-22T15:50:37.376 INFO:tasks.workunit.client.0.smithi019.stdout:6/572: dwrite d6/f39 [4194304,4194304] 0 2024-02-22T15:50:37.376 INFO:tasks.workunit.client.0.smithi019.stdout:4/529: creat d2/db/d14/d23/d9e/fab x:0 0 0 2024-02-22T15:50:37.379 INFO:tasks.workunit.client.0.smithi019.stdout:8/544: getdents d9/d14/d17 0 2024-02-22T15:50:37.381 INFO:tasks.workunit.client.0.smithi019.stdout:7/690: sync 2024-02-22T15:50:37.382 INFO:tasks.workunit.client.0.smithi019.stdout:5/584: sync 2024-02-22T15:50:37.384 INFO:tasks.workunit.client.0.smithi019.stdout:4/530: mkdir d2/db/d14/d23/d9e/dac 0 2024-02-22T15:50:37.386 INFO:tasks.workunit.client.0.smithi019.stdout:8/545: unlink d9/d14/f53 0 2024-02-22T15:50:37.386 INFO:tasks.workunit.client.0.smithi019.stdout:6/573: rename d6/d4b/d6d/fa8 to d6/d4b/d6d/da3/faf 0 2024-02-22T15:50:37.387 INFO:tasks.workunit.client.0.smithi019.stdout:7/691: rmdir d9/d4c/d5b 39 2024-02-22T15:50:37.388 INFO:tasks.workunit.client.0.smithi019.stdout:1/602: dread d8/d15/d34/f65 [0,4194304] 0 2024-02-22T15:50:37.389 INFO:tasks.workunit.client.1.smithi062.stdout:8/308: rmdir d8/d9/d44 39 2024-02-22T15:50:37.394 INFO:tasks.workunit.client.1.smithi062.stdout:4/297: dread d4/f39 [0,4194304] 0 2024-02-22T15:50:37.396 INFO:tasks.workunit.client.0.smithi019.stdout:4/531: mknod d2/db/d14/d61/cad 0 2024-02-22T15:50:37.396 INFO:tasks.workunit.client.0.smithi019.stdout:8/546: unlink d9/d14/d17/d3d/f9f 0 2024-02-22T15:50:37.400 INFO:tasks.workunit.client.1.smithi062.stdout:0/392: mkdir d1/d3/de/d3f/d2c/d87 0 2024-02-22T15:50:37.401 INFO:tasks.workunit.client.1.smithi062.stdout:5/168: getdents d0/d3 0 2024-02-22T15:50:37.402 INFO:tasks.workunit.client.1.smithi062.stdout:9/276: truncate d4/d15/f44 3945230 0 2024-02-22T15:50:37.402 INFO:tasks.workunit.client.0.smithi019.stdout:6/574: unlink d6/d4b/da9/cad 0 2024-02-22T15:50:37.403 INFO:tasks.workunit.client.0.smithi019.stdout:5/585: mkdir d8/dd/d19/d3a/d46/d7e/dbe/dc3 0 2024-02-22T15:50:37.403 INFO:tasks.workunit.client.0.smithi019.stdout:4/532: dwrite d2/db/d14/f18 [0,4194304] 0 2024-02-22T15:50:37.405 INFO:tasks.workunit.client.0.smithi019.stdout:4/533: chown d2/db/d14/d23/d4b 1391 1 2024-02-22T15:50:37.407 INFO:tasks.workunit.client.1.smithi062.stdout:3/311: fdatasync d5/f6 0 2024-02-22T15:50:37.407 INFO:tasks.workunit.client.0.smithi019.stdout:4/534: chown d2/f6 314 1 2024-02-22T15:50:37.408 INFO:tasks.workunit.client.0.smithi019.stdout:7/692: rename d9/d26/d77/fc2 to d9/d38/d41/d45/fe7 0 2024-02-22T15:50:37.409 INFO:tasks.workunit.client.1.smithi062.stdout:4/298: dread d4/f51 [0,4194304] 0 2024-02-22T15:50:37.410 INFO:tasks.workunit.client.1.smithi062.stdout:1/300: write d1/d6/d14/f45 [1039648,70782] 0 2024-02-22T15:50:37.414 INFO:tasks.workunit.client.0.smithi019.stdout:6/575: rename d6/f22 to d6/d31/d43/fb0 0 2024-02-22T15:50:37.415 INFO:tasks.workunit.client.1.smithi062.stdout:0/393: dread - d1/d3/d5f/f57 zero size 2024-02-22T15:50:37.415 INFO:tasks.workunit.client.1.smithi062.stdout:5/169: fsync d0/d3/f10 0 2024-02-22T15:50:37.416 INFO:tasks.workunit.client.0.smithi019.stdout:4/535: mknod d2/db/d14/d23/d21/d35/d3d/cae 0 2024-02-22T15:50:37.417 INFO:tasks.workunit.client.1.smithi062.stdout:9/277: mknod d4/d1c/c4c 0 2024-02-22T15:50:37.418 INFO:tasks.workunit.client.0.smithi019.stdout:7/693: symlink d9/d3a/d49/d5d/d8f/le8 0 2024-02-22T15:50:37.420 INFO:tasks.workunit.client.1.smithi062.stdout:6/413: creat d5/df/f84 x:0 0 0 2024-02-22T15:50:37.421 INFO:tasks.workunit.client.0.smithi019.stdout:7/694: write d9/d38/d41/d45/f70 [1169144,24215] 0 2024-02-22T15:50:37.421 INFO:tasks.workunit.client.0.smithi019.stdout:4/536: unlink d2/db/d14/d23/d4b/fa1 0 2024-02-22T15:50:37.421 INFO:tasks.workunit.client.0.smithi019.stdout:6/576: dread d6/d4b/d6d/d7b/d78/f7e [0,4194304] 0 2024-02-22T15:50:37.422 INFO:tasks.workunit.client.1.smithi062.stdout:3/312: creat d5/d16/d21/f60 x:0 0 0 2024-02-22T15:50:37.423 INFO:tasks.workunit.client.1.smithi062.stdout:6/414: chown d5/df/d40/d44/d4a/l64 526061676 1 2024-02-22T15:50:37.423 INFO:tasks.workunit.client.1.smithi062.stdout:2/379: dwrite d1/d47/d39/f4b [0,4194304] 0 2024-02-22T15:50:37.424 INFO:tasks.workunit.client.0.smithi019.stdout:7/695: write d9/d26/f87 [2073951,18063] 0 2024-02-22T15:50:37.425 INFO:tasks.workunit.client.0.smithi019.stdout:6/577: mknod d6/d4b/cb1 0 2024-02-22T15:50:37.429 INFO:tasks.workunit.client.1.smithi062.stdout:1/301: symlink d1/d6/d7/l62 0 2024-02-22T15:50:37.432 INFO:tasks.workunit.client.0.smithi019.stdout:5/586: truncate d8/f23 3189203 0 2024-02-22T15:50:37.432 INFO:tasks.workunit.client.1.smithi062.stdout:8/309: dread d8/d9/f16 [0,4194304] 0 2024-02-22T15:50:37.433 INFO:tasks.workunit.client.1.smithi062.stdout:5/170: mknod d0/c2c 0 2024-02-22T15:50:37.435 INFO:tasks.workunit.client.0.smithi019.stdout:4/537: dwrite d2/db/d14/d23/f4d [0,4194304] 0 2024-02-22T15:50:37.438 INFO:tasks.workunit.client.1.smithi062.stdout:6/415: mkdir d5/df/d40/d44/d4a/d41/d85 0 2024-02-22T15:50:37.441 INFO:tasks.workunit.client.0.smithi019.stdout:4/538: symlink d2/d3f/laf 0 2024-02-22T15:50:37.441 INFO:tasks.workunit.client.1.smithi062.stdout:1/302: unlink d1/d6/d7/d2b/d47/f2f 0 2024-02-22T15:50:37.443 INFO:tasks.workunit.client.0.smithi019.stdout:5/587: link d8/dd/f67 d8/d11/d33/fc4 0 2024-02-22T15:50:37.444 INFO:tasks.workunit.client.0.smithi019.stdout:6/578: dwrite d6/d4b/d6d/da3/faf [0,4194304] 0 2024-02-22T15:50:37.444 INFO:tasks.workunit.client.0.smithi019.stdout:1/603: dwrite d8/d18/f21 [4194304,4194304] 0 2024-02-22T15:50:37.446 INFO:tasks.workunit.client.0.smithi019.stdout:7/696: dwrite d9/d38/d41/d45/d61/d6a/fdd [0,4194304] 0 2024-02-22T15:50:37.448 INFO:tasks.workunit.client.0.smithi019.stdout:6/579: fsync d6/d4b/da9/d67/f9a 0 2024-02-22T15:50:37.449 INFO:tasks.workunit.client.1.smithi062.stdout:6/416: fdatasync d5/df/d40/d44/d78/f5c 0 2024-02-22T15:50:37.449 INFO:tasks.workunit.client.0.smithi019.stdout:4/539: mknod d2/d3f/d48/d76/cb0 0 2024-02-22T15:50:37.450 INFO:tasks.workunit.client.1.smithi062.stdout:3/313: dwrite d5/f10 [0,4194304] 0 2024-02-22T15:50:37.451 INFO:tasks.workunit.client.1.smithi062.stdout:9/278: mknod d4/c4d 0 2024-02-22T15:50:37.454 INFO:tasks.workunit.client.0.smithi019.stdout:1/604: mknod d8/d2a/d75/da0/cce 0 2024-02-22T15:50:37.454 INFO:tasks.workunit.client.1.smithi062.stdout:1/303: mkdir d1/d6/d7/d2b/d47/d63 0 2024-02-22T15:50:37.455 INFO:tasks.workunit.client.0.smithi019.stdout:7/697: symlink d9/d38/d41/d45/db0/le9 0 2024-02-22T15:50:37.456 INFO:tasks.workunit.client.0.smithi019.stdout:7/698: fdatasync d9/d26/d77/f95 0 2024-02-22T15:50:37.457 INFO:tasks.workunit.client.1.smithi062.stdout:8/310: mkdir d8/d9/d44/d58 0 2024-02-22T15:50:37.458 INFO:tasks.workunit.client.1.smithi062.stdout:2/380: rename d1/d47/d32/c69 to d1/c71 0 2024-02-22T15:50:37.458 INFO:tasks.workunit.client.0.smithi019.stdout:7/699: fdatasync d9/d4c/dc9/fdc 0 2024-02-22T15:50:37.460 INFO:tasks.workunit.client.1.smithi062.stdout:1/304: chown d1/d6/d36/d37/l39 11194894 1 2024-02-22T15:50:37.460 INFO:tasks.workunit.client.0.smithi019.stdout:1/605: dread d8/d2a/f3f [0,4194304] 0 2024-02-22T15:50:37.461 INFO:tasks.workunit.client.1.smithi062.stdout:6/417: symlink d5/df/d40/d44/d4a/l86 0 2024-02-22T15:50:37.464 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:37 smithi062 ceph-mon[48130]: pgmap v34: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 190 MiB/s wr, 404 op/s 2024-02-22T15:50:37.471 INFO:tasks.workunit.client.1.smithi062.stdout:8/311: creat d8/d9/d2c/f59 x:0 0 0 2024-02-22T15:50:37.473 INFO:tasks.workunit.client.1.smithi062.stdout:6/418: write d5/df/d40/d44/d76/d56/f77 [846158,65227] 0 2024-02-22T15:50:37.476 INFO:tasks.workunit.client.1.smithi062.stdout:9/279: creat d4/d15/d31/f4e x:0 0 0 2024-02-22T15:50:37.479 INFO:tasks.workunit.client.1.smithi062.stdout:9/280: chown d4/d15/d3c 13 1 2024-02-22T15:50:37.479 INFO:tasks.workunit.client.1.smithi062.stdout:8/312: unlink d8/d9/f2e 0 2024-02-22T15:50:37.483 INFO:tasks.workunit.client.1.smithi062.stdout:3/314: getdents d5 0 2024-02-22T15:50:37.483 INFO:tasks.workunit.client.1.smithi062.stdout:8/313: read - d8/f39 zero size 2024-02-22T15:50:37.484 INFO:tasks.workunit.client.1.smithi062.stdout:0/394: dwrite d1/d3/f39 [0,4194304] 0 2024-02-22T15:50:37.491 INFO:tasks.workunit.client.1.smithi062.stdout:8/314: fdatasync d8/d9/d44/f56 0 2024-02-22T15:50:37.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:37 smithi019 ceph-mon[67418]: pgmap v34: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 190 MiB/s wr, 404 op/s 2024-02-22T15:50:37.496 INFO:tasks.workunit.client.0.smithi019.stdout:1/606: fdatasync d8/d18/d64/f89 0 2024-02-22T15:50:37.499 INFO:tasks.workunit.client.1.smithi062.stdout:0/395: dread - d1/d13/f31 zero size 2024-02-22T15:50:37.499 INFO:tasks.workunit.client.1.smithi062.stdout:2/381: dwrite d1/f2c [0,4194304] 0 2024-02-22T15:50:37.506 INFO:tasks.workunit.client.1.smithi062.stdout:1/305: write d1/d6/d10/f4b [990372,127708] 0 2024-02-22T15:50:37.506 INFO:tasks.workunit.client.1.smithi062.stdout:6/419: truncate d5/df/f21 2907765 0 2024-02-22T15:50:37.507 INFO:tasks.workunit.client.1.smithi062.stdout:3/315: dwrite d5/d16/f44 [0,4194304] 0 2024-02-22T15:50:37.510 INFO:tasks.workunit.client.0.smithi019.stdout:5/588: rename d8/dd/d8e/d2c/cb5 to d8/dd/d19/d77/d7a/cc5 0 2024-02-22T15:50:37.511 INFO:tasks.workunit.client.0.smithi019.stdout:6/580: link d6/d4b/cb1 d6/d4b/da9/d67/d8a/cb2 0 2024-02-22T15:50:37.511 INFO:tasks.workunit.client.1.smithi062.stdout:8/315: symlink d8/d9/d44/d4d/d51/d55/l5a 0 2024-02-22T15:50:37.512 INFO:tasks.workunit.client.0.smithi019.stdout:6/581: write d6/d4b/f52 [110010,60426] 0 2024-02-22T15:50:37.515 INFO:tasks.workunit.client.0.smithi019.stdout:4/540: getdents d2/db/d14/d23/d21/d42 0 2024-02-22T15:50:37.515 INFO:tasks.workunit.client.1.smithi062.stdout:5/171: dread d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:37.516 INFO:tasks.workunit.client.0.smithi019.stdout:4/541: fsync d2/db/d14/f81 0 2024-02-22T15:50:37.520 INFO:tasks.workunit.client.1.smithi062.stdout:2/382: write d1/d8/df/f31 [3268424,40568] 0 2024-02-22T15:50:37.521 INFO:tasks.workunit.client.0.smithi019.stdout:4/542: write d2/db/d92/f8a [1657366,42525] 0 2024-02-22T15:50:37.521 INFO:tasks.workunit.client.0.smithi019.stdout:6/582: mknod d6/d58/d6a/cb3 0 2024-02-22T15:50:37.521 INFO:tasks.workunit.client.0.smithi019.stdout:7/700: link d9/f16 d9/d38/d41/d45/db0/fea 0 2024-02-22T15:50:37.521 INFO:tasks.workunit.client.0.smithi019.stdout:7/701: truncate d9/d38/d41/d45/d61/fb1 914071 0 2024-02-22T15:50:37.522 INFO:tasks.workunit.client.1.smithi062.stdout:9/281: dwrite d4/f12 [4194304,4194304] 0 2024-02-22T15:50:37.523 INFO:tasks.workunit.client.1.smithi062.stdout:8/316: mknod d8/d9/d2c/c5b 0 2024-02-22T15:50:37.523 INFO:tasks.workunit.client.1.smithi062.stdout:3/316: chown d5/d16/d21/d36/c37 14 1 2024-02-22T15:50:37.527 INFO:tasks.workunit.client.1.smithi062.stdout:6/420: symlink d5/df/d40/d44/d76/d56/l87 0 2024-02-22T15:50:37.527 INFO:tasks.workunit.client.1.smithi062.stdout:5/172: creat d0/d3/da/d13/f2d x:0 0 0 2024-02-22T15:50:37.528 INFO:tasks.workunit.client.0.smithi019.stdout:9/542: sync 2024-02-22T15:50:37.528 INFO:tasks.workunit.client.0.smithi019.stdout:2/501: sync 2024-02-22T15:50:37.529 INFO:tasks.workunit.client.1.smithi062.stdout:2/383: creat d1/d47/f72 x:0 0 0 2024-02-22T15:50:37.531 INFO:tasks.workunit.client.1.smithi062.stdout:1/306: rename d1/d6/d36/d37/c4c to d1/c64 0 2024-02-22T15:50:37.531 INFO:tasks.workunit.client.1.smithi062.stdout:0/396: dwrite d1/d38/f7b [0,4194304] 0 2024-02-22T15:50:37.534 INFO:tasks.workunit.client.1.smithi062.stdout:5/173: write d0/f7 [1155779,128302] 0 2024-02-22T15:50:37.534 INFO:tasks.workunit.client.1.smithi062.stdout:3/317: creat d5/d26/f61 x:0 0 0 2024-02-22T15:50:37.534 INFO:tasks.workunit.client.0.smithi019.stdout:6/583: dwrite d6/d4b/da9/d67/f84 [0,4194304] 0 2024-02-22T15:50:37.535 INFO:tasks.workunit.client.0.smithi019.stdout:5/589: getdents d8/db1 0 2024-02-22T15:50:37.536 INFO:tasks.workunit.client.1.smithi062.stdout:9/282: dread d4/f10 [0,4194304] 0 2024-02-22T15:50:37.538 INFO:tasks.workunit.client.0.smithi019.stdout:6/584: truncate d6/d31/d69/d8e/fab 328198 0 2024-02-22T15:50:37.538 INFO:tasks.workunit.client.0.smithi019.stdout:5/590: write d8/dd/d19/d38/d47/f98 [544559,50581] 0 2024-02-22T15:50:37.539 INFO:tasks.workunit.client.1.smithi062.stdout:0/397: write d1/d3/d4/d1e/f79 [486720,51239] 0 2024-02-22T15:50:37.539 INFO:tasks.workunit.client.1.smithi062.stdout:2/384: write d1/f16 [1312360,79893] 0 2024-02-22T15:50:37.546 INFO:tasks.workunit.client.1.smithi062.stdout:2/385: dread - d1/d8/d62/f68 zero size 2024-02-22T15:50:37.548 INFO:tasks.workunit.client.0.smithi019.stdout:6/585: dwrite d6/d4b/f52 [0,4194304] 0 2024-02-22T15:50:37.548 INFO:tasks.workunit.client.1.smithi062.stdout:3/318: write d5/d16/d21/f22 [1142943,22002] 0 2024-02-22T15:50:37.551 INFO:tasks.workunit.client.0.smithi019.stdout:6/586: fsync d6/d4b/da9/d67/f9a 0 2024-02-22T15:50:37.554 INFO:tasks.workunit.client.1.smithi062.stdout:3/319: rmdir d5/d16 39 2024-02-22T15:50:37.555 INFO:tasks.workunit.client.0.smithi019.stdout:2/502: creat d1/d17/fb7 x:0 0 0 2024-02-22T15:50:37.555 INFO:tasks.workunit.client.1.smithi062.stdout:2/386: mkdir d1/d8/d62/d73 0 2024-02-22T15:50:37.556 INFO:tasks.workunit.client.0.smithi019.stdout:9/543: creat d2/d1f/d2e/d57/d60/d8c/fc3 x:0 0 0 2024-02-22T15:50:37.558 INFO:tasks.workunit.client.1.smithi062.stdout:9/283: dread d4/d35/f36 [0,4194304] 0 2024-02-22T15:50:37.559 INFO:tasks.workunit.client.0.smithi019.stdout:7/702: truncate d9/f16 1568683 0 2024-02-22T15:50:37.560 INFO:tasks.workunit.client.0.smithi019.stdout:1/607: dread d8/d18/d64/f83 [0,4194304] 0 2024-02-22T15:50:37.563 INFO:tasks.workunit.client.0.smithi019.stdout:2/503: symlink d1/d17/lb8 0 2024-02-22T15:50:37.565 INFO:tasks.workunit.client.0.smithi019.stdout:6/587: dwrite d6/d58/d6a/f85 [0,4194304] 0 2024-02-22T15:50:37.565 INFO:tasks.workunit.client.1.smithi062.stdout:9/284: dread d4/d35/d3f/f25 [0,4194304] 0 2024-02-22T15:50:37.566 INFO:tasks.workunit.client.1.smithi062.stdout:2/387: creat d1/d1e/d38/f74 x:0 0 0 2024-02-22T15:50:37.567 INFO:tasks.workunit.client.0.smithi019.stdout:1/608: readlink d8/d15/l3d 0 2024-02-22T15:50:37.568 INFO:tasks.workunit.client.1.smithi062.stdout:8/317: dread d8/d9/d2c/f30 [0,4194304] 0 2024-02-22T15:50:37.568 INFO:tasks.workunit.client.1.smithi062.stdout:2/388: chown d1/d8/df/l4d 17825346 1 2024-02-22T15:50:37.569 INFO:tasks.workunit.client.1.smithi062.stdout:5/174: dread d0/d3/ff [0,4194304] 0 2024-02-22T15:50:37.569 INFO:tasks.workunit.client.1.smithi062.stdout:3/320: mkdir d5/d1a/d3c/d62 0 2024-02-22T15:50:37.571 INFO:tasks.workunit.client.0.smithi019.stdout:2/504: rmdir d1/d17/d21/d51 39 2024-02-22T15:50:37.572 INFO:tasks.workunit.client.1.smithi062.stdout:9/285: write d4/d35/f36 [2152856,83096] 0 2024-02-22T15:50:37.573 INFO:tasks.workunit.client.0.smithi019.stdout:7/703: mkdir d9/d3a/d49/d5d/d74/d9a/de3/deb 0 2024-02-22T15:50:37.573 INFO:tasks.workunit.client.1.smithi062.stdout:5/175: fsync d0/d3/da/f18 0 2024-02-22T15:50:37.573 INFO:tasks.workunit.client.1.smithi062.stdout:1/307: dread d1/d6/d10/f4b [0,4194304] 0 2024-02-22T15:50:37.576 INFO:tasks.workunit.client.0.smithi019.stdout:1/609: dwrite d8/d15/d34/f39 [0,4194304] 0 2024-02-22T15:50:37.577 INFO:tasks.workunit.client.1.smithi062.stdout:5/176: write d0/f26 [636367,99806] 0 2024-02-22T15:50:37.578 INFO:tasks.workunit.client.0.smithi019.stdout:2/505: chown d1/d17/d21/d51/f77 279 1 2024-02-22T15:50:37.579 INFO:tasks.workunit.client.1.smithi062.stdout:8/318: getdents d8/d28 0 2024-02-22T15:50:37.580 INFO:tasks.workunit.client.0.smithi019.stdout:6/588: mknod d6/d4b/da9/d67/d8a/d90/cb4 0 2024-02-22T15:50:37.581 INFO:tasks.workunit.client.0.smithi019.stdout:4/543: rmdir d2/db/d14/d61 39 2024-02-22T15:50:37.583 INFO:tasks.workunit.client.0.smithi019.stdout:2/506: write d1/d17/d4b/f4a [146605,73264] 0 2024-02-22T15:50:37.585 INFO:tasks.workunit.client.0.smithi019.stdout:6/589: chown d6/d31/f3b 107604828 1 2024-02-22T15:50:37.585 INFO:tasks.workunit.client.0.smithi019.stdout:4/544: chown d2/db/d14/d23/d4b 5979766 1 2024-02-22T15:50:37.586 INFO:tasks.workunit.client.0.smithi019.stdout:7/704: dwrite d9/d38/f52 [0,4194304] 0 2024-02-22T15:50:37.589 INFO:tasks.workunit.client.0.smithi019.stdout:2/507: mkdir d1/d17/d1c/d66/db9 0 2024-02-22T15:50:37.589 INFO:tasks.workunit.client.0.smithi019.stdout:4/545: write d2/db/d14/d23/d9e/fab [18273,38382] 0 2024-02-22T15:50:37.589 INFO:tasks.workunit.client.0.smithi019.stdout:7/705: chown d9/d26/d77/f95 723173 1 2024-02-22T15:50:37.591 INFO:tasks.workunit.client.1.smithi062.stdout:2/389: creat d1/d8/d62/d73/f75 x:0 0 0 2024-02-22T15:50:37.592 INFO:tasks.workunit.client.1.smithi062.stdout:1/308: read d1/f2e [1608660,1524] 0 2024-02-22T15:50:37.593 INFO:tasks.workunit.client.0.smithi019.stdout:5/591: write d8/ff [5051060,108745] 0 2024-02-22T15:50:37.593 INFO:tasks.workunit.client.0.smithi019.stdout:7/706: chown d9/d38/d41/d45/d61/d6a/fd0 11 1 2024-02-22T15:50:37.596 INFO:tasks.workunit.client.1.smithi062.stdout:0/398: dread d1/d3/de/d3f/d5d/f37 [0,4194304] 0 2024-02-22T15:50:37.597 INFO:tasks.workunit.client.0.smithi019.stdout:2/508: rename d1/d17/d21/d51/f5d to d1/d17/d21/d71/d60/fba 0 2024-02-22T15:50:37.597 INFO:tasks.workunit.client.1.smithi062.stdout:6/421: write d5/df/d40/d44/d78/f31 [563519,16531] 0 2024-02-22T15:50:37.599 INFO:tasks.workunit.client.1.smithi062.stdout:5/177: link d0/c4 d0/d3/da/c2e 0 2024-02-22T15:50:37.600 INFO:tasks.workunit.client.1.smithi062.stdout:9/286: creat d4/f4f x:0 0 0 2024-02-22T15:50:37.600 INFO:tasks.workunit.client.0.smithi019.stdout:6/590: dwrite d6/d31/d69/fa6 [0,4194304] 0 2024-02-22T15:50:37.602 INFO:tasks.workunit.client.0.smithi019.stdout:7/707: rmdir d9/d3a/d49/d5d/d74/d9a 39 2024-02-22T15:50:37.602 INFO:tasks.workunit.client.0.smithi019.stdout:6/591: chown d6/d31/c9c 10917030 1 2024-02-22T15:50:37.602 INFO:tasks.workunit.client.1.smithi062.stdout:5/178: readlink d0/le 0 2024-02-22T15:50:37.607 INFO:tasks.workunit.client.1.smithi062.stdout:6/422: mkdir d5/df/d40/d44/d4a/d41/d85/d88 0 2024-02-22T15:50:37.608 INFO:tasks.workunit.client.0.smithi019.stdout:4/546: rename d2/db/d14/d23/d4b/d8e to d2/db/d14/d23/d33/db1 0 2024-02-22T15:50:37.608 INFO:tasks.workunit.client.0.smithi019.stdout:9/544: truncate d2/d7/dc/d27/d68/f73 2701703 0 2024-02-22T15:50:37.609 INFO:tasks.workunit.client.1.smithi062.stdout:8/319: dwrite d8/d26/f34 [0,4194304] 0 2024-02-22T15:50:37.610 INFO:tasks.workunit.client.0.smithi019.stdout:9/545: readlink d2/d1f/d2e/l5f 0 2024-02-22T15:50:37.613 INFO:tasks.workunit.client.0.smithi019.stdout:9/546: write d2/d1f/d95/f9e [254353,91276] 0 2024-02-22T15:50:37.615 INFO:tasks.workunit.client.1.smithi062.stdout:1/309: dwrite d1/d6/f1a [0,4194304] 0 2024-02-22T15:50:37.616 INFO:tasks.workunit.client.1.smithi062.stdout:6/423: write d5/fb [5138717,32130] 0 2024-02-22T15:50:37.616 INFO:tasks.workunit.client.0.smithi019.stdout:5/592: write d8/dd/d19/d38/d54/f6b [180601,66433] 0 2024-02-22T15:50:37.618 INFO:tasks.workunit.client.1.smithi062.stdout:3/321: dread d5/f41 [0,4194304] 0 2024-02-22T15:50:37.618 INFO:tasks.workunit.client.0.smithi019.stdout:7/708: getdents d9/d4c/dc9 0 2024-02-22T15:50:37.619 INFO:tasks.workunit.client.0.smithi019.stdout:9/547: write d2/d1f/d2e/f42 [6417799,39463] 0 2024-02-22T15:50:37.621 INFO:tasks.workunit.client.0.smithi019.stdout:5/593: chown d8/d11/d33/l44 424709 1 2024-02-22T15:50:37.621 INFO:tasks.workunit.client.1.smithi062.stdout:5/179: dread d0/d3/da/d13/f1b [0,4194304] 0 2024-02-22T15:50:37.625 INFO:tasks.workunit.client.0.smithi019.stdout:5/594: readlink d8/dd/d8e/l58 0 2024-02-22T15:50:37.627 INFO:tasks.workunit.client.1.smithi062.stdout:9/287: dwrite d4/f1b [4194304,4194304] 0 2024-02-22T15:50:37.627 INFO:tasks.workunit.client.0.smithi019.stdout:1/610: dwrite d8/d15/f43 [0,4194304] 0 2024-02-22T15:50:37.628 INFO:tasks.workunit.client.1.smithi062.stdout:3/322: write d5/d1a/f33 [526700,77218] 0 2024-02-22T15:50:37.628 INFO:tasks.workunit.client.1.smithi062.stdout:5/180: mkdir d0/d3/d2f 0 2024-02-22T15:50:37.632 INFO:tasks.workunit.client.1.smithi062.stdout:6/424: mknod d5/df/c89 0 2024-02-22T15:50:37.632 INFO:tasks.workunit.client.1.smithi062.stdout:0/399: dread d1/d3/de/ff [4194304,4194304] 0 2024-02-22T15:50:37.633 INFO:tasks.workunit.client.0.smithi019.stdout:1/611: symlink d8/d15/d34/d9e/lcf 0 2024-02-22T15:50:37.633 INFO:tasks.workunit.client.1.smithi062.stdout:1/310: mkdir d1/d6/d7/d4a/d65 0 2024-02-22T15:50:37.635 INFO:tasks.workunit.client.0.smithi019.stdout:1/612: symlink d8/d2a/d75/ld0 0 2024-02-22T15:50:37.637 INFO:tasks.workunit.client.1.smithi062.stdout:9/288: fdatasync d4/d28/f42 0 2024-02-22T15:50:37.637 INFO:tasks.workunit.client.0.smithi019.stdout:2/509: truncate d1/d17/d21/f6f 167081 0 2024-02-22T15:50:37.638 INFO:tasks.workunit.client.1.smithi062.stdout:5/181: fdatasync d0/d3/da/d13/f2d 0 2024-02-22T15:50:37.639 INFO:tasks.workunit.client.1.smithi062.stdout:3/323: mknod d5/d16/d21/c63 0 2024-02-22T15:50:37.640 INFO:tasks.workunit.client.1.smithi062.stdout:9/289: chown d4/d15/f23 2981 1 2024-02-22T15:50:37.641 INFO:tasks.workunit.client.0.smithi019.stdout:2/510: symlink d1/de/lbb 0 2024-02-22T15:50:37.641 INFO:tasks.workunit.client.1.smithi062.stdout:6/425: mkdir d5/d8a 0 2024-02-22T15:50:37.642 INFO:tasks.workunit.client.1.smithi062.stdout:1/311: mkdir d1/d66 0 2024-02-22T15:50:37.642 INFO:tasks.workunit.client.1.smithi062.stdout:2/390: dread d1/d1e/d38/f54 [0,4194304] 0 2024-02-22T15:50:37.644 INFO:tasks.workunit.client.1.smithi062.stdout:5/182: mkdir d0/d3/d30 0 2024-02-22T15:50:37.645 INFO:tasks.workunit.client.1.smithi062.stdout:8/320: dwrite d8/f20 [0,4194304] 0 2024-02-22T15:50:37.646 INFO:tasks.workunit.client.0.smithi019.stdout:2/511: rename d1/d17/d38/c44 to d1/d17/d21/cbc 0 2024-02-22T15:50:37.647 INFO:tasks.workunit.client.1.smithi062.stdout:6/426: write d5/df/f30 [2495957,78723] 0 2024-02-22T15:50:37.647 INFO:tasks.workunit.client.0.smithi019.stdout:7/709: write d9/f1b [145016,67509] 0 2024-02-22T15:50:37.648 INFO:tasks.workunit.client.0.smithi019.stdout:2/512: stat d1/d17/d21/d51/f77 0 2024-02-22T15:50:37.649 INFO:tasks.workunit.client.1.smithi062.stdout:9/290: unlink d4/d1c/d48/l4b 0 2024-02-22T15:50:37.650 INFO:tasks.workunit.client.1.smithi062.stdout:1/312: symlink d1/d66/l67 0 2024-02-22T15:50:37.650 INFO:tasks.workunit.client.0.smithi019.stdout:2/513: rmdir d1/d86 39 2024-02-22T15:50:37.652 INFO:tasks.workunit.client.0.smithi019.stdout:2/514: chown d1/d17/d1c/d7f 14 1 2024-02-22T15:50:37.653 INFO:tasks.workunit.client.1.smithi062.stdout:3/324: mknod d5/d1a/d3c/d62/c64 0 2024-02-22T15:50:37.653 INFO:tasks.workunit.client.0.smithi019.stdout:7/710: rename d9/d38/d41/d45/d61/d6a/l6f to d9/d38/d41/lec 0 2024-02-22T15:50:37.654 INFO:tasks.workunit.client.1.smithi062.stdout:8/321: creat d8/d9/d44/f5c x:0 0 0 2024-02-22T15:50:37.655 INFO:tasks.workunit.client.0.smithi019.stdout:6/592: dwrite d6/d31/f32 [0,4194304] 0 2024-02-22T15:50:37.659 INFO:tasks.workunit.client.1.smithi062.stdout:6/427: write f0 [780617,115149] 0 2024-02-22T15:50:37.659 INFO:tasks.workunit.client.1.smithi062.stdout:3/325: fsync d5/d26/f61 0 2024-02-22T15:50:37.659 INFO:tasks.workunit.client.1.smithi062.stdout:5/183: dwrite d0/f26 [0,4194304] 0 2024-02-22T15:50:37.662 INFO:tasks.workunit.client.1.smithi062.stdout:1/313: creat d1/d6/d7/d2b/d47/d63/f68 x:0 0 0 2024-02-22T15:50:37.663 INFO:tasks.workunit.client.0.smithi019.stdout:4/547: dread d2/db/d14/d23/d21/f3a [0,4194304] 0 2024-02-22T15:50:37.664 INFO:tasks.workunit.client.0.smithi019.stdout:2/515: creat d1/d17/fbd x:0 0 0 2024-02-22T15:50:37.665 INFO:tasks.workunit.client.0.smithi019.stdout:7/711: mkdir d9/d3a/d49/d5d/d74/d9a/de3/deb/ded 0 2024-02-22T15:50:37.665 INFO:tasks.workunit.client.0.smithi019.stdout:2/516: write d1/d17/fb1 [764490,31393] 0 2024-02-22T15:50:37.667 INFO:tasks.workunit.client.1.smithi062.stdout:1/314: write d1/d6/de/f17 [3008895,52443] 0 2024-02-22T15:50:37.667 INFO:tasks.workunit.client.1.smithi062.stdout:5/184: chown d0/d3/da/d13/l15 0 1 2024-02-22T15:50:37.667 INFO:tasks.workunit.client.0.smithi019.stdout:6/593: symlink d6/d4b/d6d/d7b/lb5 0 2024-02-22T15:50:37.668 INFO:tasks.workunit.client.1.smithi062.stdout:8/322: link l5 d8/d42/l5d 0 2024-02-22T15:50:37.668 INFO:tasks.workunit.client.0.smithi019.stdout:7/712: chown d9/d38/d41/d45/d92/da1/ca2 65905499 1 2024-02-22T15:50:37.668 INFO:tasks.workunit.client.0.smithi019.stdout:6/594: write d6/f3c [138851,108190] 0 2024-02-22T15:50:37.670 INFO:tasks.workunit.client.1.smithi062.stdout:6/428: dread d5/df/d32/f35 [0,4194304] 0 2024-02-22T15:50:37.672 INFO:tasks.workunit.client.1.smithi062.stdout:8/323: chown d8/d28 1780378 1 2024-02-22T15:50:37.673 INFO:tasks.workunit.client.0.smithi019.stdout:2/517: creat d1/fbe x:0 0 0 2024-02-22T15:50:37.673 INFO:tasks.workunit.client.1.smithi062.stdout:5/185: mkdir d0/d3/da/d13/d23/d31 0 2024-02-22T15:50:37.675 INFO:tasks.workunit.client.0.smithi019.stdout:6/595: unlink d6/d4b/da9/d67/f9a 0 2024-02-22T15:50:37.675 INFO:tasks.workunit.client.1.smithi062.stdout:9/291: dwrite d4/d1c/f2c [0,4194304] 0 2024-02-22T15:50:37.675 INFO:tasks.workunit.client.1.smithi062.stdout:1/315: write d1/d6/d36/f5d [4545977,93238] 0 2024-02-22T15:50:37.676 INFO:tasks.workunit.client.0.smithi019.stdout:7/713: rename d9/d38/lac to d9/d4c/ddf/lee 0 2024-02-22T15:50:37.677 INFO:tasks.workunit.client.1.smithi062.stdout:8/324: mkdir d8/d9/d2c/d49/d5e 0 2024-02-22T15:50:37.679 INFO:tasks.workunit.client.1.smithi062.stdout:5/186: mkdir d0/d3/da/d32 0 2024-02-22T15:50:37.679 INFO:tasks.workunit.client.1.smithi062.stdout:8/325: chown d8/d28 323285556 1 2024-02-22T15:50:37.679 INFO:tasks.workunit.client.0.smithi019.stdout:2/518: mknod d1/d17/d1c/cbf 0 2024-02-22T15:50:37.680 INFO:tasks.workunit.client.1.smithi062.stdout:6/429: truncate d5/df/d40/d44/d76/d56/f61 231957 0 2024-02-22T15:50:37.680 INFO:tasks.workunit.client.0.smithi019.stdout:6/596: rename d6/d58/f76 to d6/d4b/da9/d67/d8a/fb6 0 2024-02-22T15:50:37.682 INFO:tasks.workunit.client.0.smithi019.stdout:7/714: symlink d9/d3a/d49/d5d/d74/d9a/lef 0 2024-02-22T15:50:37.687 INFO:tasks.workunit.client.0.smithi019.stdout:6/597: rename d6/d4b/d6d/d7b/f92 to d6/d3e/fb7 0 2024-02-22T15:50:37.689 INFO:tasks.workunit.client.0.smithi019.stdout:7/715: rename d9/la0 to d9/lf0 0 2024-02-22T15:50:37.691 INFO:tasks.workunit.client.0.smithi019.stdout:6/598: rename d6/f39 to d6/d4b/d6d/d7b/d7f/fb8 0 2024-02-22T15:50:37.692 INFO:tasks.workunit.client.0.smithi019.stdout:9/548: dread d2/d7/dc/d27/d68/f73 [0,4194304] 0 2024-02-22T15:50:37.692 INFO:tasks.workunit.client.1.smithi062.stdout:1/316: rename d1/d6/d7/d2b/d47/d24/c3d to d1/d6/d36/d37/c69 0 2024-02-22T15:50:37.692 INFO:tasks.workunit.client.1.smithi062.stdout:5/187: dwrite d0/d3/f14 [0,4194304] 0 2024-02-22T15:50:37.693 INFO:tasks.workunit.client.0.smithi019.stdout:6/599: dread - d6/d31/d43/f8b zero size 2024-02-22T15:50:37.695 INFO:tasks.workunit.client.0.smithi019.stdout:1/613: write d8/f14 [4627796,88455] 0 2024-02-22T15:50:37.699 INFO:tasks.workunit.client.1.smithi062.stdout:8/326: getdents d8 0 2024-02-22T15:50:37.700 INFO:tasks.workunit.client.0.smithi019.stdout:7/716: rename d9/f25 to d9/ff1 0 2024-02-22T15:50:37.700 INFO:tasks.workunit.client.1.smithi062.stdout:5/188: dread d0/d3/ff [0,4194304] 0 2024-02-22T15:50:37.701 INFO:tasks.workunit.client.1.smithi062.stdout:1/317: chown d1/d6/de/c48 13943047 1 2024-02-22T15:50:37.701 INFO:tasks.workunit.client.1.smithi062.stdout:3/326: dread d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:37.703 INFO:tasks.workunit.client.1.smithi062.stdout:2/391: dread d1/f16 [0,4194304] 0 2024-02-22T15:50:37.704 INFO:tasks.workunit.client.0.smithi019.stdout:2/519: dread d1/f2e [0,4194304] 0 2024-02-22T15:50:37.704 INFO:tasks.workunit.client.0.smithi019.stdout:7/717: write d9/d3a/d49/d5d/f9f [2471338,20004] 0 2024-02-22T15:50:37.704 INFO:tasks.workunit.client.1.smithi062.stdout:5/189: write d0/f9 [3220950,40456] 0 2024-02-22T15:50:37.707 INFO:tasks.workunit.client.0.smithi019.stdout:2/520: mknod d1/d17/d38/cc0 0 2024-02-22T15:50:37.708 INFO:tasks.workunit.client.1.smithi062.stdout:0/400: dwrite d1/d3/de/d3f/d2c/d84/f26 [0,4194304] 0 2024-02-22T15:50:37.709 INFO:tasks.workunit.client.0.smithi019.stdout:7/718: rmdir d9/d38/d41/d45/d92 39 2024-02-22T15:50:37.714 INFO:tasks.workunit.client.1.smithi062.stdout:4/299: sync 2024-02-22T15:50:37.714 INFO:tasks.workunit.client.1.smithi062.stdout:7/291: sync 2024-02-22T15:50:37.714 INFO:tasks.workunit.client.1.smithi062.stdout:6/430: truncate d5/f1e 1399373 0 2024-02-22T15:50:37.716 INFO:tasks.workunit.client.0.smithi019.stdout:7/719: creat d9/d4c/d5b/ff2 x:0 0 0 2024-02-22T15:50:37.716 INFO:tasks.workunit.client.1.smithi062.stdout:5/190: dread d0/d3/da/f18 [0,4194304] 0 2024-02-22T15:50:37.716 INFO:tasks.workunit.client.1.smithi062.stdout:8/327: creat d8/d9/f5f x:0 0 0 2024-02-22T15:50:37.717 INFO:tasks.workunit.client.0.smithi019.stdout:2/521: dwrite d1/d17/d21/d51/f6a [0,4194304] 0 2024-02-22T15:50:37.718 INFO:tasks.workunit.client.1.smithi062.stdout:2/392: mkdir d1/d1e/d38/d76 0 2024-02-22T15:50:37.719 INFO:tasks.workunit.client.1.smithi062.stdout:6/431: chown d5/df/d32/f35 0 1 2024-02-22T15:50:37.720 INFO:tasks.workunit.client.1.smithi062.stdout:6/432: dread - d5/f73 zero size 2024-02-22T15:50:37.721 INFO:tasks.workunit.client.1.smithi062.stdout:1/318: link d1/d6/de/f31 d1/d6/d7/d2b/d49/f6a 0 2024-02-22T15:50:37.721 INFO:tasks.workunit.client.1.smithi062.stdout:4/300: chown d4/d13/f43 61023 1 2024-02-22T15:50:37.724 INFO:tasks.workunit.client.0.smithi019.stdout:6/600: dread d6/fc [4194304,4194304] 0 2024-02-22T15:50:37.724 INFO:tasks.workunit.client.0.smithi019.stdout:2/522: symlink d1/de/d9a/lc1 0 2024-02-22T15:50:37.725 INFO:tasks.workunit.client.1.smithi062.stdout:2/393: fdatasync d1/d1e/f2d 0 2024-02-22T15:50:37.727 INFO:tasks.workunit.client.1.smithi062.stdout:5/191: dread d0/d3/da/f18 [0,4194304] 0 2024-02-22T15:50:37.727 INFO:tasks.workunit.client.0.smithi019.stdout:2/523: write d1/d17/d21/d51/f7c [1068592,56498] 0 2024-02-22T15:50:37.728 INFO:tasks.workunit.client.0.smithi019.stdout:9/549: dread d2/f19 [4194304,4194304] 0 2024-02-22T15:50:37.731 INFO:tasks.workunit.client.0.smithi019.stdout:6/601: unlink d6/f2a 0 2024-02-22T15:50:37.732 INFO:tasks.workunit.client.1.smithi062.stdout:7/292: mkdir d0/d6/d2e/d4a/d5a/d5e 0 2024-02-22T15:50:37.734 INFO:tasks.workunit.client.1.smithi062.stdout:2/394: dread - d1/d8/d62/d73/f75 zero size 2024-02-22T15:50:37.734 INFO:tasks.workunit.client.0.smithi019.stdout:7/720: write d9/d4c/f66 [4922207,81550] 0 2024-02-22T15:50:37.735 INFO:tasks.workunit.client.0.smithi019.stdout:6/602: symlink d6/d4b/d8d/lb9 0 2024-02-22T15:50:37.736 INFO:tasks.workunit.client.1.smithi062.stdout:9/292: dread d4/fc [0,4194304] 0 2024-02-22T15:50:37.737 INFO:tasks.workunit.client.1.smithi062.stdout:1/319: read d1/d6/d10/f1c [1336327,129885] 0 2024-02-22T15:50:37.737 INFO:tasks.workunit.client.1.smithi062.stdout:6/433: creat d5/df/d40/d44/d76/f8b x:0 0 0 2024-02-22T15:50:37.737 INFO:tasks.workunit.client.0.smithi019.stdout:9/550: rmdir d2/d1f/d2e/d57/d60/dc1 0 2024-02-22T15:50:37.738 INFO:tasks.workunit.client.1.smithi062.stdout:8/328: write d8/d9/f16 [1807676,59260] 0 2024-02-22T15:50:37.738 INFO:tasks.workunit.client.0.smithi019.stdout:9/551: write d2/fab [293606,25792] 0 2024-02-22T15:50:37.739 INFO:tasks.workunit.client.0.smithi019.stdout:7/721: mkdir d9/d38/df3 0 2024-02-22T15:50:37.740 INFO:tasks.workunit.client.1.smithi062.stdout:7/293: fdatasync d0/d6/f52 0 2024-02-22T15:50:37.740 INFO:tasks.workunit.client.0.smithi019.stdout:9/552: fsync d2/d1f/d95/fa2 0 2024-02-22T15:50:37.741 INFO:tasks.workunit.client.0.smithi019.stdout:2/524: truncate d1/d17/f30 2589951 0 2024-02-22T15:50:37.741 INFO:tasks.workunit.client.1.smithi062.stdout:4/301: unlink d4/d2a/f4d 0 2024-02-22T15:50:37.742 INFO:tasks.workunit.client.1.smithi062.stdout:9/293: chown d4/d28/f30 2 1 2024-02-22T15:50:37.742 INFO:tasks.workunit.client.0.smithi019.stdout:9/553: stat d2/d1f/d2e/d57 0 2024-02-22T15:50:37.743 INFO:tasks.workunit.client.1.smithi062.stdout:0/401: rename d1/d3/de/d3f/d2c/d84/f81 to d1/d3/de/d3f/f88 0 2024-02-22T15:50:37.743 INFO:tasks.workunit.client.1.smithi062.stdout:3/327: dread d5/d16/f48 [0,4194304] 0 2024-02-22T15:50:37.743 INFO:tasks.workunit.client.1.smithi062.stdout:8/329: fdatasync d8/d9/d2c/f30 0 2024-02-22T15:50:37.744 INFO:tasks.workunit.client.1.smithi062.stdout:6/434: write d5/df/d40/d44/f6b [140196,16789] 0 2024-02-22T15:50:37.745 INFO:tasks.workunit.client.1.smithi062.stdout:5/192: mkdir d0/d3/da/d32/d33 0 2024-02-22T15:50:37.745 INFO:tasks.workunit.client.0.smithi019.stdout:2/525: creat d1/d17/d21/d51/fc2 x:0 0 0 2024-02-22T15:50:37.746 INFO:tasks.workunit.client.1.smithi062.stdout:4/302: fdatasync d4/fd 0 2024-02-22T15:50:37.747 INFO:tasks.workunit.client.0.smithi019.stdout:9/554: creat d2/d1f/d2e/d48/fc4 x:0 0 0 2024-02-22T15:50:37.747 INFO:tasks.workunit.client.1.smithi062.stdout:9/294: mkdir d4/d28/d50 0 2024-02-22T15:50:37.747 INFO:tasks.workunit.client.1.smithi062.stdout:8/330: fsync d8/f39 0 2024-02-22T15:50:37.747 INFO:tasks.workunit.client.1.smithi062.stdout:1/320: unlink d1/d6/d7/d2b/c40 0 2024-02-22T15:50:37.748 INFO:tasks.workunit.client.0.smithi019.stdout:2/526: write d1/d17/fb2 [39640,96653] 0 2024-02-22T15:50:37.749 INFO:tasks.workunit.client.1.smithi062.stdout:7/294: mknod d0/d6/c5f 0 2024-02-22T15:50:37.750 INFO:tasks.workunit.client.0.smithi019.stdout:7/722: dwrite d9/d38/d41/f78 [0,4194304] 0 2024-02-22T15:50:37.751 INFO:tasks.workunit.client.1.smithi062.stdout:5/193: rmdir d0/d3/da/d32 39 2024-02-22T15:50:37.751 INFO:tasks.workunit.client.1.smithi062.stdout:6/435: mknod d5/dd/d33/d66/c8c 0 2024-02-22T15:50:37.752 INFO:tasks.workunit.client.1.smithi062.stdout:2/395: rename d1/d8/df/l4d to d1/d8/d62/d73/l77 0 2024-02-22T15:50:37.752 INFO:tasks.workunit.client.1.smithi062.stdout:3/328: fsync d5/d16/d21/d32/f51 0 2024-02-22T15:50:37.752 INFO:tasks.workunit.client.1.smithi062.stdout:8/331: chown d8/cd 20628 1 2024-02-22T15:50:37.753 INFO:tasks.workunit.client.1.smithi062.stdout:2/396: read - d1/d47/f72 zero size 2024-02-22T15:50:37.754 INFO:tasks.workunit.client.0.smithi019.stdout:6/603: dread d6/d4b/d6d/d7b/f60 [0,4194304] 0 2024-02-22T15:50:37.756 INFO:tasks.workunit.client.1.smithi062.stdout:8/332: chown d8/f29 24721911 1 2024-02-22T15:50:37.756 INFO:tasks.workunit.client.0.smithi019.stdout:9/555: mkdir d2/d1f/d2e/d57/d60/dc5 0 2024-02-22T15:50:37.757 INFO:tasks.workunit.client.0.smithi019.stdout:6/604: readlink d6/d4b/d6d/l9b 0 2024-02-22T15:50:37.758 INFO:tasks.workunit.client.1.smithi062.stdout:7/295: chown d0/d14/d19/l31 657902 1 2024-02-22T15:50:37.758 INFO:tasks.workunit.client.0.smithi019.stdout:7/723: symlink d9/d3a/d49/d5d/d74/lf4 0 2024-02-22T15:50:37.759 INFO:tasks.workunit.client.1.smithi062.stdout:6/436: dread - d5/df/d40/d44/d78/d59/f7f zero size 2024-02-22T15:50:37.760 INFO:tasks.workunit.client.1.smithi062.stdout:4/303: creat d4/d1c/d30/d3b/f67 x:0 0 0 2024-02-22T15:50:37.762 INFO:tasks.workunit.client.1.smithi062.stdout:2/397: creat d1/d8/df/f78 x:0 0 0 2024-02-22T15:50:37.762 INFO:tasks.workunit.client.0.smithi019.stdout:7/724: read d9/d4c/d5b/f64 [2704838,15296] 0 2024-02-22T15:50:37.764 INFO:tasks.workunit.client.1.smithi062.stdout:8/333: chown d8/d9/d2c/f59 31733 1 2024-02-22T15:50:37.765 INFO:tasks.workunit.client.1.smithi062.stdout:7/296: mkdir d0/d6/d5b/d60 0 2024-02-22T15:50:37.765 INFO:tasks.workunit.client.0.smithi019.stdout:9/556: dwrite d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:37.768 INFO:tasks.workunit.client.1.smithi062.stdout:0/402: dwrite d1/d3/de/d3f/f4c [0,4194304] 0 2024-02-22T15:50:37.770 INFO:tasks.workunit.client.0.smithi019.stdout:2/527: truncate d1/f1f 2716552 0 2024-02-22T15:50:37.771 INFO:tasks.workunit.client.1.smithi062.stdout:5/194: rename d0/d3/da/d13/f1b to d0/d3/da/d13/d23/d31/f34 0 2024-02-22T15:50:37.771 INFO:tasks.workunit.client.1.smithi062.stdout:5/195: chown d0/f7 24009 1 2024-02-22T15:50:37.773 INFO:tasks.workunit.client.0.smithi019.stdout:6/605: dread d6/d31/d43/fb0 [0,4194304] 0 2024-02-22T15:50:37.773 INFO:tasks.workunit.client.0.smithi019.stdout:2/528: dread - d1/d17/d1c/d66/f68 zero size 2024-02-22T15:50:37.774 INFO:tasks.workunit.client.1.smithi062.stdout:4/304: rmdir d4/d34 39 2024-02-22T15:50:37.777 INFO:tasks.workunit.client.0.smithi019.stdout:9/557: dwrite d2/f30 [4194304,4194304] 0 2024-02-22T15:50:37.778 INFO:tasks.workunit.client.1.smithi062.stdout:3/329: dwrite d5/d26/d2d/f47 [0,4194304] 0 2024-02-22T15:50:37.778 INFO:tasks.workunit.client.0.smithi019.stdout:2/529: creat d1/d17/d21/daf/fc3 x:0 0 0 2024-02-22T15:50:37.780 INFO:tasks.workunit.client.1.smithi062.stdout:2/398: symlink d1/d1e/d38/d55/l79 0 2024-02-22T15:50:37.782 INFO:tasks.workunit.client.1.smithi062.stdout:2/399: write d1/d8/f6e [826107,1302] 0 2024-02-22T15:50:37.782 INFO:tasks.workunit.client.1.smithi062.stdout:1/321: dwrite d1/d6/d7/f54 [0,4194304] 0 2024-02-22T15:50:37.782 INFO:tasks.workunit.client.1.smithi062.stdout:6/437: dread d5/dd/d33/f49 [4194304,4194304] 0 2024-02-22T15:50:37.785 INFO:tasks.workunit.client.0.smithi019.stdout:9/558: creat d2/d1f/d2e/d57/d60/d8c/d66/fc6 x:0 0 0 2024-02-22T15:50:37.786 INFO:tasks.workunit.client.1.smithi062.stdout:6/438: chown d5/df/d40/d44/d76 38852508 1 2024-02-22T15:50:37.787 INFO:tasks.workunit.client.0.smithi019.stdout:6/606: dwrite d6/f1d [0,4194304] 0 2024-02-22T15:50:37.787 INFO:tasks.workunit.client.0.smithi019.stdout:9/559: write d2/f6 [1175781,85491] 0 2024-02-22T15:50:37.787 INFO:tasks.workunit.client.0.smithi019.stdout:7/725: dwrite d9/d38/f3c [0,4194304] 0 2024-02-22T15:50:37.789 INFO:tasks.workunit.client.0.smithi019.stdout:9/560: read d2/d7/dc/f49 [2124297,108337] 0 2024-02-22T15:50:37.790 INFO:tasks.workunit.client.0.smithi019.stdout:7/726: write d9/d38/d41/fb7 [2592514,32221] 0 2024-02-22T15:50:37.791 INFO:tasks.workunit.client.1.smithi062.stdout:5/196: unlink d0/le 0 2024-02-22T15:50:37.792 INFO:tasks.workunit.client.0.smithi019.stdout:7/727: stat d9/d3a/d49/d5d/d74/d9a/lef 0 2024-02-22T15:50:37.792 INFO:tasks.workunit.client.1.smithi062.stdout:9/295: truncate d4/d15/f23 1163704 0 2024-02-22T15:50:37.794 INFO:tasks.workunit.client.1.smithi062.stdout:8/334: mknod d8/d9/d2c/d49/d5e/c60 0 2024-02-22T15:50:37.795 INFO:tasks.workunit.client.1.smithi062.stdout:8/335: dread - d8/d26/f41 zero size 2024-02-22T15:50:37.795 INFO:tasks.workunit.client.0.smithi019.stdout:6/607: read d6/ff [29999,127757] 0 2024-02-22T15:50:37.796 INFO:tasks.workunit.client.1.smithi062.stdout:8/336: chown l7 481864 1 2024-02-22T15:50:37.796 INFO:tasks.workunit.client.0.smithi019.stdout:7/728: rmdir d9/d4c/d5b 39 2024-02-22T15:50:37.797 INFO:tasks.workunit.client.0.smithi019.stdout:9/561: write d2/d7/dc/d27/d68/f6b [1635684,25310] 0 2024-02-22T15:50:37.798 INFO:tasks.workunit.client.1.smithi062.stdout:3/330: symlink d5/d16/d56/l65 0 2024-02-22T15:50:37.799 INFO:tasks.workunit.client.0.smithi019.stdout:7/729: rename d9/f21 to d9/d38/d41/d45/db0/ff5 0 2024-02-22T15:50:37.800 INFO:tasks.workunit.client.1.smithi062.stdout:3/331: fdatasync d5/d16/f44 0 2024-02-22T15:50:37.801 INFO:tasks.workunit.client.1.smithi062.stdout:6/439: symlink d5/dd/d33/l8d 0 2024-02-22T15:50:37.803 INFO:tasks.workunit.client.1.smithi062.stdout:8/337: symlink d8/d9/d2c/d49/l61 0 2024-02-22T15:50:37.803 INFO:tasks.workunit.client.1.smithi062.stdout:3/332: truncate d5/d26/f61 239122 0 2024-02-22T15:50:37.804 INFO:tasks.workunit.client.1.smithi062.stdout:6/440: chown d5/df/d40/d44/d4a/d41/d85 118 1 2024-02-22T15:50:37.804 INFO:tasks.workunit.client.1.smithi062.stdout:1/322: mknod d1/c6b 0 2024-02-22T15:50:37.804 INFO:tasks.workunit.client.1.smithi062.stdout:9/296: write d4/d1c/f1d [3858400,85161] 0 2024-02-22T15:50:37.806 INFO:tasks.workunit.client.1.smithi062.stdout:7/297: creat d0/d6/d2e/f61 x:0 0 0 2024-02-22T15:50:37.806 INFO:tasks.workunit.client.1.smithi062.stdout:2/400: mkdir d1/d1e/d7a 0 2024-02-22T15:50:37.807 INFO:tasks.workunit.client.1.smithi062.stdout:5/197: dread d0/f16 [0,4194304] 0 2024-02-22T15:50:37.807 INFO:tasks.workunit.client.1.smithi062.stdout:1/323: dread - d1/f44 zero size 2024-02-22T15:50:37.809 INFO:tasks.workunit.client.1.smithi062.stdout:4/305: truncate d4/d1c/d30/d3b/f67 955620 0 2024-02-22T15:50:37.810 INFO:tasks.workunit.client.1.smithi062.stdout:8/338: truncate d8/d9/d44/f45 499433 0 2024-02-22T15:50:37.810 INFO:tasks.workunit.client.1.smithi062.stdout:5/198: read d0/d3/ff [1201734,124479] 0 2024-02-22T15:50:37.812 INFO:tasks.workunit.client.1.smithi062.stdout:3/333: creat d5/d16/d21/d32/f66 x:0 0 0 2024-02-22T15:50:37.813 INFO:tasks.workunit.client.1.smithi062.stdout:7/298: mknod d0/d6/d2e/d2f/c62 0 2024-02-22T15:50:37.814 INFO:tasks.workunit.client.1.smithi062.stdout:4/306: symlink d4/d1c/d2c/l68 0 2024-02-22T15:50:37.815 INFO:tasks.workunit.client.1.smithi062.stdout:1/324: fdatasync d1/d6/d7/f50 0 2024-02-22T15:50:37.815 INFO:tasks.workunit.client.1.smithi062.stdout:5/199: mknod d0/c35 0 2024-02-22T15:50:37.816 INFO:tasks.workunit.client.1.smithi062.stdout:1/325: stat d1/d6/de/d9 0 2024-02-22T15:50:37.816 INFO:tasks.workunit.client.1.smithi062.stdout:3/334: readlink d5/ld 0 2024-02-22T15:50:37.817 INFO:tasks.workunit.client.1.smithi062.stdout:1/326: readlink d1/d6/d36/l5a 0 2024-02-22T15:50:37.819 INFO:tasks.workunit.client.1.smithi062.stdout:4/307: write d4/f3f [399107,129037] 0 2024-02-22T15:50:37.820 INFO:tasks.workunit.client.1.smithi062.stdout:4/308: write d4/f3f [1126966,38380] 0 2024-02-22T15:50:37.822 INFO:tasks.workunit.client.0.smithi019.stdout:3/431: sync 2024-02-22T15:50:37.822 INFO:tasks.workunit.client.1.smithi062.stdout:3/335: rename d5/ld to d5/d26/d2d/d3f/l67 0 2024-02-22T15:50:37.824 INFO:tasks.workunit.client.1.smithi062.stdout:1/327: mknod d1/d6/d36/c6c 0 2024-02-22T15:50:37.826 INFO:tasks.workunit.client.0.smithi019.stdout:9/562: creat d2/d1f/d2e/d57/fc7 x:0 0 0 2024-02-22T15:50:37.826 INFO:tasks.workunit.client.1.smithi062.stdout:8/339: link c6 d8/d9/d44/d4d/d54/c62 0 2024-02-22T15:50:37.827 INFO:tasks.workunit.client.1.smithi062.stdout:2/401: getdents d1/d8/d62/d73 0 2024-02-22T15:50:37.827 INFO:tasks.workunit.client.0.smithi019.stdout:3/432: dread - d9/d22/d27/d6d/d72/f81 zero size 2024-02-22T15:50:37.828 INFO:tasks.workunit.client.0.smithi019.stdout:7/730: unlink d9/d38/d41/d45/db0/ccf 0 2024-02-22T15:50:37.830 INFO:tasks.workunit.client.0.smithi019.stdout:6/608: getdents d6/d4b/da9/d67/d8a/d90 0 2024-02-22T15:50:37.830 INFO:tasks.workunit.client.0.smithi019.stdout:3/433: dread - d9/d26/d3f/d5c/f6e zero size 2024-02-22T15:50:37.832 INFO:tasks.workunit.client.1.smithi062.stdout:5/200: rename d0/d3/da/c2e to d0/d3/da/d13/d23/d31/c36 0 2024-02-22T15:50:37.833 INFO:tasks.workunit.client.1.smithi062.stdout:2/402: truncate d1/d8/df/f4c 782474 0 2024-02-22T15:50:37.833 INFO:tasks.workunit.client.0.smithi019.stdout:3/434: write d9/d26/d3f/d59/d66/f7f [259390,69230] 0 2024-02-22T15:50:37.833 INFO:tasks.workunit.client.0.smithi019.stdout:7/731: creat d9/ff6 x:0 0 0 2024-02-22T15:50:37.834 INFO:tasks.workunit.client.0.smithi019.stdout:6/609: symlink d6/d31/lba 0 2024-02-22T15:50:37.835 INFO:tasks.workunit.client.1.smithi062.stdout:4/309: creat d4/d2a/d2d/d2e/f69 x:0 0 0 2024-02-22T15:50:37.836 INFO:tasks.workunit.client.1.smithi062.stdout:9/297: dread d4/f26 [0,4194304] 0 2024-02-22T15:50:37.837 INFO:tasks.workunit.client.0.smithi019.stdout:7/732: mkdir d9/d38/d41/d45/db0/df7 0 2024-02-22T15:50:37.837 INFO:tasks.workunit.client.0.smithi019.stdout:2/530: dread d1/d17/d21/f41 [0,4194304] 0 2024-02-22T15:50:37.837 INFO:tasks.workunit.client.0.smithi019.stdout:3/435: read d9/d26/d32/f43 [1765738,54724] 0 2024-02-22T15:50:37.838 INFO:tasks.workunit.client.1.smithi062.stdout:3/336: getdents d5/d26/d2d 0 2024-02-22T15:50:37.838 INFO:tasks.workunit.client.1.smithi062.stdout:5/201: symlink d0/d3/da/d13/l37 0 2024-02-22T15:50:37.839 INFO:tasks.workunit.client.1.smithi062.stdout:1/328: dwrite d1/d6/d10/f5c [0,4194304] 0 2024-02-22T15:50:37.840 INFO:tasks.workunit.client.0.smithi019.stdout:6/610: dread d6/d3e/fb7 [0,4194304] 0 2024-02-22T15:50:37.840 INFO:tasks.workunit.client.0.smithi019.stdout:3/436: write d9/d11/f4d [1216637,52027] 0 2024-02-22T15:50:37.841 INFO:tasks.workunit.client.1.smithi062.stdout:2/403: mknod d1/d47/d39/d5d/c7b 0 2024-02-22T15:50:37.841 INFO:tasks.workunit.client.1.smithi062.stdout:9/298: write d4/f14 [467622,68125] 0 2024-02-22T15:50:37.843 INFO:tasks.workunit.client.1.smithi062.stdout:3/337: chown d5/d1a/f1e 432 1 2024-02-22T15:50:37.847 INFO:tasks.workunit.client.1.smithi062.stdout:4/310: mkdir d4/d2a/d2d/d2e/d6a 0 2024-02-22T15:50:37.848 INFO:tasks.workunit.client.0.smithi019.stdout:7/733: dread d9/d26/d77/f9b [0,4194304] 0 2024-02-22T15:50:37.849 INFO:tasks.workunit.client.0.smithi019.stdout:2/531: dwrite d1/d46/f5e [4194304,4194304] 0 2024-02-22T15:50:37.852 INFO:tasks.workunit.client.1.smithi062.stdout:5/202: write d0/d3/da/f1d [1373942,4528] 0 2024-02-22T15:50:37.853 INFO:tasks.workunit.client.1.smithi062.stdout:8/340: getdents d8/d9/d48 0 2024-02-22T15:50:37.855 INFO:tasks.workunit.client.0.smithi019.stdout:7/734: truncate d9/d3a/d49/d5d/d74/f88 2118225 0 2024-02-22T15:50:37.856 INFO:tasks.workunit.client.1.smithi062.stdout:3/338: unlink d5/d16/d21/d36/c37 0 2024-02-22T15:50:37.857 INFO:tasks.workunit.client.0.smithi019.stdout:2/532: creat d1/fc4 x:0 0 0 2024-02-22T15:50:37.857 INFO:tasks.workunit.client.1.smithi062.stdout:8/341: chown d8/d9/d2c/d49 0 1 2024-02-22T15:50:37.857 INFO:tasks.workunit.client.1.smithi062.stdout:1/329: dwrite d1/f30 [0,4194304] 0 2024-02-22T15:50:37.859 INFO:tasks.workunit.client.1.smithi062.stdout:4/311: mkdir d4/d1c/d2c/d6b 0 2024-02-22T15:50:37.860 INFO:tasks.workunit.client.1.smithi062.stdout:1/330: stat d1/d6/d7/d2b/d47 0 2024-02-22T15:50:37.863 INFO:tasks.workunit.client.0.smithi019.stdout:7/735: dread d9/d38/f59 [0,4194304] 0 2024-02-22T15:50:37.863 INFO:tasks.workunit.client.1.smithi062.stdout:2/404: rename d1/d28/c37 to d1/d8/c7c 0 2024-02-22T15:50:37.865 INFO:tasks.workunit.client.1.smithi062.stdout:2/405: fsync d1/d47/d39/f4b 0 2024-02-22T15:50:37.865 INFO:tasks.workunit.client.1.smithi062.stdout:5/203: mkdir d0/d3/d2f/d38 0 2024-02-22T15:50:37.866 INFO:tasks.workunit.client.1.smithi062.stdout:6/441: write d5/df/d40/d44/d4a/d41/f72 [218687,63415] 0 2024-02-22T15:50:37.867 INFO:tasks.workunit.client.1.smithi062.stdout:8/342: dread d8/d9/d2c/f30 [0,4194304] 0 2024-02-22T15:50:37.869 INFO:tasks.workunit.client.1.smithi062.stdout:8/343: chown d8/d26/f47 248906 1 2024-02-22T15:50:37.871 INFO:tasks.workunit.client.0.smithi019.stdout:2/533: write d1/d17/f47 [1672385,39006] 0 2024-02-22T15:50:37.871 INFO:tasks.workunit.client.0.smithi019.stdout:2/534: stat d1/de/lbb 0 2024-02-22T15:50:37.872 INFO:tasks.workunit.client.0.smithi019.stdout:2/535: fdatasync d1/fbe 0 2024-02-22T15:50:37.872 INFO:tasks.workunit.client.1.smithi062.stdout:5/204: creat d0/f39 x:0 0 0 2024-02-22T15:50:37.873 INFO:tasks.workunit.client.0.smithi019.stdout:7/736: symlink d9/d38/d41/d45/lf8 0 2024-02-22T15:50:37.874 INFO:tasks.workunit.client.0.smithi019.stdout:7/737: write d9/ff6 [707436,14657] 0 2024-02-22T15:50:37.874 INFO:tasks.workunit.client.1.smithi062.stdout:1/331: write d1/d6/d10/f1c [1816198,30891] 0 2024-02-22T15:50:37.874 INFO:tasks.workunit.client.0.smithi019.stdout:2/536: mknod d1/d17/d21/daf/cc5 0 2024-02-22T15:50:37.875 INFO:tasks.workunit.client.0.smithi019.stdout:2/537: dread - d1/d17/d21/d51/fc2 zero size 2024-02-22T15:50:37.877 INFO:tasks.workunit.client.0.smithi019.stdout:6/611: dread d6/d58/f66 [0,4194304] 0 2024-02-22T15:50:37.878 INFO:tasks.workunit.client.0.smithi019.stdout:7/738: unlink d9/d38/d41/d45/f8c 0 2024-02-22T15:50:37.879 INFO:tasks.workunit.client.0.smithi019.stdout:6/612: fdatasync d6/d98/fa5 0 2024-02-22T15:50:37.880 INFO:tasks.workunit.client.1.smithi062.stdout:3/339: getdents d5/d16/d21 0 2024-02-22T15:50:37.880 INFO:tasks.workunit.client.1.smithi062.stdout:5/205: creat d0/d3/d2f/f3a x:0 0 0 2024-02-22T15:50:37.882 INFO:tasks.workunit.client.1.smithi062.stdout:2/406: dwrite d1/d28/f5c [0,4194304] 0 2024-02-22T15:50:37.882 INFO:tasks.workunit.client.0.smithi019.stdout:2/538: symlink d1/d17/d21/d71/lc6 0 2024-02-22T15:50:37.884 INFO:tasks.workunit.client.0.smithi019.stdout:7/739: dread d9/d38/f3c [0,4194304] 0 2024-02-22T15:50:37.884 INFO:tasks.workunit.client.0.smithi019.stdout:6/613: fsync d6/d31/f35 0 2024-02-22T15:50:37.886 INFO:tasks.workunit.client.0.smithi019.stdout:2/539: symlink d1/de/lc7 0 2024-02-22T15:50:37.887 INFO:tasks.workunit.client.1.smithi062.stdout:9/299: truncate d4/d1c/f2c 358942 0 2024-02-22T15:50:37.888 INFO:tasks.workunit.client.0.smithi019.stdout:2/540: stat d1/d17/d21/daf 0 2024-02-22T15:50:37.889 INFO:tasks.workunit.client.1.smithi062.stdout:8/344: dwrite d8/d9/f16 [0,4194304] 0 2024-02-22T15:50:37.890 INFO:tasks.workunit.client.1.smithi062.stdout:2/407: mkdir d1/d28/d3f/d7d 0 2024-02-22T15:50:37.891 INFO:tasks.workunit.client.1.smithi062.stdout:8/345: stat d8/f53 0 2024-02-22T15:50:37.891 INFO:tasks.workunit.client.0.smithi019.stdout:7/740: symlink d9/d38/d41/d45/d61/lf9 0 2024-02-22T15:50:37.892 INFO:tasks.workunit.client.1.smithi062.stdout:3/340: mknod d5/d16/d21/d32/c68 0 2024-02-22T15:50:37.893 INFO:tasks.workunit.client.1.smithi062.stdout:5/206: creat d0/d3/da/d13/d23/f3b x:0 0 0 2024-02-22T15:50:37.894 INFO:tasks.workunit.client.0.smithi019.stdout:7/741: dread - d9/d3a/d49/d5d/d74/f9e zero size 2024-02-22T15:50:37.894 INFO:tasks.workunit.client.0.smithi019.stdout:6/614: dwrite d6/d58/d6a/f85 [0,4194304] 0 2024-02-22T15:50:37.895 INFO:tasks.workunit.client.1.smithi062.stdout:4/312: getdents d4/d13 0 2024-02-22T15:50:37.896 INFO:tasks.workunit.client.1.smithi062.stdout:9/300: symlink d4/d35/d3f/l51 0 2024-02-22T15:50:37.897 INFO:tasks.workunit.client.1.smithi062.stdout:1/332: dwrite d1/d6/d10/f4b [0,4194304] 0 2024-02-22T15:50:37.898 INFO:tasks.workunit.client.1.smithi062.stdout:5/207: chown d0/d3/d2f/d38 253 1 2024-02-22T15:50:37.902 INFO:tasks.workunit.client.1.smithi062.stdout:6/442: dread d5/df/d40/d44/d76/f36 [0,4194304] 0 2024-02-22T15:50:37.902 INFO:tasks.workunit.client.0.smithi019.stdout:6/615: creat d6/d4b/d6d/fbb x:0 0 0 2024-02-22T15:50:37.902 INFO:tasks.workunit.client.0.smithi019.stdout:2/541: creat d1/d17/fc8 x:0 0 0 2024-02-22T15:50:37.903 INFO:tasks.workunit.client.1.smithi062.stdout:5/208: chown d0/d3/da/d13/l1f 69507 1 2024-02-22T15:50:37.904 INFO:tasks.workunit.client.1.smithi062.stdout:8/346: mknod d8/d9/d44/d4d/d51/c63 0 2024-02-22T15:50:37.905 INFO:tasks.workunit.client.0.smithi019.stdout:2/542: rmdir d1/d17/d67 39 2024-02-22T15:50:37.905 INFO:tasks.workunit.client.1.smithi062.stdout:2/408: rename d1/d8/df/f4c to d1/d28/d3f/d65/f7e 0 2024-02-22T15:50:37.907 INFO:tasks.workunit.client.0.smithi019.stdout:2/543: chown d1/de/l13 0 1 2024-02-22T15:50:37.908 INFO:tasks.workunit.client.1.smithi062.stdout:6/443: creat d5/df/d40/f8e x:0 0 0 2024-02-22T15:50:37.910 INFO:tasks.workunit.client.0.smithi019.stdout:2/544: creat d1/d17/d21/d51/fc9 x:0 0 0 2024-02-22T15:50:37.911 INFO:tasks.workunit.client.1.smithi062.stdout:9/301: creat d4/d28/d50/f52 x:0 0 0 2024-02-22T15:50:37.913 INFO:tasks.workunit.client.0.smithi019.stdout:2/545: creat d1/db4/fca x:0 0 0 2024-02-22T15:50:37.914 INFO:tasks.workunit.client.1.smithi062.stdout:6/444: write d5/df/d40/d44/d76/f8b [763118,3086] 0 2024-02-22T15:50:37.914 INFO:tasks.workunit.client.0.smithi019.stdout:2/546: chown d1/f76 468604 1 2024-02-22T15:50:37.915 INFO:tasks.workunit.client.0.smithi019.stdout:7/742: truncate d9/d38/f52 2436915 0 2024-02-22T15:50:37.915 INFO:tasks.workunit.client.1.smithi062.stdout:8/347: readlink l5 0 2024-02-22T15:50:37.915 INFO:tasks.workunit.client.0.smithi019.stdout:2/547: fdatasync d1/d17/d21/daf/fc3 0 2024-02-22T15:50:37.918 INFO:tasks.workunit.client.0.smithi019.stdout:7/743: read d9/f24 [533449,87469] 0 2024-02-22T15:50:37.918 INFO:tasks.workunit.client.1.smithi062.stdout:2/409: chown d1/d8/c7c 520810 1 2024-02-22T15:50:37.918 INFO:tasks.workunit.client.1.smithi062.stdout:9/302: creat d4/d1c/f53 x:0 0 0 2024-02-22T15:50:37.919 INFO:tasks.workunit.client.0.smithi019.stdout:7/744: readlink d9/d3a/d49/d5d/d74/d9a/lef 0 2024-02-22T15:50:37.919 INFO:tasks.workunit.client.1.smithi062.stdout:4/313: dread d4/d1c/f23 [0,4194304] 0 2024-02-22T15:50:37.921 INFO:tasks.workunit.client.1.smithi062.stdout:8/348: chown d8/d9/d44/d4d/d51/d55 14702 1 2024-02-22T15:50:37.921 INFO:tasks.workunit.client.1.smithi062.stdout:1/333: dread d1/f30 [0,4194304] 0 2024-02-22T15:50:37.922 INFO:tasks.workunit.client.0.smithi019.stdout:7/745: creat d9/d38/d41/d45/d92/ffa x:0 0 0 2024-02-22T15:50:37.923 INFO:tasks.workunit.client.1.smithi062.stdout:6/445: rename d5/df/d40/d44/d78/d59 to d5/df/d32/d8f 0 2024-02-22T15:50:37.923 INFO:tasks.workunit.client.1.smithi062.stdout:4/314: read d4/d1c/f3d [112765,57314] 0 2024-02-22T15:50:37.924 INFO:tasks.workunit.client.1.smithi062.stdout:5/209: dwrite d0/f26 [0,4194304] 0 2024-02-22T15:50:37.925 INFO:tasks.workunit.client.0.smithi019.stdout:7/746: unlink d9/d38/d41/d45/db0/fde 0 2024-02-22T15:50:37.927 INFO:tasks.workunit.client.1.smithi062.stdout:6/446: fsync d5/df/d40/d44/d76/f52 0 2024-02-22T15:50:37.927 INFO:tasks.workunit.client.1.smithi062.stdout:2/410: creat d1/d28/d3f/d65/f7f x:0 0 0 2024-02-22T15:50:37.928 INFO:tasks.workunit.client.0.smithi019.stdout:6/616: dread d6/d3e/f50 [4194304,4194304] 0 2024-02-22T15:50:37.931 INFO:tasks.workunit.client.0.smithi019.stdout:7/747: rename d9/d3a/d49/d5d/d74/d9a to d9/d38/d41/d45/db0/df7/dfb 0 2024-02-22T15:50:37.934 INFO:tasks.workunit.client.0.smithi019.stdout:6/617: rmdir d6/d4b 39 2024-02-22T15:50:37.934 INFO:tasks.workunit.client.1.smithi062.stdout:5/210: write d0/d3/da/d13/f2d [824414,80906] 0 2024-02-22T15:50:37.935 INFO:tasks.workunit.client.0.smithi019.stdout:7/748: chown d9/d38/d41/lec 107 1 2024-02-22T15:50:37.937 INFO:tasks.workunit.client.1.smithi062.stdout:9/303: dread d4/d15/d31/f4a [0,4194304] 0 2024-02-22T15:50:37.937 INFO:tasks.workunit.client.1.smithi062.stdout:2/411: mkdir d1/d47/d32/d52/d80 0 2024-02-22T15:50:37.937 INFO:tasks.workunit.client.1.smithi062.stdout:6/447: dread d5/df/d40/d44/d76/d56/f61 [0,4194304] 0 2024-02-22T15:50:37.940 INFO:tasks.workunit.client.1.smithi062.stdout:8/349: link l7 d8/d42/l64 0 2024-02-22T15:50:37.940 INFO:tasks.workunit.client.1.smithi062.stdout:4/315: dwrite d4/d1c/d30/d3b/f67 [0,4194304] 0 2024-02-22T15:50:37.941 INFO:tasks.workunit.client.0.smithi019.stdout:6/618: creat d6/d31/fbc x:0 0 0 2024-02-22T15:50:37.950 INFO:tasks.workunit.client.0.smithi019.stdout:6/619: dwrite d6/d31/d43/f8b [0,4194304] 0 2024-02-22T15:50:37.950 INFO:tasks.workunit.client.1.smithi062.stdout:3/341: truncate d5/d26/f35 8163982 0 2024-02-22T15:50:37.950 INFO:tasks.workunit.client.1.smithi062.stdout:5/211: write d0/d3/da/d13/d23/f3b [46672,110535] 0 2024-02-22T15:50:37.950 INFO:tasks.workunit.client.1.smithi062.stdout:2/412: creat d1/d1e/d38/f81 x:0 0 0 2024-02-22T15:50:37.951 INFO:tasks.workunit.client.0.smithi019.stdout:6/620: read - d6/d31/d48/f51 zero size 2024-02-22T15:50:37.952 INFO:tasks.workunit.client.0.smithi019.stdout:6/621: stat d6/d98/fa5 0 2024-02-22T15:50:37.953 INFO:tasks.workunit.client.1.smithi062.stdout:2/413: dread - d1/d8/df/f78 zero size 2024-02-22T15:50:37.953 INFO:tasks.workunit.client.1.smithi062.stdout:4/316: mknod d4/d1c/d2c/d6b/c6c 0 2024-02-22T15:50:37.953 INFO:tasks.workunit.client.1.smithi062.stdout:5/212: fsync d0/f39 0 2024-02-22T15:50:37.954 INFO:tasks.workunit.client.1.smithi062.stdout:1/334: write d1/f2e [817632,2453] 0 2024-02-22T15:50:37.954 INFO:tasks.workunit.client.0.smithi019.stdout:7/749: dread d9/d26/f32 [0,4194304] 0 2024-02-22T15:50:37.954 INFO:tasks.workunit.client.0.smithi019.stdout:6/622: write d6/d31/d43/f8b [3836754,58476] 0 2024-02-22T15:50:37.955 INFO:tasks.workunit.client.1.smithi062.stdout:3/342: chown d5/d16/l2c 76491269 1 2024-02-22T15:50:37.955 INFO:tasks.workunit.client.0.smithi019.stdout:6/623: readlink d6/l53 0 2024-02-22T15:50:37.956 INFO:tasks.workunit.client.0.smithi019.stdout:6/624: read d6/d98/fa5 [3394283,7117] 0 2024-02-22T15:50:37.957 INFO:tasks.workunit.client.1.smithi062.stdout:8/350: dread d8/d26/f47 [0,4194304] 0 2024-02-22T15:50:37.958 INFO:tasks.workunit.client.0.smithi019.stdout:7/750: mkdir d9/d38/d41/d45/db0/df7/dfb/de3/dfc 0 2024-02-22T15:50:37.960 INFO:tasks.workunit.client.1.smithi062.stdout:5/213: read - d0/d3/d2f/f3a zero size 2024-02-22T15:50:37.960 INFO:tasks.workunit.client.1.smithi062.stdout:6/448: dwrite d5/df/d32/d47/f50 [0,4194304] 0 2024-02-22T15:50:37.961 INFO:tasks.workunit.client.0.smithi019.stdout:6/625: mknod d6/d4b/da9/d67/cbd 0 2024-02-22T15:50:37.962 INFO:tasks.workunit.client.0.smithi019.stdout:7/751: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/ffd x:0 0 0 2024-02-22T15:50:37.963 INFO:tasks.workunit.client.1.smithi062.stdout:6/449: readlink d5/df/d40/d44/d78/l7b 0 2024-02-22T15:50:37.964 INFO:tasks.workunit.client.1.smithi062.stdout:6/450: read d5/df/d32/d8f/f7e [3096585,30685] 0 2024-02-22T15:50:37.971 INFO:tasks.workunit.client.1.smithi062.stdout:5/214: mknod d0/d3/da/d13/c3c 0 2024-02-22T15:50:37.973 INFO:tasks.workunit.client.1.smithi062.stdout:8/351: chown d8/d9/f18 660003 1 2024-02-22T15:50:37.975 INFO:tasks.workunit.client.1.smithi062.stdout:5/215: fsync d0/d3/da/d13/d23/f3b 0 2024-02-22T15:50:37.976 INFO:tasks.workunit.client.1.smithi062.stdout:2/414: rename d1/d8/d62/f68 to d1/f82 0 2024-02-22T15:50:37.976 INFO:tasks.workunit.client.1.smithi062.stdout:8/352: write d8/d9/d44/f56 [746027,18852] 0 2024-02-22T15:50:37.977 INFO:tasks.workunit.client.0.smithi019.stdout:7/752: dread d9/d38/d41/f48 [0,4194304] 0 2024-02-22T15:50:37.978 INFO:tasks.workunit.client.1.smithi062.stdout:1/335: creat d1/d6/d14/f6d x:0 0 0 2024-02-22T15:50:37.979 INFO:tasks.workunit.client.1.smithi062.stdout:6/451: dwrite d5/df/d40/d44/d76/f6a [0,4194304] 0 2024-02-22T15:50:37.979 INFO:tasks.workunit.client.0.smithi019.stdout:7/753: stat d9/d3a/d49/d5d/dd6 0 2024-02-22T15:50:37.979 INFO:tasks.workunit.client.0.smithi019.stdout:6/626: dread d6/d31/f36 [0,4194304] 0 2024-02-22T15:50:37.980 INFO:tasks.workunit.client.0.smithi019.stdout:7/754: dread - d9/d38/d41/d45/d61/fc0 zero size 2024-02-22T15:50:37.980 INFO:tasks.workunit.client.1.smithi062.stdout:8/353: chown d8/d9/fe 2808544 1 2024-02-22T15:50:37.981 INFO:tasks.workunit.client.0.smithi019.stdout:7/755: fdatasync d9/ff6 0 2024-02-22T15:50:37.982 INFO:tasks.workunit.client.1.smithi062.stdout:9/304: dread d4/f8 [0,4194304] 0 2024-02-22T15:50:37.983 INFO:tasks.workunit.client.1.smithi062.stdout:1/336: chown d1/d6/d36/c6c 9 1 2024-02-22T15:50:37.983 INFO:tasks.workunit.client.0.smithi019.stdout:6/627: truncate d6/d3e/f42 3739307 0 2024-02-22T15:50:37.986 INFO:tasks.workunit.client.1.smithi062.stdout:8/354: fsync d8/d26/f41 0 2024-02-22T15:50:37.986 INFO:tasks.workunit.client.0.smithi019.stdout:7/756: unlink d9/d79/c85 0 2024-02-22T15:50:37.987 INFO:tasks.workunit.client.0.smithi019.stdout:6/628: mknod d6/d4b/d8d/cbe 0 2024-02-22T15:50:37.987 INFO:tasks.workunit.client.1.smithi062.stdout:4/317: rename d4/d13/d20/f25 to d4/d1c/d30/f6d 0 2024-02-22T15:50:37.988 INFO:tasks.workunit.client.1.smithi062.stdout:4/318: chown d4/d13 810 1 2024-02-22T15:50:37.989 INFO:tasks.workunit.client.0.smithi019.stdout:6/629: write d6/d4b/d6d/f81 [1881791,43704] 0 2024-02-22T15:50:37.992 INFO:tasks.workunit.client.0.smithi019.stdout:6/630: chown d6/f1c 602365 1 2024-02-22T15:50:37.992 INFO:tasks.workunit.client.1.smithi062.stdout:6/452: mknod d5/df/d32/c90 0 2024-02-22T15:50:37.993 INFO:tasks.workunit.client.0.smithi019.stdout:6/631: rmdir d6/d4b/da9/d67 39 2024-02-22T15:50:37.994 INFO:tasks.workunit.client.1.smithi062.stdout:9/305: rmdir d4 39 2024-02-22T15:50:37.994 INFO:tasks.workunit.client.0.smithi019.stdout:7/757: link d9/c12 d9/d38/d41/d45/d61/d6a/cfe 0 2024-02-22T15:50:37.996 INFO:tasks.workunit.client.1.smithi062.stdout:1/337: symlink d1/d6/d7/d4a/l6e 0 2024-02-22T15:50:37.996 INFO:tasks.workunit.client.1.smithi062.stdout:6/453: write d5/df/d40/d44/d4a/f5e [214402,112460] 0 2024-02-22T15:50:37.996 INFO:tasks.workunit.client.1.smithi062.stdout:3/343: dwrite d5/d1a/d3c/f53 [4194304,4194304] 0 2024-02-22T15:50:37.998 INFO:tasks.workunit.client.0.smithi019.stdout:7/758: mknod d9/d4c/d5b/cff 0 2024-02-22T15:50:37.999 INFO:tasks.workunit.client.1.smithi062.stdout:6/454: chown d5/df/d40 180141 1 2024-02-22T15:50:38.000 INFO:tasks.workunit.client.0.smithi019.stdout:7/759: read d9/d4c/f66 [2132688,10901] 0 2024-02-22T15:50:38.003 INFO:tasks.workunit.client.1.smithi062.stdout:5/216: link d0/d3/f14 d0/d3/da/d13/f3d 0 2024-02-22T15:50:38.005 INFO:tasks.workunit.client.1.smithi062.stdout:4/319: dread d4/f39 [0,4194304] 0 2024-02-22T15:50:38.006 INFO:tasks.workunit.client.1.smithi062.stdout:7/299: sync 2024-02-22T15:50:38.006 INFO:tasks.workunit.client.1.smithi062.stdout:0/403: sync 2024-02-22T15:50:38.008 INFO:tasks.workunit.client.0.smithi019.stdout:7/760: dread d9/f34 [0,4194304] 0 2024-02-22T15:50:38.010 INFO:tasks.workunit.client.1.smithi062.stdout:4/320: readlink d4/d1c/d30/d37/l42 0 2024-02-22T15:50:38.012 INFO:tasks.workunit.client.0.smithi019.stdout:7/761: link d9/d26/dc4/fda d9/d38/d41/d45/db0/df7/f100 0 2024-02-22T15:50:38.014 INFO:tasks.workunit.client.1.smithi062.stdout:6/455: dwrite d5/df/d32/f3d [0,4194304] 0 2024-02-22T15:50:38.014 INFO:tasks.workunit.client.1.smithi062.stdout:2/415: getdents d1/d8/d62 0 2024-02-22T15:50:38.015 INFO:tasks.workunit.client.0.smithi019.stdout:6/632: dread d6/d4b/d6d/d7b/d7f/fb8 [0,4194304] 0 2024-02-22T15:50:38.016 INFO:tasks.workunit.client.1.smithi062.stdout:2/416: readlink d1/d8/d62/d73/l77 0 2024-02-22T15:50:38.016 INFO:tasks.workunit.client.1.smithi062.stdout:5/217: creat d0/d3/da/d13/d23/f3e x:0 0 0 2024-02-22T15:50:38.016 INFO:tasks.workunit.client.0.smithi019.stdout:7/762: getdents d9/d3a/d49 0 2024-02-22T15:50:38.016 INFO:tasks.workunit.client.0.smithi019.stdout:6/633: write d6/d4b/f52 [1547210,30562] 0 2024-02-22T15:50:38.017 INFO:tasks.workunit.client.1.smithi062.stdout:0/404: symlink d1/dd/l89 0 2024-02-22T15:50:38.017 INFO:tasks.workunit.client.0.smithi019.stdout:6/634: dread - d6/d4b/d71/f7d zero size 2024-02-22T15:50:38.017 INFO:tasks.workunit.client.1.smithi062.stdout:5/218: dread - d0/f39 zero size 2024-02-22T15:50:38.019 INFO:tasks.workunit.client.0.smithi019.stdout:7/763: creat d9/d26/f101 x:0 0 0 2024-02-22T15:50:38.019 INFO:tasks.workunit.client.0.smithi019.stdout:6/635: chown d6/d31/d43/f5f 817093363 1 2024-02-22T15:50:38.020 INFO:tasks.workunit.client.1.smithi062.stdout:8/355: getdents d8/d9/d44 0 2024-02-22T15:50:38.020 INFO:tasks.workunit.client.0.smithi019.stdout:7/764: fdatasync d9/f23 0 2024-02-22T15:50:38.020 INFO:tasks.workunit.client.1.smithi062.stdout:1/338: link d1/d66/l67 d1/d6/d10/l6f 0 2024-02-22T15:50:38.021 INFO:tasks.workunit.client.0.smithi019.stdout:0/451: sync 2024-02-22T15:50:38.025 INFO:tasks.workunit.client.1.smithi062.stdout:7/300: rename d0/d14/d28/f47 to d0/d14/d19/d30/d37/f63 0 2024-02-22T15:50:38.026 INFO:tasks.workunit.client.1.smithi062.stdout:3/344: truncate d5/d26/f27 224000 0 2024-02-22T15:50:38.026 INFO:tasks.workunit.client.1.smithi062.stdout:9/306: link d4/f10 d4/d35/d3f/f54 0 2024-02-22T15:50:38.028 INFO:tasks.workunit.client.0.smithi019.stdout:6/636: symlink d6/d4b/d6d/lbf 0 2024-02-22T15:50:38.029 INFO:tasks.workunit.client.1.smithi062.stdout:6/456: write d5/df/d40/d44/d76/f36 [2459907,69605] 0 2024-02-22T15:50:38.029 INFO:tasks.workunit.client.1.smithi062.stdout:8/356: symlink d8/d9/d2c/l65 0 2024-02-22T15:50:38.029 INFO:tasks.workunit.client.0.smithi019.stdout:6/637: stat d6/d31/d48/f4a 0 2024-02-22T15:50:38.029 INFO:tasks.workunit.client.0.smithi019.stdout:7/765: mkdir d9/d38/d41/d102 0 2024-02-22T15:50:38.031 INFO:tasks.workunit.client.0.smithi019.stdout:7/766: fdatasync d9/d3a/d49/d5d/d74/f9e 0 2024-02-22T15:50:38.032 INFO:tasks.workunit.client.1.smithi062.stdout:7/301: mknod d0/d14/d13/c64 0 2024-02-22T15:50:38.032 INFO:tasks.workunit.client.0.smithi019.stdout:0/452: creat d4/db/d24/d80/fa0 x:0 0 0 2024-02-22T15:50:38.032 INFO:tasks.workunit.client.1.smithi062.stdout:3/345: fdatasync d5/d16/f48 0 2024-02-22T15:50:38.034 INFO:tasks.workunit.client.0.smithi019.stdout:6/638: mkdir d6/d4b/d71/d88/dc0 0 2024-02-22T15:50:38.034 INFO:tasks.workunit.client.1.smithi062.stdout:5/219: mkdir d0/d3/d2f/d38/d3f 0 2024-02-22T15:50:38.035 INFO:tasks.workunit.client.0.smithi019.stdout:6/639: readlink d6/d31/d69/l9e 0 2024-02-22T15:50:38.036 INFO:tasks.workunit.client.1.smithi062.stdout:2/417: link d1/d8/df/f4a d1/d47/f83 0 2024-02-22T15:50:38.038 INFO:tasks.workunit.client.1.smithi062.stdout:9/307: creat d4/d28/d50/f55 x:0 0 0 2024-02-22T15:50:38.040 INFO:tasks.workunit.client.1.smithi062.stdout:5/220: dread d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:38.040 INFO:tasks.workunit.client.1.smithi062.stdout:4/321: rmdir d4/d2a/d5e 0 2024-02-22T15:50:38.043 INFO:tasks.workunit.client.0.smithi019.stdout:6/640: dwrite d6/d4b/d8d/f96 [0,4194304] 0 2024-02-22T15:50:38.044 INFO:tasks.workunit.client.1.smithi062.stdout:5/221: readlink d0/d3/da/d13/l37 0 2024-02-22T15:50:38.044 INFO:tasks.workunit.client.1.smithi062.stdout:2/418: dread d1/d1e/f66 [0,4194304] 0 2024-02-22T15:50:38.045 INFO:tasks.workunit.client.1.smithi062.stdout:6/457: creat d5/df/d32/d8f/f91 x:0 0 0 2024-02-22T15:50:38.045 INFO:tasks.workunit.client.0.smithi019.stdout:6/641: mknod d6/d31/d69/d8e/cc1 0 2024-02-22T15:50:38.049 INFO:tasks.workunit.client.1.smithi062.stdout:3/346: rmdir d5/d1a/d3c 39 2024-02-22T15:50:38.049 INFO:tasks.workunit.client.0.smithi019.stdout:6/642: mkdir d6/d4b/d71/dc2 0 2024-02-22T15:50:38.051 INFO:tasks.workunit.client.1.smithi062.stdout:3/347: readlink d5/d16/d21/d36/l4b 0 2024-02-22T15:50:38.052 INFO:tasks.workunit.client.0.smithi019.stdout:6/643: creat d6/d4b/da9/d82/fc3 x:0 0 0 2024-02-22T15:50:38.053 INFO:tasks.workunit.client.0.smithi019.stdout:0/453: dread d4/d32/f46 [0,4194304] 0 2024-02-22T15:50:38.053 INFO:tasks.workunit.client.0.smithi019.stdout:0/454: fsync d4/d5/d52/f62 0 2024-02-22T15:50:38.054 INFO:tasks.workunit.client.1.smithi062.stdout:4/322: unlink d4/d13/f43 0 2024-02-22T15:50:38.056 INFO:tasks.workunit.client.1.smithi062.stdout:1/339: dwrite d1/d6/d7/d2b/d49/f6a [0,4194304] 0 2024-02-22T15:50:38.056 INFO:tasks.workunit.client.1.smithi062.stdout:0/405: link d1/d3/d4/d1a/c6e d1/d3/d4/d1a/d1d/d3b/c8a 0 2024-02-22T15:50:38.057 INFO:tasks.workunit.client.1.smithi062.stdout:7/302: dwrite d0/d14/d13/f36 [0,4194304] 0 2024-02-22T15:50:38.058 INFO:tasks.workunit.client.0.smithi019.stdout:6/644: rename d6/d4b/da9/d82/fc3 to d6/d4b/d6d/d7b/d78/fc4 0 2024-02-22T15:50:38.059 INFO:tasks.workunit.client.0.smithi019.stdout:0/455: mkdir d4/db/d7f/da1 0 2024-02-22T15:50:38.059 INFO:tasks.workunit.client.1.smithi062.stdout:0/406: fdatasync d1/d3/de/d3f/f80 0 2024-02-22T15:50:38.060 INFO:tasks.workunit.client.0.smithi019.stdout:7/767: dread d9/d38/d41/d45/f70 [0,4194304] 0 2024-02-22T15:50:38.061 INFO:tasks.workunit.client.1.smithi062.stdout:7/303: truncate d0/d6/d2e/d58/f5c 613368 0 2024-02-22T15:50:38.061 INFO:tasks.workunit.client.1.smithi062.stdout:0/407: write d1/d13/d43/f53 [5007393,3759] 0 2024-02-22T15:50:38.061 INFO:tasks.workunit.client.0.smithi019.stdout:7/768: fdatasync d9/d38/d41/f78 0 2024-02-22T15:50:38.062 INFO:tasks.workunit.client.0.smithi019.stdout:0/456: mknod d4/db/d49/ca2 0 2024-02-22T15:50:38.065 INFO:tasks.workunit.client.1.smithi062.stdout:5/222: unlink d0/d3/da/d13/l37 0 2024-02-22T15:50:38.065 INFO:tasks.workunit.client.0.smithi019.stdout:0/457: fdatasync d4/d5/fe 0 2024-02-22T15:50:38.065 INFO:tasks.workunit.client.0.smithi019.stdout:6/645: symlink d6/lc5 0 2024-02-22T15:50:38.069 INFO:tasks.workunit.client.0.smithi019.stdout:7/769: dread d9/d38/f3c [0,4194304] 0 2024-02-22T15:50:38.069 INFO:tasks.workunit.client.0.smithi019.stdout:6/646: chown d6/d4b/d6d/d7b/d7f/f89 274468 1 2024-02-22T15:50:38.069 INFO:tasks.workunit.client.1.smithi062.stdout:2/419: rename d1/d1e/d7a to d1/d28/d3f/d7d/d84 0 2024-02-22T15:50:38.070 INFO:tasks.workunit.client.1.smithi062.stdout:6/458: mknod d5/df/d32/d47/c92 0 2024-02-22T15:50:38.070 INFO:tasks.workunit.client.1.smithi062.stdout:5/223: dread d0/d3/da/d13/d23/f3b [0,4194304] 0 2024-02-22T15:50:38.070 INFO:tasks.workunit.client.1.smithi062.stdout:1/340: fdatasync d1/f30 0 2024-02-22T15:50:38.070 INFO:tasks.workunit.client.1.smithi062.stdout:3/348: write d5/d1a/d3c/f54 [347499,10578] 0 2024-02-22T15:50:38.072 INFO:tasks.workunit.client.0.smithi019.stdout:7/770: mknod d9/d26/c103 0 2024-02-22T15:50:38.072 INFO:tasks.workunit.client.1.smithi062.stdout:0/408: unlink d1/d3/d5f/f46 0 2024-02-22T15:50:38.073 INFO:tasks.workunit.client.1.smithi062.stdout:6/459: chown d5/df/d40/d44/d4a/f1c 2241 1 2024-02-22T15:50:38.074 INFO:tasks.workunit.client.1.smithi062.stdout:0/409: chown d1/d3/d4/d1a/f72 47487 1 2024-02-22T15:50:38.076 INFO:tasks.workunit.client.1.smithi062.stdout:7/304: creat d0/d6/d2e/d55/f65 x:0 0 0 2024-02-22T15:50:38.078 INFO:tasks.workunit.client.0.smithi019.stdout:7/771: creat d9/d4c/f104 x:0 0 0 2024-02-22T15:50:38.079 INFO:tasks.workunit.client.1.smithi062.stdout:5/224: unlink d0/l17 0 2024-02-22T15:50:38.083 INFO:tasks.workunit.client.1.smithi062.stdout:3/349: symlink d5/d26/l69 0 2024-02-22T15:50:38.084 INFO:tasks.workunit.client.0.smithi019.stdout:7/772: truncate d9/f34 641481 0 2024-02-22T15:50:38.085 INFO:tasks.workunit.client.1.smithi062.stdout:9/308: getdents d4/d28 0 2024-02-22T15:50:38.085 INFO:tasks.workunit.client.1.smithi062.stdout:9/309: chown d4/d15/l3d 8021517 1 2024-02-22T15:50:38.085 INFO:tasks.workunit.client.1.smithi062.stdout:4/323: link d4/d13/l22 d4/d2a/d2d/d46/l6e 0 2024-02-22T15:50:38.086 INFO:tasks.workunit.client.1.smithi062.stdout:9/310: readlink d4/d15/l3d 0 2024-02-22T15:50:38.086 INFO:tasks.workunit.client.1.smithi062.stdout:0/410: write d1/d3/de/f3a [2575226,24502] 0 2024-02-22T15:50:38.087 INFO:tasks.workunit.client.1.smithi062.stdout:6/460: mknod d5/df/d40/d44/d4a/d41/c93 0 2024-02-22T15:50:38.088 INFO:tasks.workunit.client.1.smithi062.stdout:2/420: getdents d1/d1e/d38/d76 0 2024-02-22T15:50:38.089 INFO:tasks.workunit.client.1.smithi062.stdout:2/421: fsync d1/d28/d3f/f46 0 2024-02-22T15:50:38.092 INFO:tasks.workunit.client.1.smithi062.stdout:4/324: chown d4/df/f14 51238 1 2024-02-22T15:50:38.092 INFO:tasks.workunit.client.1.smithi062.stdout:9/311: creat d4/d1c/f56 x:0 0 0 2024-02-22T15:50:38.094 INFO:tasks.workunit.client.1.smithi062.stdout:0/411: unlink d1/d3/de/d3f/f80 0 2024-02-22T15:50:38.096 INFO:tasks.workunit.client.1.smithi062.stdout:7/305: truncate d0/d14/d13/fc 1153263 0 2024-02-22T15:50:38.097 INFO:tasks.workunit.client.1.smithi062.stdout:0/412: dread - d1/d13/f31 zero size 2024-02-22T15:50:38.098 INFO:tasks.workunit.client.1.smithi062.stdout:2/422: dwrite d1/d1e/d38/f4f [4194304,4194304] 0 2024-02-22T15:50:38.100 INFO:tasks.workunit.client.1.smithi062.stdout:5/225: rename d0/c35 to d0/d3/da/d32/d33/c40 0 2024-02-22T15:50:38.107 INFO:tasks.workunit.client.1.smithi062.stdout:7/306: dwrite d0/f24 [0,4194304] 0 2024-02-22T15:50:38.110 INFO:tasks.workunit.client.1.smithi062.stdout:1/341: getdents d1/d6 0 2024-02-22T15:50:38.111 INFO:tasks.workunit.client.1.smithi062.stdout:1/342: chown d1/d6/d7/d2b 6 1 2024-02-22T15:50:38.115 INFO:tasks.workunit.client.1.smithi062.stdout:2/423: fsync d1/f82 0 2024-02-22T15:50:38.115 INFO:tasks.workunit.client.1.smithi062.stdout:2/424: chown d1/d47/d39/d5d 13843673 1 2024-02-22T15:50:38.116 INFO:tasks.workunit.client.1.smithi062.stdout:9/312: rename d4/d1c/d48 to d4/d35/d3f/d57 0 2024-02-22T15:50:38.117 INFO:tasks.workunit.client.1.smithi062.stdout:9/313: chown d4/d28/f30 9 1 2024-02-22T15:50:38.117 INFO:tasks.workunit.client.1.smithi062.stdout:6/461: creat d5/dd/f94 x:0 0 0 2024-02-22T15:50:38.119 INFO:tasks.workunit.client.1.smithi062.stdout:9/314: read d4/d35/f36 [846872,40961] 0 2024-02-22T15:50:38.119 INFO:tasks.workunit.client.1.smithi062.stdout:5/226: creat d0/d3/d2f/d38/f41 x:0 0 0 2024-02-22T15:50:38.121 INFO:tasks.workunit.client.1.smithi062.stdout:4/325: creat d4/d66/f6f x:0 0 0 2024-02-22T15:50:38.121 INFO:tasks.workunit.client.1.smithi062.stdout:1/343: symlink d1/d6/d36/l70 0 2024-02-22T15:50:38.123 INFO:tasks.workunit.client.1.smithi062.stdout:2/425: creat d1/d8/df/f85 x:0 0 0 2024-02-22T15:50:38.125 INFO:tasks.workunit.client.1.smithi062.stdout:0/413: rename d1/d3/de/d3f/l40 to d1/d5e/l8b 0 2024-02-22T15:50:38.128 INFO:tasks.workunit.client.1.smithi062.stdout:3/350: dwrite d5/f1d [0,4194304] 0 2024-02-22T15:50:38.128 INFO:tasks.workunit.client.1.smithi062.stdout:6/462: creat d5/df/d40/d44/f95 x:0 0 0 2024-02-22T15:50:38.136 INFO:tasks.workunit.client.1.smithi062.stdout:1/344: dread - d1/f27 zero size 2024-02-22T15:50:38.136 INFO:tasks.workunit.client.1.smithi062.stdout:4/326: mkdir d4/d1c/d2c/d6b/d70 0 2024-02-22T15:50:38.136 INFO:tasks.workunit.client.1.smithi062.stdout:6/463: readlink d5/df/d40/d44/d4a/l64 0 2024-02-22T15:50:38.136 INFO:tasks.workunit.client.1.smithi062.stdout:2/426: symlink d1/d47/l86 0 2024-02-22T15:50:38.136 INFO:tasks.workunit.client.1.smithi062.stdout:4/327: fdatasync d4/d1c/d2c/f56 0 2024-02-22T15:50:38.136 INFO:tasks.workunit.client.1.smithi062.stdout:7/307: rename d0/d6/f54 to d0/d6/d5b/d60/f66 0 2024-02-22T15:50:38.137 INFO:tasks.workunit.client.1.smithi062.stdout:6/464: write d5/df/d32/f4e [2020094,57243] 0 2024-02-22T15:50:38.137 INFO:tasks.workunit.client.1.smithi062.stdout:0/414: fdatasync d1/d3/de/d3f/d2c/d84/f2a 0 2024-02-22T15:50:38.139 INFO:tasks.workunit.client.1.smithi062.stdout:3/351: mkdir d5/d16/d21/d32/d6a 0 2024-02-22T15:50:38.144 INFO:tasks.workunit.client.1.smithi062.stdout:7/308: write d0/d14/f25 [693160,111081] 0 2024-02-22T15:50:38.144 INFO:tasks.workunit.client.1.smithi062.stdout:5/227: dread d0/d3/da/fb [0,4194304] 0 2024-02-22T15:50:38.146 INFO:tasks.workunit.client.1.smithi062.stdout:1/345: mknod d1/d6/d10/c71 0 2024-02-22T15:50:38.147 INFO:tasks.workunit.client.1.smithi062.stdout:4/328: mknod d4/d1c/d30/d3b/c71 0 2024-02-22T15:50:38.149 INFO:tasks.workunit.client.1.smithi062.stdout:7/309: write d0/d14/d13/f4d [182026,84397] 0 2024-02-22T15:50:38.150 INFO:tasks.workunit.client.1.smithi062.stdout:7/310: chown d0/d14/f29 240608962 1 2024-02-22T15:50:38.152 INFO:tasks.workunit.client.1.smithi062.stdout:1/346: readlink d1/d66/l67 0 2024-02-22T15:50:38.154 INFO:tasks.workunit.client.1.smithi062.stdout:0/415: rename d1/d3/de/d3f to d1/d3/d4/d1a/d76/d8c 0 2024-02-22T15:50:38.158 INFO:tasks.workunit.client.1.smithi062.stdout:4/329: fdatasync d4/d1c/f23 0 2024-02-22T15:50:38.161 INFO:tasks.workunit.client.0.smithi019.stdout:0/458: dread d4/d32/f9c [0,4194304] 0 2024-02-22T15:50:38.162 INFO:tasks.workunit.client.1.smithi062.stdout:5/228: dwrite d0/f7 [0,4194304] 0 2024-02-22T15:50:38.163 INFO:tasks.workunit.client.1.smithi062.stdout:0/416: rmdir d1/d21 39 2024-02-22T15:50:38.165 INFO:tasks.workunit.client.0.smithi019.stdout:6/647: dread d6/f1c [0,4194304] 0 2024-02-22T15:50:38.165 INFO:tasks.workunit.client.0.smithi019.stdout:7/773: dread d9/d26/f87 [0,4194304] 0 2024-02-22T15:50:38.166 INFO:tasks.workunit.client.1.smithi062.stdout:0/417: fdatasync d1/d3/d4/d1a/d76/d8c/d2c/d84/f2a 0 2024-02-22T15:50:38.166 INFO:tasks.workunit.client.1.smithi062.stdout:4/330: mknod d4/d66/c72 0 2024-02-22T15:50:38.168 INFO:tasks.workunit.client.1.smithi062.stdout:4/331: write d4/d1c/f23 [160266,66088] 0 2024-02-22T15:50:38.168 INFO:tasks.workunit.client.0.smithi019.stdout:7/774: mkdir d9/d4c/ddf/d105 0 2024-02-22T15:50:38.169 INFO:tasks.workunit.client.0.smithi019.stdout:6/648: symlink d6/d4b/da9/d67/d8a/d90/lc6 0 2024-02-22T15:50:38.170 INFO:tasks.workunit.client.1.smithi062.stdout:2/427: write d1/d47/f83 [850046,28747] 0 2024-02-22T15:50:38.171 INFO:tasks.workunit.client.1.smithi062.stdout:4/332: fsync d4/d2a/d2d/d2e/f69 0 2024-02-22T15:50:38.171 INFO:tasks.workunit.client.0.smithi019.stdout:6/649: creat d6/d58/d6a/fc7 x:0 0 0 2024-02-22T15:50:38.172 INFO:tasks.workunit.client.1.smithi062.stdout:3/352: rename d5/d16/d21/d36/c5c to d5/d16/d56/c6b 0 2024-02-22T15:50:38.172 INFO:tasks.workunit.client.0.smithi019.stdout:7/775: dread d9/ff6 [0,4194304] 0 2024-02-22T15:50:38.173 INFO:tasks.workunit.client.1.smithi062.stdout:4/333: read - d4/d66/f6f zero size 2024-02-22T15:50:38.173 INFO:tasks.workunit.client.1.smithi062.stdout:7/311: creat d0/d14/f67 x:0 0 0 2024-02-22T15:50:38.174 INFO:tasks.workunit.client.1.smithi062.stdout:9/315: dwrite d4/f5 [0,4194304] 0 2024-02-22T15:50:38.174 INFO:tasks.workunit.client.1.smithi062.stdout:3/353: readlink d5/d1a/l40 0 2024-02-22T15:50:38.176 INFO:tasks.workunit.client.1.smithi062.stdout:7/312: write d0/f24 [1316595,84857] 0 2024-02-22T15:50:38.177 INFO:tasks.workunit.client.1.smithi062.stdout:5/229: mkdir d0/d3/da/d32/d42 0 2024-02-22T15:50:38.178 INFO:tasks.workunit.client.1.smithi062.stdout:6/465: truncate d5/f1a 441612 0 2024-02-22T15:50:38.178 INFO:tasks.workunit.client.1.smithi062.stdout:5/230: stat d0/d3/da/f27 0 2024-02-22T15:50:38.180 INFO:tasks.workunit.client.1.smithi062.stdout:0/418: rmdir d1/dd 39 2024-02-22T15:50:38.180 INFO:tasks.workunit.client.1.smithi062.stdout:6/466: readlink d5/dd/l2b 0 2024-02-22T15:50:38.180 INFO:tasks.workunit.client.1.smithi062.stdout:7/313: write d0/d14/d13/f36 [1086556,23882] 0 2024-02-22T15:50:38.182 INFO:tasks.workunit.client.0.smithi019.stdout:6/650: dwrite d6/d31/f32 [0,4194304] 0 2024-02-22T15:50:38.184 INFO:tasks.workunit.client.1.smithi062.stdout:2/428: symlink d1/d28/d3f/d7d/l87 0 2024-02-22T15:50:38.184 INFO:tasks.workunit.client.1.smithi062.stdout:1/347: link d1/d6/f61 d1/d6/de/f72 0 2024-02-22T15:50:38.185 INFO:tasks.workunit.client.1.smithi062.stdout:7/314: fsync d0/d6/d2e/d4a/f1f 0 2024-02-22T15:50:38.188 INFO:tasks.workunit.client.0.smithi019.stdout:6/651: rename d6/l45 to d6/d4b/d6d/d7b/d97/lc8 0 2024-02-22T15:50:38.189 INFO:tasks.workunit.client.0.smithi019.stdout:7/776: truncate d9/d3a/d49/d5d/d8f/fa3 836412 0 2024-02-22T15:50:38.190 INFO:tasks.workunit.client.0.smithi019.stdout:7/777: chown d9/d4c 2 1 2024-02-22T15:50:38.193 INFO:tasks.workunit.client.0.smithi019.stdout:7/778: dread - d9/d38/d41/d45/d92/fa6 zero size 2024-02-22T15:50:38.193 INFO:tasks.workunit.client.1.smithi062.stdout:6/467: fdatasync d5/df/d40/d44/d4a/d41/f55 0 2024-02-22T15:50:38.196 INFO:tasks.workunit.client.0.smithi019.stdout:6/652: rename d6/d31/fbc to d6/d31/fc9 0 2024-02-22T15:50:38.197 INFO:tasks.workunit.client.1.smithi062.stdout:6/468: chown d5/df/d32/d8f/l83 261171543 1 2024-02-22T15:50:38.197 INFO:tasks.workunit.client.1.smithi062.stdout:9/316: truncate d4/d15/f23 1186774 0 2024-02-22T15:50:38.198 INFO:tasks.workunit.client.0.smithi019.stdout:6/653: chown d6/d4b/d6d/fbb 260065 1 2024-02-22T15:50:38.198 INFO:tasks.workunit.client.0.smithi019.stdout:7/779: symlink d9/d38/d41/d45/db0/df7/dfb/de3/deb/l106 0 2024-02-22T15:50:38.198 INFO:tasks.workunit.client.0.smithi019.stdout:7/780: write d9/d4c/d5b/f65 [2637502,20265] 0 2024-02-22T15:50:38.200 INFO:tasks.workunit.client.1.smithi062.stdout:5/231: link d0/f9 d0/d3/da/d13/d23/d31/f43 0 2024-02-22T15:50:38.201 INFO:tasks.workunit.client.0.smithi019.stdout:0/459: dread d4/f5c [0,4194304] 0 2024-02-22T15:50:38.201 INFO:tasks.workunit.client.0.smithi019.stdout:7/781: write d9/d38/d41/fb7 [5105766,37840] 0 2024-02-22T15:50:38.202 INFO:tasks.workunit.client.0.smithi019.stdout:6/654: rename d6/d4b/l5d to d6/d31/d69/lca 0 2024-02-22T15:50:38.203 INFO:tasks.workunit.client.0.smithi019.stdout:0/460: dread - d4/db/d24/d80/fa0 zero size 2024-02-22T15:50:38.203 INFO:tasks.workunit.client.0.smithi019.stdout:7/782: chown d9/d26/f2e 3 1 2024-02-22T15:50:38.203 INFO:tasks.workunit.client.1.smithi062.stdout:2/429: link d1/d1e/d38/f74 d1/d47/f88 0 2024-02-22T15:50:38.203 INFO:tasks.workunit.client.0.smithi019.stdout:8/547: sync 2024-02-22T15:50:38.204 INFO:tasks.workunit.client.0.smithi019.stdout:5/595: sync 2024-02-22T15:50:38.204 INFO:tasks.workunit.client.0.smithi019.stdout:6/655: fdatasync d6/d31/d43/f56 0 2024-02-22T15:50:38.206 INFO:tasks.workunit.client.1.smithi062.stdout:2/430: write d1/d28/d3f/f46 [847918,41271] 0 2024-02-22T15:50:38.206 INFO:tasks.workunit.client.0.smithi019.stdout:0/461: fsync d4/d23/d7d/f85 0 2024-02-22T15:50:38.206 INFO:tasks.workunit.client.0.smithi019.stdout:6/656: dread - d6/d4b/d8d/f95 zero size 2024-02-22T15:50:38.207 INFO:tasks.workunit.client.0.smithi019.stdout:5/596: creat d8/db1/fc6 x:0 0 0 2024-02-22T15:50:38.208 INFO:tasks.workunit.client.1.smithi062.stdout:0/419: mkdir d1/d21/d85/d8d 0 2024-02-22T15:50:38.209 INFO:tasks.workunit.client.0.smithi019.stdout:8/548: creat d9/d14/d6f/d7e/d94/d9a/fba x:0 0 0 2024-02-22T15:50:38.209 INFO:tasks.workunit.client.0.smithi019.stdout:7/783: truncate d9/d3a/d49/d5d/fbd 1409748 0 2024-02-22T15:50:38.209 INFO:tasks.workunit.client.0.smithi019.stdout:6/657: chown d6/d31/d43/fb0 1 1 2024-02-22T15:50:38.210 INFO:tasks.workunit.client.1.smithi062.stdout:5/232: dwrite d0/f26 [0,4194304] 0 2024-02-22T15:50:38.211 INFO:tasks.workunit.client.0.smithi019.stdout:8/549: read - d9/d14/d6f/d7e/d94/db8/faf zero size 2024-02-22T15:50:38.212 INFO:tasks.workunit.client.0.smithi019.stdout:0/462: rename d4/d22/d2c/d68/d98 to d4/d5/da3 0 2024-02-22T15:50:38.212 INFO:tasks.workunit.client.0.smithi019.stdout:8/550: fsync d9/d14/d1f/d5c/fa2 0 2024-02-22T15:50:38.213 INFO:tasks.workunit.client.0.smithi019.stdout:5/597: mkdir d8/dd/d19/d3a/dc7 0 2024-02-22T15:50:38.215 INFO:tasks.workunit.client.0.smithi019.stdout:5/598: fsync d8/dd/d19/d3a/d46/f53 0 2024-02-22T15:50:38.215 INFO:tasks.workunit.client.1.smithi062.stdout:2/431: symlink d1/d1e/d38/l89 0 2024-02-22T15:50:38.216 INFO:tasks.workunit.client.0.smithi019.stdout:6/658: symlink d6/d4b/d6d/d7b/lcb 0 2024-02-22T15:50:38.218 INFO:tasks.workunit.client.0.smithi019.stdout:5/599: rmdir d8/dd/d19/d77/d7a 39 2024-02-22T15:50:38.218 INFO:tasks.workunit.client.1.smithi062.stdout:5/233: write d0/f9 [9659865,30455] 0 2024-02-22T15:50:38.223 INFO:tasks.workunit.client.0.smithi019.stdout:8/551: getdents d9/d14/d17/d3d/d3f 0 2024-02-22T15:50:38.224 INFO:tasks.workunit.client.0.smithi019.stdout:6/659: mknod d6/d4b/d71/dc2/ccc 0 2024-02-22T15:50:38.225 INFO:tasks.workunit.client.1.smithi062.stdout:4/334: dwrite d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:38.225 INFO:tasks.workunit.client.1.smithi062.stdout:7/315: write d0/d14/d13/fc [1306085,99260] 0 2024-02-22T15:50:38.228 INFO:tasks.workunit.client.1.smithi062.stdout:1/348: dread d1/d6/de/f42 [0,4194304] 0 2024-02-22T15:50:38.229 INFO:tasks.workunit.client.0.smithi019.stdout:5/600: dwrite d8/dd/d19/d38/fb9 [0,4194304] 0 2024-02-22T15:50:38.230 INFO:tasks.workunit.client.0.smithi019.stdout:6/660: truncate d6/f20 427053 0 2024-02-22T15:50:38.230 INFO:tasks.workunit.client.1.smithi062.stdout:2/432: rename d1/d8/c18 to d1/d1e/d38/d76/c8a 0 2024-02-22T15:50:38.233 INFO:tasks.workunit.client.1.smithi062.stdout:5/234: creat d0/d3/d2f/f44 x:0 0 0 2024-02-22T15:50:38.235 INFO:tasks.workunit.client.1.smithi062.stdout:1/349: dread - d1/f27 zero size 2024-02-22T15:50:38.237 INFO:tasks.workunit.client.1.smithi062.stdout:2/433: stat d1/d47/d39/l64 0 2024-02-22T15:50:38.238 INFO:tasks.workunit.client.1.smithi062.stdout:3/354: dwrite d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:38.238 INFO:tasks.workunit.client.1.smithi062.stdout:0/420: dwrite d1/dd/f6b [0,4194304] 0 2024-02-22T15:50:38.239 INFO:tasks.workunit.client.0.smithi019.stdout:8/552: getdents d9/d14/d6f/d7e 0 2024-02-22T15:50:38.243 INFO:tasks.workunit.client.0.smithi019.stdout:7/784: dread d9/d3a/f6b [0,4194304] 0 2024-02-22T15:50:38.244 INFO:tasks.workunit.client.0.smithi019.stdout:7/785: chown d9/d26/cce 50361526 1 2024-02-22T15:50:38.246 INFO:tasks.workunit.client.1.smithi062.stdout:9/317: dread d4/f12 [0,4194304] 0 2024-02-22T15:50:38.247 INFO:tasks.workunit.client.1.smithi062.stdout:5/235: fdatasync d0/d3/da/d13/d23/d31/f43 0 2024-02-22T15:50:38.248 INFO:tasks.workunit.client.0.smithi019.stdout:8/553: write d9/d14/d17/d1c/f48 [1656986,86260] 0 2024-02-22T15:50:38.249 INFO:tasks.workunit.client.1.smithi062.stdout:0/421: mkdir d1/d21/d85/d8e 0 2024-02-22T15:50:38.249 INFO:tasks.workunit.client.1.smithi062.stdout:4/335: symlink d4/d34/l73 0 2024-02-22T15:50:38.249 INFO:tasks.workunit.client.1.smithi062.stdout:3/355: write d5/d26/f61 [288401,89959] 0 2024-02-22T15:50:38.250 INFO:tasks.workunit.client.0.smithi019.stdout:6/661: dwrite d6/d4b/d6d/d7b/d78/fc4 [0,4194304] 0 2024-02-22T15:50:38.251 INFO:tasks.workunit.client.1.smithi062.stdout:9/318: creat d4/d1c/f58 x:0 0 0 2024-02-22T15:50:38.251 INFO:tasks.workunit.client.1.smithi062.stdout:5/236: chown d0/d3/da/d32/d42 100454723 1 2024-02-22T15:50:38.254 INFO:tasks.workunit.client.1.smithi062.stdout:5/237: chown d0/d3/da/fb 286 1 2024-02-22T15:50:38.254 INFO:tasks.workunit.client.0.smithi019.stdout:8/554: write d9/d14/d17/d1c/f48 [27937,94548] 0 2024-02-22T15:50:38.255 INFO:tasks.workunit.client.1.smithi062.stdout:2/434: rename d1/d28/f56 to d1/d8/df/f8b 0 2024-02-22T15:50:38.258 INFO:tasks.workunit.client.0.smithi019.stdout:6/662: creat d6/d4b/da9/d67/d8a/fcd x:0 0 0 2024-02-22T15:50:38.259 INFO:tasks.workunit.client.0.smithi019.stdout:7/786: rmdir d9/d38/d41/d45/db0/df7/dfb/de3/dfc 0 2024-02-22T15:50:38.260 INFO:tasks.workunit.client.0.smithi019.stdout:7/787: chown d9/d38/faa 633 1 2024-02-22T15:50:38.261 INFO:tasks.workunit.client.1.smithi062.stdout:4/336: chown l0 63861489 1 2024-02-22T15:50:38.261 INFO:tasks.workunit.client.0.smithi019.stdout:8/555: rename d9/d14/d1f/d6d/f7c to d9/d14/d17/d3d/fbb 0 2024-02-22T15:50:38.262 INFO:tasks.workunit.client.0.smithi019.stdout:6/663: creat d6/d3e/fce x:0 0 0 2024-02-22T15:50:38.263 INFO:tasks.workunit.client.0.smithi019.stdout:8/556: truncate d9/d88/f90 442182 0 2024-02-22T15:50:38.264 INFO:tasks.workunit.client.0.smithi019.stdout:7/788: chown d9/d38/d41/d45/d61/d6a/cfe 628147 1 2024-02-22T15:50:38.266 INFO:tasks.workunit.client.0.smithi019.stdout:7/789: stat d9/d26/d77/c82 0 2024-02-22T15:50:38.267 INFO:tasks.workunit.client.0.smithi019.stdout:8/557: mknod d9/d14/d1f/cbc 0 2024-02-22T15:50:38.269 INFO:tasks.workunit.client.0.smithi019.stdout:7/790: fsync d9/d4c/f68 0 2024-02-22T15:50:38.269 INFO:tasks.workunit.client.1.smithi062.stdout:5/238: creat d0/f45 x:0 0 0 2024-02-22T15:50:38.270 INFO:tasks.workunit.client.1.smithi062.stdout:2/435: symlink d1/d8/d62/l8c 0 2024-02-22T15:50:38.270 INFO:tasks.workunit.client.0.smithi019.stdout:4/548: sync 2024-02-22T15:50:38.270 INFO:tasks.workunit.client.0.smithi019.stdout:5/601: dread d8/fe [0,4194304] 0 2024-02-22T15:50:38.270 INFO:tasks.workunit.client.1.smithi062.stdout:9/319: write d4/f12 [3914884,105595] 0 2024-02-22T15:50:38.271 INFO:tasks.workunit.client.1.smithi062.stdout:7/316: dwrite d0/d6/d5b/d60/f66 [0,4194304] 0 2024-02-22T15:50:38.273 INFO:tasks.workunit.client.0.smithi019.stdout:6/664: dread d6/d4b/d6d/da3/faf [0,4194304] 0 2024-02-22T15:50:38.273 INFO:tasks.workunit.client.0.smithi019.stdout:7/791: chown d9/d38/d41/d45/d92/fa6 2 1 2024-02-22T15:50:38.274 INFO:tasks.workunit.client.1.smithi062.stdout:1/350: dwrite d1/d6/d14/f45 [0,4194304] 0 2024-02-22T15:50:38.274 INFO:tasks.workunit.client.0.smithi019.stdout:4/549: fdatasync d2/db/d44/fa2 0 2024-02-22T15:50:38.280 INFO:tasks.workunit.client.1.smithi062.stdout:0/422: unlink d1/l66 0 2024-02-22T15:50:38.281 INFO:tasks.workunit.client.0.smithi019.stdout:8/558: creat d9/d14/d6f/d7e/d94/daa/fbd x:0 0 0 2024-02-22T15:50:38.282 INFO:tasks.workunit.client.1.smithi062.stdout:5/239: rmdir d0/d3/d2f/d38 39 2024-02-22T15:50:38.284 INFO:tasks.workunit.client.0.smithi019.stdout:7/792: mkdir d9/d38/d41/d45/db0/df7/dfb/de3/d107 0 2024-02-22T15:50:38.285 INFO:tasks.workunit.client.1.smithi062.stdout:9/320: creat d4/d15/d3c/f59 x:0 0 0 2024-02-22T15:50:38.285 INFO:tasks.workunit.client.1.smithi062.stdout:5/240: fdatasync d0/d3/da/d13/d23/f3e 0 2024-02-22T15:50:38.285 INFO:tasks.workunit.client.1.smithi062.stdout:7/317: read d0/d14/f17 [1240083,22255] 0 2024-02-22T15:50:38.285 INFO:tasks.workunit.client.0.smithi019.stdout:6/665: mkdir d6/d31/d69/d8e/dcf 0 2024-02-22T15:50:38.286 INFO:tasks.workunit.client.1.smithi062.stdout:1/351: symlink d1/d6/d36/l73 0 2024-02-22T15:50:38.288 INFO:tasks.workunit.client.1.smithi062.stdout:2/436: dread d1/f16 [0,4194304] 0 2024-02-22T15:50:38.288 INFO:tasks.workunit.client.0.smithi019.stdout:4/550: truncate d2/db/d14/d23/d33/f7d 47653 0 2024-02-22T15:50:38.292 INFO:tasks.workunit.client.1.smithi062.stdout:0/423: write d1/d3/d4/d1a/d1d/d3b/f55 [569795,20689] 0 2024-02-22T15:50:38.293 INFO:tasks.workunit.client.1.smithi062.stdout:7/318: read d0/d6/f23 [264511,115140] 0 2024-02-22T15:50:38.293 INFO:tasks.workunit.client.1.smithi062.stdout:1/352: mkdir d1/d6/d7/d2b/d49/d74 0 2024-02-22T15:50:38.294 INFO:tasks.workunit.client.0.smithi019.stdout:4/551: mkdir d2/db/d14/d23/d33/db2 0 2024-02-22T15:50:38.296 INFO:tasks.workunit.client.1.smithi062.stdout:1/353: readlink d1/d6/d7/l4f 0 2024-02-22T15:50:38.297 INFO:tasks.workunit.client.0.smithi019.stdout:5/602: creat d8/dd/fc8 x:0 0 0 2024-02-22T15:50:38.297 INFO:tasks.workunit.client.0.smithi019.stdout:5/603: write d8/dd/d8e/d2c/f5d [1330,43597] 0 2024-02-22T15:50:38.297 INFO:tasks.workunit.client.0.smithi019.stdout:5/604: dread - d8/f9b zero size 2024-02-22T15:50:38.299 INFO:tasks.workunit.client.0.smithi019.stdout:7/793: rename d9/d3a/d49/d5d/d74 to d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108 0 2024-02-22T15:50:38.299 INFO:tasks.workunit.client.0.smithi019.stdout:5/605: fdatasync d8/dd/d8e/f22 0 2024-02-22T15:50:38.300 INFO:tasks.workunit.client.1.smithi062.stdout:1/354: dread - d1/d6/d14/f6d zero size 2024-02-22T15:50:38.300 INFO:tasks.workunit.client.0.smithi019.stdout:6/666: getdents d6/d4b/da9/d67/d8a/d90 0 2024-02-22T15:50:38.301 INFO:tasks.workunit.client.1.smithi062.stdout:5/241: creat d0/d3/d30/f46 x:0 0 0 2024-02-22T15:50:38.301 INFO:tasks.workunit.client.1.smithi062.stdout:7/319: fdatasync d0/d14/d19/d30/f42 0 2024-02-22T15:50:38.302 INFO:tasks.workunit.client.1.smithi062.stdout:9/321: dwrite d4/d35/d3f/f39 [0,4194304] 0 2024-02-22T15:50:38.303 INFO:tasks.workunit.client.1.smithi062.stdout:3/356: dread d5/f8 [0,4194304] 0 2024-02-22T15:50:38.307 INFO:tasks.workunit.client.0.smithi019.stdout:4/552: rename d2/db/d92/f8a to d2/db/d14/d23/d21/fb3 0 2024-02-22T15:50:38.312 INFO:tasks.workunit.client.0.smithi019.stdout:4/553: symlink d2/d3f/d48/d76/lb4 0 2024-02-22T15:50:38.314 INFO:tasks.workunit.client.1.smithi062.stdout:4/337: dread d4/d1c/d2c/f56 [0,4194304] 0 2024-02-22T15:50:38.315 INFO:tasks.workunit.client.0.smithi019.stdout:4/554: mknod d2/cb5 0 2024-02-22T15:50:38.316 INFO:tasks.workunit.client.0.smithi019.stdout:4/555: fdatasync d2/db/fa3 0 2024-02-22T15:50:38.317 INFO:tasks.workunit.client.1.smithi062.stdout:0/424: unlink d1/d3/d4/d1a/d76/d8c/d2c/d84/c44 0 2024-02-22T15:50:38.319 INFO:tasks.workunit.client.1.smithi062.stdout:8/357: sync 2024-02-22T15:50:38.319 INFO:tasks.workunit.client.1.smithi062.stdout:7/320: write d0/d6/d2e/d58/f5c [807838,13144] 0 2024-02-22T15:50:38.319 INFO:tasks.workunit.client.0.smithi019.stdout:5/606: dread d8/f23 [0,4194304] 0 2024-02-22T15:50:38.320 INFO:tasks.workunit.client.0.smithi019.stdout:5/607: fdatasync d8/dd/f5b 0 2024-02-22T15:50:38.322 INFO:tasks.workunit.client.1.smithi062.stdout:2/437: dwrite d1/d1e/d38/f5f [0,4194304] 0 2024-02-22T15:50:38.324 INFO:tasks.workunit.client.0.smithi019.stdout:6/667: write d6/d3e/f42 [4310202,44561] 0 2024-02-22T15:50:38.325 INFO:tasks.workunit.client.1.smithi062.stdout:1/355: dwrite d1/d6/d7/f28 [4194304,4194304] 0 2024-02-22T15:50:38.325 INFO:tasks.workunit.client.1.smithi062.stdout:7/321: stat d0/d14/l18 0 2024-02-22T15:50:38.326 INFO:tasks.workunit.client.0.smithi019.stdout:7/794: write d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/f88 [1143150,116191] 0 2024-02-22T15:50:38.326 INFO:tasks.workunit.client.0.smithi019.stdout:4/556: dwrite d2/db/d14/d61/f73 [0,4194304] 0 2024-02-22T15:50:38.328 INFO:tasks.workunit.client.0.smithi019.stdout:6/668: chown d6/d4b/d6d/f81 4391971 1 2024-02-22T15:50:38.331 INFO:tasks.workunit.client.1.smithi062.stdout:1/356: creat d1/d66/f75 x:0 0 0 2024-02-22T15:50:38.332 INFO:tasks.workunit.client.1.smithi062.stdout:0/425: fdatasync d1/d3/d4/d1a/d76/d8c/f4c 0 2024-02-22T15:50:38.333 INFO:tasks.workunit.client.1.smithi062.stdout:8/358: symlink d8/d9/l66 0 2024-02-22T15:50:38.336 INFO:tasks.workunit.client.0.smithi019.stdout:7/795: rename d9/ff6 to d9/d4c/ddf/f109 0 2024-02-22T15:50:38.336 INFO:tasks.workunit.client.0.smithi019.stdout:6/669: symlink d6/d4b/d71/dc2/ld0 0 2024-02-22T15:50:38.337 INFO:tasks.workunit.client.0.smithi019.stdout:7/796: chown d9/d4c/f68 496683453 1 2024-02-22T15:50:38.344 INFO:tasks.workunit.client.0.smithi019.stdout:6/670: dwrite d6/d4b/da9/d67/f84 [0,4194304] 0 2024-02-22T15:50:38.345 INFO:tasks.workunit.client.1.smithi062.stdout:7/322: symlink d0/d6/d2e/d2f/l68 0 2024-02-22T15:50:38.347 INFO:tasks.workunit.client.1.smithi062.stdout:1/357: mkdir d1/d6/d36/d76 0 2024-02-22T15:50:38.347 INFO:tasks.workunit.client.1.smithi062.stdout:2/438: dread d1/d28/d3f/d65/f7e [0,4194304] 0 2024-02-22T15:50:38.350 INFO:tasks.workunit.client.0.smithi019.stdout:6/671: rename d6/d31/d48/f4a to d6/fd1 0 2024-02-22T15:50:38.352 INFO:tasks.workunit.client.1.smithi062.stdout:3/357: dread d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:38.353 INFO:tasks.workunit.client.0.smithi019.stdout:6/672: symlink d6/d31/d48/ld2 0 2024-02-22T15:50:38.354 INFO:tasks.workunit.client.1.smithi062.stdout:4/338: dwrite d4/df/f14 [8388608,4194304] 0 2024-02-22T15:50:38.355 INFO:tasks.workunit.client.0.smithi019.stdout:6/673: fsync d6/ff 0 2024-02-22T15:50:38.355 INFO:tasks.workunit.client.1.smithi062.stdout:5/242: dwrite d0/d3/da/f27 [0,4194304] 0 2024-02-22T15:50:38.358 INFO:tasks.workunit.client.0.smithi019.stdout:7/797: truncate d9/d26/f32 2076688 0 2024-02-22T15:50:38.358 INFO:tasks.workunit.client.0.smithi019.stdout:6/674: truncate d6/d31/d48/f51 800552 0 2024-02-22T15:50:38.360 INFO:tasks.workunit.client.1.smithi062.stdout:1/358: write d1/f32 [250246,53081] 0 2024-02-22T15:50:38.362 INFO:tasks.workunit.client.1.smithi062.stdout:9/322: dwrite d4/d1c/f1d [0,4194304] 0 2024-02-22T15:50:38.362 INFO:tasks.workunit.client.1.smithi062.stdout:0/426: creat d1/d3/d4/f8f x:0 0 0 2024-02-22T15:50:38.364 INFO:tasks.workunit.client.1.smithi062.stdout:2/439: mknod d1/d8/d62/c8d 0 2024-02-22T15:50:38.366 INFO:tasks.workunit.client.1.smithi062.stdout:5/243: creat d0/d3/d30/f47 x:0 0 0 2024-02-22T15:50:38.366 INFO:tasks.workunit.client.1.smithi062.stdout:3/358: rmdir d5/d1a/d3b 39 2024-02-22T15:50:38.368 INFO:tasks.workunit.client.0.smithi019.stdout:4/557: dread d2/fa [0,4194304] 0 2024-02-22T15:50:38.368 INFO:tasks.workunit.client.0.smithi019.stdout:7/798: link d9/d79/cd5 d9/d38/d41/d45/db0/c10a 0 2024-02-22T15:50:38.369 INFO:tasks.workunit.client.1.smithi062.stdout:1/359: truncate d1/d6/d7/d2b/d49/f6a 5004678 0 2024-02-22T15:50:38.369 INFO:tasks.workunit.client.0.smithi019.stdout:7/799: fsync d9/d38/f5a 0 2024-02-22T15:50:38.370 INFO:tasks.workunit.client.1.smithi062.stdout:4/339: fdatasync d4/f51 0 2024-02-22T15:50:38.372 INFO:tasks.workunit.client.1.smithi062.stdout:7/323: dread d0/d6/f23 [4194304,4194304] 0 2024-02-22T15:50:38.372 INFO:tasks.workunit.client.0.smithi019.stdout:7/800: dread d9/d38/f59 [0,4194304] 0 2024-02-22T15:50:38.375 INFO:tasks.workunit.client.0.smithi019.stdout:6/675: truncate d6/d31/d43/f8b 1003789 0 2024-02-22T15:50:38.375 INFO:tasks.workunit.client.1.smithi062.stdout:5/244: symlink d0/d3/d2f/d38/l48 0 2024-02-22T15:50:38.376 INFO:tasks.workunit.client.1.smithi062.stdout:1/360: write d1/d6/d7/f50 [4107262,106564] 0 2024-02-22T15:50:38.378 INFO:tasks.workunit.client.0.smithi019.stdout:6/676: write d6/d31/f35 [7210301,3167] 0 2024-02-22T15:50:38.379 INFO:tasks.workunit.client.1.smithi062.stdout:4/340: truncate d4/d66/f6f 728126 0 2024-02-22T15:50:38.379 INFO:tasks.workunit.client.1.smithi062.stdout:7/324: write d0/d14/d19/d30/f42 [1194040,59673] 0 2024-02-22T15:50:38.381 INFO:tasks.workunit.client.1.smithi062.stdout:9/323: dwrite d4/d15/d3c/f3e [0,4194304] 0 2024-02-22T15:50:38.381 INFO:tasks.workunit.client.0.smithi019.stdout:4/558: dwrite d2/db/d14/d23/d21/f9b [0,4194304] 0 2024-02-22T15:50:38.382 INFO:tasks.workunit.client.1.smithi062.stdout:1/361: mknod d1/d66/c77 0 2024-02-22T15:50:38.383 INFO:tasks.workunit.client.1.smithi062.stdout:9/324: dread - d4/d1c/f53 zero size 2024-02-22T15:50:38.384 INFO:tasks.workunit.client.1.smithi062.stdout:2/440: dread d1/d8/df/f31 [0,4194304] 0 2024-02-22T15:50:38.386 INFO:tasks.workunit.client.1.smithi062.stdout:7/325: write d0/d14/d13/fc [1267491,12416] 0 2024-02-22T15:50:38.386 INFO:tasks.workunit.client.1.smithi062.stdout:5/245: creat d0/d3/da/d13/d23/f49 x:0 0 0 2024-02-22T15:50:38.388 INFO:tasks.workunit.client.0.smithi019.stdout:7/801: dwrite d9/d26/d77/f8e [0,4194304] 0 2024-02-22T15:50:38.389 INFO:tasks.workunit.client.0.smithi019.stdout:4/559: chown d2/db/d14/d23/d33/l40 110471285 1 2024-02-22T15:50:38.390 INFO:tasks.workunit.client.1.smithi062.stdout:8/359: truncate d8/f3a 405625 0 2024-02-22T15:50:38.392 INFO:tasks.workunit.client.0.smithi019.stdout:4/560: rmdir d2/db/d14/d23/d33 39 2024-02-22T15:50:38.392 INFO:tasks.workunit.client.1.smithi062.stdout:5/246: write d0/d3/da/d13/d23/d31/f43 [1338669,95110] 0 2024-02-22T15:50:38.392 INFO:tasks.workunit.client.1.smithi062.stdout:4/341: rename d4/d2a/c58 to d4/df/c74 0 2024-02-22T15:50:38.394 INFO:tasks.workunit.client.0.smithi019.stdout:7/802: write d9/d26/d77/f8e [941447,128859] 0 2024-02-22T15:50:38.395 INFO:tasks.workunit.client.1.smithi062.stdout:7/326: write d0/d14/f17 [1458141,13904] 0 2024-02-22T15:50:38.396 INFO:tasks.workunit.client.1.smithi062.stdout:2/441: mknod d1/d47/d32/d52/c8e 0 2024-02-22T15:50:38.397 INFO:tasks.workunit.client.1.smithi062.stdout:1/362: dread d1/d6/d7/d2b/d47/d24/f2a [0,4194304] 0 2024-02-22T15:50:38.398 INFO:tasks.workunit.client.1.smithi062.stdout:4/342: chown d4/d1c/d30/d3b/d49 4366469 1 2024-02-22T15:50:38.401 INFO:tasks.workunit.client.1.smithi062.stdout:2/442: chown d1/c2e 138522814 1 2024-02-22T15:50:38.402 INFO:tasks.workunit.client.0.smithi019.stdout:4/561: creat d2/fb6 x:0 0 0 2024-02-22T15:50:38.403 INFO:tasks.workunit.client.1.smithi062.stdout:2/443: chown d1/d47/d39 295 1 2024-02-22T15:50:38.403 INFO:tasks.workunit.client.1.smithi062.stdout:5/247: rename d0/d3/da/d13/f2d to d0/d3/d30/f4a 0 2024-02-22T15:50:38.404 INFO:tasks.workunit.client.1.smithi062.stdout:4/343: mkdir d4/d2a/d75 0 2024-02-22T15:50:38.405 INFO:tasks.workunit.client.1.smithi062.stdout:8/360: dwrite d8/d26/f34 [0,4194304] 0 2024-02-22T15:50:38.406 INFO:tasks.workunit.client.1.smithi062.stdout:7/327: write d0/d14/d19/d30/d37/f63 [280148,106542] 0 2024-02-22T15:50:38.410 INFO:tasks.workunit.client.1.smithi062.stdout:1/363: link d1/f44 d1/d6/d7/d2b/d47/d24/f78 0 2024-02-22T15:50:38.410 INFO:tasks.workunit.client.1.smithi062.stdout:2/444: mkdir d1/d8/d8f 0 2024-02-22T15:50:38.413 INFO:tasks.workunit.client.1.smithi062.stdout:5/248: creat d0/d3/d2f/d38/d3f/f4b x:0 0 0 2024-02-22T15:50:38.413 INFO:tasks.workunit.client.0.smithi019.stdout:7/803: creat d9/d4c/ddf/d105/f10b x:0 0 0 2024-02-22T15:50:38.414 INFO:tasks.workunit.client.1.smithi062.stdout:1/364: chown d1/d6/d7/l62 3274 1 2024-02-22T15:50:38.414 INFO:tasks.workunit.client.1.smithi062.stdout:8/361: symlink d8/d9/d44/d4d/d54/l67 0 2024-02-22T15:50:38.416 INFO:tasks.workunit.client.1.smithi062.stdout:4/344: creat d4/d1c/d2c/d6b/d70/f76 x:0 0 0 2024-02-22T15:50:38.417 INFO:tasks.workunit.client.1.smithi062.stdout:8/362: chown d8/d9/fe 1613668 1 2024-02-22T15:50:38.418 INFO:tasks.workunit.client.1.smithi062.stdout:1/365: unlink d1/d66/c77 0 2024-02-22T15:50:38.418 INFO:tasks.workunit.client.0.smithi019.stdout:7/804: rename d9/d38/d41/c50 to d9/d26/dc4/c10c 0 2024-02-22T15:50:38.419 INFO:tasks.workunit.client.1.smithi062.stdout:5/249: symlink d0/d3/d2f/d38/l4c 0 2024-02-22T15:50:38.421 INFO:tasks.workunit.client.0.smithi019.stdout:4/562: dwrite d2/db/d14/d23/d67/f68 [0,4194304] 0 2024-02-22T15:50:38.422 INFO:tasks.workunit.client.1.smithi062.stdout:4/345: creat d4/d2a/d2d/d2e/f77 x:0 0 0 2024-02-22T15:50:38.422 INFO:tasks.workunit.client.1.smithi062.stdout:7/328: dwrite d0/d14/d13/f41 [0,4194304] 0 2024-02-22T15:50:38.422 INFO:tasks.workunit.client.1.smithi062.stdout:2/445: rename d1/d47/d32/c41 to d1/d47/d32/c90 0 2024-02-22T15:50:38.424 INFO:tasks.workunit.client.0.smithi019.stdout:7/805: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/f10d x:0 0 0 2024-02-22T15:50:38.426 INFO:tasks.workunit.client.0.smithi019.stdout:4/563: symlink d2/db/d14/d23/lb7 0 2024-02-22T15:50:38.426 INFO:tasks.workunit.client.1.smithi062.stdout:1/366: chown d1/d6/de/d9/c34 0 1 2024-02-22T15:50:38.426 INFO:tasks.workunit.client.1.smithi062.stdout:4/346: write f2 [2580109,42053] 0 2024-02-22T15:50:38.427 INFO:tasks.workunit.client.0.smithi019.stdout:4/564: truncate d2/d3f/d48/d76/fa8 116075 0 2024-02-22T15:50:38.430 INFO:tasks.workunit.client.1.smithi062.stdout:3/359: dread d5/d26/f31 [0,4194304] 0 2024-02-22T15:50:38.432 INFO:tasks.workunit.client.0.smithi019.stdout:4/565: mknod d2/d3f/d48/d76/cb8 0 2024-02-22T15:50:38.432 INFO:tasks.workunit.client.0.smithi019.stdout:6/677: dread d6/f2c [0,4194304] 0 2024-02-22T15:50:38.432 INFO:tasks.workunit.client.1.smithi062.stdout:8/363: rename d8/d9/d2c/f59 to d8/d9/d44/d4d/d51/f68 0 2024-02-22T15:50:38.432 INFO:tasks.workunit.client.1.smithi062.stdout:0/427: dread d1/d13/d43/f53 [0,4194304] 0 2024-02-22T15:50:38.434 INFO:tasks.workunit.client.1.smithi062.stdout:2/446: mknod d1/d8/d62/d73/c91 0 2024-02-22T15:50:38.435 INFO:tasks.workunit.client.0.smithi019.stdout:4/566: mkdir d2/db/db9 0 2024-02-22T15:50:38.435 INFO:tasks.workunit.client.1.smithi062.stdout:5/250: link d0/l25 d0/d3/da/d13/l4d 0 2024-02-22T15:50:38.435 INFO:tasks.workunit.client.1.smithi062.stdout:1/367: creat d1/d6/d7/d2b/d47/f79 x:0 0 0 2024-02-22T15:50:38.436 INFO:tasks.workunit.client.1.smithi062.stdout:2/447: stat d1/d8/l17 0 2024-02-22T15:50:38.437 INFO:tasks.workunit.client.0.smithi019.stdout:4/567: readlink d2/l4f 0 2024-02-22T15:50:38.438 INFO:tasks.workunit.client.1.smithi062.stdout:3/360: read d5/d16/f48 [4011791,129993] 0 2024-02-22T15:50:38.438 INFO:tasks.workunit.client.0.smithi019.stdout:6/678: link d6/d4b/d6d/l9b d6/d98/ld3 0 2024-02-22T15:50:38.439 INFO:tasks.workunit.client.1.smithi062.stdout:7/329: link d0/d14/d19/d30/d37/f53 d0/d6/d2e/d55/f69 0 2024-02-22T15:50:38.441 INFO:tasks.workunit.client.0.smithi019.stdout:4/568: rename d2/c10 to d2/db/d14/d23/d21/d35/cba 0 2024-02-22T15:50:38.442 INFO:tasks.workunit.client.1.smithi062.stdout:2/448: dread - d1/d1e/d38/d55/f5e zero size 2024-02-22T15:50:38.442 INFO:tasks.workunit.client.1.smithi062.stdout:3/361: write d5/d1a/d3c/f53 [9339333,34148] 0 2024-02-22T15:50:38.442 INFO:tasks.workunit.client.1.smithi062.stdout:5/251: read d0/d3/da/d13/d23/f3b [7141,57740] 0 2024-02-22T15:50:38.443 INFO:tasks.workunit.client.1.smithi062.stdout:1/368: dread d1/d6/d7/d2b/d49/f51 [0,4194304] 0 2024-02-22T15:50:38.446 INFO:tasks.workunit.client.1.smithi062.stdout:2/449: write d1/d1e/f2d [1737103,58689] 0 2024-02-22T15:50:38.448 INFO:tasks.workunit.client.0.smithi019.stdout:6/679: rename d6/d31/f32 to d6/d3e/fd4 0 2024-02-22T15:50:38.448 INFO:tasks.workunit.client.1.smithi062.stdout:3/362: creat d5/d26/f6c x:0 0 0 2024-02-22T15:50:38.449 INFO:tasks.workunit.client.1.smithi062.stdout:5/252: chown d0/d3/da/l20 209225 1 2024-02-22T15:50:38.449 INFO:tasks.workunit.client.1.smithi062.stdout:7/330: mkdir d0/d14/d19/d30/d45/d6a 0 2024-02-22T15:50:38.449 INFO:tasks.workunit.client.1.smithi062.stdout:1/369: stat d1/l1e 0 2024-02-22T15:50:38.450 INFO:tasks.workunit.client.0.smithi019.stdout:6/680: write d6/d4b/d6d/d7b/d78/fc4 [118469,75103] 0 2024-02-22T15:50:38.453 INFO:tasks.workunit.client.1.smithi062.stdout:2/450: unlink d1/d8/l17 0 2024-02-22T15:50:38.454 INFO:tasks.workunit.client.0.smithi019.stdout:4/569: dwrite d2/db/d14/d23/d21/f9b [0,4194304] 0 2024-02-22T15:50:38.454 INFO:tasks.workunit.client.1.smithi062.stdout:5/253: read d0/d3/da/f18 [185287,12484] 0 2024-02-22T15:50:38.457 INFO:tasks.workunit.client.1.smithi062.stdout:0/428: dwrite d1/d13/d43/f53 [4194304,4194304] 0 2024-02-22T15:50:38.459 INFO:tasks.workunit.client.0.smithi019.stdout:6/681: mknod d6/d4b/d71/d88/cd5 0 2024-02-22T15:50:38.459 INFO:tasks.workunit.client.1.smithi062.stdout:5/254: dread - d0/d3/d30/f46 zero size 2024-02-22T15:50:38.461 INFO:tasks.workunit.client.1.smithi062.stdout:1/370: creat d1/d6/d7/f7a x:0 0 0 2024-02-22T15:50:38.463 INFO:tasks.workunit.client.0.smithi019.stdout:4/570: mkdir d2/db/d54/d8c/dbb 0 2024-02-22T15:50:38.463 INFO:tasks.workunit.client.1.smithi062.stdout:2/451: creat d1/d1e/d38/d76/f92 x:0 0 0 2024-02-22T15:50:38.464 INFO:tasks.workunit.client.0.smithi019.stdout:1/614: sync 2024-02-22T15:50:38.464 INFO:tasks.workunit.client.0.smithi019.stdout:2/548: sync 2024-02-22T15:50:38.464 INFO:tasks.workunit.client.0.smithi019.stdout:7/806: write d9/d26/f33 [9883563,13626] 0 2024-02-22T15:50:38.467 INFO:tasks.workunit.client.1.smithi062.stdout:5/255: mkdir d0/d3/da/d13/d23/d4e 0 2024-02-22T15:50:38.468 INFO:tasks.workunit.client.1.smithi062.stdout:1/371: symlink d1/d6/d36/l7b 0 2024-02-22T15:50:38.469 INFO:tasks.workunit.client.1.smithi062.stdout:2/452: creat d1/d28/d3f/d7d/f93 x:0 0 0 2024-02-22T15:50:38.471 INFO:tasks.workunit.client.0.smithi019.stdout:1/615: stat d8/d2a/d75/d81/d8e/lbb 0 2024-02-22T15:50:38.474 INFO:tasks.workunit.client.0.smithi019.stdout:2/549: creat d1/de/d9a/fcb x:0 0 0 2024-02-22T15:50:38.475 INFO:tasks.workunit.client.1.smithi062.stdout:0/429: creat d1/d3/de/f90 x:0 0 0 2024-02-22T15:50:38.475 INFO:tasks.workunit.client.1.smithi062.stdout:0/430: stat d1/d3/d4/d1a 0 2024-02-22T15:50:38.476 INFO:tasks.workunit.client.1.smithi062.stdout:0/431: stat d1/d3/d4/d1e/f79 0 2024-02-22T15:50:38.477 INFO:tasks.workunit.client.0.smithi019.stdout:6/682: dwrite d6/d31/d69/d8e/fab [0,4194304] 0 2024-02-22T15:50:38.477 INFO:tasks.workunit.client.0.smithi019.stdout:2/550: readlink d1/da4/lb0 0 2024-02-22T15:50:38.478 INFO:tasks.workunit.client.1.smithi062.stdout:1/372: link d1/c6b d1/d6/d7/d2b/d49/c7c 0 2024-02-22T15:50:38.480 INFO:tasks.workunit.client.1.smithi062.stdout:2/453: creat d1/d1e/f94 x:0 0 0 2024-02-22T15:50:38.481 INFO:tasks.workunit.client.0.smithi019.stdout:7/807: creat d9/d38/d41/d45/d61/d6a/f10e x:0 0 0 2024-02-22T15:50:38.481 INFO:tasks.workunit.client.0.smithi019.stdout:4/571: fsync d2/db/d14/d23/d33/f74 0 2024-02-22T15:50:38.482 INFO:tasks.workunit.client.0.smithi019.stdout:6/683: symlink d6/d4b/d6d/d7b/d7f/ld6 0 2024-02-22T15:50:38.482 INFO:tasks.workunit.client.1.smithi062.stdout:4/347: truncate d4/f59 3291010 0 2024-02-22T15:50:38.482 INFO:tasks.workunit.client.1.smithi062.stdout:0/432: symlink d1/d3/d4/d1a/l91 0 2024-02-22T15:50:38.482 INFO:tasks.workunit.client.1.smithi062.stdout:4/348: chown d4/d13/d20/l31 119 1 2024-02-22T15:50:38.485 INFO:tasks.workunit.client.0.smithi019.stdout:9/563: sync 2024-02-22T15:50:38.485 INFO:tasks.workunit.client.0.smithi019.stdout:3/437: sync 2024-02-22T15:50:38.487 INFO:tasks.workunit.client.1.smithi062.stdout:2/454: rename d1/d28/d3f/d65/f7f to d1/d8/d8f/f95 0 2024-02-22T15:50:38.488 INFO:tasks.workunit.client.0.smithi019.stdout:2/551: rename d1/d17/d1c/f4f to d1/d17/d21/d51/fcc 0 2024-02-22T15:50:38.488 INFO:tasks.workunit.client.0.smithi019.stdout:9/564: fsync d2/d7/dc/d27/d31/d79/fb2 0 2024-02-22T15:50:38.490 INFO:tasks.workunit.client.0.smithi019.stdout:6/684: fdatasync d6/ff 0 2024-02-22T15:50:38.493 INFO:tasks.workunit.client.1.smithi062.stdout:0/433: mkdir d1/d3/d4/d1a/d76/d8c/d92 0 2024-02-22T15:50:38.493 INFO:tasks.workunit.client.0.smithi019.stdout:9/565: write d2/d1f/d2e/d57/fc7 [653147,8174] 0 2024-02-22T15:50:38.494 INFO:tasks.workunit.client.1.smithi062.stdout:1/373: getdents d1/d6/d7/d2b/d47/d24 0 2024-02-22T15:50:38.494 INFO:tasks.workunit.client.0.smithi019.stdout:3/438: dread d9/f2f [0,4194304] 0 2024-02-22T15:50:38.497 INFO:tasks.workunit.client.0.smithi019.stdout:4/572: dread d2/db/d14/d23/d21/f3a [0,4194304] 0 2024-02-22T15:50:38.497 INFO:tasks.workunit.client.0.smithi019.stdout:9/566: dread d2/d1f/d95/f9e [0,4194304] 0 2024-02-22T15:50:38.498 INFO:tasks.workunit.client.0.smithi019.stdout:2/552: mkdir d1/d17/d1c/d7f/dcd 0 2024-02-22T15:50:38.500 INFO:tasks.workunit.client.0.smithi019.stdout:6/685: symlink d6/d4b/d71/dc2/ld7 0 2024-02-22T15:50:38.500 INFO:tasks.workunit.client.0.smithi019.stdout:1/616: write d8/d2a/d75/d81/f96 [940285,128991] 0 2024-02-22T15:50:38.502 INFO:tasks.workunit.client.1.smithi062.stdout:5/256: write d0/d3/da/f1d [3311349,50716] 0 2024-02-22T15:50:38.505 INFO:tasks.workunit.client.0.smithi019.stdout:6/686: write d6/ff [1018966,15616] 0 2024-02-22T15:50:38.506 INFO:tasks.workunit.client.0.smithi019.stdout:2/553: creat d1/d17/d38/fce x:0 0 0 2024-02-22T15:50:38.508 INFO:tasks.workunit.client.0.smithi019.stdout:1/617: creat d8/d44/d5d/fd1 x:0 0 0 2024-02-22T15:50:38.508 INFO:tasks.workunit.client.1.smithi062.stdout:2/455: rmdir d1/d8 39 2024-02-22T15:50:38.510 INFO:tasks.workunit.client.1.smithi062.stdout:1/374: write d1/f44 [915311,45364] 0 2024-02-22T15:50:38.513 INFO:tasks.workunit.client.1.smithi062.stdout:4/349: dread d4/fd [0,4194304] 0 2024-02-22T15:50:38.515 INFO:tasks.workunit.client.1.smithi062.stdout:1/375: chown d1/d6/d7/d2b/d47/d63/f68 2659 1 2024-02-22T15:50:38.515 INFO:tasks.workunit.client.1.smithi062.stdout:0/434: dread d1/d21/f47 [0,4194304] 0 2024-02-22T15:50:38.516 INFO:tasks.workunit.client.0.smithi019.stdout:7/808: dwrite d9/d38/d41/d45/d61/d6a/f93 [0,4194304] 0 2024-02-22T15:50:38.516 INFO:tasks.workunit.client.0.smithi019.stdout:9/567: getdents d2/d1f/d2e/d57/d60 0 2024-02-22T15:50:38.517 INFO:tasks.workunit.client.1.smithi062.stdout:2/456: rename d1/d1e/f27 to d1/d8/d8f/f96 0 2024-02-22T15:50:38.517 INFO:tasks.workunit.client.1.smithi062.stdout:5/257: dwrite d0/f7 [0,4194304] 0 2024-02-22T15:50:38.517 INFO:tasks.workunit.client.0.smithi019.stdout:6/687: read d6/f20 [325175,77417] 0 2024-02-22T15:50:38.518 INFO:tasks.workunit.client.0.smithi019.stdout:1/618: mkdir d8/d18/dd2 0 2024-02-22T15:50:38.519 INFO:tasks.workunit.client.0.smithi019.stdout:1/619: chown d8/d23/d33 0 1 2024-02-22T15:50:38.519 INFO:tasks.workunit.client.1.smithi062.stdout:4/350: symlink d4/d1c/d2c/d6b/d70/l78 0 2024-02-22T15:50:38.520 INFO:tasks.workunit.client.0.smithi019.stdout:7/809: creat d9/d4c/dc9/f10f x:0 0 0 2024-02-22T15:50:38.521 INFO:tasks.workunit.client.0.smithi019.stdout:4/573: truncate d2/db/d14/d23/d21/fb3 892879 0 2024-02-22T15:50:38.522 INFO:tasks.workunit.client.0.smithi019.stdout:2/554: dread d1/d17/d21/d71/f37 [0,4194304] 0 2024-02-22T15:50:38.523 INFO:tasks.workunit.client.0.smithi019.stdout:1/620: truncate d8/d15/fa6 327793 0 2024-02-22T15:50:38.523 INFO:tasks.workunit.client.1.smithi062.stdout:1/376: symlink d1/d6/d7/d2b/d56/l7d 0 2024-02-22T15:50:38.523 INFO:tasks.workunit.client.1.smithi062.stdout:2/457: creat d1/d1e/d38/d55/f97 x:0 0 0 2024-02-22T15:50:38.524 INFO:tasks.workunit.client.0.smithi019.stdout:6/688: mknod d6/d4b/da9/cd8 0 2024-02-22T15:50:38.525 INFO:tasks.workunit.client.1.smithi062.stdout:5/258: truncate d0/d3/d2f/d38/f41 3173 0 2024-02-22T15:50:38.525 INFO:tasks.workunit.client.1.smithi062.stdout:0/435: rename d1/d5e/c60 to d1/d3/d4/d1a/c93 0 2024-02-22T15:50:38.525 INFO:tasks.workunit.client.1.smithi062.stdout:1/377: write d1/d6/d7/f28 [3525005,44611] 0 2024-02-22T15:50:38.525 INFO:tasks.workunit.client.0.smithi019.stdout:3/439: dread d9/d11/f17 [0,4194304] 0 2024-02-22T15:50:38.526 INFO:tasks.workunit.client.1.smithi062.stdout:5/259: fdatasync d0/d3/d2f/f44 0 2024-02-22T15:50:38.526 INFO:tasks.workunit.client.1.smithi062.stdout:4/351: mkdir d4/d1c/d2c/d6b/d70/d79 0 2024-02-22T15:50:38.527 INFO:tasks.workunit.client.0.smithi019.stdout:9/568: rename d2/d1f/d95 to d2/dc8 0 2024-02-22T15:50:38.529 INFO:tasks.workunit.client.0.smithi019.stdout:7/810: truncate d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/f88 2226609 0 2024-02-22T15:50:38.529 INFO:tasks.workunit.client.1.smithi062.stdout:0/436: mkdir d1/d21/d85/d8e/d94 0 2024-02-22T15:50:38.529 INFO:tasks.workunit.client.0.smithi019.stdout:1/621: creat d8/d18/d72/fd3 x:0 0 0 2024-02-22T15:50:38.531 INFO:tasks.workunit.client.1.smithi062.stdout:0/437: dread - d1/d3/d4/d1a/d76/d8c/f88 zero size 2024-02-22T15:50:38.531 INFO:tasks.workunit.client.0.smithi019.stdout:6/689: fdatasync d6/f41 0 2024-02-22T15:50:38.532 INFO:tasks.workunit.client.1.smithi062.stdout:1/378: mkdir d1/d6/d7/d2b/d47/d7e 0 2024-02-22T15:50:38.534 INFO:tasks.workunit.client.1.smithi062.stdout:1/379: write d1/d6/de/f31 [3252119,26868] 0 2024-02-22T15:50:38.534 INFO:tasks.workunit.client.1.smithi062.stdout:1/380: readlink d1/d6/d36/l5a 0 2024-02-22T15:50:38.536 INFO:tasks.workunit.client.1.smithi062.stdout:1/381: write d1/d6/d10/f4b [95079,72863] 0 2024-02-22T15:50:38.537 INFO:tasks.workunit.client.0.smithi019.stdout:1/622: creat d8/d23/d33/fd4 x:0 0 0 2024-02-22T15:50:38.537 INFO:tasks.workunit.client.0.smithi019.stdout:2/555: rename d1/d17/d21/d71/l2b to d1/d17/d1c/d7f/dcd/lcf 0 2024-02-22T15:50:38.538 INFO:tasks.workunit.client.0.smithi019.stdout:6/690: symlink d6/d4b/ld9 0 2024-02-22T15:50:38.539 INFO:tasks.workunit.client.0.smithi019.stdout:3/440: creat d9/d11/f8a x:0 0 0 2024-02-22T15:50:38.539 INFO:tasks.workunit.client.1.smithi062.stdout:0/438: dwrite d1/dd/f36 [0,4194304] 0 2024-02-22T15:50:38.541 INFO:tasks.workunit.client.0.smithi019.stdout:1/623: creat d8/d15/d84/fd5 x:0 0 0 2024-02-22T15:50:38.542 INFO:tasks.workunit.client.0.smithi019.stdout:1/624: chown d8/d18/d72/c87 294 1 2024-02-22T15:50:38.543 INFO:tasks.workunit.client.0.smithi019.stdout:6/691: mkdir d6/d4b/d6d/dda 0 2024-02-22T15:50:38.544 INFO:tasks.workunit.client.0.smithi019.stdout:6/692: fsync d6/d31/d48/f51 0 2024-02-22T15:50:38.545 INFO:tasks.workunit.client.1.smithi062.stdout:0/439: mknod d1/d3/d4/d1a/d76/c95 0 2024-02-22T15:50:38.545 INFO:tasks.workunit.client.0.smithi019.stdout:3/441: mknod d9/d22/d27/d6d/c8b 0 2024-02-22T15:50:38.545 INFO:tasks.workunit.client.0.smithi019.stdout:6/693: stat d6/d31/f35 0 2024-02-22T15:50:38.546 INFO:tasks.workunit.client.1.smithi062.stdout:1/382: dwrite d1/d6/d7/f7a [0,4194304] 0 2024-02-22T15:50:38.548 INFO:tasks.workunit.client.0.smithi019.stdout:1/625: dread d8/d15/d34/f74 [0,4194304] 0 2024-02-22T15:50:38.548 INFO:tasks.workunit.client.1.smithi062.stdout:6/469: sync 2024-02-22T15:50:38.548 INFO:tasks.workunit.client.1.smithi062.stdout:0/440: write d1/d3/d4/d1a/f34 [8564834,73413] 0 2024-02-22T15:50:38.550 INFO:tasks.workunit.client.0.smithi019.stdout:7/811: write d9/d38/faa [2751910,43393] 0 2024-02-22T15:50:38.554 INFO:tasks.workunit.client.0.smithi019.stdout:7/812: chown d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/f9e 852 1 2024-02-22T15:50:38.557 INFO:tasks.workunit.client.1.smithi062.stdout:6/470: unlink d5/df/d40/d44/d4a/l3f 0 2024-02-22T15:50:38.557 INFO:tasks.workunit.client.1.smithi062.stdout:4/352: truncate d4/d1c/d30/f6d 829533 0 2024-02-22T15:50:38.558 INFO:tasks.workunit.client.0.smithi019.stdout:3/442: getdents d9/d26 0 2024-02-22T15:50:38.560 INFO:tasks.workunit.client.0.smithi019.stdout:7/813: fdatasync d9/d38/d41/d45/db0/df7/f100 0 2024-02-22T15:50:38.561 INFO:tasks.workunit.client.1.smithi062.stdout:4/353: write d4/f53 [4037759,113346] 0 2024-02-22T15:50:38.565 INFO:tasks.workunit.client.0.smithi019.stdout:7/814: mknod d9/c110 0 2024-02-22T15:50:38.565 INFO:tasks.workunit.client.1.smithi062.stdout:0/441: dwrite d1/d3/de/ff [8388608,4194304] 0 2024-02-22T15:50:38.566 INFO:tasks.workunit.client.0.smithi019.stdout:7/815: chown d9/d26/f87 15538 1 2024-02-22T15:50:38.566 INFO:tasks.workunit.client.0.smithi019.stdout:1/626: write d8/d15/d34/d9e/fbe [4329588,92066] 0 2024-02-22T15:50:38.566 INFO:tasks.workunit.client.0.smithi019.stdout:4/574: dread d2/db/d14/d23/f2f [0,4194304] 0 2024-02-22T15:50:38.568 INFO:tasks.workunit.client.0.smithi019.stdout:6/694: write d6/d31/d43/f8b [1974125,95283] 0 2024-02-22T15:50:38.569 INFO:tasks.workunit.client.0.smithi019.stdout:1/627: unlink d8/f80 0 2024-02-22T15:50:38.570 INFO:tasks.workunit.client.0.smithi019.stdout:7/816: write d9/d4c/ddf/d105/f10b [220363,127012] 0 2024-02-22T15:50:38.570 INFO:tasks.workunit.client.0.smithi019.stdout:6/695: stat d6/d4b 0 2024-02-22T15:50:38.571 INFO:tasks.workunit.client.1.smithi062.stdout:1/383: truncate d1/d6/de/d9/fd 2200574 0 2024-02-22T15:50:38.571 INFO:tasks.workunit.client.1.smithi062.stdout:6/471: dread d5/df/d40/d44/d78/f31 [0,4194304] 0 2024-02-22T15:50:38.573 INFO:tasks.workunit.client.0.smithi019.stdout:4/575: mknod d2/db/d14/d23/d9e/dac/cbc 0 2024-02-22T15:50:38.574 INFO:tasks.workunit.client.0.smithi019.stdout:7/817: fdatasync d9/d4c/dc9/f10f 0 2024-02-22T15:50:38.574 INFO:tasks.workunit.client.0.smithi019.stdout:1/628: mkdir d8/d15/d5b/dbc/dd6 0 2024-02-22T15:50:38.575 INFO:tasks.workunit.client.0.smithi019.stdout:3/443: write d9/d26/d2d/d53/f56 [612902,19624] 0 2024-02-22T15:50:38.578 INFO:tasks.workunit.client.1.smithi062.stdout:6/472: rmdir d5/dd/d33/d66 39 2024-02-22T15:50:38.578 INFO:tasks.workunit.client.0.smithi019.stdout:9/569: dread d2/d7/dc/f24 [12582912,4194304] 0 2024-02-22T15:50:38.579 INFO:tasks.workunit.client.1.smithi062.stdout:0/442: rename d1/d3/d4/f8f to d1/d3/d4/d1a/d76/d8c/d92/f96 0 2024-02-22T15:50:38.580 INFO:tasks.workunit.client.1.smithi062.stdout:1/384: creat d1/d6/d7/d4a/f7f x:0 0 0 2024-02-22T15:50:38.581 INFO:tasks.workunit.client.0.smithi019.stdout:6/696: rename d6/d4b/da9/d82 to d6/d4b/d6d/da3/ddb 0 2024-02-22T15:50:38.581 INFO:tasks.workunit.client.0.smithi019.stdout:1/629: symlink d8/d15/d84/ld7 0 2024-02-22T15:50:38.582 INFO:tasks.workunit.client.1.smithi062.stdout:1/385: chown d1/d6/d7/d4a/f7f 11880795 1 2024-02-22T15:50:38.582 INFO:tasks.workunit.client.1.smithi062.stdout:0/443: mkdir d1/d21/d85/d8e/d97 0 2024-02-22T15:50:38.583 INFO:tasks.workunit.client.1.smithi062.stdout:1/386: fdatasync d1/d6/d7/d2b/d49/f57 0 2024-02-22T15:50:38.584 INFO:tasks.workunit.client.0.smithi019.stdout:4/576: symlink d2/db/d14/d23/d67/d8f/lbd 0 2024-02-22T15:50:38.585 INFO:tasks.workunit.client.0.smithi019.stdout:3/444: truncate d9/d11/f1e 188130 0 2024-02-22T15:50:38.585 INFO:tasks.workunit.client.0.smithi019.stdout:4/577: fdatasync d2/fb6 0 2024-02-22T15:50:38.585 INFO:tasks.workunit.client.1.smithi062.stdout:4/354: truncate d4/d1c/d30/d3b/f67 1979461 0 2024-02-22T15:50:38.586 INFO:tasks.workunit.client.0.smithi019.stdout:3/445: stat d9/d26/d2d/l40 0 2024-02-22T15:50:38.587 INFO:tasks.workunit.client.1.smithi062.stdout:0/444: rmdir d1/d3/de/d50 39 2024-02-22T15:50:38.587 INFO:tasks.workunit.client.1.smithi062.stdout:4/355: dread d4/d1c/f23 [0,4194304] 0 2024-02-22T15:50:38.588 INFO:tasks.workunit.client.1.smithi062.stdout:6/473: mknod d5/df/d40/d44/d4a/d41/d85/d88/c96 0 2024-02-22T15:50:38.588 INFO:tasks.workunit.client.0.smithi019.stdout:6/697: mkdir d6/d4b/d6d/da3/ddb/ddc 0 2024-02-22T15:50:38.588 INFO:tasks.workunit.client.0.smithi019.stdout:7/818: write d9/d38/d41/d45/f70 [3025887,38656] 0 2024-02-22T15:50:38.590 INFO:tasks.workunit.client.0.smithi019.stdout:3/446: chown d9/c44 239506973 1 2024-02-22T15:50:38.592 INFO:tasks.workunit.client.0.smithi019.stdout:1/630: rename d8/d15/f24 to d8/d18/d64/fd8 0 2024-02-22T15:50:38.595 INFO:tasks.workunit.client.0.smithi019.stdout:3/447: write d9/d26/d3f/d59/f74 [5171674,26218] 0 2024-02-22T15:50:38.596 INFO:tasks.workunit.client.0.smithi019.stdout:6/698: creat d6/d4b/d6d/da3/ddb/fdd x:0 0 0 2024-02-22T15:50:38.596 INFO:tasks.workunit.client.0.smithi019.stdout:9/570: dwrite d2/f4 [4194304,4194304] 0 2024-02-22T15:50:38.598 INFO:tasks.workunit.client.0.smithi019.stdout:1/631: chown d8/d2a/d75/d81/f96 163 1 2024-02-22T15:50:38.599 INFO:tasks.workunit.client.0.smithi019.stdout:7/819: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/f111 x:0 0 0 2024-02-22T15:50:38.603 INFO:tasks.workunit.client.0.smithi019.stdout:6/699: creat d6/d4b/da9/d67/d8a/fde x:0 0 0 2024-02-22T15:50:38.604 INFO:tasks.workunit.client.1.smithi062.stdout:1/387: truncate d1/d6/d36/f5d 6729586 0 2024-02-22T15:50:38.605 INFO:tasks.workunit.client.1.smithi062.stdout:4/356: write d4/f39 [782120,121577] 0 2024-02-22T15:50:38.606 INFO:tasks.workunit.client.0.smithi019.stdout:1/632: creat d8/d44/d5d/fd9 x:0 0 0 2024-02-22T15:50:38.610 INFO:tasks.workunit.client.0.smithi019.stdout:9/571: mknod d2/d7/dc/d97/cc9 0 2024-02-22T15:50:38.611 INFO:tasks.workunit.client.1.smithi062.stdout:0/445: dwrite d1/d3/d4/d1a/d76/d8c/f64 [0,4194304] 0 2024-02-22T15:50:38.611 INFO:tasks.workunit.client.0.smithi019.stdout:6/700: creat d6/d31/d43/fdf x:0 0 0 2024-02-22T15:50:38.612 INFO:tasks.workunit.client.0.smithi019.stdout:7/820: dwrite d9/d38/d41/d45/d61/fc0 [0,4194304] 0 2024-02-22T15:50:38.613 INFO:tasks.workunit.client.0.smithi019.stdout:6/701: read - d6/d4b/d8d/f95 zero size 2024-02-22T15:50:38.617 INFO:tasks.workunit.client.1.smithi062.stdout:4/357: symlink d4/d2a/d75/l7a 0 2024-02-22T15:50:38.617 INFO:tasks.workunit.client.0.smithi019.stdout:3/448: creat d9/d26/d3f/f8c x:0 0 0 2024-02-22T15:50:38.620 INFO:tasks.workunit.client.0.smithi019.stdout:4/578: dread d2/fd [0,4194304] 0 2024-02-22T15:50:38.622 INFO:tasks.workunit.client.1.smithi062.stdout:1/388: dwrite d1/d6/de/f31 [4194304,4194304] 0 2024-02-22T15:50:38.622 INFO:tasks.workunit.client.0.smithi019.stdout:6/702: write d6/f2c [1713701,77553] 0 2024-02-22T15:50:38.625 INFO:tasks.workunit.client.0.smithi019.stdout:4/579: symlink d2/db/d14/d23/d33/lbe 0 2024-02-22T15:50:38.625 INFO:tasks.workunit.client.0.smithi019.stdout:6/703: rmdir d6/d4b 39 2024-02-22T15:50:38.629 INFO:tasks.workunit.client.0.smithi019.stdout:3/449: dread d9/d11/f17 [0,4194304] 0 2024-02-22T15:50:38.631 INFO:tasks.workunit.client.0.smithi019.stdout:9/572: dwrite d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:38.633 INFO:tasks.workunit.client.0.smithi019.stdout:9/573: chown d2/d7/dc/d27/dbc 57397606 1 2024-02-22T15:50:38.635 INFO:tasks.workunit.client.0.smithi019.stdout:9/574: fsync d2/d7/dc/d27/d87/fb1 0 2024-02-22T15:50:38.636 INFO:tasks.workunit.client.1.smithi062.stdout:6/474: dwrite d5/df/d40/d44/d4a/d41/f55 [0,4194304] 0 2024-02-22T15:50:38.637 INFO:tasks.workunit.client.0.smithi019.stdout:6/704: rename d6/d4b/d6d/d7b/lb5 to d6/d4b/d71/dc2/le0 0 2024-02-22T15:50:38.641 INFO:tasks.workunit.client.0.smithi019.stdout:9/575: mkdir d2/d7/dc/d27/d68/dca 0 2024-02-22T15:50:38.641 INFO:tasks.workunit.client.1.smithi062.stdout:0/446: dwrite d1/d3/d5f/f57 [0,4194304] 0 2024-02-22T15:50:38.644 INFO:tasks.workunit.client.0.smithi019.stdout:9/576: mknod d2/d7/dc/d27/d87/ccb 0 2024-02-22T15:50:38.644 INFO:tasks.workunit.client.0.smithi019.stdout:9/577: stat d2/d7/f1b 0 2024-02-22T15:50:38.649 INFO:tasks.workunit.client.1.smithi062.stdout:4/358: dwrite d4/d1c/d2c/d6b/d70/f76 [0,4194304] 0 2024-02-22T15:50:38.649 INFO:tasks.workunit.client.0.smithi019.stdout:7/821: dread d9/d26/f33 [4194304,4194304] 0 2024-02-22T15:50:38.650 INFO:tasks.workunit.client.0.smithi019.stdout:9/578: rename d2/d1f/d2e/d48/da6 to d2/d7/dc/d27/dbc/dcc 0 2024-02-22T15:50:38.650 INFO:tasks.workunit.client.0.smithi019.stdout:1/633: dread d8/d15/d34/f39 [0,4194304] 0 2024-02-22T15:50:38.652 INFO:tasks.workunit.client.1.smithi062.stdout:0/447: creat d1/d38/f98 x:0 0 0 2024-02-22T15:50:38.655 INFO:tasks.workunit.client.0.smithi019.stdout:9/579: mkdir d2/d7/d52/dc0/dcd 0 2024-02-22T15:50:38.660 INFO:tasks.workunit.client.0.smithi019.stdout:7/822: creat d9/d38/d41/d45/f112 x:0 0 0 2024-02-22T15:50:38.660 INFO:tasks.workunit.client.0.smithi019.stdout:9/580: mknod d2/d1f/d2e/d48/cce 0 2024-02-22T15:50:38.660 INFO:tasks.workunit.client.1.smithi062.stdout:6/475: link d5/df/f84 d5/dd/d33/f97 0 2024-02-22T15:50:38.660 INFO:tasks.workunit.client.1.smithi062.stdout:4/359: truncate d4/d1c/d30/f6d 483538 0 2024-02-22T15:50:38.660 INFO:tasks.workunit.client.1.smithi062.stdout:4/360: chown d4/d1c/d30/d3b/d49 2286084 1 2024-02-22T15:50:38.662 INFO:tasks.workunit.client.0.smithi019.stdout:4/580: write d2/db/d14/f87 [1289480,130095] 0 2024-02-22T15:50:38.664 INFO:tasks.workunit.client.0.smithi019.stdout:9/581: rename d2/d1f/d2e/d57/d60/d8c/d66/l67 to d2/d7/dc/d27/d31/d79/lcf 0 2024-02-22T15:50:38.664 INFO:tasks.workunit.client.0.smithi019.stdout:4/581: read d2/db/d54/d8c/fa4 [487039,115328] 0 2024-02-22T15:50:38.665 INFO:tasks.workunit.client.1.smithi062.stdout:4/361: dwrite d4/f39 [0,4194304] 0 2024-02-22T15:50:38.667 INFO:tasks.workunit.client.0.smithi019.stdout:4/582: mkdir d2/db/d14/d23/d21/dbf 0 2024-02-22T15:50:38.669 INFO:tasks.workunit.client.1.smithi062.stdout:4/362: mkdir d4/d2a/d2d/d2e/d6a/d7b 0 2024-02-22T15:50:38.669 INFO:tasks.workunit.client.0.smithi019.stdout:4/583: write d2/db/d14/d23/d21/f9b [750099,77059] 0 2024-02-22T15:50:38.669 INFO:tasks.workunit.client.1.smithi062.stdout:4/363: read d4/f39 [2329819,51083] 0 2024-02-22T15:50:38.670 INFO:tasks.workunit.client.0.smithi019.stdout:9/582: getdents d2/d1f/d2e/d57 0 2024-02-22T15:50:38.672 INFO:tasks.workunit.client.0.smithi019.stdout:9/583: read d2/d1f/d2e/d57/d60/d8c/f6a [1303021,51128] 0 2024-02-22T15:50:38.675 INFO:tasks.workunit.client.1.smithi062.stdout:4/364: link d4/d1c/d30/d37/l42 d4/d1c/d30/d37/l7c 0 2024-02-22T15:50:38.675 INFO:tasks.workunit.client.0.smithi019.stdout:9/584: truncate d2/d7/dc/d27/d31/d79/fb2 4626207 0 2024-02-22T15:50:38.675 INFO:tasks.workunit.client.1.smithi062.stdout:4/365: write d4/d1c/d30/f32 [4079385,1201] 0 2024-02-22T15:50:38.677 INFO:tasks.workunit.client.1.smithi062.stdout:4/366: readlink d4/d1c/d30/d3b/d49/l61 0 2024-02-22T15:50:38.677 INFO:tasks.workunit.client.0.smithi019.stdout:9/585: unlink d2/d1f/d2e/d57/d60/d8c/d66/la1 0 2024-02-22T15:50:38.679 INFO:tasks.workunit.client.0.smithi019.stdout:9/586: unlink d2/fab 0 2024-02-22T15:50:38.687 INFO:tasks.workunit.client.1.smithi062.stdout:1/389: truncate d1/d6/f1a 129643 0 2024-02-22T15:50:38.689 INFO:tasks.workunit.client.1.smithi062.stdout:0/448: dread d1/dd/f63 [0,4194304] 0 2024-02-22T15:50:38.690 INFO:tasks.workunit.client.0.smithi019.stdout:3/450: dread d9/d11/f1f [4194304,4194304] 0 2024-02-22T15:50:38.691 INFO:tasks.workunit.client.0.smithi019.stdout:6/705: truncate d6/d4b/f52 2266087 0 2024-02-22T15:50:38.691 INFO:tasks.workunit.client.1.smithi062.stdout:4/367: dread d4/f3f [0,4194304] 0 2024-02-22T15:50:38.696 INFO:tasks.workunit.client.0.smithi019.stdout:7/823: write d9/d38/d41/d45/db0/df7/f100 [30490,106431] 0 2024-02-22T15:50:38.699 INFO:tasks.workunit.client.0.smithi019.stdout:7/824: write d9/fe2 [785163,76176] 0 2024-02-22T15:50:38.700 INFO:tasks.workunit.client.0.smithi019.stdout:8/559: sync 2024-02-22T15:50:38.701 INFO:tasks.workunit.client.0.smithi019.stdout:1/634: dwrite d8/d2a/f3f [0,4194304] 0 2024-02-22T15:50:38.702 INFO:tasks.workunit.client.0.smithi019.stdout:7/825: read - d9/d38/d41/d45/d92/ffa zero size 2024-02-22T15:50:38.702 INFO:tasks.workunit.client.1.smithi062.stdout:1/390: dwrite d1/d6/d7/d2b/f5b [0,4194304] 0 2024-02-22T15:50:38.703 INFO:tasks.workunit.client.0.smithi019.stdout:8/560: dread - d9/d14/d17/f85 zero size 2024-02-22T15:50:38.704 INFO:tasks.workunit.client.0.smithi019.stdout:3/451: dwrite d9/d14/f3c [8388608,4194304] 0 2024-02-22T15:50:38.705 INFO:tasks.workunit.client.0.smithi019.stdout:8/561: chown d9/d14/c7a 24349847 1 2024-02-22T15:50:38.707 INFO:tasks.workunit.client.0.smithi019.stdout:3/452: dread - d9/d26/d3f/d5c/f6e zero size 2024-02-22T15:50:38.707 INFO:tasks.workunit.client.1.smithi062.stdout:4/368: truncate d4/d1c/d30/d3b/f67 1059264 0 2024-02-22T15:50:38.707 INFO:tasks.workunit.client.1.smithi062.stdout:6/476: dwrite d5/f6 [0,4194304] 0 2024-02-22T15:50:38.707 INFO:tasks.workunit.client.0.smithi019.stdout:3/453: readlink d9/d11/l60 0 2024-02-22T15:50:38.708 INFO:tasks.workunit.client.1.smithi062.stdout:1/391: mknod d1/d6/d7/d2b/d47/c80 0 2024-02-22T15:50:38.709 INFO:tasks.workunit.client.0.smithi019.stdout:7/826: creat d9/d38/df3/f113 x:0 0 0 2024-02-22T15:50:38.710 INFO:tasks.workunit.client.0.smithi019.stdout:3/454: chown c8 10997 1 2024-02-22T15:50:38.711 INFO:tasks.workunit.client.1.smithi062.stdout:6/477: write d5/df/d40/d44/d78/f31 [1377575,52209] 0 2024-02-22T15:50:38.711 INFO:tasks.workunit.client.1.smithi062.stdout:1/392: write d1/d6/d7/f29 [2946609,95055] 0 2024-02-22T15:50:38.713 INFO:tasks.workunit.client.0.smithi019.stdout:8/562: rename d9/d14/f89 to d9/d14/d17/fbe 0 2024-02-22T15:50:38.714 INFO:tasks.workunit.client.0.smithi019.stdout:3/455: symlink d9/d26/d2d/d39/l8d 0 2024-02-22T15:50:38.714 INFO:tasks.workunit.client.0.smithi019.stdout:7/827: write d9/ff1 [4113355,39452] 0 2024-02-22T15:50:38.719 INFO:tasks.workunit.client.0.smithi019.stdout:7/828: rmdir d9/d4c 39 2024-02-22T15:50:38.719 INFO:tasks.workunit.client.0.smithi019.stdout:0/463: sync 2024-02-22T15:50:38.722 INFO:tasks.workunit.client.1.smithi062.stdout:6/478: dread - d5/df/d40/d44/f95 zero size 2024-02-22T15:50:38.722 INFO:tasks.workunit.client.1.smithi062.stdout:6/479: fdatasync d5/df/d40/d44/d4a/f5e 0 2024-02-22T15:50:38.722 INFO:tasks.workunit.client.1.smithi062.stdout:0/449: dwrite d1/d3/d4/d1a/d76/d8c/d5d/f37 [4194304,4194304] 0 2024-02-22T15:50:38.722 INFO:tasks.workunit.client.1.smithi062.stdout:0/450: read d1/dd/f63 [1609328,25392] 0 2024-02-22T15:50:38.723 INFO:tasks.workunit.client.0.smithi019.stdout:7/829: read - d9/d26/d77/f95 zero size 2024-02-22T15:50:38.723 INFO:tasks.workunit.client.0.smithi019.stdout:8/563: fsync f6 0 2024-02-22T15:50:38.723 INFO:tasks.workunit.client.0.smithi019.stdout:0/464: chown d4/db/d1b/d63/f87 18154042 1 2024-02-22T15:50:38.724 INFO:tasks.workunit.client.0.smithi019.stdout:7/830: write d9/d38/f4e [4050576,126057] 0 2024-02-22T15:50:38.724 INFO:tasks.workunit.client.1.smithi062.stdout:4/369: unlink d4/d2a/d2d/f35 0 2024-02-22T15:50:38.727 INFO:tasks.workunit.client.0.smithi019.stdout:8/564: chown d9/f12 11 1 2024-02-22T15:50:38.727 INFO:tasks.workunit.client.0.smithi019.stdout:0/465: symlink d4/d5/d52/la4 0 2024-02-22T15:50:38.727 INFO:tasks.workunit.client.0.smithi019.stdout:7/831: dread - d9/d38/d41/d45/d61/d6a/fd0 zero size 2024-02-22T15:50:38.729 INFO:tasks.workunit.client.0.smithi019.stdout:7/832: write d9/d38/d41/f78 [434404,116835] 0 2024-02-22T15:50:38.733 INFO:tasks.workunit.client.1.smithi062.stdout:4/370: rename d4/d1c/d2c/d6b/d70/d79 to d4/d2a/d7d 0 2024-02-22T15:50:38.733 INFO:tasks.workunit.client.0.smithi019.stdout:6/706: dread d6/d4b/d6d/d7b/f60 [0,4194304] 0 2024-02-22T15:50:38.737 INFO:tasks.workunit.client.0.smithi019.stdout:0/466: dwrite d4/d22/d2c/f9a [0,4194304] 0 2024-02-22T15:50:38.738 INFO:tasks.workunit.client.1.smithi062.stdout:4/371: unlink d4/f59 0 2024-02-22T15:50:38.738 INFO:tasks.workunit.client.1.smithi062.stdout:0/451: dwrite d1/d13/f31 [0,4194304] 0 2024-02-22T15:50:38.739 INFO:tasks.workunit.client.1.smithi062.stdout:6/480: dwrite d5/df/d40/d44/d76/f6a [0,4194304] 0 2024-02-22T15:50:38.740 INFO:tasks.workunit.client.1.smithi062.stdout:0/452: fsync d1/d21/f48 0 2024-02-22T15:50:38.740 INFO:tasks.workunit.client.1.smithi062.stdout:6/481: chown d5/d6e/c7d 0 1 2024-02-22T15:50:38.748 INFO:tasks.workunit.client.1.smithi062.stdout:6/482: creat d5/df/d40/d44/d76/d56/f98 x:0 0 0 2024-02-22T15:50:38.749 INFO:tasks.workunit.client.1.smithi062.stdout:0/453: creat d1/d3/d4/f99 x:0 0 0 2024-02-22T15:50:38.749 INFO:tasks.workunit.client.1.smithi062.stdout:4/372: dread d4/fd [0,4194304] 0 2024-02-22T15:50:38.757 INFO:tasks.workunit.client.1.smithi062.stdout:0/454: mknod d1/d5e/c9a 0 2024-02-22T15:50:38.757 INFO:tasks.workunit.client.1.smithi062.stdout:4/373: creat d4/d1c/d2c/d6b/f7e x:0 0 0 2024-02-22T15:50:38.757 INFO:tasks.workunit.client.1.smithi062.stdout:4/374: fdatasync d4/d1c/d2c/d6b/f7e 0 2024-02-22T15:50:38.757 INFO:tasks.workunit.client.1.smithi062.stdout:1/393: dread d1/d6/d7/f54 [0,4194304] 0 2024-02-22T15:50:38.758 INFO:tasks.workunit.client.1.smithi062.stdout:6/483: symlink d5/df/d40/d44/d4a/d41/d43/l99 0 2024-02-22T15:50:38.759 INFO:tasks.workunit.client.1.smithi062.stdout:0/455: chown d1/d3/d4/d1e/d74/d68 167282332 1 2024-02-22T15:50:38.760 INFO:tasks.workunit.client.1.smithi062.stdout:0/456: chown d1/d13/d43 27 1 2024-02-22T15:50:38.768 INFO:tasks.workunit.client.1.smithi062.stdout:4/375: dwrite d4/d1c/d2c/d6b/f7e [0,4194304] 0 2024-02-22T15:50:38.769 INFO:tasks.workunit.client.0.smithi019.stdout:3/456: rmdir d9/d26/d2d/d39 39 2024-02-22T15:50:38.769 INFO:tasks.workunit.client.0.smithi019.stdout:3/457: dread - d9/d26/d3f/f8c zero size 2024-02-22T15:50:38.770 INFO:tasks.workunit.client.1.smithi062.stdout:0/457: symlink d1/d3/d4/d1a/d1d/l9b 0 2024-02-22T15:50:38.771 INFO:tasks.workunit.client.1.smithi062.stdout:4/376: mknod d4/d2a/d2d/d2e/c7f 0 2024-02-22T15:50:38.775 INFO:tasks.workunit.client.0.smithi019.stdout:3/458: symlink d9/d22/d27/d6d/d72/l8e 0 2024-02-22T15:50:38.775 INFO:tasks.workunit.client.1.smithi062.stdout:6/484: dread d5/df/d40/d44/d4a/f28 [0,4194304] 0 2024-02-22T15:50:38.776 INFO:tasks.workunit.client.1.smithi062.stdout:0/458: symlink d1/d3/d4/d1a/d1d/l9c 0 2024-02-22T15:50:38.779 INFO:tasks.workunit.client.0.smithi019.stdout:3/459: dread - d9/d14/f86 zero size 2024-02-22T15:50:38.779 INFO:tasks.workunit.client.1.smithi062.stdout:0/459: mknod d1/d5e/c9d 0 2024-02-22T15:50:38.782 INFO:tasks.workunit.client.0.smithi019.stdout:8/565: dwrite d9/d14/d17/f4a [4194304,4194304] 0 2024-02-22T15:50:38.782 INFO:tasks.workunit.client.1.smithi062.stdout:4/377: dwrite d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:38.783 INFO:tasks.workunit.client.0.smithi019.stdout:6/707: dwrite d6/f1c [0,4194304] 0 2024-02-22T15:50:38.783 INFO:tasks.workunit.client.0.smithi019.stdout:7/833: dwrite d9/d38/f59 [0,4194304] 0 2024-02-22T15:50:38.785 INFO:tasks.workunit.client.0.smithi019.stdout:6/708: chown d6/f44 9703 1 2024-02-22T15:50:38.786 INFO:tasks.workunit.client.1.smithi062.stdout:1/394: write d1/d6/f1a [1064680,2773] 0 2024-02-22T15:50:38.786 INFO:tasks.workunit.client.1.smithi062.stdout:4/378: readlink d4/l5a 0 2024-02-22T15:50:38.786 INFO:tasks.workunit.client.0.smithi019.stdout:0/467: write f1 [573927,76903] 0 2024-02-22T15:50:38.789 INFO:tasks.workunit.client.1.smithi062.stdout:0/460: getdents d1/dd 0 2024-02-22T15:50:38.790 INFO:tasks.workunit.client.1.smithi062.stdout:1/395: symlink d1/d6/d36/l81 0 2024-02-22T15:50:38.795 INFO:tasks.workunit.client.0.smithi019.stdout:7/834: mknod d9/c114 0 2024-02-22T15:50:38.796 INFO:tasks.workunit.client.1.smithi062.stdout:4/379: rename d4/d13/d20/l2f to d4/d2a/d2d/l80 0 2024-02-22T15:50:38.796 INFO:tasks.workunit.client.0.smithi019.stdout:8/566: mknod d9/d14/d17/d3d/d3f/cbf 0 2024-02-22T15:50:38.797 INFO:tasks.workunit.client.1.smithi062.stdout:4/380: chown d4/d13/d20/l31 116172 1 2024-02-22T15:50:38.798 INFO:tasks.workunit.client.0.smithi019.stdout:0/468: chown d4/db/d76 1150 1 2024-02-22T15:50:38.798 INFO:tasks.workunit.client.1.smithi062.stdout:1/396: creat d1/f82 x:0 0 0 2024-02-22T15:50:38.799 INFO:tasks.workunit.client.1.smithi062.stdout:0/461: getdents d1/d3/d5f 0 2024-02-22T15:50:38.799 INFO:tasks.workunit.client.1.smithi062.stdout:1/397: write d1/f82 [840436,32660] 0 2024-02-22T15:50:38.800 INFO:tasks.workunit.client.0.smithi019.stdout:3/460: write d9/d26/d3f/d59/d66/f69 [414127,107317] 0 2024-02-22T15:50:38.801 INFO:tasks.workunit.client.1.smithi062.stdout:4/381: creat d4/d1c/d2c/f81 x:0 0 0 2024-02-22T15:50:38.802 INFO:tasks.workunit.client.0.smithi019.stdout:8/567: symlink d9/d14/d6f/d7e/d94/db8/lc0 0 2024-02-22T15:50:38.802 INFO:tasks.workunit.client.1.smithi062.stdout:6/485: write d5/df/d40/d44/d78/f70 [1209307,68607] 0 2024-02-22T15:50:38.803 INFO:tasks.workunit.client.0.smithi019.stdout:0/469: chown d4/db/d1b/f1c 2 1 2024-02-22T15:50:38.803 INFO:tasks.workunit.client.0.smithi019.stdout:6/709: dwrite d6/d4b/d8d/f95 [0,4194304] 0 2024-02-22T15:50:38.804 INFO:tasks.workunit.client.0.smithi019.stdout:0/470: fdatasync d4/f3e 0 2024-02-22T15:50:38.804 INFO:tasks.workunit.client.0.smithi019.stdout:3/461: read - d9/d65/f7d zero size 2024-02-22T15:50:38.805 INFO:tasks.workunit.client.1.smithi062.stdout:6/486: write d5/df/f30 [3040244,6153] 0 2024-02-22T15:50:38.805 INFO:tasks.workunit.client.1.smithi062.stdout:9/325: sync 2024-02-22T15:50:38.806 INFO:tasks.workunit.client.0.smithi019.stdout:7/835: mknod d9/d38/d41/d45/db0/df7/dfb/de3/c115 0 2024-02-22T15:50:38.807 INFO:tasks.workunit.client.1.smithi062.stdout:1/398: chown d1/d6/d36/f5d 11 1 2024-02-22T15:50:38.808 INFO:tasks.workunit.client.1.smithi062.stdout:6/487: write d5/f6c [1013416,104047] 0 2024-02-22T15:50:38.809 INFO:tasks.workunit.client.1.smithi062.stdout:4/382: unlink d4/d1c/d30/f6d 0 2024-02-22T15:50:38.810 INFO:tasks.workunit.client.0.smithi019.stdout:3/462: creat d9/d16/f8f x:0 0 0 2024-02-22T15:50:38.810 INFO:tasks.workunit.client.0.smithi019.stdout:6/710: symlink d6/d4b/d71/le1 0 2024-02-22T15:50:38.811 INFO:tasks.workunit.client.1.smithi062.stdout:9/326: mkdir d4/d28/d50/d5a 0 2024-02-22T15:50:38.813 INFO:tasks.workunit.client.0.smithi019.stdout:0/471: mkdir d4/da5 0 2024-02-22T15:50:38.813 INFO:tasks.workunit.client.1.smithi062.stdout:9/327: fdatasync d4/d1c/f58 0 2024-02-22T15:50:38.816 INFO:tasks.workunit.client.1.smithi062.stdout:6/488: dread d5/df/d40/d44/d76/d56/f61 [0,4194304] 0 2024-02-22T15:50:38.817 INFO:tasks.workunit.client.0.smithi019.stdout:8/568: dwrite d9/d14/d1f/d6d/f8f [0,4194304] 0 2024-02-22T15:50:38.818 INFO:tasks.workunit.client.1.smithi062.stdout:0/462: dwrite d1/d3/d4/d1a/d76/d8c/d2c/d84/f26 [0,4194304] 0 2024-02-22T15:50:38.818 INFO:tasks.workunit.client.1.smithi062.stdout:1/399: dread d1/f44 [0,4194304] 0 2024-02-22T15:50:38.818 INFO:tasks.workunit.client.1.smithi062.stdout:4/383: mknod d4/d2a/d2d/d64/c82 0 2024-02-22T15:50:38.818 INFO:tasks.workunit.client.0.smithi019.stdout:0/472: fdatasync d4/db/d1b/f1c 0 2024-02-22T15:50:38.819 INFO:tasks.workunit.client.1.smithi062.stdout:6/489: chown d5/df/d40/d44/d78 2523260 1 2024-02-22T15:50:38.826 INFO:tasks.workunit.client.0.smithi019.stdout:6/711: link d6/d3e/c49 d6/d31/d69/d8e/ce2 0 2024-02-22T15:50:38.827 INFO:tasks.workunit.client.1.smithi062.stdout:1/400: write d1/d6/d7/d2b/d47/d63/f68 [313536,100682] 0 2024-02-22T15:50:38.827 INFO:tasks.workunit.client.1.smithi062.stdout:6/490: mkdir d5/dd/d33/d71/d9a 0 2024-02-22T15:50:38.827 INFO:tasks.workunit.client.1.smithi062.stdout:0/463: mknod d1/d3/d4/d1a/d76/d8c/c9e 0 2024-02-22T15:50:38.827 INFO:tasks.workunit.client.1.smithi062.stdout:4/384: symlink d4/d2a/d2d/d2e/d6a/l83 0 2024-02-22T15:50:38.828 INFO:tasks.workunit.client.0.smithi019.stdout:7/836: getdents d9/d38/d41/d45/db0/df7/dfb/de3/deb 0 2024-02-22T15:50:38.829 INFO:tasks.workunit.client.1.smithi062.stdout:1/401: write d1/d6/d7/d2b/d47/d24/f2a [1729554,43839] 0 2024-02-22T15:50:38.829 INFO:tasks.workunit.client.0.smithi019.stdout:0/473: creat d4/d22/d2c/d68/fa6 x:0 0 0 2024-02-22T15:50:38.829 INFO:tasks.workunit.client.1.smithi062.stdout:4/385: read - d4/d2a/d2d/d2e/f77 zero size 2024-02-22T15:50:38.830 INFO:tasks.workunit.client.0.smithi019.stdout:3/463: dwrite f4 [8388608,4194304] 0 2024-02-22T15:50:38.830 INFO:tasks.workunit.client.1.smithi062.stdout:6/491: symlink d5/dd/d33/d71/d9a/l9b 0 2024-02-22T15:50:38.832 INFO:tasks.workunit.client.0.smithi019.stdout:6/712: creat d6/d58/fe3 x:0 0 0 2024-02-22T15:50:38.832 INFO:tasks.workunit.client.0.smithi019.stdout:0/474: chown d4/d22/d3d/f73 725 1 2024-02-22T15:50:38.833 INFO:tasks.workunit.client.0.smithi019.stdout:6/713: chown d6/d4b/c91 432186 1 2024-02-22T15:50:38.834 INFO:tasks.workunit.client.1.smithi062.stdout:1/402: mkdir d1/d6/d7/d4a/d65/d83 0 2024-02-22T15:50:38.835 INFO:tasks.workunit.client.0.smithi019.stdout:3/464: rename d9/c83 to d9/d14/d19/c90 0 2024-02-22T15:50:38.835 INFO:tasks.workunit.client.0.smithi019.stdout:6/714: truncate d6/d3e/f42 4919492 0 2024-02-22T15:50:38.836 INFO:tasks.workunit.client.1.smithi062.stdout:6/492: creat d5/dd/d33/d71/d9a/f9c x:0 0 0 2024-02-22T15:50:38.836 INFO:tasks.workunit.client.0.smithi019.stdout:0/475: write d4/d32/f9c [2172874,16478] 0 2024-02-22T15:50:38.837 INFO:tasks.workunit.client.0.smithi019.stdout:7/837: creat d9/f116 x:0 0 0 2024-02-22T15:50:38.842 INFO:tasks.workunit.client.0.smithi019.stdout:0/476: mkdir d4/d9/da7 0 2024-02-22T15:50:38.842 INFO:tasks.workunit.client.1.smithi062.stdout:4/386: link d4/d1c/f23 d4/d2a/d2d/d2e/f84 0 2024-02-22T15:50:38.842 INFO:tasks.workunit.client.1.smithi062.stdout:0/464: dwrite d1/d3/d4/d1a/d76/d8c/f88 [0,4194304] 0 2024-02-22T15:50:38.843 INFO:tasks.workunit.client.0.smithi019.stdout:7/838: dread - d9/d38/d41/d45/db0/df7/dfb/de3/deb/f10d zero size 2024-02-22T15:50:38.843 INFO:tasks.workunit.client.1.smithi062.stdout:1/403: symlink d1/d6/d36/d76/l84 0 2024-02-22T15:50:38.843 INFO:tasks.workunit.client.0.smithi019.stdout:0/477: chown d4/db/d1b/d40/c61 12 1 2024-02-22T15:50:38.844 INFO:tasks.workunit.client.0.smithi019.stdout:6/715: link d6/d31/d48/f51 d6/d4b/d6d/da3/fe4 0 2024-02-22T15:50:38.846 INFO:tasks.workunit.client.0.smithi019.stdout:3/465: creat d9/d26/d2d/f91 x:0 0 0 2024-02-22T15:50:38.846 INFO:tasks.workunit.client.0.smithi019.stdout:7/839: dread d9/d26/f87 [0,4194304] 0 2024-02-22T15:50:38.847 INFO:tasks.workunit.client.0.smithi019.stdout:0/478: symlink d4/d32/la8 0 2024-02-22T15:50:38.849 INFO:tasks.workunit.client.0.smithi019.stdout:6/716: fdatasync d6/d58/f66 0 2024-02-22T15:50:38.850 INFO:tasks.workunit.client.0.smithi019.stdout:3/466: symlink d9/d22/d49/l92 0 2024-02-22T15:50:38.851 INFO:tasks.workunit.client.0.smithi019.stdout:7/840: fsync d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/fc8 0 2024-02-22T15:50:38.851 INFO:tasks.workunit.client.0.smithi019.stdout:3/467: dread - d9/d11/f8a zero size 2024-02-22T15:50:38.852 INFO:tasks.workunit.client.1.smithi062.stdout:4/387: dwrite d4/f53 [0,4194304] 0 2024-02-22T15:50:38.852 INFO:tasks.workunit.client.0.smithi019.stdout:0/479: creat d4/d22/fa9 x:0 0 0 2024-02-22T15:50:38.853 INFO:tasks.workunit.client.1.smithi062.stdout:1/404: unlink d1/d6/d7/d2b/d47/c80 0 2024-02-22T15:50:38.855 INFO:tasks.workunit.client.0.smithi019.stdout:0/480: dread - d4/f8e zero size 2024-02-22T15:50:38.856 INFO:tasks.workunit.client.0.smithi019.stdout:3/468: unlink d9/d11/f17 0 2024-02-22T15:50:38.858 INFO:tasks.workunit.client.0.smithi019.stdout:8/569: dread d9/d14/d17/f4a [0,4194304] 0 2024-02-22T15:50:38.859 INFO:tasks.workunit.client.0.smithi019.stdout:6/717: dwrite d6/f2c [0,4194304] 0 2024-02-22T15:50:38.859 INFO:tasks.workunit.client.1.smithi062.stdout:0/465: dread d1/dd/f36 [0,4194304] 0 2024-02-22T15:50:38.860 INFO:tasks.workunit.client.1.smithi062.stdout:4/388: dread d4/f3f [0,4194304] 0 2024-02-22T15:50:38.861 INFO:tasks.workunit.client.1.smithi062.stdout:4/389: write d4/f53 [5082818,24252] 0 2024-02-22T15:50:38.861 INFO:tasks.workunit.client.1.smithi062.stdout:9/328: dwrite d4/fd [0,4194304] 0 2024-02-22T15:50:38.862 INFO:tasks.workunit.client.1.smithi062.stdout:6/493: rename d5/dd/d33/d71/d9a to d5/df/d40/d44/d4a/d41/d85/d9d 0 2024-02-22T15:50:38.864 INFO:tasks.workunit.client.1.smithi062.stdout:9/329: truncate d4/d1c/f58 56492 0 2024-02-22T15:50:38.864 INFO:tasks.workunit.client.0.smithi019.stdout:6/718: mkdir d6/d3e/de5 0 2024-02-22T15:50:38.868 INFO:tasks.workunit.client.1.smithi062.stdout:9/330: fsync d4/d1c/f1d 0 2024-02-22T15:50:38.872 INFO:tasks.workunit.client.1.smithi062.stdout:4/390: rmdir d4/d1c 39 2024-02-22T15:50:38.872 INFO:tasks.workunit.client.1.smithi062.stdout:0/466: mknod d1/d3/d4/d1a/d1d/c9f 0 2024-02-22T15:50:38.873 INFO:tasks.workunit.client.0.smithi019.stdout:3/469: dwrite d9/d22/d27/d6d/f82 [0,4194304] 0 2024-02-22T15:50:38.874 INFO:tasks.workunit.client.1.smithi062.stdout:9/331: read d4/d28/f38 [6828419,116832] 0 2024-02-22T15:50:38.877 INFO:tasks.workunit.client.0.smithi019.stdout:8/570: dwrite d9/d14/d17/f47 [4194304,4194304] 0 2024-02-22T15:50:38.877 INFO:tasks.workunit.client.0.smithi019.stdout:3/470: unlink d9/d11/c5b 0 2024-02-22T15:50:38.878 INFO:tasks.workunit.client.0.smithi019.stdout:3/471: fsync d9/d26/d76/f77 0 2024-02-22T15:50:38.880 INFO:tasks.workunit.client.0.smithi019.stdout:3/472: dread - d9/d26/d3f/d59/d66/f87 zero size 2024-02-22T15:50:38.880 INFO:tasks.workunit.client.1.smithi062.stdout:9/332: dread d4/d1c/f1e [0,4194304] 0 2024-02-22T15:50:38.881 INFO:tasks.workunit.client.1.smithi062.stdout:0/467: creat d1/d3/d4/d1a/d1d/fa0 x:0 0 0 2024-02-22T15:50:38.886 INFO:tasks.workunit.client.1.smithi062.stdout:0/468: chown d1/d3/l19 0 1 2024-02-22T15:50:38.887 INFO:tasks.workunit.client.1.smithi062.stdout:4/391: link d4/d1c/d30/f32 d4/d2a/d2d/f85 0 2024-02-22T15:50:38.888 INFO:tasks.workunit.client.0.smithi019.stdout:3/473: rename d9/d26/d32/l4e to d9/d26/d2d/l93 0 2024-02-22T15:50:38.889 INFO:tasks.workunit.client.0.smithi019.stdout:0/481: dread d4/f56 [0,4194304] 0 2024-02-22T15:50:38.889 INFO:tasks.workunit.client.1.smithi062.stdout:4/392: chown d4/d1c/d2c/d6b/f7e 242793 1 2024-02-22T15:50:38.891 INFO:tasks.workunit.client.0.smithi019.stdout:8/571: dwrite d9/d14/d17/d1c/f9d [0,4194304] 0 2024-02-22T15:50:38.891 INFO:tasks.workunit.client.0.smithi019.stdout:0/482: chown d4/db/d1b/f1c 736809 1 2024-02-22T15:50:38.891 INFO:tasks.workunit.client.1.smithi062.stdout:9/333: getdents d4/d35/d3f 0 2024-02-22T15:50:38.892 INFO:tasks.workunit.client.0.smithi019.stdout:3/474: symlink d9/d26/d76/l94 0 2024-02-22T15:50:38.893 INFO:tasks.workunit.client.1.smithi062.stdout:0/469: creat d1/d21/d85/d8d/fa1 x:0 0 0 2024-02-22T15:50:38.894 INFO:tasks.workunit.client.1.smithi062.stdout:4/393: mknod d4/d1c/c86 0 2024-02-22T15:50:38.897 INFO:tasks.workunit.client.0.smithi019.stdout:3/475: read - d9/d26/d3f/d59/f6b zero size 2024-02-22T15:50:38.898 INFO:tasks.workunit.client.1.smithi062.stdout:9/334: chown d4/cb 1584 1 2024-02-22T15:50:38.899 INFO:tasks.workunit.client.1.smithi062.stdout:4/394: creat d4/d1c/d2c/f87 x:0 0 0 2024-02-22T15:50:38.903 INFO:tasks.workunit.client.1.smithi062.stdout:1/405: dwrite d1/d6/d7/d2b/d49/f51 [0,4194304] 0 2024-02-22T15:50:38.903 INFO:tasks.workunit.client.1.smithi062.stdout:0/470: creat d1/d3/d4/d1a/d76/d8c/d2c/d87/fa2 x:0 0 0 2024-02-22T15:50:38.904 INFO:tasks.workunit.client.0.smithi019.stdout:3/476: rename d9/d11/fd to d9/d14/f95 0 2024-02-22T15:50:38.906 INFO:tasks.workunit.client.1.smithi062.stdout:1/406: truncate d1/d6/d7/f54 5207760 0 2024-02-22T15:50:38.906 INFO:tasks.workunit.client.1.smithi062.stdout:9/335: link d4/d15/d3c/f59 d4/d15/d3c/f5b 0 2024-02-22T15:50:38.907 INFO:tasks.workunit.client.1.smithi062.stdout:6/494: dread d5/dd/f20 [0,4194304] 0 2024-02-22T15:50:38.907 INFO:tasks.workunit.client.1.smithi062.stdout:0/471: truncate d1/d3/d4/d1e/f79 866231 0 2024-02-22T15:50:38.908 INFO:tasks.workunit.client.0.smithi019.stdout:8/572: dwrite d9/d14/f38 [0,4194304] 0 2024-02-22T15:50:38.910 INFO:tasks.workunit.client.1.smithi062.stdout:9/336: creat d4/d15/f5c x:0 0 0 2024-02-22T15:50:38.911 INFO:tasks.workunit.client.0.smithi019.stdout:7/841: dwrite d9/f34 [0,4194304] 0 2024-02-22T15:50:38.912 INFO:tasks.workunit.client.1.smithi062.stdout:6/495: symlink d5/df/d40/l9e 0 2024-02-22T15:50:38.912 INFO:tasks.workunit.client.1.smithi062.stdout:9/337: dread - d4/d15/d31/f4e zero size 2024-02-22T15:50:38.912 INFO:tasks.workunit.client.0.smithi019.stdout:8/573: chown d9/d14/d17/d3d/f91 8985 1 2024-02-22T15:50:38.915 INFO:tasks.workunit.client.1.smithi062.stdout:6/496: fsync d5/df/d40/d44/d4a/d41/f55 0 2024-02-22T15:50:38.915 INFO:tasks.workunit.client.1.smithi062.stdout:9/338: chown d4/d28/f42 2521574 1 2024-02-22T15:50:38.918 INFO:tasks.workunit.client.0.smithi019.stdout:6/719: dread d6/d31/d43/f56 [0,4194304] 0 2024-02-22T15:50:38.919 INFO:tasks.workunit.client.1.smithi062.stdout:0/472: dwrite d1/d3/d4/d1a/d76/d8c/d2c/d84/f2a [4194304,4194304] 0 2024-02-22T15:50:38.919 INFO:tasks.workunit.client.1.smithi062.stdout:0/473: fsync d1/d13/f41 0 2024-02-22T15:50:38.923 INFO:tasks.workunit.client.0.smithi019.stdout:7/842: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/f117 x:0 0 0 2024-02-22T15:50:38.923 INFO:tasks.workunit.client.0.smithi019.stdout:8/574: mknod d9/d14/d6f/d7e/d94/d9c/cc1 0 2024-02-22T15:50:38.924 INFO:tasks.workunit.client.0.smithi019.stdout:6/720: unlink d6/l19 0 2024-02-22T15:50:38.926 INFO:tasks.workunit.client.0.smithi019.stdout:7/843: chown d9/d38/d41/d45/db0/fea 29 1 2024-02-22T15:50:38.927 INFO:tasks.workunit.client.1.smithi062.stdout:0/474: creat d1/d21/d85/fa3 x:0 0 0 2024-02-22T15:50:38.928 INFO:tasks.workunit.client.1.smithi062.stdout:9/339: mknod d4/d15/c5d 0 2024-02-22T15:50:38.928 INFO:tasks.workunit.client.1.smithi062.stdout:0/475: chown d1/dd/l7f 46340 1 2024-02-22T15:50:38.928 INFO:tasks.workunit.client.0.smithi019.stdout:8/575: truncate d9/d14/d6f/d7e/d94/d9a/fb7 932402 0 2024-02-22T15:50:38.929 INFO:tasks.workunit.client.1.smithi062.stdout:0/476: chown d1/d3/d4/d1a/f56 306723148 1 2024-02-22T15:50:38.930 INFO:tasks.workunit.client.0.smithi019.stdout:6/721: creat d6/d4b/fe6 x:0 0 0 2024-02-22T15:50:38.932 INFO:tasks.workunit.client.0.smithi019.stdout:8/576: readlink d9/d14/d1f/l23 0 2024-02-22T15:50:38.932 INFO:tasks.workunit.client.0.smithi019.stdout:7/844: symlink d9/d38/d41/d45/d92/l118 0 2024-02-22T15:50:38.934 INFO:tasks.workunit.client.1.smithi062.stdout:0/477: rename d1/d3/d4/d1a/d1d/d3b/f55 to d1/d21/d85/d8e/fa4 0 2024-02-22T15:50:38.935 INFO:tasks.workunit.client.1.smithi062.stdout:9/340: dwrite d4/d35/f37 [0,4194304] 0 2024-02-22T15:50:38.941 INFO:tasks.workunit.client.0.smithi019.stdout:7/845: dwrite d9/d4c/ddf/d105/f10b [0,4194304] 0 2024-02-22T15:50:38.941 INFO:tasks.workunit.client.0.smithi019.stdout:0/483: dread d4/db/d1b/f1c [0,4194304] 0 2024-02-22T15:50:38.945 INFO:tasks.workunit.client.1.smithi062.stdout:9/341: dwrite d4/d1c/f1e [0,4194304] 0 2024-02-22T15:50:38.946 INFO:tasks.workunit.client.1.smithi062.stdout:9/342: readlink d4/l11 0 2024-02-22T15:50:38.946 INFO:tasks.workunit.client.0.smithi019.stdout:0/484: unlink d4/d22/d3d/f73 0 2024-02-22T15:50:38.947 INFO:tasks.workunit.client.0.smithi019.stdout:7/846: mknod d9/d38/d41/d45/db0/df7/c119 0 2024-02-22T15:50:38.949 INFO:tasks.workunit.client.0.smithi019.stdout:7/847: write d9/d4c/dc9/f10f [125597,74563] 0 2024-02-22T15:50:38.949 INFO:tasks.workunit.client.0.smithi019.stdout:0/485: getdents d4/d22/d78 0 2024-02-22T15:50:38.953 INFO:tasks.workunit.client.1.smithi062.stdout:9/343: mkdir d4/d15/d5e 0 2024-02-22T15:50:38.954 INFO:tasks.workunit.client.0.smithi019.stdout:7/848: rename d9/d38/d41/d102 to d9/d38/d41/d45/d92/d11a 0 2024-02-22T15:50:38.954 INFO:tasks.workunit.client.1.smithi062.stdout:9/344: dread - d4/d15/f5c zero size 2024-02-22T15:50:38.955 INFO:tasks.workunit.client.1.smithi062.stdout:9/345: truncate d4/f5 4534482 0 2024-02-22T15:50:38.957 INFO:tasks.workunit.client.1.smithi062.stdout:9/346: mkdir d4/d15/d5f 0 2024-02-22T15:50:38.957 INFO:tasks.workunit.client.0.smithi019.stdout:0/486: link d4/d22/d2c/f57 d4/db/d76/d77/faa 0 2024-02-22T15:50:38.958 INFO:tasks.workunit.client.1.smithi062.stdout:9/347: dread - d4/d28/d50/f55 zero size 2024-02-22T15:50:38.958 INFO:tasks.workunit.client.0.smithi019.stdout:7/849: chown d9/d3a/d49/d5d/dd6 5 1 2024-02-22T15:50:38.959 INFO:tasks.workunit.client.1.smithi062.stdout:9/348: dread - d4/d28/d50/f55 zero size 2024-02-22T15:50:38.959 INFO:tasks.workunit.client.0.smithi019.stdout:7/850: fsync d9/fe2 0 2024-02-22T15:50:38.960 INFO:tasks.workunit.client.0.smithi019.stdout:0/487: mkdir d4/db/d76/d77/dab 0 2024-02-22T15:50:38.962 INFO:tasks.workunit.client.1.smithi062.stdout:9/349: read d4/d1c/f2c [270712,55673] 0 2024-02-22T15:50:38.962 INFO:tasks.workunit.client.0.smithi019.stdout:7/851: mkdir d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/d11b 0 2024-02-22T15:50:38.965 INFO:tasks.workunit.client.1.smithi062.stdout:9/350: creat d4/d15/d31/f60 x:0 0 0 2024-02-22T15:50:38.967 INFO:tasks.workunit.client.1.smithi062.stdout:9/351: getdents d4/d28/d50/d5a 0 2024-02-22T15:50:38.968 INFO:tasks.workunit.client.0.smithi019.stdout:7/852: dwrite d9/d38/d41/f78 [0,4194304] 0 2024-02-22T15:50:38.969 INFO:tasks.workunit.client.1.smithi062.stdout:9/352: fsync d4/d35/d3f/f54 0 2024-02-22T15:50:38.970 INFO:tasks.workunit.client.1.smithi062.stdout:9/353: chown d4/d15/d5f 76346 1 2024-02-22T15:50:38.971 INFO:tasks.workunit.client.1.smithi062.stdout:0/478: dread d1/d3/d4/d1a/d76/d8c/f62 [0,4194304] 0 2024-02-22T15:50:38.971 INFO:tasks.workunit.client.0.smithi019.stdout:7/853: creat d9/f11c x:0 0 0 2024-02-22T15:50:38.972 INFO:tasks.workunit.client.0.smithi019.stdout:7/854: fdatasync d9/f23 0 2024-02-22T15:50:38.972 INFO:tasks.workunit.client.1.smithi062.stdout:9/354: read d4/d35/d3f/f25 [48733,112450] 0 2024-02-22T15:50:38.974 INFO:tasks.workunit.client.1.smithi062.stdout:0/479: mkdir d1/d3/d4/d1a/d76/da5 0 2024-02-22T15:50:38.975 INFO:tasks.workunit.client.1.smithi062.stdout:9/355: symlink d4/d15/d31/l61 0 2024-02-22T15:50:38.975 INFO:tasks.workunit.client.0.smithi019.stdout:7/855: rename d9/lf0 to d9/d38/d41/d45/db0/df7/dfb/de3/l11d 0 2024-02-22T15:50:38.976 INFO:tasks.workunit.client.0.smithi019.stdout:7/856: truncate d9/d4c/dc9/fdc 1490284 0 2024-02-22T15:50:38.977 INFO:tasks.workunit.client.1.smithi062.stdout:0/480: rename d1/d13/c4f to d1/d38/ca6 0 2024-02-22T15:50:38.978 INFO:tasks.workunit.client.0.smithi019.stdout:7/857: creat d9/d38/d41/d45/d92/f11e x:0 0 0 2024-02-22T15:50:38.978 INFO:tasks.workunit.client.1.smithi062.stdout:9/356: getdents d4/d15/d5f 0 2024-02-22T15:50:38.979 INFO:tasks.workunit.client.0.smithi019.stdout:7/858: fsync d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/ffd 0 2024-02-22T15:50:38.980 INFO:tasks.workunit.client.0.smithi019.stdout:7/859: chown d9/d3a/f60 8818 1 2024-02-22T15:50:38.980 INFO:tasks.workunit.client.1.smithi062.stdout:9/357: mkdir d4/d35/d3f/d57/d62 0 2024-02-22T15:50:38.981 INFO:tasks.workunit.client.0.smithi019.stdout:7/860: mknod d9/d26/dc4/c11f 0 2024-02-22T15:50:38.981 INFO:tasks.workunit.client.1.smithi062.stdout:0/481: dread d1/d3/d4/d1a/f34 [4194304,4194304] 0 2024-02-22T15:50:38.982 INFO:tasks.workunit.client.0.smithi019.stdout:5/608: sync 2024-02-22T15:50:38.983 INFO:tasks.workunit.client.0.smithi019.stdout:7/861: write d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/f111 [540922,72321] 0 2024-02-22T15:50:38.983 INFO:tasks.workunit.client.1.smithi062.stdout:0/482: write d1/dd/f6c [702013,103028] 0 2024-02-22T15:50:38.986 INFO:tasks.workunit.client.1.smithi062.stdout:0/483: creat d1/d3/d4/d1a/d1d/fa7 x:0 0 0 2024-02-22T15:50:38.989 INFO:tasks.workunit.client.0.smithi019.stdout:7/862: chown d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/fc8 3401 1 2024-02-22T15:50:38.991 INFO:tasks.workunit.client.1.smithi062.stdout:9/358: truncate d4/f14 2565844 0 2024-02-22T15:50:38.993 INFO:tasks.workunit.client.0.smithi019.stdout:5/609: dwrite d8/dd/d8e/f8a [0,4194304] 0 2024-02-22T15:50:38.994 INFO:tasks.workunit.client.0.smithi019.stdout:7/863: creat d9/d38/d41/d45/d92/da1/f120 x:0 0 0 2024-02-22T15:50:38.995 INFO:tasks.workunit.client.1.smithi062.stdout:0/484: dwrite d1/dd/f63 [0,4194304] 0 2024-02-22T15:50:38.998 INFO:tasks.workunit.client.1.smithi062.stdout:9/359: dread d4/fd [0,4194304] 0 2024-02-22T15:50:38.999 INFO:tasks.workunit.client.1.smithi062.stdout:0/485: creat d1/d13/d43/fa8 x:0 0 0 2024-02-22T15:50:39.001 INFO:tasks.workunit.client.0.smithi019.stdout:5/610: unlink d8/d11/f7d 0 2024-02-22T15:50:39.003 INFO:tasks.workunit.client.0.smithi019.stdout:7/864: rename d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/f117 to d9/d38/d41/d45/db0/f121 0 2024-02-22T15:50:39.005 INFO:tasks.workunit.client.1.smithi062.stdout:0/486: creat d1/d5e/fa9 x:0 0 0 2024-02-22T15:50:39.005 INFO:tasks.workunit.client.1.smithi062.stdout:9/360: write d4/d1c/f1e [1400823,94349] 0 2024-02-22T15:50:39.006 INFO:tasks.workunit.client.0.smithi019.stdout:5/611: rename d8/d11/c9a to d8/dd/d19/d3a/d46/d7e/cc9 0 2024-02-22T15:50:39.007 INFO:tasks.workunit.client.1.smithi062.stdout:0/487: write d1/d3/d4/d1e/f79 [769623,7841] 0 2024-02-22T15:50:39.008 INFO:tasks.workunit.client.1.smithi062.stdout:0/488: dread - d1/d3/d4/d1a/d1d/fa0 zero size 2024-02-22T15:50:39.008 INFO:tasks.workunit.client.1.smithi062.stdout:9/361: fdatasync d4/f10 0 2024-02-22T15:50:39.009 INFO:tasks.workunit.client.0.smithi019.stdout:5/612: chown d8/d11/d33/fa9 36147 1 2024-02-22T15:50:39.011 INFO:tasks.workunit.client.1.smithi062.stdout:0/489: creat d1/d21/d85/d8e/d97/faa x:0 0 0 2024-02-22T15:50:39.012 INFO:tasks.workunit.client.0.smithi019.stdout:5/613: write d8/fe [943648,94292] 0 2024-02-22T15:50:39.012 INFO:tasks.workunit.client.0.smithi019.stdout:7/865: dwrite d9/d38/d41/d45/d61/d6a/f10e [0,4194304] 0 2024-02-22T15:50:39.013 INFO:tasks.workunit.client.1.smithi062.stdout:9/362: dwrite d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:39.015 INFO:tasks.workunit.client.1.smithi062.stdout:8/364: sync 2024-02-22T15:50:39.016 INFO:tasks.workunit.client.0.smithi019.stdout:6/722: dread d6/f9 [0,4194304] 0 2024-02-22T15:50:39.019 INFO:tasks.workunit.client.0.smithi019.stdout:7/866: getdents d9/d4c/d5b 0 2024-02-22T15:50:39.020 INFO:tasks.workunit.client.1.smithi062.stdout:8/365: creat d8/d9/d44/d4d/f69 x:0 0 0 2024-02-22T15:50:39.021 INFO:tasks.workunit.client.1.smithi062.stdout:9/363: mkdir d4/d15/d5e/d63 0 2024-02-22T15:50:39.024 INFO:tasks.workunit.client.0.smithi019.stdout:5/614: link d8/c9 d8/dd/d19/d3a/d46/d7e/dbe/dc3/cca 0 2024-02-22T15:50:39.024 INFO:tasks.workunit.client.0.smithi019.stdout:5/615: stat d8/dd/d8e/d2c 0 2024-02-22T15:50:39.024 INFO:tasks.workunit.client.0.smithi019.stdout:5/616: stat d8/d11/c6a 0 2024-02-22T15:50:39.024 INFO:tasks.workunit.client.0.smithi019.stdout:6/723: write d6/fd1 [3072105,72420] 0 2024-02-22T15:50:39.027 INFO:tasks.workunit.client.0.smithi019.stdout:6/724: mkdir d6/d31/d43/de7 0 2024-02-22T15:50:39.028 INFO:tasks.workunit.client.1.smithi062.stdout:8/366: creat d8/d9/d2c/d49/d5e/f6a x:0 0 0 2024-02-22T15:50:39.029 INFO:tasks.workunit.client.0.smithi019.stdout:6/725: write d6/d31/d43/fdf [740055,51127] 0 2024-02-22T15:50:39.029 INFO:tasks.workunit.client.1.smithi062.stdout:8/367: write d8/f40 [1221454,79001] 0 2024-02-22T15:50:39.031 INFO:tasks.workunit.client.0.smithi019.stdout:5/617: rename d8/dd/f67 to d8/d11/d33/fcb 0 2024-02-22T15:50:39.031 INFO:tasks.workunit.client.1.smithi062.stdout:8/368: creat d8/d9/d2c/d49/f6b x:0 0 0 2024-02-22T15:50:39.033 INFO:tasks.workunit.client.1.smithi062.stdout:8/369: creat d8/d26/f6c x:0 0 0 2024-02-22T15:50:39.034 INFO:tasks.workunit.client.0.smithi019.stdout:5/618: creat d8/dd/d19/d3a/d46/d7e/fcc x:0 0 0 2024-02-22T15:50:39.034 INFO:tasks.workunit.client.0.smithi019.stdout:5/619: dread - d8/d11/fbb zero size 2024-02-22T15:50:39.035 INFO:tasks.workunit.client.1.smithi062.stdout:8/370: creat d8/d9/d44/d4d/d51/d55/f6d x:0 0 0 2024-02-22T15:50:39.035 INFO:tasks.workunit.client.0.smithi019.stdout:6/726: link d6/d4b/da9/d67/d8a/fde d6/d4b/d6d/da3/ddb/fe8 0 2024-02-22T15:50:39.036 INFO:tasks.workunit.client.1.smithi062.stdout:8/371: chown l7 7232 1 2024-02-22T15:50:39.042 INFO:tasks.workunit.client.0.smithi019.stdout:5/620: dwrite d8/dd/d8e/f8a [0,4194304] 0 2024-02-22T15:50:39.043 INFO:tasks.workunit.client.1.smithi062.stdout:8/372: rmdir d8/d9/d44/d4d/d51/d55 39 2024-02-22T15:50:39.043 INFO:tasks.workunit.client.0.smithi019.stdout:0/488: truncate d4/f92 3875607 0 2024-02-22T15:50:39.045 INFO:tasks.workunit.client.0.smithi019.stdout:0/489: chown d4/d9/f13 773 1 2024-02-22T15:50:39.045 INFO:tasks.workunit.client.1.smithi062.stdout:9/364: dwrite d4/d1c/f2c [0,4194304] 0 2024-02-22T15:50:39.052 INFO:tasks.workunit.client.0.smithi019.stdout:5/621: link d8/dd/d19/l27 d8/dd/d19/d3a/d46/d7e/dbe/lcd 0 2024-02-22T15:50:39.056 INFO:tasks.workunit.client.0.smithi019.stdout:0/490: dwrite d4/d22/d2c/f9a [0,4194304] 0 2024-02-22T15:50:39.057 INFO:tasks.workunit.client.1.smithi062.stdout:9/365: write d4/d35/d3f/f25 [406852,96062] 0 2024-02-22T15:50:39.057 INFO:tasks.workunit.client.0.smithi019.stdout:5/622: mkdir d8/dd/d19/d3a/dce 0 2024-02-22T15:50:39.060 INFO:tasks.workunit.client.0.smithi019.stdout:5/623: chown d8/dd/d19/d3a/d46/f61 451098 1 2024-02-22T15:50:39.061 INFO:tasks.workunit.client.1.smithi062.stdout:9/366: rename d4/d15/d31/f4a to d4/d15/d5e/d63/f64 0 2024-02-22T15:50:39.062 INFO:tasks.workunit.client.0.smithi019.stdout:7/867: truncate d9/d3a/d49/d5d/f72 149331 0 2024-02-22T15:50:39.063 INFO:tasks.workunit.client.0.smithi019.stdout:7/868: read d9/d38/d41/fb7 [1906297,28821] 0 2024-02-22T15:50:39.064 INFO:tasks.workunit.client.1.smithi062.stdout:9/367: creat d4/d35/d3f/d57/f65 x:0 0 0 2024-02-22T15:50:39.066 INFO:tasks.workunit.client.0.smithi019.stdout:6/727: rmdir d6/d4b/da9/d67/d8a 39 2024-02-22T15:50:39.066 INFO:tasks.workunit.client.1.smithi062.stdout:9/368: mknod d4/d1c/c66 0 2024-02-22T15:50:39.067 INFO:tasks.workunit.client.1.smithi062.stdout:9/369: truncate d4/d28/f42 1259840 0 2024-02-22T15:50:39.075 INFO:tasks.workunit.client.0.smithi019.stdout:6/728: rename d6/d4b/da9/d67/d8a/fde to d6/fe9 0 2024-02-22T15:50:39.075 INFO:tasks.workunit.client.0.smithi019.stdout:0/491: dread d4/d22/f51 [0,4194304] 0 2024-02-22T15:50:39.078 INFO:tasks.workunit.client.0.smithi019.stdout:7/869: truncate d9/d4c/d5b/fca 677933 0 2024-02-22T15:50:39.082 INFO:tasks.workunit.client.0.smithi019.stdout:7/870: unlink d9/d3a/ccc 0 2024-02-22T15:50:39.086 INFO:tasks.workunit.client.0.smithi019.stdout:5/624: dread d8/f48 [0,4194304] 0 2024-02-22T15:50:39.086 INFO:tasks.workunit.client.0.smithi019.stdout:0/492: dwrite d4/d5/d52/f58 [0,4194304] 0 2024-02-22T15:50:39.092 INFO:tasks.workunit.client.0.smithi019.stdout:0/493: creat d4/d22/d2c/fac x:0 0 0 2024-02-22T15:50:39.096 INFO:tasks.workunit.client.0.smithi019.stdout:6/729: dwrite d6/f41 [0,4194304] 0 2024-02-22T15:50:39.097 INFO:tasks.workunit.client.0.smithi019.stdout:0/494: dread - d4/d5/dc/d41/f90 zero size 2024-02-22T15:50:39.097 INFO:tasks.workunit.client.0.smithi019.stdout:7/871: dwrite d9/d38/d41/fb7 [4194304,4194304] 0 2024-02-22T15:50:39.101 INFO:tasks.workunit.client.0.smithi019.stdout:6/730: write d6/f10 [4545754,68674] 0 2024-02-22T15:50:39.101 INFO:tasks.workunit.client.0.smithi019.stdout:0/495: mknod d4/d5/cad 0 2024-02-22T15:50:39.102 INFO:tasks.workunit.client.0.smithi019.stdout:6/731: fsync d6/d31/d69/d8e/fab 0 2024-02-22T15:50:39.103 INFO:tasks.workunit.client.0.smithi019.stdout:7/872: mkdir d9/d38/d41/d45/d92/d122 0 2024-02-22T15:50:39.104 INFO:tasks.workunit.client.0.smithi019.stdout:7/873: fdatasync d9/d38/f4e 0 2024-02-22T15:50:39.105 INFO:tasks.workunit.client.0.smithi019.stdout:0/496: mkdir d4/db/d76/dae 0 2024-02-22T15:50:39.107 INFO:tasks.workunit.client.0.smithi019.stdout:7/874: symlink d9/d4c/ddf/l123 0 2024-02-22T15:50:39.108 INFO:tasks.workunit.client.0.smithi019.stdout:0/497: read d4/db/d76/d77/faa [3032534,34609] 0 2024-02-22T15:50:39.110 INFO:tasks.workunit.client.0.smithi019.stdout:5/625: dread d8/dd/d19/d38/d54/f6b [0,4194304] 0 2024-02-22T15:50:39.111 INFO:tasks.workunit.client.0.smithi019.stdout:7/875: readlink d9/d3a/d49/d5d/d8f/le8 0 2024-02-22T15:50:39.111 INFO:tasks.workunit.client.0.smithi019.stdout:0/498: truncate d4/d23/d7d/f88 145560 0 2024-02-22T15:50:39.113 INFO:tasks.workunit.client.0.smithi019.stdout:7/876: write d9/d38/d41/d45/f70 [2095730,77141] 0 2024-02-22T15:50:39.114 INFO:tasks.workunit.client.0.smithi019.stdout:0/499: write d4/d23/d7d/f88 [282767,129067] 0 2024-02-22T15:50:39.114 INFO:tasks.workunit.client.0.smithi019.stdout:5/626: read d8/d11/f3e [799888,25182] 0 2024-02-22T15:50:39.119 INFO:tasks.workunit.client.0.smithi019.stdout:0/500: write d4/db/d1b/d40/f5d [1077832,85939] 0 2024-02-22T15:50:39.122 INFO:tasks.workunit.client.0.smithi019.stdout:0/501: fsync d4/d23/d7d/f88 0 2024-02-22T15:50:39.123 INFO:tasks.workunit.client.0.smithi019.stdout:7/877: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108/f124 x:0 0 0 2024-02-22T15:50:39.124 INFO:tasks.workunit.client.0.smithi019.stdout:7/878: write d9/d38/d41/d45/d92/fd4 [944009,34054] 0 2024-02-22T15:50:39.126 INFO:tasks.workunit.client.0.smithi019.stdout:0/502: getdents d4/db/d1b/d40/d8c 0 2024-02-22T15:50:39.126 INFO:tasks.workunit.client.0.smithi019.stdout:7/879: write d9/d4c/f68 [779619,28390] 0 2024-02-22T15:50:39.128 INFO:tasks.workunit.client.0.smithi019.stdout:6/732: write d6/d4b/d6d/d7b/f62 [156745,104058] 0 2024-02-22T15:50:39.130 INFO:tasks.workunit.client.0.smithi019.stdout:7/880: creat d9/d38/d41/f125 x:0 0 0 2024-02-22T15:50:39.132 INFO:tasks.workunit.client.0.smithi019.stdout:6/733: symlink d6/d4b/d71/d88/lea 0 2024-02-22T15:50:39.133 INFO:tasks.workunit.client.0.smithi019.stdout:7/881: write d9/ff1 [306006,84328] 0 2024-02-22T15:50:39.135 INFO:tasks.workunit.client.0.smithi019.stdout:6/734: chown d6/d4b/d71/dc2/ccc 6 1 2024-02-22T15:50:39.137 INFO:tasks.workunit.client.0.smithi019.stdout:7/882: creat d9/d26/d77/f126 x:0 0 0 2024-02-22T15:50:39.138 INFO:tasks.workunit.client.0.smithi019.stdout:0/503: dwrite d4/db/d1b/d40/f89 [0,4194304] 0 2024-02-22T15:50:39.139 INFO:tasks.workunit.client.0.smithi019.stdout:0/504: fsync d4/db/d1b/d40/f5d 0 2024-02-22T15:50:39.140 INFO:tasks.workunit.client.0.smithi019.stdout:6/735: truncate d6/d4b/da9/d67/d8a/fb6 257343 0 2024-02-22T15:50:39.143 INFO:tasks.workunit.client.0.smithi019.stdout:6/736: symlink d6/d4b/d8d/leb 0 2024-02-22T15:50:39.146 INFO:tasks.workunit.client.0.smithi019.stdout:0/505: dread d4/f56 [0,4194304] 0 2024-02-22T15:50:39.147 INFO:tasks.workunit.client.0.smithi019.stdout:6/737: creat d6/d31/d69/d8e/dcf/fec x:0 0 0 2024-02-22T15:50:39.147 INFO:tasks.workunit.client.0.smithi019.stdout:5/627: dread d8/dd/f75 [0,4194304] 0 2024-02-22T15:50:39.149 INFO:tasks.workunit.client.0.smithi019.stdout:0/506: write d4/d5/dc/d41/f90 [907734,78051] 0 2024-02-22T15:50:39.149 INFO:tasks.workunit.client.0.smithi019.stdout:6/738: truncate d6/d4b/fe6 885076 0 2024-02-22T15:50:39.149 INFO:tasks.workunit.client.0.smithi019.stdout:5/628: write d8/d11/f99 [488209,122768] 0 2024-02-22T15:50:39.151 INFO:tasks.workunit.client.0.smithi019.stdout:6/739: write d6/d3e/fb7 [831332,37130] 0 2024-02-22T15:50:39.156 INFO:tasks.workunit.client.0.smithi019.stdout:0/507: rename d4/d22/d78 to d4/db/d24/d80/daf 0 2024-02-22T15:50:39.156 INFO:tasks.workunit.client.0.smithi019.stdout:6/740: mkdir d6/d4b/d71/d88/ded 0 2024-02-22T15:50:39.157 INFO:tasks.workunit.client.0.smithi019.stdout:0/508: readlink d4/db/d1b/d40/d44/d75/l9f 0 2024-02-22T15:50:39.160 INFO:tasks.workunit.client.0.smithi019.stdout:0/509: rename d4/db/d7f to d4/db/d1b/d63/db0 0 2024-02-22T15:50:39.161 INFO:tasks.workunit.client.0.smithi019.stdout:7/883: write d9/d38/d41/d45/db0/df7/dfb/fb2 [1675087,79658] 0 2024-02-22T15:50:39.162 INFO:tasks.workunit.client.0.smithi019.stdout:0/510: creat d4/db/d49/fb1 x:0 0 0 2024-02-22T15:50:39.164 INFO:tasks.workunit.client.0.smithi019.stdout:0/511: write d4/d22/d2c/d68/fa6 [901631,18151] 0 2024-02-22T15:50:39.164 INFO:tasks.workunit.client.0.smithi019.stdout:7/884: unlink d9/d26/c35 0 2024-02-22T15:50:39.165 INFO:tasks.workunit.client.0.smithi019.stdout:6/741: rmdir d6/d31/d69/d8e/dcf 39 2024-02-22T15:50:39.165 INFO:tasks.workunit.client.0.smithi019.stdout:0/512: chown d4/d5/d7/l54 12 1 2024-02-22T15:50:39.166 INFO:tasks.workunit.client.0.smithi019.stdout:5/629: write d8/d11/d33/fcb [3763395,13338] 0 2024-02-22T15:50:39.167 INFO:tasks.workunit.client.0.smithi019.stdout:7/885: read d9/ff1 [3352538,80834] 0 2024-02-22T15:50:39.168 INFO:tasks.workunit.client.0.smithi019.stdout:0/513: write d4/db/f43 [4280817,15046] 0 2024-02-22T15:50:39.169 INFO:tasks.workunit.client.0.smithi019.stdout:7/886: chown d9/d26/dc4/fc7 32227446 1 2024-02-22T15:50:39.171 INFO:tasks.workunit.client.0.smithi019.stdout:0/514: fsync d4/d23/f4a 0 2024-02-22T15:50:39.174 INFO:tasks.workunit.client.0.smithi019.stdout:5/630: rename d8/d11/d33/l44 to d8/dd/d19/d77/d7a/lcf 0 2024-02-22T15:50:39.175 INFO:tasks.workunit.client.0.smithi019.stdout:6/742: dwrite d6/f44 [0,4194304] 0 2024-02-22T15:50:39.176 INFO:tasks.workunit.client.0.smithi019.stdout:5/631: write d8/f9b [945053,67718] 0 2024-02-22T15:50:39.177 INFO:tasks.workunit.client.0.smithi019.stdout:0/515: mkdir d4/d9/da7/db2 0 2024-02-22T15:50:39.178 INFO:tasks.workunit.client.0.smithi019.stdout:5/632: dread - d8/dd/d19/d3a/d46/d7e/f8c zero size 2024-02-22T15:50:39.179 INFO:tasks.workunit.client.0.smithi019.stdout:0/516: rmdir d4/db/d24 39 2024-02-22T15:50:39.183 INFO:tasks.workunit.client.0.smithi019.stdout:5/633: rmdir d8/dd/d19/d77/d7a 39 2024-02-22T15:50:39.188 INFO:tasks.workunit.client.1.smithi062.stdout:2/458: sync 2024-02-22T15:50:39.188 INFO:tasks.workunit.client.1.smithi062.stdout:3/363: sync 2024-02-22T15:50:39.188 INFO:tasks.workunit.client.1.smithi062.stdout:7/331: sync 2024-02-22T15:50:39.189 INFO:tasks.workunit.client.1.smithi062.stdout:2/459: readlink d1/d8/d62/d73/l77 0 2024-02-22T15:50:39.189 INFO:tasks.workunit.client.1.smithi062.stdout:3/364: stat d5/c14 0 2024-02-22T15:50:39.189 INFO:tasks.workunit.client.0.smithi019.stdout:0/517: dwrite d4/d22/d2c/d68/fa6 [0,4194304] 0 2024-02-22T15:50:39.190 INFO:tasks.workunit.client.1.smithi062.stdout:3/365: write d5/d1a/f25 [3456486,98323] 0 2024-02-22T15:50:39.192 INFO:tasks.workunit.client.1.smithi062.stdout:2/460: mknod d1/d8/c98 0 2024-02-22T15:50:39.193 INFO:tasks.workunit.client.0.smithi019.stdout:7/887: truncate d9/d38/d41/d45/d61/d6a/f93 2232281 0 2024-02-22T15:50:39.193 INFO:tasks.workunit.client.1.smithi062.stdout:7/332: read d0/d6/d2e/d4a/f1f [2481624,58057] 0 2024-02-22T15:50:39.194 INFO:tasks.workunit.client.0.smithi019.stdout:5/634: creat d8/dd/d19/d77/fd0 x:0 0 0 2024-02-22T15:50:39.194 INFO:tasks.workunit.client.1.smithi062.stdout:3/366: mknod d5/d16/d21/c6d 0 2024-02-22T15:50:39.195 INFO:tasks.workunit.client.0.smithi019.stdout:5/635: dread - d8/d11/fb8 zero size 2024-02-22T15:50:39.195 INFO:tasks.workunit.client.0.smithi019.stdout:0/518: rmdir d4/db/d1b/d40 39 2024-02-22T15:50:39.195 INFO:tasks.workunit.client.1.smithi062.stdout:2/461: creat d1/d1e/d38/d76/f99 x:0 0 0 2024-02-22T15:50:39.196 INFO:tasks.workunit.client.1.smithi062.stdout:2/462: dread - d1/d47/f72 zero size 2024-02-22T15:50:39.196 INFO:tasks.workunit.client.1.smithi062.stdout:7/333: truncate d0/d14/d13/f48 102852 0 2024-02-22T15:50:39.197 INFO:tasks.workunit.client.1.smithi062.stdout:7/334: fsync d0/d14/f67 0 2024-02-22T15:50:39.198 INFO:tasks.workunit.client.0.smithi019.stdout:5/636: truncate d8/dd/d19/d38/d54/fb2 581542 0 2024-02-22T15:50:39.199 INFO:tasks.workunit.client.0.smithi019.stdout:7/888: unlink d9/d4c/d5b/lbc 0 2024-02-22T15:50:39.200 INFO:tasks.workunit.client.1.smithi062.stdout:2/463: creat d1/d1e/f9a x:0 0 0 2024-02-22T15:50:39.201 INFO:tasks.workunit.client.0.smithi019.stdout:0/519: mknod d4/db/d49/cb3 0 2024-02-22T15:50:39.204 INFO:tasks.workunit.client.1.smithi062.stdout:3/367: dwrite d5/d16/d21/f2b [4194304,4194304] 0 2024-02-22T15:50:39.205 INFO:tasks.workunit.client.0.smithi019.stdout:6/743: dwrite d6/f17 [0,4194304] 0 2024-02-22T15:50:39.207 INFO:tasks.workunit.client.1.smithi062.stdout:7/335: link d0/d6/d2e/d4a/f4c d0/d14/d19/d30/d37/f6b 0 2024-02-22T15:50:39.208 INFO:tasks.workunit.client.0.smithi019.stdout:5/637: rename f1 to d8/dd/d19/d3a/d46/d7e/fd1 0 2024-02-22T15:50:39.208 INFO:tasks.workunit.client.0.smithi019.stdout:0/520: mknod d4/d22/cb4 0 2024-02-22T15:50:39.212 INFO:tasks.workunit.client.0.smithi019.stdout:7/889: symlink d9/d38/d41/d45/l127 0 2024-02-22T15:50:39.213 INFO:tasks.workunit.client.0.smithi019.stdout:0/521: truncate d4/d22/f51 2117827 0 2024-02-22T15:50:39.220 INFO:tasks.workunit.client.0.smithi019.stdout:7/890: symlink d9/d26/d77/l128 0 2024-02-22T15:50:39.220 INFO:tasks.workunit.client.1.smithi062.stdout:2/464: rename d1/d28/f35 to d1/d47/d32/d52/d80/f9b 0 2024-02-22T15:50:39.221 INFO:tasks.workunit.client.0.smithi019.stdout:0/522: mknod d4/db/d49/d65/cb5 0 2024-02-22T15:50:39.221 INFO:tasks.workunit.client.0.smithi019.stdout:3/477: sync 2024-02-22T15:50:39.221 INFO:tasks.workunit.client.0.smithi019.stdout:1/635: sync 2024-02-22T15:50:39.224 INFO:tasks.workunit.client.0.smithi019.stdout:6/744: dwrite d6/d58/d6a/fc7 [0,4194304] 0 2024-02-22T15:50:39.225 INFO:tasks.workunit.client.0.smithi019.stdout:7/891: write d9/d38/d41/d45/d92/f11e [806407,24343] 0 2024-02-22T15:50:39.226 INFO:tasks.workunit.client.0.smithi019.stdout:1/636: chown d8/d18/d64/d7f/d94 943196384 1 2024-02-22T15:50:39.226 INFO:tasks.workunit.client.0.smithi019.stdout:8/577: sync 2024-02-22T15:50:39.226 INFO:tasks.workunit.client.0.smithi019.stdout:9/587: sync 2024-02-22T15:50:39.226 INFO:tasks.workunit.client.0.smithi019.stdout:4/584: sync 2024-02-22T15:50:39.226 INFO:tasks.workunit.client.0.smithi019.stdout:2/556: sync 2024-02-22T15:50:39.227 INFO:tasks.workunit.client.1.smithi062.stdout:3/368: dread d5/d1a/f25 [0,4194304] 0 2024-02-22T15:50:39.230 INFO:tasks.workunit.client.0.smithi019.stdout:0/523: symlink d4/db/d1b/lb6 0 2024-02-22T15:50:39.230 INFO:tasks.workunit.client.0.smithi019.stdout:8/578: fsync d9/fa 0 2024-02-22T15:50:39.231 INFO:tasks.workunit.client.0.smithi019.stdout:5/638: dwrite d8/dd/d19/d38/d54/db0/fbd [0,4194304] 0 2024-02-22T15:50:39.234 INFO:tasks.workunit.client.0.smithi019.stdout:9/588: write d2/fb [8978781,110784] 0 2024-02-22T15:50:39.238 INFO:tasks.workunit.client.0.smithi019.stdout:6/745: mknod d6/d4b/d6d/da3/cee 0 2024-02-22T15:50:39.239 INFO:tasks.workunit.client.0.smithi019.stdout:6/746: truncate d6/d4b/d8d/f96 4393000 0 2024-02-22T15:50:39.241 INFO:tasks.workunit.client.0.smithi019.stdout:1/637: creat d8/d18/fda x:0 0 0 2024-02-22T15:50:39.244 INFO:tasks.workunit.client.1.smithi062.stdout:7/336: creat d0/d6/d2e/d4a/d5a/d5e/f6c x:0 0 0 2024-02-22T15:50:39.248 INFO:tasks.workunit.client.1.smithi062.stdout:7/337: getdents d0/d14/d19/d30/d45/d6a 0 2024-02-22T15:50:39.281 INFO:tasks.workunit.client.1.smithi062.stdout:7/338: write d0/f24 [4672655,46728] 0 2024-02-22T15:50:39.287 INFO:tasks.workunit.client.1.smithi062.stdout:7/339: dwrite d0/d6/d2e/d2f/f3a [0,4194304] 0 2024-02-22T15:50:39.289 INFO:tasks.workunit.client.1.smithi062.stdout:7/340: write d0/d14/d19/f1a [928682,125980] 0 2024-02-22T15:50:39.292 INFO:tasks.workunit.client.1.smithi062.stdout:7/341: dread d0/d14/d19/d30/d37/f63 [0,4194304] 0 2024-02-22T15:50:39.294 INFO:tasks.workunit.client.1.smithi062.stdout:7/342: symlink d0/d14/l6d 0 2024-02-22T15:50:39.295 INFO:tasks.workunit.client.1.smithi062.stdout:7/343: stat d0/d14/d19/d30/d37/l39 0 2024-02-22T15:50:39.301 INFO:tasks.workunit.client.1.smithi062.stdout:5/260: sync 2024-02-22T15:50:39.301 INFO:tasks.workunit.client.1.smithi062.stdout:1/407: sync 2024-02-22T15:50:39.301 INFO:tasks.workunit.client.1.smithi062.stdout:4/395: sync 2024-02-22T15:50:39.301 INFO:tasks.workunit.client.1.smithi062.stdout:7/344: dwrite d0/d14/d19/d30/d37/f63 [0,4194304] 0 2024-02-22T15:50:39.304 INFO:tasks.workunit.client.1.smithi062.stdout:4/396: dread d4/f51 [0,4194304] 0 2024-02-22T15:50:39.306 INFO:tasks.workunit.client.1.smithi062.stdout:1/408: creat d1/d6/d7/d2b/d49/d74/f85 x:0 0 0 2024-02-22T15:50:39.307 INFO:tasks.workunit.client.1.smithi062.stdout:1/409: readlink d1/d6/d36/l70 0 2024-02-22T15:50:39.308 INFO:tasks.workunit.client.1.smithi062.stdout:4/397: mknod d4/d1c/c88 0 2024-02-22T15:50:39.308 INFO:tasks.workunit.client.1.smithi062.stdout:1/410: chown d1/d6/d10/f4b 641449 1 2024-02-22T15:50:39.309 INFO:tasks.workunit.client.1.smithi062.stdout:4/398: chown d4/d2a/d75 1541008 1 2024-02-22T15:50:39.312 INFO:tasks.workunit.client.1.smithi062.stdout:4/399: mknod d4/d1c/d30/c89 0 2024-02-22T15:50:39.313 INFO:tasks.workunit.client.1.smithi062.stdout:5/261: dwrite d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:39.316 INFO:tasks.workunit.client.1.smithi062.stdout:4/400: mkdir d4/d2a/d2d/d2e/d6a/d8a 0 2024-02-22T15:50:39.319 INFO:tasks.workunit.client.1.smithi062.stdout:4/401: creat d4/d2a/f8b x:0 0 0 2024-02-22T15:50:39.322 INFO:tasks.workunit.client.1.smithi062.stdout:5/262: dwrite d0/d3/da/d13/d23/f24 [0,4194304] 0 2024-02-22T15:50:39.323 INFO:tasks.workunit.client.1.smithi062.stdout:4/402: mknod d4/d2a/d2d/c8c 0 2024-02-22T15:50:39.324 INFO:tasks.workunit.client.1.smithi062.stdout:5/263: dread - d0/d3/da/d13/d23/f49 zero size 2024-02-22T15:50:39.324 INFO:tasks.workunit.client.1.smithi062.stdout:4/403: stat d4/d1c/d2c/d6b 0 2024-02-22T15:50:39.325 INFO:tasks.workunit.client.0.smithi019.stdout:7/892: rename d9/d38/d41/lec to d9/d38/d41/d45/db0/df7/dfb/l129 0 2024-02-22T15:50:39.326 INFO:tasks.workunit.client.1.smithi062.stdout:5/264: chown d0/d3/da/d32 10874977 1 2024-02-22T15:50:39.326 INFO:tasks.workunit.client.1.smithi062.stdout:4/404: fdatasync d4/d1c/d30/f65 0 2024-02-22T15:50:39.326 INFO:tasks.workunit.client.0.smithi019.stdout:9/589: creat d2/d1f/d2e/d48/d45/d5a/fd0 x:0 0 0 2024-02-22T15:50:39.327 INFO:tasks.workunit.client.0.smithi019.stdout:8/579: mknod d9/d14/d6f/d7e/d94/d9c/cc2 0 2024-02-22T15:50:39.328 INFO:tasks.workunit.client.1.smithi062.stdout:4/405: fsync d4/d1c/d2c/f56 0 2024-02-22T15:50:39.331 INFO:tasks.workunit.client.0.smithi019.stdout:1/638: mkdir d8/d2a/d75/d81/ddb 0 2024-02-22T15:50:39.332 INFO:tasks.workunit.client.0.smithi019.stdout:2/557: mkdir d1/d17/d1c/d66/db9/dd0 0 2024-02-22T15:50:39.334 INFO:tasks.workunit.client.1.smithi062.stdout:5/265: dwrite d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:39.334 INFO:tasks.workunit.client.0.smithi019.stdout:1/639: truncate d8/f88 1650994 0 2024-02-22T15:50:39.335 INFO:tasks.workunit.client.0.smithi019.stdout:6/747: rename d6/d31/d69/d8e/fab to d6/d31/d69/d8e/dcf/fef 0 2024-02-22T15:50:39.335 INFO:tasks.workunit.client.0.smithi019.stdout:9/590: readlink d2/d7/dc/d27/dbc/lbe 0 2024-02-22T15:50:39.337 INFO:tasks.workunit.client.0.smithi019.stdout:8/580: fdatasync d9/d14/f36 0 2024-02-22T15:50:39.340 INFO:tasks.workunit.client.1.smithi062.stdout:1/411: dread d1/d6/d14/f45 [0,4194304] 0 2024-02-22T15:50:39.340 INFO:tasks.workunit.client.1.smithi062.stdout:4/406: dwrite d4/d1c/d2c/f87 [0,4194304] 0 2024-02-22T15:50:39.341 INFO:tasks.workunit.client.0.smithi019.stdout:4/585: mkdir d2/db/d14/d23/d21/d35/dc0 0 2024-02-22T15:50:39.343 INFO:tasks.workunit.client.0.smithi019.stdout:0/524: rename d4/d5/dc/d41 to d4/db/d1b/d40/d44/d75/db7 0 2024-02-22T15:50:39.346 INFO:tasks.workunit.client.1.smithi062.stdout:1/412: fsync d1/d6/f61 0 2024-02-22T15:50:39.346 INFO:tasks.workunit.client.1.smithi062.stdout:4/407: write d4/f11 [4668979,117802] 0 2024-02-22T15:50:39.348 INFO:tasks.workunit.client.1.smithi062.stdout:5/266: dwrite d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:39.350 INFO:tasks.workunit.client.1.smithi062.stdout:1/413: creat d1/d6/d36/d37/f86 x:0 0 0 2024-02-22T15:50:39.351 INFO:tasks.workunit.client.1.smithi062.stdout:4/408: rmdir d4/d1c/d2c 39 2024-02-22T15:50:39.351 INFO:tasks.workunit.client.0.smithi019.stdout:9/591: dread d2/d7/dc/f29 [0,4194304] 0 2024-02-22T15:50:39.351 INFO:tasks.workunit.client.0.smithi019.stdout:2/558: mkdir d1/d17/d21/d71/dd1 0 2024-02-22T15:50:39.352 INFO:tasks.workunit.client.0.smithi019.stdout:3/478: dwrite d9/d11/f41 [0,4194304] 0 2024-02-22T15:50:39.353 INFO:tasks.workunit.client.1.smithi062.stdout:5/267: mkdir d0/d3/da/d32/d4f 0 2024-02-22T15:50:39.356 INFO:tasks.workunit.client.1.smithi062.stdout:4/409: symlink d4/d2a/d2d/d2e/d6a/d8a/l8d 0 2024-02-22T15:50:39.359 INFO:tasks.workunit.client.1.smithi062.stdout:1/414: stat d1/d6/d7/c53 0 2024-02-22T15:50:39.361 INFO:tasks.workunit.client.0.smithi019.stdout:0/525: creat d4/d9/da7/fb8 x:0 0 0 2024-02-22T15:50:39.364 INFO:tasks.workunit.client.0.smithi019.stdout:8/581: creat d9/fc3 x:0 0 0 2024-02-22T15:50:39.366 INFO:tasks.workunit.client.0.smithi019.stdout:1/640: dwrite d8/d18/d64/f67 [0,4194304] 0 2024-02-22T15:50:39.367 INFO:tasks.workunit.client.1.smithi062.stdout:5/268: dwrite d0/d3/d30/f46 [0,4194304] 0 2024-02-22T15:50:39.369 INFO:tasks.workunit.client.0.smithi019.stdout:0/526: fdatasync d4/d22/d3d/f4f 0 2024-02-22T15:50:39.370 INFO:tasks.workunit.client.0.smithi019.stdout:3/479: mkdir d9/d22/d27/d96 0 2024-02-22T15:50:39.371 INFO:tasks.workunit.client.0.smithi019.stdout:6/748: dwrite d6/f1d [4194304,4194304] 0 2024-02-22T15:50:39.371 INFO:tasks.workunit.client.0.smithi019.stdout:9/592: mknod d2/d7/dc/d27/d68/dca/cd1 0 2024-02-22T15:50:39.372 INFO:tasks.workunit.client.0.smithi019.stdout:7/893: dread d9/d3a/f6b [4194304,4194304] 0 2024-02-22T15:50:39.375 INFO:tasks.workunit.client.1.smithi062.stdout:4/410: dwrite d4/d1c/d2c/d6b/f7e [0,4194304] 0 2024-02-22T15:50:39.375 INFO:tasks.workunit.client.1.smithi062.stdout:5/269: getdents d0/d3/d30 0 2024-02-22T15:50:39.375 INFO:tasks.workunit.client.0.smithi019.stdout:1/641: fsync d8/d18/d64/f66 0 2024-02-22T15:50:39.377 INFO:tasks.workunit.client.0.smithi019.stdout:8/582: unlink d9/d88/f90 0 2024-02-22T15:50:39.377 INFO:tasks.workunit.client.0.smithi019.stdout:7/894: fsync d9/d26/f101 0 2024-02-22T15:50:39.383 INFO:tasks.workunit.client.1.smithi062.stdout:5/270: creat d0/d3/da/d32/d4f/f50 x:0 0 0 2024-02-22T15:50:39.385 INFO:tasks.workunit.client.0.smithi019.stdout:7/895: readlink d9/d38/d41/d45/db0/df7/dfb/l9c 0 2024-02-22T15:50:39.385 INFO:tasks.workunit.client.0.smithi019.stdout:8/583: write d9/f64 [720020,72930] 0 2024-02-22T15:50:39.389 INFO:tasks.workunit.client.1.smithi062.stdout:4/411: link d4/d1c/d30/d3b/d49/l61 d4/d2a/d2d/d46/l8e 0 2024-02-22T15:50:39.389 INFO:tasks.workunit.client.1.smithi062.stdout:5/271: truncate d0/d3/da/d13/d23/f3e 650398 0 2024-02-22T15:50:39.389 INFO:tasks.workunit.client.1.smithi062.stdout:5/272: fsync d0/f26 0 2024-02-22T15:50:39.390 INFO:tasks.workunit.client.0.smithi019.stdout:2/559: dwrite d1/de/f16 [4194304,4194304] 0 2024-02-22T15:50:39.391 INFO:tasks.workunit.client.1.smithi062.stdout:4/412: chown d4/d13/l22 0 1 2024-02-22T15:50:39.391 INFO:tasks.workunit.client.0.smithi019.stdout:0/527: rename d4/db/d49/c60 to d4/db/d24/cb9 0 2024-02-22T15:50:39.393 INFO:tasks.workunit.client.1.smithi062.stdout:5/273: write d0/f39 [457821,126413] 0 2024-02-22T15:50:39.393 INFO:tasks.workunit.client.1.smithi062.stdout:5/274: fdatasync d0/d3/da/f27 0 2024-02-22T15:50:39.395 INFO:tasks.workunit.client.1.smithi062.stdout:4/413: mkdir d4/d8f 0 2024-02-22T15:50:39.398 INFO:tasks.workunit.client.0.smithi019.stdout:3/480: getdents d9/d22/d49 0 2024-02-22T15:50:39.398 INFO:tasks.workunit.client.0.smithi019.stdout:1/642: dwrite d8/fb4 [0,4194304] 0 2024-02-22T15:50:39.398 INFO:tasks.workunit.client.0.smithi019.stdout:4/586: write d2/db/d14/d23/d4b/f86 [1976449,96219] 0 2024-02-22T15:50:39.400 INFO:tasks.workunit.client.0.smithi019.stdout:7/896: rmdir d9/d38/d41/d45/d92/d11a 0 2024-02-22T15:50:39.401 INFO:tasks.workunit.client.0.smithi019.stdout:8/584: symlink d9/d14/d17/d5d/lc4 0 2024-02-22T15:50:39.401 INFO:tasks.workunit.client.1.smithi062.stdout:5/275: dwrite d0/d3/da/fb [4194304,4194304] 0 2024-02-22T15:50:39.402 INFO:tasks.workunit.client.1.smithi062.stdout:4/414: rename d4/df/c74 to d4/d66/c90 0 2024-02-22T15:50:39.409 INFO:tasks.workunit.client.1.smithi062.stdout:4/415: rename d4/d2a/d2d/d46/l8e to d4/d13/l91 0 2024-02-22T15:50:39.410 INFO:tasks.workunit.client.0.smithi019.stdout:6/749: getdents d6/d4b/d71 0 2024-02-22T15:50:39.411 INFO:tasks.workunit.client.1.smithi062.stdout:5/276: link d0/d3/f14 d0/d3/d2f/d38/d3f/f51 0 2024-02-22T15:50:39.413 INFO:tasks.workunit.client.1.smithi062.stdout:4/416: mkdir d4/d2a/d2d/d2e/d6a/d8a/d92 0 2024-02-22T15:50:39.413 INFO:tasks.workunit.client.0.smithi019.stdout:3/481: dread - d9/d22/d27/d6d/d72/f81 zero size 2024-02-22T15:50:39.413 INFO:tasks.workunit.client.1.smithi062.stdout:5/277: truncate d0/d3/d2f/f44 933960 0 2024-02-22T15:50:39.414 INFO:tasks.workunit.client.1.smithi062.stdout:5/278: fdatasync d0/f26 0 2024-02-22T15:50:39.416 INFO:tasks.workunit.client.0.smithi019.stdout:0/528: truncate d4/d23/d7d/f88 433330 0 2024-02-22T15:50:39.419 INFO:tasks.workunit.client.1.smithi062.stdout:5/279: rename d0/d3/da/d13/d23/d31/c36 to d0/c52 0 2024-02-22T15:50:39.422 INFO:tasks.workunit.client.1.smithi062.stdout:5/280: creat d0/d3/da/f53 x:0 0 0 2024-02-22T15:50:39.423 INFO:tasks.workunit.client.0.smithi019.stdout:7/897: dwrite d9/f34 [4194304,4194304] 0 2024-02-22T15:50:39.427 INFO:tasks.workunit.client.1.smithi062.stdout:5/281: mknod d0/d3/d2f/d38/d3f/c54 0 2024-02-22T15:50:39.428 INFO:tasks.workunit.client.0.smithi019.stdout:9/593: dwrite d2/d7/dc/d27/dbc/f3f [0,4194304] 0 2024-02-22T15:50:39.435 INFO:tasks.workunit.client.0.smithi019.stdout:2/560: write d1/d17/fae [502187,64291] 0 2024-02-22T15:50:39.436 INFO:tasks.workunit.client.0.smithi019.stdout:4/587: symlink d2/d3f/d6e/lc1 0 2024-02-22T15:50:39.437 INFO:tasks.workunit.client.0.smithi019.stdout:8/585: creat d9/d14/d1f/d6d/fc5 x:0 0 0 2024-02-22T15:50:39.438 INFO:tasks.workunit.client.0.smithi019.stdout:8/586: dread - d9/d14/d17/d3d/d3f/d7d/f98 zero size 2024-02-22T15:50:39.442 INFO:tasks.workunit.client.0.smithi019.stdout:6/750: creat d6/d4b/d6d/da3/ff0 x:0 0 0 2024-02-22T15:50:39.443 INFO:tasks.workunit.client.1.smithi062.stdout:4/417: dread d4/df/f14 [4194304,4194304] 0 2024-02-22T15:50:39.446 INFO:tasks.workunit.client.0.smithi019.stdout:0/529: creat d4/d5/fba x:0 0 0 2024-02-22T15:50:39.446 INFO:tasks.workunit.client.0.smithi019.stdout:2/561: creat d1/db4/fd2 x:0 0 0 2024-02-22T15:50:39.446 INFO:tasks.workunit.client.1.smithi062.stdout:4/418: creat d4/d1c/d2c/d6b/d70/f93 x:0 0 0 2024-02-22T15:50:39.447 INFO:tasks.workunit.client.1.smithi062.stdout:4/419: chown d4/d1c/f27 10 1 2024-02-22T15:50:39.449 INFO:tasks.workunit.client.0.smithi019.stdout:9/594: link d2/d7/dc/d27/fb4 d2/d1f/d2e/d57/d60/dc5/fd2 0 2024-02-22T15:50:39.449 INFO:tasks.workunit.client.1.smithi062.stdout:4/420: mkdir d4/d2a/d2d/d2e/d6a/d94 0 2024-02-22T15:50:39.450 INFO:tasks.workunit.client.0.smithi019.stdout:4/588: write d2/db/d14/d23/d21/d35/f64 [351782,51473] 0 2024-02-22T15:50:39.452 INFO:tasks.workunit.client.1.smithi062.stdout:4/421: creat d4/d1c/d30/d3b/d49/f95 x:0 0 0 2024-02-22T15:50:39.456 INFO:tasks.workunit.client.0.smithi019.stdout:6/751: getdents d6/d4b/d71/dc2 0 2024-02-22T15:50:39.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:39 smithi062 ceph-mon[48130]: pgmap v35: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 33 MiB/s rd, 179 MiB/s wr, 304 op/s 2024-02-22T15:50:39.458 INFO:tasks.workunit.client.1.smithi062.stdout:4/422: link d4/d34/l73 d4/d34/l96 0 2024-02-22T15:50:39.460 INFO:tasks.workunit.client.0.smithi019.stdout:1/643: dwrite d8/d18/d85/f98 [0,4194304] 0 2024-02-22T15:50:39.462 INFO:tasks.workunit.client.0.smithi019.stdout:9/595: write d2/d7/dc/d27/fb4 [1008183,40456] 0 2024-02-22T15:50:39.463 INFO:tasks.workunit.client.0.smithi019.stdout:2/562: rename d1/de/d15/c99 to d1/d17/d67/cd3 0 2024-02-22T15:50:39.465 INFO:tasks.workunit.client.0.smithi019.stdout:4/589: chown d2/db/d14/d23/d21/d35/d3d 25 1 2024-02-22T15:50:39.465 INFO:tasks.workunit.client.0.smithi019.stdout:7/898: dwrite d9/d4c/f66 [0,4194304] 0 2024-02-22T15:50:39.465 INFO:tasks.workunit.client.0.smithi019.stdout:6/752: mkdir d6/d4b/d8d/df1 0 2024-02-22T15:50:39.466 INFO:tasks.workunit.client.1.smithi062.stdout:4/423: truncate d4/d1c/d2c/d6b/f7e 707070 0 2024-02-22T15:50:39.466 INFO:tasks.workunit.client.0.smithi019.stdout:1/644: write d8/d18/f21 [5576647,15266] 0 2024-02-22T15:50:39.468 INFO:tasks.workunit.client.1.smithi062.stdout:5/282: dwrite d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:39.469 INFO:tasks.workunit.client.1.smithi062.stdout:4/424: creat d4/d2a/d2d/d2e/f97 x:0 0 0 2024-02-22T15:50:39.469 INFO:tasks.workunit.client.0.smithi019.stdout:2/563: symlink d1/d17/d1c/d7f/ld4 0 2024-02-22T15:50:39.469 INFO:tasks.workunit.client.0.smithi019.stdout:6/753: dread - d6/d4b/d6d/da3/ddb/fe8 zero size 2024-02-22T15:50:39.470 INFO:tasks.workunit.client.1.smithi062.stdout:5/283: mknod d0/c55 0 2024-02-22T15:50:39.470 INFO:tasks.workunit.client.0.smithi019.stdout:9/596: write d2/d1f/d2e/d48/d45/d5a/fa0 [1185894,82259] 0 2024-02-22T15:50:39.472 INFO:tasks.workunit.client.1.smithi062.stdout:4/425: getdents d4/d2a/d2d/d2e/d6a/d94 0 2024-02-22T15:50:39.476 INFO:tasks.workunit.client.1.smithi062.stdout:5/284: mkdir d0/d3/d2f/d38/d3f/d56 0 2024-02-22T15:50:39.480 INFO:tasks.workunit.client.1.smithi062.stdout:5/285: write d0/f26 [651265,19108] 0 2024-02-22T15:50:39.480 INFO:tasks.workunit.client.0.smithi019.stdout:2/564: dwrite d1/f14 [0,4194304] 0 2024-02-22T15:50:39.482 INFO:tasks.workunit.client.1.smithi062.stdout:5/286: mkdir d0/d3/da/d13/d57 0 2024-02-22T15:50:39.482 INFO:tasks.workunit.client.1.smithi062.stdout:8/373: sync 2024-02-22T15:50:39.482 INFO:tasks.workunit.client.1.smithi062.stdout:4/426: link d4/d1c/d2c/d6b/c6c d4/d1c/d30/c98 0 2024-02-22T15:50:39.485 INFO:tasks.workunit.client.0.smithi019.stdout:1/645: mknod d8/d15/cdc 0 2024-02-22T15:50:39.485 INFO:tasks.workunit.client.0.smithi019.stdout:1/646: read - d8/d15/d5b/fb2 zero size 2024-02-22T15:50:39.485 INFO:tasks.workunit.client.0.smithi019.stdout:4/590: mkdir d2/db/d14/d23/d9e/dac/dc2 0 2024-02-22T15:50:39.486 INFO:tasks.workunit.client.1.smithi062.stdout:5/287: symlink d0/d3/d2f/d38/l58 0 2024-02-22T15:50:39.487 INFO:tasks.workunit.client.1.smithi062.stdout:8/374: truncate d8/f29 3302286 0 2024-02-22T15:50:39.488 INFO:tasks.workunit.client.0.smithi019.stdout:6/754: chown d6/d4b/da9/d67/c7c 3953730 1 2024-02-22T15:50:39.489 INFO:tasks.workunit.client.0.smithi019.stdout:9/597: creat d2/d1f/d2e/d48/d45/fd3 x:0 0 0 2024-02-22T15:50:39.489 INFO:tasks.workunit.client.0.smithi019.stdout:2/565: rmdir d1/d17/d67 39 2024-02-22T15:50:39.490 INFO:tasks.workunit.client.0.smithi019.stdout:1/647: symlink d8/d2a/d97/ldd 0 2024-02-22T15:50:39.492 INFO:tasks.workunit.client.1.smithi062.stdout:8/375: rename d8/d9/d2c/d49/l61 to d8/l6e 0 2024-02-22T15:50:39.493 INFO:tasks.workunit.client.0.smithi019.stdout:7/899: getdents d9/d38/d41/d45/db0 0 2024-02-22T15:50:39.493 INFO:tasks.workunit.client.0.smithi019.stdout:6/755: dread - d6/d4b/d6d/da3/ddb/fe8 zero size 2024-02-22T15:50:39.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:39 smithi019 ceph-mon[67418]: pgmap v35: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 33 MiB/s rd, 179 MiB/s wr, 304 op/s 2024-02-22T15:50:39.495 INFO:tasks.workunit.client.0.smithi019.stdout:1/648: chown d8/d2a/lca 62 1 2024-02-22T15:50:39.495 INFO:tasks.workunit.client.1.smithi062.stdout:6/497: sync 2024-02-22T15:50:39.495 INFO:tasks.workunit.client.1.smithi062.stdout:9/370: sync 2024-02-22T15:50:39.495 INFO:tasks.workunit.client.1.smithi062.stdout:0/490: sync 2024-02-22T15:50:39.498 INFO:tasks.workunit.client.0.smithi019.stdout:9/598: truncate d2/d1f/d2e/d57/f94 230241 0 2024-02-22T15:50:39.498 INFO:tasks.workunit.client.1.smithi062.stdout:6/498: write d5/df/d40/d44/d4a/d41/d43/f58 [752839,89132] 0 2024-02-22T15:50:39.499 INFO:tasks.workunit.client.1.smithi062.stdout:0/491: creat d1/d21/d85/d8e/d97/fab x:0 0 0 2024-02-22T15:50:39.500 INFO:tasks.workunit.client.0.smithi019.stdout:3/482: truncate d9/d26/d32/f33 190388 0 2024-02-22T15:50:39.500 INFO:tasks.workunit.client.1.smithi062.stdout:6/499: truncate d5/dd/d33/f46 943545 0 2024-02-22T15:50:39.501 INFO:tasks.workunit.client.0.smithi019.stdout:2/566: dread - d1/d46/f98 zero size 2024-02-22T15:50:39.501 INFO:tasks.workunit.client.0.smithi019.stdout:8/587: truncate d9/d14/d1f/d6d/f95 2199604 0 2024-02-22T15:50:39.501 INFO:tasks.workunit.client.0.smithi019.stdout:6/756: fsync d6/d31/d69/d8e/dcf/fec 0 2024-02-22T15:50:39.502 INFO:tasks.workunit.client.0.smithi019.stdout:4/591: dwrite d2/db/d14/d23/fa7 [0,4194304] 0 2024-02-22T15:50:39.502 INFO:tasks.workunit.client.0.smithi019.stdout:5/639: sync 2024-02-22T15:50:39.504 INFO:tasks.workunit.client.0.smithi019.stdout:3/483: write d9/d14/f86 [601122,78832] 0 2024-02-22T15:50:39.505 INFO:tasks.workunit.client.1.smithi062.stdout:4/427: rmdir d4/d2a/d2d/d2e 39 2024-02-22T15:50:39.505 INFO:tasks.workunit.client.0.smithi019.stdout:3/484: chown d9/d26/d3f 362 1 2024-02-22T15:50:39.507 INFO:tasks.workunit.client.0.smithi019.stdout:7/900: truncate d9/d38/f69 1536520 0 2024-02-22T15:50:39.508 INFO:tasks.workunit.client.1.smithi062.stdout:0/492: dwrite d1/d3/de/f90 [0,4194304] 0 2024-02-22T15:50:39.510 INFO:tasks.workunit.client.0.smithi019.stdout:1/649: mkdir d8/d2a/dde 0 2024-02-22T15:50:39.511 INFO:tasks.workunit.client.0.smithi019.stdout:0/530: truncate d4/db/d1b/d40/d44/d75/db7/f90 292271 0 2024-02-22T15:50:39.513 INFO:tasks.workunit.client.1.smithi062.stdout:4/428: dread - d4/d2a/d2d/d2e/f77 zero size 2024-02-22T15:50:39.515 INFO:tasks.workunit.client.0.smithi019.stdout:4/592: write d2/db/d14/d23/d21/f2c [1574821,107453] 0 2024-02-22T15:50:39.515 INFO:tasks.workunit.client.0.smithi019.stdout:1/650: chown d8/d2a/d75/da0/lc6 0 1 2024-02-22T15:50:39.516 INFO:tasks.workunit.client.1.smithi062.stdout:4/429: write f2 [3957231,87133] 0 2024-02-22T15:50:39.518 INFO:tasks.workunit.client.1.smithi062.stdout:5/288: truncate d0/d3/d2f/f44 538771 0 2024-02-22T15:50:39.520 INFO:tasks.workunit.client.0.smithi019.stdout:9/599: rename d2/d1f/d2e/d57/d60/dc5 to d2/d7/dc/d27/d31/dd4 0 2024-02-22T15:50:39.523 INFO:tasks.workunit.client.1.smithi062.stdout:4/430: mknod d4/d1c/d30/d37/c99 0 2024-02-22T15:50:39.523 INFO:tasks.workunit.client.0.smithi019.stdout:0/531: unlink d4/db/d1b/d63/f87 0 2024-02-22T15:50:39.524 INFO:tasks.workunit.client.0.smithi019.stdout:2/567: rmdir d1/d17/d67 39 2024-02-22T15:50:39.526 INFO:tasks.workunit.client.1.smithi062.stdout:0/493: link d1/dd/l89 d1/d3/d4/d1a/d76/d8c/d2c/lac 0 2024-02-22T15:50:39.527 INFO:tasks.workunit.client.0.smithi019.stdout:8/588: dread d9/f29 [0,4194304] 0 2024-02-22T15:50:39.527 INFO:tasks.workunit.client.1.smithi062.stdout:4/431: stat d4/d2a/d2d/d2e/d6a/d8a/d92 0 2024-02-22T15:50:39.530 INFO:tasks.workunit.client.1.smithi062.stdout:4/432: chown d4/d2a/d2d/d2e/d6a/d94 4073 1 2024-02-22T15:50:39.531 INFO:tasks.workunit.client.1.smithi062.stdout:5/289: dread d0/d3/da/d13/d23/f24 [0,4194304] 0 2024-02-22T15:50:39.531 INFO:tasks.workunit.client.1.smithi062.stdout:4/433: readlink d4/d13/d20/l24 0 2024-02-22T15:50:39.532 INFO:tasks.workunit.client.0.smithi019.stdout:3/485: getdents d9/d65 0 2024-02-22T15:50:39.534 INFO:tasks.workunit.client.0.smithi019.stdout:6/757: dwrite d6/d31/f6b [0,4194304] 0 2024-02-22T15:50:39.535 INFO:tasks.workunit.client.0.smithi019.stdout:6/758: chown d6/d4b/d8d/f95 6329 1 2024-02-22T15:50:39.536 INFO:tasks.workunit.client.1.smithi062.stdout:8/376: dwrite d8/d9/d2c/f30 [0,4194304] 0 2024-02-22T15:50:39.537 INFO:tasks.workunit.client.0.smithi019.stdout:7/901: rename d9/d3a/d49 to d9/d38/d41/d45/db0/df7/dfb/d12a 0 2024-02-22T15:50:39.537 INFO:tasks.workunit.client.1.smithi062.stdout:4/434: dread d4/d2a/d2d/d2e/f84 [0,4194304] 0 2024-02-22T15:50:39.538 INFO:tasks.workunit.client.0.smithi019.stdout:2/568: rmdir d1/d17/d21/d51 39 2024-02-22T15:50:39.541 INFO:tasks.workunit.client.0.smithi019.stdout:8/589: mkdir d9/d14/d6f/d7e/d94/d9c/dc6 0 2024-02-22T15:50:39.543 INFO:tasks.workunit.client.0.smithi019.stdout:7/902: chown d9/d26/f31 0 1 2024-02-22T15:50:39.543 INFO:tasks.workunit.client.1.smithi062.stdout:5/290: mkdir d0/d3/d2f/d59 0 2024-02-22T15:50:39.544 INFO:tasks.workunit.client.1.smithi062.stdout:0/494: dwrite d1/d3/d4/d1a/d76/d8c/d2c/d84/f26 [0,4194304] 0 2024-02-22T15:50:39.544 INFO:tasks.workunit.client.0.smithi019.stdout:6/759: rmdir d6/d4b/da9/d67/d8a/d90 39 2024-02-22T15:50:39.546 INFO:tasks.workunit.client.1.smithi062.stdout:4/435: creat d4/d2a/d2d/d2e/d6a/d8a/f9a x:0 0 0 2024-02-22T15:50:39.546 INFO:tasks.workunit.client.0.smithi019.stdout:2/569: mknod d1/d17/d1c/d66/cd5 0 2024-02-22T15:50:39.547 INFO:tasks.workunit.client.1.smithi062.stdout:0/495: chown d1/d21/d85/d8d/fa1 3560344 1 2024-02-22T15:50:39.549 INFO:tasks.workunit.client.0.smithi019.stdout:8/590: symlink d9/d14/d6f/lc7 0 2024-02-22T15:50:39.549 INFO:tasks.workunit.client.1.smithi062.stdout:8/377: mknod d8/d28/c6f 0 2024-02-22T15:50:39.551 INFO:tasks.workunit.client.0.smithi019.stdout:5/640: dwrite d8/dd/d19/d3a/f3f [0,4194304] 0 2024-02-22T15:50:39.551 INFO:tasks.workunit.client.1.smithi062.stdout:0/496: mknod d1/d21/d85/cad 0 2024-02-22T15:50:39.553 INFO:tasks.workunit.client.0.smithi019.stdout:8/591: chown d9/d14/d1f/d6d/fc5 302530069 1 2024-02-22T15:50:39.553 INFO:tasks.workunit.client.0.smithi019.stdout:4/593: dread d2/db/d14/d23/f4d [0,4194304] 0 2024-02-22T15:50:39.554 INFO:tasks.workunit.client.1.smithi062.stdout:8/378: mkdir d8/d28/d70 0 2024-02-22T15:50:39.554 INFO:tasks.workunit.client.0.smithi019.stdout:9/600: rename d2/d1f/d2e/d48/cce to d2/cd5 0 2024-02-22T15:50:39.554 INFO:tasks.workunit.client.1.smithi062.stdout:9/371: dread d4/d1c/f1d [0,4194304] 0 2024-02-22T15:50:39.561 INFO:tasks.workunit.client.0.smithi019.stdout:1/651: dread d8/d15/f69 [0,4194304] 0 2024-02-22T15:50:39.561 INFO:tasks.workunit.client.1.smithi062.stdout:4/436: rmdir d4/d2a/d2d/d2e/d6a/d8a/d92 0 2024-02-22T15:50:39.563 INFO:tasks.workunit.client.1.smithi062.stdout:9/372: dread d4/d1c/f2c [0,4194304] 0 2024-02-22T15:50:39.563 INFO:tasks.workunit.client.1.smithi062.stdout:9/373: dread - d4/d15/d31/f4e zero size 2024-02-22T15:50:39.564 INFO:tasks.workunit.client.1.smithi062.stdout:9/374: truncate d4/d15/d5e/d63/f64 2537801 0 2024-02-22T15:50:39.565 INFO:tasks.workunit.client.0.smithi019.stdout:4/594: dwrite d2/db/d54/d8c/f96 [0,4194304] 0 2024-02-22T15:50:39.565 INFO:tasks.workunit.client.0.smithi019.stdout:5/641: mkdir d8/dd/d19/d77/dd2 0 2024-02-22T15:50:39.566 INFO:tasks.workunit.client.0.smithi019.stdout:6/760: rename d6/d4b/d6d/d7b/d7f/ld6 to d6/d4b/d6d/da3/lf2 0 2024-02-22T15:50:39.568 INFO:tasks.workunit.client.0.smithi019.stdout:7/903: getdents d9/d38/d41/d45/db0/df7/dfb/d12a/d5d/d8f 0 2024-02-22T15:50:39.569 INFO:tasks.workunit.client.1.smithi062.stdout:9/375: rmdir d4/d35/d3f/d57/d62 0 2024-02-22T15:50:39.571 INFO:tasks.workunit.client.0.smithi019.stdout:9/601: rename d2/d1f/d2e/d57/d60/d8c/d66/d9d to d2/d7/dc/dd6 0 2024-02-22T15:50:39.572 INFO:tasks.workunit.client.1.smithi062.stdout:9/376: creat d4/d28/f67 x:0 0 0 2024-02-22T15:50:39.574 INFO:tasks.workunit.client.0.smithi019.stdout:5/642: symlink d8/dd/d19/d77/dd2/ld3 0 2024-02-22T15:50:39.574 INFO:tasks.workunit.client.0.smithi019.stdout:9/602: dread d2/d1f/d2e/d57/f94 [0,4194304] 0 2024-02-22T15:50:39.575 INFO:tasks.workunit.client.0.smithi019.stdout:5/643: chown d8/dd/d19/d3a/d46/d7e/dbe/dc3 39561536 1 2024-02-22T15:50:39.577 INFO:tasks.workunit.client.0.smithi019.stdout:5/644: write d8/dd/d19/d38/d47/f98 [223906,54906] 0 2024-02-22T15:50:39.578 INFO:tasks.workunit.client.0.smithi019.stdout:9/603: chown d2/d7/dc/d27/dbc 3831 1 2024-02-22T15:50:39.590 INFO:tasks.workunit.client.1.smithi062.stdout:4/437: dread d4/d1c/d2c/d6b/f7e [0,4194304] 0 2024-02-22T15:50:39.591 INFO:tasks.workunit.client.1.smithi062.stdout:0/497: getdents d1/d21/d85 0 2024-02-22T15:50:39.594 INFO:tasks.workunit.client.1.smithi062.stdout:0/498: mknod d1/d21/d83/cae 0 2024-02-22T15:50:39.595 INFO:tasks.workunit.client.1.smithi062.stdout:8/379: truncate d8/d9/d2c/f30 1820907 0 2024-02-22T15:50:39.600 INFO:tasks.workunit.client.1.smithi062.stdout:9/377: rmdir d4/d28 39 2024-02-22T15:50:39.604 INFO:tasks.workunit.client.1.smithi062.stdout:8/380: link d8/d26/f47 d8/d9/f71 0 2024-02-22T15:50:39.608 INFO:tasks.workunit.client.1.smithi062.stdout:4/438: dread d4/d2a/d2d/f85 [0,4194304] 0 2024-02-22T15:50:39.609 INFO:tasks.workunit.client.1.smithi062.stdout:0/499: truncate d1/d3/d4/d1a/d76/d8c/d5d/f37 3510866 0 2024-02-22T15:50:39.610 INFO:tasks.workunit.client.1.smithi062.stdout:0/500: chown d1/f45 393 1 2024-02-22T15:50:39.611 INFO:tasks.workunit.client.1.smithi062.stdout:8/381: link d8/d9/d2c/c5b d8/d9/d44/d4d/c72 0 2024-02-22T15:50:39.613 INFO:tasks.workunit.client.1.smithi062.stdout:0/501: write d1/d3/d4/d1a/d76/d8c/f62 [2934960,47764] 0 2024-02-22T15:50:39.614 INFO:tasks.workunit.client.1.smithi062.stdout:8/382: write d8/d9/f37 [793266,96416] 0 2024-02-22T15:50:39.615 INFO:tasks.workunit.client.1.smithi062.stdout:8/383: read - d8/d9/d44/f5c zero size 2024-02-22T15:50:39.616 INFO:tasks.workunit.client.1.smithi062.stdout:0/502: unlink d1/d5e/fa9 0 2024-02-22T15:50:39.617 INFO:tasks.workunit.client.1.smithi062.stdout:8/384: chown d8/d26/c31 833375648 1 2024-02-22T15:50:39.617 INFO:tasks.workunit.client.1.smithi062.stdout:9/378: truncate d4/d1c/f1e 991139 0 2024-02-22T15:50:39.622 INFO:tasks.workunit.client.1.smithi062.stdout:4/439: write d4/d1c/f23 [1401845,52002] 0 2024-02-22T15:50:39.623 INFO:tasks.workunit.client.1.smithi062.stdout:8/385: mknod d8/d9/d2c/d49/d5e/c73 0 2024-02-22T15:50:39.625 INFO:tasks.workunit.client.1.smithi062.stdout:4/440: chown d4/d13/l22 115743 1 2024-02-22T15:50:39.625 INFO:tasks.workunit.client.1.smithi062.stdout:9/379: getdents d4/d35/d3f 0 2024-02-22T15:50:39.627 INFO:tasks.workunit.client.1.smithi062.stdout:0/503: dwrite d1/d3/de/f3a [8388608,4194304] 0 2024-02-22T15:50:39.636 INFO:tasks.workunit.client.0.smithi019.stdout:1/652: creat d8/fdf x:0 0 0 2024-02-22T15:50:39.639 INFO:tasks.workunit.client.1.smithi062.stdout:8/386: dwrite d8/d9/d44/d4d/d51/f68 [0,4194304] 0 2024-02-22T15:50:39.640 INFO:tasks.workunit.client.1.smithi062.stdout:0/504: dwrite d1/d3/d4/d1a/d1d/fa7 [0,4194304] 0 2024-02-22T15:50:39.641 INFO:tasks.workunit.client.1.smithi062.stdout:3/369: sync 2024-02-22T15:50:39.641 INFO:tasks.workunit.client.1.smithi062.stdout:7/345: sync 2024-02-22T15:50:39.642 INFO:tasks.workunit.client.0.smithi019.stdout:8/592: getdents d9/d14/d17/d1c 0 2024-02-22T15:50:39.642 INFO:tasks.workunit.client.0.smithi019.stdout:6/761: symlink d6/d4b/d71/d88/dc0/lf3 0 2024-02-22T15:50:39.643 INFO:tasks.workunit.client.1.smithi062.stdout:2/465: sync 2024-02-22T15:50:39.646 INFO:tasks.workunit.client.1.smithi062.stdout:3/370: symlink d5/d16/d21/l6e 0 2024-02-22T15:50:39.648 INFO:tasks.workunit.client.1.smithi062.stdout:8/387: rmdir d8/d42 39 2024-02-22T15:50:39.650 INFO:tasks.workunit.client.0.smithi019.stdout:4/595: creat d2/db/d14/d23/fc3 x:0 0 0 2024-02-22T15:50:39.651 INFO:tasks.workunit.client.0.smithi019.stdout:9/604: unlink d2/d7/dc/d27/fb4 0 2024-02-22T15:50:39.652 INFO:tasks.workunit.client.1.smithi062.stdout:7/346: dwrite d0/d6/d2e/d4a/f1f [4194304,4194304] 0 2024-02-22T15:50:39.652 INFO:tasks.workunit.client.1.smithi062.stdout:0/505: unlink d1/d3/l5b 0 2024-02-22T15:50:39.655 INFO:tasks.workunit.client.1.smithi062.stdout:0/506: fsync d1/d3/de/f90 0 2024-02-22T15:50:39.656 INFO:tasks.workunit.client.1.smithi062.stdout:3/371: write f3 [4274877,32918] 0 2024-02-22T15:50:39.659 INFO:tasks.workunit.client.1.smithi062.stdout:9/380: dwrite d4/f14 [0,4194304] 0 2024-02-22T15:50:39.660 INFO:tasks.workunit.client.1.smithi062.stdout:3/372: rmdir d5/d16/d21 39 2024-02-22T15:50:39.661 INFO:tasks.workunit.client.1.smithi062.stdout:0/507: readlink d1/d3/d4/d1a/d76/d8c/d2c/l78 0 2024-02-22T15:50:39.661 INFO:tasks.workunit.client.1.smithi062.stdout:7/347: dread d0/d14/f25 [0,4194304] 0 2024-02-22T15:50:39.662 INFO:tasks.workunit.client.1.smithi062.stdout:0/508: chown d1/d13 393164 1 2024-02-22T15:50:39.662 INFO:tasks.workunit.client.0.smithi019.stdout:1/653: unlink d8/d23/f55 0 2024-02-22T15:50:39.662 INFO:tasks.workunit.client.0.smithi019.stdout:0/532: dwrite d4/d22/d2c/f57 [0,4194304] 0 2024-02-22T15:50:39.663 INFO:tasks.workunit.client.1.smithi062.stdout:8/388: rmdir d8/d9/d2c/d49/d5e 39 2024-02-22T15:50:39.665 INFO:tasks.workunit.client.1.smithi062.stdout:7/348: chown d0/d6/d2e/f61 128107 1 2024-02-22T15:50:39.666 INFO:tasks.workunit.client.1.smithi062.stdout:3/373: fsync d5/d16/d21/d55/d5e/f5f 0 2024-02-22T15:50:39.667 INFO:tasks.workunit.client.1.smithi062.stdout:4/441: dread d4/d1c/d2c/f56 [0,4194304] 0 2024-02-22T15:50:39.668 INFO:tasks.workunit.client.1.smithi062.stdout:8/389: mkdir d8/d9/d2c/d74 0 2024-02-22T15:50:39.670 INFO:tasks.workunit.client.1.smithi062.stdout:4/442: write d4/d1c/d2c/d6b/d70/f76 [876051,19771] 0 2024-02-22T15:50:39.671 INFO:tasks.workunit.client.1.smithi062.stdout:4/443: write f2 [3330209,79254] 0 2024-02-22T15:50:39.672 INFO:tasks.workunit.client.1.smithi062.stdout:9/381: link d4/f26 d4/d35/d3f/d57/f68 0 2024-02-22T15:50:39.672 INFO:tasks.workunit.client.1.smithi062.stdout:4/444: chown d4/d1c/d2c 134 1 2024-02-22T15:50:39.674 INFO:tasks.workunit.client.1.smithi062.stdout:7/349: symlink d0/d6/d2e/d4a/d5a/d5e/l6e 0 2024-02-22T15:50:39.676 INFO:tasks.workunit.client.1.smithi062.stdout:0/509: read d1/dd/f6b [3808446,126708] 0 2024-02-22T15:50:39.678 INFO:tasks.workunit.client.1.smithi062.stdout:9/382: rmdir d4/d15/d5e 39 2024-02-22T15:50:39.681 INFO:tasks.workunit.client.0.smithi019.stdout:1/654: creat d8/d15/d5b/dbc/fe0 x:0 0 0 2024-02-22T15:50:39.682 INFO:tasks.workunit.client.0.smithi019.stdout:1/655: chown d8/d15/f20 4 1 2024-02-22T15:50:39.682 INFO:tasks.workunit.client.0.smithi019.stdout:1/656: stat d8/d23/c6e 0 2024-02-22T15:50:39.683 INFO:tasks.workunit.client.0.smithi019.stdout:0/533: rename d4/d5 to d4/db/d1b/d63/db0/da1/dbb 0 2024-02-22T15:50:39.685 INFO:tasks.workunit.client.0.smithi019.stdout:0/534: write d4/db/d1b/d40/f5d [4663794,35437] 0 2024-02-22T15:50:39.686 INFO:tasks.workunit.client.1.smithi062.stdout:7/350: symlink d0/d6/d2e/d55/l6f 0 2024-02-22T15:50:39.687 INFO:tasks.workunit.client.1.smithi062.stdout:8/390: creat d8/d9/d44/d4d/d51/d55/f75 x:0 0 0 2024-02-22T15:50:39.688 INFO:tasks.workunit.client.0.smithi019.stdout:0/535: fsync d4/d23/d7d/f85 0 2024-02-22T15:50:39.688 INFO:tasks.workunit.client.0.smithi019.stdout:3/486: fdatasync d9/d26/d32/f33 0 2024-02-22T15:50:39.689 INFO:tasks.workunit.client.1.smithi062.stdout:3/374: creat d5/f6f x:0 0 0 2024-02-22T15:50:39.689 INFO:tasks.workunit.client.1.smithi062.stdout:2/466: dwrite d1/d47/f2b [8388608,4194304] 0 2024-02-22T15:50:39.689 INFO:tasks.workunit.client.1.smithi062.stdout:7/351: creat d0/d6/d2e/d35/f70 x:0 0 0 2024-02-22T15:50:39.690 INFO:tasks.workunit.client.0.smithi019.stdout:5/645: creat d8/dd/d19/d77/d7a/d9e/fd4 x:0 0 0 2024-02-22T15:50:39.690 INFO:tasks.workunit.client.1.smithi062.stdout:0/510: symlink d1/d21/d85/d8e/d94/laf 0 2024-02-22T15:50:39.694 INFO:tasks.workunit.client.1.smithi062.stdout:8/391: creat d8/d9/d44/d4d/d54/f76 x:0 0 0 2024-02-22T15:50:39.695 INFO:tasks.workunit.client.1.smithi062.stdout:4/445: dwrite d4/f3f [0,4194304] 0 2024-02-22T15:50:39.697 INFO:tasks.workunit.client.1.smithi062.stdout:3/375: fsync d5/d16/d21/d32/f51 0 2024-02-22T15:50:39.699 INFO:tasks.workunit.client.0.smithi019.stdout:6/762: dread d6/d31/f3b [0,4194304] 0 2024-02-22T15:50:39.700 INFO:tasks.workunit.client.0.smithi019.stdout:2/570: dwrite d1/d17/d21/d51/fcc [0,4194304] 0 2024-02-22T15:50:39.701 INFO:tasks.workunit.client.1.smithi062.stdout:9/383: getdents d4/d15 0 2024-02-22T15:50:39.704 INFO:tasks.workunit.client.1.smithi062.stdout:0/511: symlink d1/d3/d4/d1a/d76/d8c/d2c/lb0 0 2024-02-22T15:50:39.704 INFO:tasks.workunit.client.0.smithi019.stdout:0/536: write d4/d22/f5f [3364632,110271] 0 2024-02-22T15:50:39.704 INFO:tasks.workunit.client.0.smithi019.stdout:1/657: fsync d8/fdf 0 2024-02-22T15:50:39.705 INFO:tasks.workunit.client.0.smithi019.stdout:8/593: write d9/d14/d17/d1c/f51 [5991290,38567] 0 2024-02-22T15:50:39.706 INFO:tasks.workunit.client.1.smithi062.stdout:2/467: write d1/d47/d39/d5d/f63 [496995,124703] 0 2024-02-22T15:50:39.707 INFO:tasks.workunit.client.0.smithi019.stdout:3/487: mknod d9/d22/d27/d6d/c97 0 2024-02-22T15:50:39.709 INFO:tasks.workunit.client.0.smithi019.stdout:2/571: truncate d1/d17/d21/f36 1472149 0 2024-02-22T15:50:39.709 INFO:tasks.workunit.client.0.smithi019.stdout:6/763: chown d6/d4b/d6d/d7b/d78/f7e 19 1 2024-02-22T15:50:39.709 INFO:tasks.workunit.client.1.smithi062.stdout:4/446: mkdir d4/d1c/d2c/d6b/d9b 0 2024-02-22T15:50:39.710 INFO:tasks.workunit.client.1.smithi062.stdout:0/512: chown d1/d13/f2f 2012 1 2024-02-22T15:50:39.712 INFO:tasks.workunit.client.1.smithi062.stdout:8/392: creat d8/d9/d44/d4d/d51/d55/f77 x:0 0 0 2024-02-22T15:50:39.712 INFO:tasks.workunit.client.1.smithi062.stdout:2/468: write d1/d8/f6e [615602,23935] 0 2024-02-22T15:50:39.712 INFO:tasks.workunit.client.0.smithi019.stdout:0/537: truncate d4/db/d1b/d40/d44/d75/f82 1554299 0 2024-02-22T15:50:39.714 INFO:tasks.workunit.client.0.smithi019.stdout:8/594: creat d9/d14/d1f/d5c/fc8 x:0 0 0 2024-02-22T15:50:39.714 INFO:tasks.workunit.client.1.smithi062.stdout:4/447: mknod d4/d2a/c9c 0 2024-02-22T15:50:39.717 INFO:tasks.workunit.client.1.smithi062.stdout:9/384: dread d4/d35/d3f/f25 [0,4194304] 0 2024-02-22T15:50:39.717 INFO:tasks.workunit.client.1.smithi062.stdout:0/513: creat d1/d3/d4/d1a/d76/fb1 x:0 0 0 2024-02-22T15:50:39.718 INFO:tasks.workunit.client.1.smithi062.stdout:8/393: creat d8/d9/d2c/f78 x:0 0 0 2024-02-22T15:50:39.719 INFO:tasks.workunit.client.0.smithi019.stdout:2/572: creat d1/d17/d21/daf/fd6 x:0 0 0 2024-02-22T15:50:39.720 INFO:tasks.workunit.client.1.smithi062.stdout:7/352: dwrite d0/d14/f67 [0,4194304] 0 2024-02-22T15:50:39.720 INFO:tasks.workunit.client.0.smithi019.stdout:6/764: creat d6/d31/d43/ff4 x:0 0 0 2024-02-22T15:50:39.722 INFO:tasks.workunit.client.0.smithi019.stdout:9/605: truncate d2/d1f/d2e/d48/d45/d5a/f72 745473 0 2024-02-22T15:50:39.724 INFO:tasks.workunit.client.0.smithi019.stdout:0/538: rmdir d4/db/d1b/d63/db0/da1/dbb/d7 39 2024-02-22T15:50:39.726 INFO:tasks.workunit.client.0.smithi019.stdout:8/595: symlink d9/d14/d1f/d20/lc9 0 2024-02-22T15:50:39.727 INFO:tasks.workunit.client.1.smithi062.stdout:7/353: symlink d0/l71 0 2024-02-22T15:50:39.729 INFO:tasks.workunit.client.1.smithi062.stdout:3/376: dwrite d5/d26/f6c [0,4194304] 0 2024-02-22T15:50:39.729 INFO:tasks.workunit.client.0.smithi019.stdout:6/765: symlink d6/d31/d69/lf5 0 2024-02-22T15:50:39.731 INFO:tasks.workunit.client.0.smithi019.stdout:3/488: dwrite d9/d22/d49/f7c [0,4194304] 0 2024-02-22T15:50:39.736 INFO:tasks.workunit.client.0.smithi019.stdout:6/766: truncate d6/d3e/fb7 899937 0 2024-02-22T15:50:39.737 INFO:tasks.workunit.client.0.smithi019.stdout:9/606: mknod d2/d1f/d2e/d7d/cd7 0 2024-02-22T15:50:39.737 INFO:tasks.workunit.client.1.smithi062.stdout:8/394: truncate d8/d9/d44/d4d/d51/d55/f75 555296 0 2024-02-22T15:50:39.738 INFO:tasks.workunit.client.1.smithi062.stdout:0/514: link d1/d13/f31 d1/d3/d4/d1e/fb2 0 2024-02-22T15:50:39.738 INFO:tasks.workunit.client.1.smithi062.stdout:7/354: write d0/d14/d13/f48 [405340,57770] 0 2024-02-22T15:50:39.738 INFO:tasks.workunit.client.0.smithi019.stdout:5/646: dread d8/d11/d33/fcb [0,4194304] 0 2024-02-22T15:50:39.738 INFO:tasks.workunit.client.0.smithi019.stdout:4/596: dwrite d2/db/d14/d23/d21/fb3 [0,4194304] 0 2024-02-22T15:50:39.742 INFO:tasks.workunit.client.1.smithi062.stdout:9/385: dread d4/d35/d3f/d57/f68 [0,4194304] 0 2024-02-22T15:50:39.744 INFO:tasks.workunit.client.1.smithi062.stdout:4/448: dread d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:39.745 INFO:tasks.workunit.client.0.smithi019.stdout:0/539: creat d4/db/d1b/d63/db0/da1/fbc x:0 0 0 2024-02-22T15:50:39.745 INFO:tasks.workunit.client.1.smithi062.stdout:3/377: creat d5/d16/d21/d32/f70 x:0 0 0 2024-02-22T15:50:39.746 INFO:tasks.workunit.client.1.smithi062.stdout:9/386: stat d4/d15/f5c 0 2024-02-22T15:50:39.746 INFO:tasks.workunit.client.0.smithi019.stdout:6/767: mknod d6/d4b/d71/d88/cf6 0 2024-02-22T15:50:39.747 INFO:tasks.workunit.client.1.smithi062.stdout:7/355: truncate d0/d6/d2e/d4a/f2d 740837 0 2024-02-22T15:50:39.749 INFO:tasks.workunit.client.0.smithi019.stdout:3/489: symlink d9/d26/d2d/d39/l98 0 2024-02-22T15:50:39.749 INFO:tasks.workunit.client.1.smithi062.stdout:8/395: dread d8/d26/f47 [0,4194304] 0 2024-02-22T15:50:39.751 INFO:tasks.workunit.client.0.smithi019.stdout:0/540: mknod d4/db/d1b/d63/db0/cbd 0 2024-02-22T15:50:39.753 INFO:tasks.workunit.client.0.smithi019.stdout:6/768: dread d6/d31/d48/f51 [0,4194304] 0 2024-02-22T15:50:39.753 INFO:tasks.workunit.client.1.smithi062.stdout:4/449: mkdir d4/d2a/d2d/d9d 0 2024-02-22T15:50:39.753 INFO:tasks.workunit.client.1.smithi062.stdout:7/356: unlink d0/d6/d2e/d55/f65 0 2024-02-22T15:50:39.753 INFO:tasks.workunit.client.1.smithi062.stdout:0/515: getdents d1/d3/d4/d1a/d1d 0 2024-02-22T15:50:39.755 INFO:tasks.workunit.client.1.smithi062.stdout:4/450: chown d4/d1c/d30/f32 80073 1 2024-02-22T15:50:39.755 INFO:tasks.workunit.client.0.smithi019.stdout:9/607: dread d2/f6f [0,4194304] 0 2024-02-22T15:50:39.755 INFO:tasks.workunit.client.0.smithi019.stdout:3/490: dread - d9/f6a zero size 2024-02-22T15:50:39.756 INFO:tasks.workunit.client.1.smithi062.stdout:9/387: write d4/d15/d5e/d63/f64 [318555,74120] 0 2024-02-22T15:50:39.756 INFO:tasks.workunit.client.0.smithi019.stdout:6/769: mknod d6/d58/cf7 0 2024-02-22T15:50:39.756 INFO:tasks.workunit.client.1.smithi062.stdout:8/396: creat d8/d9/d48/f79 x:0 0 0 2024-02-22T15:50:39.757 INFO:tasks.workunit.client.1.smithi062.stdout:9/388: readlink d4/d15/l3d 0 2024-02-22T15:50:39.757 INFO:tasks.workunit.client.1.smithi062.stdout:3/378: creat d5/d1a/d3b/f71 x:0 0 0 2024-02-22T15:50:39.758 INFO:tasks.workunit.client.0.smithi019.stdout:6/770: write d6/f3c [237328,116271] 0 2024-02-22T15:50:39.758 INFO:tasks.workunit.client.1.smithi062.stdout:8/397: stat d8/d9/d44/d4d/d51/d55/f6d 0 2024-02-22T15:50:39.759 INFO:tasks.workunit.client.0.smithi019.stdout:0/541: mknod d4/db/d1b/d63/db0/da1/dbb/d7/cbe 0 2024-02-22T15:50:39.759 INFO:tasks.workunit.client.1.smithi062.stdout:0/516: creat d1/d3/d4/d1a/d1d/fb3 x:0 0 0 2024-02-22T15:50:39.760 INFO:tasks.workunit.client.0.smithi019.stdout:6/771: chown d6/fc 7 1 2024-02-22T15:50:39.761 INFO:tasks.workunit.client.1.smithi062.stdout:3/379: write d5/d16/d21/d32/f66 [227840,86148] 0 2024-02-22T15:50:39.761 INFO:tasks.workunit.client.0.smithi019.stdout:6/772: write d6/d31/d69/d8e/dcf/fec [673194,113710] 0 2024-02-22T15:50:39.762 INFO:tasks.workunit.client.1.smithi062.stdout:7/357: mkdir d0/d6/d2e/d55/d72 0 2024-02-22T15:50:39.762 INFO:tasks.workunit.client.0.smithi019.stdout:3/491: creat d9/d26/d3f/d59/f99 x:0 0 0 2024-02-22T15:50:39.764 INFO:tasks.workunit.client.0.smithi019.stdout:9/608: creat d2/d7/dc/d27/dbc/fd8 x:0 0 0 2024-02-22T15:50:39.768 INFO:tasks.workunit.client.1.smithi062.stdout:4/451: dread d4/df/f14 [8388608,4194304] 0 2024-02-22T15:50:39.768 INFO:tasks.workunit.client.1.smithi062.stdout:2/469: dread d1/d28/d3f/f46 [0,4194304] 0 2024-02-22T15:50:39.768 INFO:tasks.workunit.client.0.smithi019.stdout:0/542: creat d4/d23/d7d/fbf x:0 0 0 2024-02-22T15:50:39.768 INFO:tasks.workunit.client.0.smithi019.stdout:6/773: symlink d6/d4b/d6d/da3/ddb/ddc/lf8 0 2024-02-22T15:50:39.768 INFO:tasks.workunit.client.0.smithi019.stdout:9/609: dread - d2/d1f/d2e/d57/d60/d8c/fc3 zero size 2024-02-22T15:50:39.769 INFO:tasks.workunit.client.1.smithi062.stdout:8/398: creat d8/f7a x:0 0 0 2024-02-22T15:50:39.770 INFO:tasks.workunit.client.1.smithi062.stdout:9/389: symlink d4/d28/d50/d5a/l69 0 2024-02-22T15:50:39.771 INFO:tasks.workunit.client.1.smithi062.stdout:2/470: write d1/f2c [460499,3105] 0 2024-02-22T15:50:39.772 INFO:tasks.workunit.client.0.smithi019.stdout:9/610: rmdir d2/d7/dc/d27/d31/dd4 39 2024-02-22T15:50:39.773 INFO:tasks.workunit.client.1.smithi062.stdout:4/452: mkdir d4/d2a/d2d/d2e/d6a/d9e 0 2024-02-22T15:50:39.774 INFO:tasks.workunit.client.1.smithi062.stdout:0/517: dread d1/d3/d4/d1a/d76/d8c/d2c/d84/f26 [0,4194304] 0 2024-02-22T15:50:39.775 INFO:tasks.workunit.client.0.smithi019.stdout:3/492: dwrite d9/d26/d76/f77 [0,4194304] 0 2024-02-22T15:50:39.776 INFO:tasks.workunit.client.0.smithi019.stdout:8/596: dread d9/d14/f3e [0,4194304] 0 2024-02-22T15:50:39.776 INFO:tasks.workunit.client.1.smithi062.stdout:7/358: link d0/d14/c1d d0/d6/d2e/d58/d5d/c73 0 2024-02-22T15:50:39.777 INFO:tasks.workunit.client.0.smithi019.stdout:9/611: dread - d2/d7/dc/d27/dbc/fd8 zero size 2024-02-22T15:50:39.778 INFO:tasks.workunit.client.1.smithi062.stdout:1/415: sync 2024-02-22T15:50:39.779 INFO:tasks.workunit.client.0.smithi019.stdout:6/774: dwrite d6/d3e/f50 [4194304,4194304] 0 2024-02-22T15:50:39.780 INFO:tasks.workunit.client.0.smithi019.stdout:3/493: symlink d9/d14/d19/l9a 0 2024-02-22T15:50:39.780 INFO:tasks.workunit.client.0.smithi019.stdout:6/775: read - d6/d3e/fce zero size 2024-02-22T15:50:39.780 INFO:tasks.workunit.client.1.smithi062.stdout:2/471: write d1/f82 [229948,125334] 0 2024-02-22T15:50:39.781 INFO:tasks.workunit.client.0.smithi019.stdout:8/597: mkdir d9/d14/d1f/dca 0 2024-02-22T15:50:39.783 INFO:tasks.workunit.client.1.smithi062.stdout:8/399: truncate d8/d9/d2c/d49/f6b 612815 0 2024-02-22T15:50:39.783 INFO:tasks.workunit.client.0.smithi019.stdout:8/598: fsync d9/d14/f38 0 2024-02-22T15:50:39.784 INFO:tasks.workunit.client.1.smithi062.stdout:3/380: dwrite d5/d16/d21/d36/d4a/f4f [0,4194304] 0 2024-02-22T15:50:39.784 INFO:tasks.workunit.client.1.smithi062.stdout:4/453: dread d4/d1c/d2c/d6b/f7e [0,4194304] 0 2024-02-22T15:50:39.784 INFO:tasks.workunit.client.1.smithi062.stdout:7/359: dread - d0/d6/d2e/d2f/f57 zero size 2024-02-22T15:50:39.784 INFO:tasks.workunit.client.0.smithi019.stdout:8/599: write d9/d14/f52 [1018729,74206] 0 2024-02-22T15:50:39.786 INFO:tasks.workunit.client.0.smithi019.stdout:9/612: mkdir d2/d7/dc/dd9 0 2024-02-22T15:50:39.787 INFO:tasks.workunit.client.1.smithi062.stdout:0/518: creat d1/d38/fb4 x:0 0 0 2024-02-22T15:50:39.788 INFO:tasks.workunit.client.0.smithi019.stdout:6/776: creat d6/d4b/d71/d88/dc0/ff9 x:0 0 0 2024-02-22T15:50:39.790 INFO:tasks.workunit.client.0.smithi019.stdout:6/777: write d6/d3e/f4e [1707995,127727] 0 2024-02-22T15:50:39.791 INFO:tasks.workunit.client.0.smithi019.stdout:8/600: creat d9/d14/d17/d3d/fcb x:0 0 0 2024-02-22T15:50:39.793 INFO:tasks.workunit.client.1.smithi062.stdout:3/381: read - d5/d1a/d3b/f71 zero size 2024-02-22T15:50:39.793 INFO:tasks.workunit.client.1.smithi062.stdout:7/360: rename d0/d14/d19 to d0/d14/d19/d30/d37/d74 22 2024-02-22T15:50:39.794 INFO:tasks.workunit.client.0.smithi019.stdout:9/613: mknod d2/d7/d52/dc0/cda 0 2024-02-22T15:50:39.794 INFO:tasks.workunit.client.1.smithi062.stdout:9/390: dwrite d4/f29 [0,4194304] 0 2024-02-22T15:50:39.794 INFO:tasks.workunit.client.1.smithi062.stdout:2/472: creat d1/d8/d62/f9c x:0 0 0 2024-02-22T15:50:39.796 INFO:tasks.workunit.client.0.smithi019.stdout:4/597: write d2/db/d14/f5e [1489720,59853] 0 2024-02-22T15:50:39.798 INFO:tasks.workunit.client.1.smithi062.stdout:1/416: creat d1/d6/d7/d2b/d47/d63/f87 x:0 0 0 2024-02-22T15:50:39.799 INFO:tasks.workunit.client.1.smithi062.stdout:6/500: sync 2024-02-22T15:50:39.799 INFO:tasks.workunit.client.1.smithi062.stdout:5/291: sync 2024-02-22T15:50:39.805 INFO:tasks.workunit.client.1.smithi062.stdout:7/361: symlink d0/d14/d28/l75 0 2024-02-22T15:50:39.805 INFO:tasks.workunit.client.0.smithi019.stdout:0/543: truncate d4/db/d1b/d63/db0/da1/dbb/d52/f58 3591343 0 2024-02-22T15:50:39.805 INFO:tasks.workunit.client.0.smithi019.stdout:8/601: truncate d9/d14/d1f/d4c/f7f 496266 0 2024-02-22T15:50:39.806 INFO:tasks.workunit.client.0.smithi019.stdout:4/598: read d2/db/d14/d23/d21/f2d [5539683,87269] 0 2024-02-22T15:50:39.807 INFO:tasks.workunit.client.0.smithi019.stdout:8/602: rmdir d9/d14/d17 39 2024-02-22T15:50:39.807 INFO:tasks.workunit.client.0.smithi019.stdout:0/544: read d4/f3e [7665214,90148] 0 2024-02-22T15:50:39.808 INFO:tasks.workunit.client.1.smithi062.stdout:0/519: rename d1/d21/d85/d8e/d94 to d1/d3/d4/d1a/d76/d8c/d2c/db5 0 2024-02-22T15:50:39.811 INFO:tasks.workunit.client.1.smithi062.stdout:4/454: dread d4/fd [0,4194304] 0 2024-02-22T15:50:39.813 INFO:tasks.workunit.client.1.smithi062.stdout:5/292: mkdir d0/d3/da/d13/d23/d4e/d5a 0 2024-02-22T15:50:39.819 INFO:tasks.workunit.client.1.smithi062.stdout:9/391: dwrite f2 [0,4194304] 0 2024-02-22T15:50:39.819 INFO:tasks.workunit.client.0.smithi019.stdout:6/778: dread d6/d58/d6a/f85 [0,4194304] 0 2024-02-22T15:50:39.821 INFO:tasks.workunit.client.1.smithi062.stdout:3/382: dwrite f3 [0,4194304] 0 2024-02-22T15:50:39.821 INFO:tasks.workunit.client.1.smithi062.stdout:4/455: truncate d4/d1c/f3d 1160934 0 2024-02-22T15:50:39.821 INFO:tasks.workunit.client.1.smithi062.stdout:8/400: write d8/f29 [115866,64113] 0 2024-02-22T15:50:39.824 INFO:tasks.workunit.client.1.smithi062.stdout:5/293: mkdir d0/d3/da/d32/d4f/d5b 0 2024-02-22T15:50:39.825 INFO:tasks.workunit.client.0.smithi019.stdout:6/779: dwrite d6/f2c [0,4194304] 0 2024-02-22T15:50:39.827 INFO:tasks.workunit.client.0.smithi019.stdout:6/780: write d6/f10 [1341875,45337] 0 2024-02-22T15:50:39.828 INFO:tasks.workunit.client.1.smithi062.stdout:8/401: unlink c3 0 2024-02-22T15:50:39.830 INFO:tasks.workunit.client.1.smithi062.stdout:3/383: mknod d5/d26/c72 0 2024-02-22T15:50:39.833 INFO:tasks.workunit.client.1.smithi062.stdout:9/392: mkdir d4/d15/d5f/d6a 0 2024-02-22T15:50:39.835 INFO:tasks.workunit.client.1.smithi062.stdout:5/294: fdatasync d0/d3/d2f/d38/f41 0 2024-02-22T15:50:39.836 INFO:tasks.workunit.client.1.smithi062.stdout:8/402: symlink d8/d9/d44/d4d/l7b 0 2024-02-22T15:50:39.836 INFO:tasks.workunit.client.1.smithi062.stdout:1/417: dwrite d1/d6/d7/d2b/d49/f51 [0,4194304] 0 2024-02-22T15:50:39.837 INFO:tasks.workunit.client.1.smithi062.stdout:6/501: truncate d5/df/d40/d44/d76/f6a 2932887 0 2024-02-22T15:50:39.839 INFO:tasks.workunit.client.1.smithi062.stdout:4/456: symlink d4/d2a/d2d/l9f 0 2024-02-22T15:50:39.842 INFO:tasks.workunit.client.1.smithi062.stdout:8/403: readlink d8/d42/l52 0 2024-02-22T15:50:39.842 INFO:tasks.workunit.client.1.smithi062.stdout:9/393: creat d4/d1c/f6b x:0 0 0 2024-02-22T15:50:39.844 INFO:tasks.workunit.client.1.smithi062.stdout:0/520: link d1/d3/d4/d1a/d1d/c77 d1/d3/d4/d1a/d76/d8c/d92/cb6 0 2024-02-22T15:50:39.849 INFO:tasks.workunit.client.1.smithi062.stdout:8/404: creat d8/d42/f7c x:0 0 0 2024-02-22T15:50:39.850 INFO:tasks.workunit.client.1.smithi062.stdout:9/394: mknod d4/d15/d31/c6c 0 2024-02-22T15:50:39.851 INFO:tasks.workunit.client.1.smithi062.stdout:2/473: dread d1/d28/d3f/f59 [0,4194304] 0 2024-02-22T15:50:39.852 INFO:tasks.workunit.client.1.smithi062.stdout:1/418: rename d1/c64 to d1/d6/d36/c88 0 2024-02-22T15:50:39.853 INFO:tasks.workunit.client.1.smithi062.stdout:7/362: dwrite d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:39.854 INFO:tasks.workunit.client.1.smithi062.stdout:5/295: rmdir d0/d3/da/d32/d4f/d5b 0 2024-02-22T15:50:39.855 INFO:tasks.workunit.client.1.smithi062.stdout:4/457: mknod d4/d1c/d2c/d6b/d9b/ca0 0 2024-02-22T15:50:39.859 INFO:tasks.workunit.client.1.smithi062.stdout:9/395: mknod d4/d15/d5e/d63/c6d 0 2024-02-22T15:50:39.861 INFO:tasks.workunit.client.1.smithi062.stdout:9/396: chown d4/d15/d31/l61 0 1 2024-02-22T15:50:39.863 INFO:tasks.workunit.client.1.smithi062.stdout:0/521: rename d1/d38/fb4 to d1/d21/d85/d8e/d97/fb7 0 2024-02-22T15:50:39.865 INFO:tasks.workunit.client.1.smithi062.stdout:2/474: dwrite d1/d47/f2b [4194304,4194304] 0 2024-02-22T15:50:39.867 INFO:tasks.workunit.client.1.smithi062.stdout:1/419: symlink d1/d66/l89 0 2024-02-22T15:50:39.873 INFO:tasks.workunit.client.1.smithi062.stdout:3/384: truncate d5/d16/d21/f22 1746524 0 2024-02-22T15:50:39.875 INFO:tasks.workunit.client.1.smithi062.stdout:1/420: mkdir d1/d6/d7/d8a 0 2024-02-22T15:50:39.878 INFO:tasks.workunit.client.1.smithi062.stdout:4/458: dwrite d4/d2a/d2d/d2e/f69 [0,4194304] 0 2024-02-22T15:50:39.879 INFO:tasks.workunit.client.1.smithi062.stdout:3/385: fdatasync d5/d16/d21/d36/d4a/f4f 0 2024-02-22T15:50:39.880 INFO:tasks.workunit.client.1.smithi062.stdout:2/475: fsync d1/d28/d3f/f46 0 2024-02-22T15:50:39.883 INFO:tasks.workunit.client.1.smithi062.stdout:8/405: link d8/d26/f34 d8/d9/d2c/d49/d5e/f7d 0 2024-02-22T15:50:39.885 INFO:tasks.workunit.client.1.smithi062.stdout:4/459: fsync d4/d1c/d2c/f56 0 2024-02-22T15:50:39.887 INFO:tasks.workunit.client.0.smithi019.stdout:4/599: mkdir d2/db/d14/d23/d21/d42/dc4 0 2024-02-22T15:50:39.888 INFO:tasks.workunit.client.0.smithi019.stdout:0/545: mknod d4/db/d1b/d63/db0/da1/dbb/d7/cc0 0 2024-02-22T15:50:39.889 INFO:tasks.workunit.client.0.smithi019.stdout:4/600: chown d2/fb6 527021141 1 2024-02-22T15:50:39.891 INFO:tasks.workunit.client.0.smithi019.stdout:4/601: write d2/db/d14/f18 [41911,62730] 0 2024-02-22T15:50:39.892 INFO:tasks.workunit.client.0.smithi019.stdout:8/603: creat d9/d14/d17/d1c/da0/fcc x:0 0 0 2024-02-22T15:50:39.893 INFO:tasks.workunit.client.0.smithi019.stdout:8/604: dread - d9/d14/d1f/d6d/fc5 zero size 2024-02-22T15:50:39.893 INFO:tasks.workunit.client.1.smithi062.stdout:5/296: dwrite d0/d3/da/d13/d23/f24 [0,4194304] 0 2024-02-22T15:50:39.894 INFO:tasks.workunit.client.1.smithi062.stdout:2/476: write d1/d8/df/f85 [852542,74432] 0 2024-02-22T15:50:39.894 INFO:tasks.workunit.client.1.smithi062.stdout:7/363: getdents d0/d14 0 2024-02-22T15:50:39.894 INFO:tasks.workunit.client.0.smithi019.stdout:8/605: stat d9/d14/d6f/d7e/d94/d9a 0 2024-02-22T15:50:39.895 INFO:tasks.workunit.client.0.smithi019.stdout:0/546: mkdir d4/db/d24/d80/daf/d95/dc1 0 2024-02-22T15:50:39.896 INFO:tasks.workunit.client.1.smithi062.stdout:6/502: dwrite d5/df/d40/d44/d76/d56/f61 [0,4194304] 0 2024-02-22T15:50:39.896 INFO:tasks.workunit.client.0.smithi019.stdout:0/547: chown d4/d32/la8 256135 1 2024-02-22T15:50:39.898 INFO:tasks.workunit.client.1.smithi062.stdout:1/421: creat d1/d6/d7/d4a/d65/d83/f8b x:0 0 0 2024-02-22T15:50:39.898 INFO:tasks.workunit.client.1.smithi062.stdout:3/386: mknod d5/d1a/d3c/c73 0 2024-02-22T15:50:39.901 INFO:tasks.workunit.client.0.smithi019.stdout:8/606: truncate d9/d14/d6f/d7e/d94/daa/fb6 347321 0 2024-02-22T15:50:39.902 INFO:tasks.workunit.client.1.smithi062.stdout:8/406: dread d8/d9/d44/f56 [0,4194304] 0 2024-02-22T15:50:39.903 INFO:tasks.workunit.client.1.smithi062.stdout:4/460: symlink d4/d1c/d2c/la1 0 2024-02-22T15:50:39.904 INFO:tasks.workunit.client.0.smithi019.stdout:0/548: unlink d4/db/d76/c8d 0 2024-02-22T15:50:39.905 INFO:tasks.workunit.client.1.smithi062.stdout:5/297: unlink d0/d3/da/d13/l15 0 2024-02-22T15:50:39.908 INFO:tasks.workunit.client.0.smithi019.stdout:9/614: dwrite d2/d1f/d2e/d48/d36/f83 [0,4194304] 0 2024-02-22T15:50:39.910 INFO:tasks.workunit.client.1.smithi062.stdout:0/522: link d1/d3/d4/d1a/c93 d1/cb8 0 2024-02-22T15:50:39.910 INFO:tasks.workunit.client.1.smithi062.stdout:3/387: write d5/d1a/d3c/f54 [645465,29615] 0 2024-02-22T15:50:39.913 INFO:tasks.workunit.client.1.smithi062.stdout:1/422: unlink d1/d6/de/l5e 0 2024-02-22T15:50:39.914 INFO:tasks.workunit.client.1.smithi062.stdout:6/503: creat d5/df/d32/f9f x:0 0 0 2024-02-22T15:50:39.917 INFO:tasks.workunit.client.0.smithi019.stdout:9/615: dwrite d2/d1f/d2e/d57/d60/d8c/d66/fba [0,4194304] 0 2024-02-22T15:50:39.918 INFO:tasks.workunit.client.1.smithi062.stdout:5/298: creat d0/d3/d2f/d38/d3f/f5c x:0 0 0 2024-02-22T15:50:39.918 INFO:tasks.workunit.client.1.smithi062.stdout:9/397: dwrite d4/d35/f36 [0,4194304] 0 2024-02-22T15:50:39.922 INFO:tasks.workunit.client.1.smithi062.stdout:3/388: mknod d5/d16/d56/c74 0 2024-02-22T15:50:39.926 INFO:tasks.workunit.client.0.smithi019.stdout:0/549: rename d4/db/d24/d80/daf/d95 to d4/db/d49/dc2 0 2024-02-22T15:50:39.926 INFO:tasks.workunit.client.1.smithi062.stdout:0/523: write d1/d3/d4/d1e/d74/f70 [367725,56682] 0 2024-02-22T15:50:39.928 INFO:tasks.workunit.client.1.smithi062.stdout:7/364: dread d0/d14/d19/d30/d37/f63 [0,4194304] 0 2024-02-22T15:50:39.929 INFO:tasks.workunit.client.1.smithi062.stdout:2/477: dwrite d1/d8/fa [0,4194304] 0 2024-02-22T15:50:39.934 INFO:tasks.workunit.client.0.smithi019.stdout:4/602: dwrite d2/db/d14/d23/d21/f29 [0,4194304] 0 2024-02-22T15:50:39.935 INFO:tasks.workunit.client.0.smithi019.stdout:8/607: dread d9/d14/d17/f4a [0,4194304] 0 2024-02-22T15:50:39.936 INFO:tasks.workunit.client.0.smithi019.stdout:8/608: chown d9/d14/d17/d5d/lc4 385489771 1 2024-02-22T15:50:39.938 INFO:tasks.workunit.client.1.smithi062.stdout:0/524: rmdir d1/d3/d4/d1a/d76/d8c/d2c/db5 39 2024-02-22T15:50:39.938 INFO:tasks.workunit.client.1.smithi062.stdout:7/365: fdatasync d0/d6/f23 0 2024-02-22T15:50:39.938 INFO:tasks.workunit.client.0.smithi019.stdout:8/609: chown d9/d14/d17/f70 601763559 1 2024-02-22T15:50:39.939 INFO:tasks.workunit.client.1.smithi062.stdout:4/461: dwrite d4/df/f14 [8388608,4194304] 0 2024-02-22T15:50:39.942 INFO:tasks.workunit.client.1.smithi062.stdout:7/366: write d0/d6/f50 [1025970,98208] 0 2024-02-22T15:50:39.942 INFO:tasks.workunit.client.0.smithi019.stdout:8/610: dread - d9/d14/d6f/d7e/d94/db8/faf zero size 2024-02-22T15:50:39.945 INFO:tasks.workunit.client.1.smithi062.stdout:9/398: mknod d4/d35/d3f/c6e 0 2024-02-22T15:50:39.945 INFO:tasks.workunit.client.1.smithi062.stdout:3/389: mkdir d5/d16/d21/d32/d75 0 2024-02-22T15:50:39.950 INFO:tasks.workunit.client.1.smithi062.stdout:3/390: fdatasync d5/d26/f6c 0 2024-02-22T15:50:39.950 INFO:tasks.workunit.client.0.smithi019.stdout:8/611: unlink d9/ce 0 2024-02-22T15:50:39.950 INFO:tasks.workunit.client.0.smithi019.stdout:8/612: stat d9/d14/l6b 0 2024-02-22T15:50:39.951 INFO:tasks.workunit.client.0.smithi019.stdout:8/613: unlink d9/d14/d6f/d7e/d94/d9a/lb0 0 2024-02-22T15:50:39.952 INFO:tasks.workunit.client.0.smithi019.stdout:8/614: stat d9/d14/d17/d1c/l2b 0 2024-02-22T15:50:39.953 INFO:tasks.workunit.client.1.smithi062.stdout:9/399: creat d4/d15/d5f/f6f x:0 0 0 2024-02-22T15:50:39.953 INFO:tasks.workunit.client.0.smithi019.stdout:8/615: readlink d9/d14/d1f/d20/l57 0 2024-02-22T15:50:39.953 INFO:tasks.workunit.client.1.smithi062.stdout:8/407: dread d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:39.954 INFO:tasks.workunit.client.0.smithi019.stdout:8/616: write d9/d14/d6f/d7e/d94/db8/faf [331370,83879] 0 2024-02-22T15:50:39.954 INFO:tasks.workunit.client.1.smithi062.stdout:2/478: symlink d1/d47/d32/l9d 0 2024-02-22T15:50:39.954 INFO:tasks.workunit.client.1.smithi062.stdout:6/504: dwrite d5/df/d40/d44/d76/d56/f98 [0,4194304] 0 2024-02-22T15:50:39.957 INFO:tasks.workunit.client.1.smithi062.stdout:1/423: link d1/d6/d7/d2b/d47/d63/f87 d1/d6/d7/d4a/d65/d83/f8c 0 2024-02-22T15:50:39.958 INFO:tasks.workunit.client.1.smithi062.stdout:6/505: truncate d5/df/d40/d44/d76/f52 4863608 0 2024-02-22T15:50:39.961 INFO:tasks.workunit.client.1.smithi062.stdout:3/391: write d5/d16/d21/d32/f70 [896100,40731] 0 2024-02-22T15:50:39.961 INFO:tasks.workunit.client.1.smithi062.stdout:4/462: creat d4/d2a/d2d/d9d/fa2 x:0 0 0 2024-02-22T15:50:39.961 INFO:tasks.workunit.client.1.smithi062.stdout:5/299: link d0/d3/d2f/f44 d0/d3/da/d13/d23/d4e/f5d 0 2024-02-22T15:50:39.962 INFO:tasks.workunit.client.0.smithi019.stdout:0/550: dread d4/d32/f9c [0,4194304] 0 2024-02-22T15:50:39.963 INFO:tasks.workunit.client.1.smithi062.stdout:7/367: fsync d0/d6/d2e/d4a/f4c 0 2024-02-22T15:50:39.963 INFO:tasks.workunit.client.1.smithi062.stdout:6/506: chown f2 6 1 2024-02-22T15:50:39.965 INFO:tasks.workunit.client.0.smithi019.stdout:0/551: mknod d4/db/d24/d80/daf/cc3 0 2024-02-22T15:50:39.965 INFO:tasks.workunit.client.0.smithi019.stdout:0/552: stat d4/d32/f46 0 2024-02-22T15:50:39.966 INFO:tasks.workunit.client.1.smithi062.stdout:1/424: mknod d1/c8d 0 2024-02-22T15:50:39.966 INFO:tasks.workunit.client.0.smithi019.stdout:0/553: write d4/d9/f11 [1078847,8600] 0 2024-02-22T15:50:39.967 INFO:tasks.workunit.client.1.smithi062.stdout:9/400: mknod d4/c70 0 2024-02-22T15:50:39.968 INFO:tasks.workunit.client.0.smithi019.stdout:0/554: symlink d4/db/d76/lc4 0 2024-02-22T15:50:39.968 INFO:tasks.workunit.client.1.smithi062.stdout:7/368: mkdir d0/d14/d76 0 2024-02-22T15:50:39.969 INFO:tasks.workunit.client.1.smithi062.stdout:3/392: write d5/f41 [1865583,109778] 0 2024-02-22T15:50:39.972 INFO:tasks.workunit.client.0.smithi019.stdout:0/555: mknod d4/db/d76/d77/cc5 0 2024-02-22T15:50:39.972 INFO:tasks.workunit.client.0.smithi019.stdout:0/556: dread - d4/db/d49/fb1 zero size 2024-02-22T15:50:39.973 INFO:tasks.workunit.client.0.smithi019.stdout:4/603: write d2/db/d14/d23/d4b/f82 [102677,2654] 0 2024-02-22T15:50:39.973 INFO:tasks.workunit.client.1.smithi062.stdout:1/425: read - d1/d6/d7/d2b/d49/f57 zero size 2024-02-22T15:50:39.973 INFO:tasks.workunit.client.1.smithi062.stdout:5/300: creat d0/d3/da/d13/d23/d4e/d5a/f5e x:0 0 0 2024-02-22T15:50:39.974 INFO:tasks.workunit.client.1.smithi062.stdout:9/401: write d4/d1c/f1d [666159,43322] 0 2024-02-22T15:50:39.976 INFO:tasks.workunit.client.1.smithi062.stdout:6/507: symlink d5/dd/d33/d66/la0 0 2024-02-22T15:50:39.978 INFO:tasks.workunit.client.0.smithi019.stdout:4/604: getdents d2/db/d14/d23/d21 0 2024-02-22T15:50:39.979 INFO:tasks.workunit.client.0.smithi019.stdout:0/557: dwrite d4/d9/da7/fb8 [0,4194304] 0 2024-02-22T15:50:39.981 INFO:tasks.workunit.client.1.smithi062.stdout:7/369: write d0/d6/d2e/d58/f5c [977992,66388] 0 2024-02-22T15:50:39.981 INFO:tasks.workunit.client.1.smithi062.stdout:1/426: creat d1/d6/d36/d37/f8e x:0 0 0 2024-02-22T15:50:39.982 INFO:tasks.workunit.client.1.smithi062.stdout:8/408: dwrite d8/d9/d44/d4d/d51/d55/f77 [0,4194304] 0 2024-02-22T15:50:39.983 INFO:tasks.workunit.client.1.smithi062.stdout:5/301: fdatasync d0/d3/ff 0 2024-02-22T15:50:39.985 INFO:tasks.workunit.client.0.smithi019.stdout:4/605: truncate d2/d3f/f7c 572187 0 2024-02-22T15:50:39.986 INFO:tasks.workunit.client.0.smithi019.stdout:0/558: rename d4/db/d76/d77/dab to d4/db/d76/dc6 0 2024-02-22T15:50:39.987 INFO:tasks.workunit.client.0.smithi019.stdout:4/606: write d2/db/d14/d23/d21/d35/f64 [1441345,23120] 0 2024-02-22T15:50:39.989 INFO:tasks.workunit.client.1.smithi062.stdout:6/508: creat d5/df/d32/d8f/fa1 x:0 0 0 2024-02-22T15:50:39.990 INFO:tasks.workunit.client.0.smithi019.stdout:4/607: chown d2/db/d14/d23/d21/d35/d3d 140088275 1 2024-02-22T15:50:39.990 INFO:tasks.workunit.client.0.smithi019.stdout:0/559: stat d4/db/d1b/d63/db0/da1/dbb/d52/la4 0 2024-02-22T15:50:39.991 INFO:tasks.workunit.client.1.smithi062.stdout:2/479: dread d1/f50 [8388608,4194304] 0 2024-02-22T15:50:39.992 INFO:tasks.workunit.client.0.smithi019.stdout:4/608: chown d2/db/d14/d23/d21/fb3 7 1 2024-02-22T15:50:39.992 INFO:tasks.workunit.client.0.smithi019.stdout:4/609: chown d2/db/d54/d8c 2101 1 2024-02-22T15:50:39.993 INFO:tasks.workunit.client.1.smithi062.stdout:2/480: chown d1/c70 0 1 2024-02-22T15:50:39.994 INFO:tasks.workunit.client.0.smithi019.stdout:0/560: creat d4/db/d49/d65/fc7 x:0 0 0 2024-02-22T15:50:39.994 INFO:tasks.workunit.client.1.smithi062.stdout:8/409: mkdir d8/d9/d44/d7e 0 2024-02-22T15:50:39.997 INFO:tasks.workunit.client.1.smithi062.stdout:6/509: stat d5/df/d40/d44/d4a/f28 0 2024-02-22T15:50:39.998 INFO:tasks.workunit.client.1.smithi062.stdout:5/302: read d0/d3/da/f1d [2242064,9199] 0 2024-02-22T15:50:39.999 INFO:tasks.workunit.client.1.smithi062.stdout:9/402: dwrite d4/d28/f38 [4194304,4194304] 0 2024-02-22T15:50:40.001 INFO:tasks.workunit.client.0.smithi019.stdout:0/561: dwrite d4/d22/fa9 [0,4194304] 0 2024-02-22T15:50:40.005 INFO:tasks.workunit.client.1.smithi062.stdout:9/403: chown d4/d1c/c4c 8606737 1 2024-02-22T15:50:40.005 INFO:tasks.workunit.client.1.smithi062.stdout:5/303: write d0/f16 [4953359,62685] 0 2024-02-22T15:50:40.007 INFO:tasks.workunit.client.1.smithi062.stdout:7/370: dread d0/d6/d2e/d4a/f1f [4194304,4194304] 0 2024-02-22T15:50:40.008 INFO:tasks.workunit.client.1.smithi062.stdout:0/525: dread d1/f45 [0,4194304] 0 2024-02-22T15:50:40.012 INFO:tasks.workunit.client.1.smithi062.stdout:6/510: unlink d5/df/d32/f4e 0 2024-02-22T15:50:40.013 INFO:tasks.workunit.client.1.smithi062.stdout:8/410: dread d8/d9/f27 [0,4194304] 0 2024-02-22T15:50:40.014 INFO:tasks.workunit.client.1.smithi062.stdout:7/371: rmdir d0/d6/d2e/d55 39 2024-02-22T15:50:40.015 INFO:tasks.workunit.client.1.smithi062.stdout:0/526: fdatasync d1/dd/f36 0 2024-02-22T15:50:40.016 INFO:tasks.workunit.client.1.smithi062.stdout:6/511: write d5/df/d40/d44/f7c [368902,457] 0 2024-02-22T15:50:40.016 INFO:tasks.workunit.client.1.smithi062.stdout:1/427: dwrite d1/d6/de/f42 [0,4194304] 0 2024-02-22T15:50:40.017 INFO:tasks.workunit.client.1.smithi062.stdout:7/372: readlink d0/l71 0 2024-02-22T15:50:40.017 INFO:tasks.workunit.client.0.smithi019.stdout:0/562: truncate d4/db/d1b/f2f 4046169 0 2024-02-22T15:50:40.018 INFO:tasks.workunit.client.1.smithi062.stdout:2/481: rename d1/d47/d32/d52/c58 to d1/c9e 0 2024-02-22T15:50:40.019 INFO:tasks.workunit.client.0.smithi019.stdout:4/610: truncate d2/db/d14/d23/d21/d35/f38 559699 0 2024-02-22T15:50:40.022 INFO:tasks.workunit.client.1.smithi062.stdout:0/527: write d1/d3/d4/d1a/d76/d8c/f88 [861886,7720] 0 2024-02-22T15:50:40.022 INFO:tasks.workunit.client.1.smithi062.stdout:6/512: creat d5/df/d32/d47/fa2 x:0 0 0 2024-02-22T15:50:40.023 INFO:tasks.workunit.client.1.smithi062.stdout:2/482: truncate d1/d1e/f66 1457850 0 2024-02-22T15:50:40.024 INFO:tasks.workunit.client.1.smithi062.stdout:1/428: creat d1/d6/d10/f8f x:0 0 0 2024-02-22T15:50:40.025 INFO:tasks.workunit.client.0.smithi019.stdout:4/611: mknod d2/db/d14/d23/d9e/dac/dc2/cc5 0 2024-02-22T15:50:40.026 INFO:tasks.workunit.client.0.smithi019.stdout:4/612: fdatasync d2/fd 0 2024-02-22T15:50:40.026 INFO:tasks.workunit.client.0.smithi019.stdout:0/563: dwrite d4/d22/d2c/d68/f6a [0,4194304] 0 2024-02-22T15:50:40.027 INFO:tasks.workunit.client.1.smithi062.stdout:0/528: write d1/d21/d85/d8d/fa1 [343316,127516] 0 2024-02-22T15:50:40.027 INFO:tasks.workunit.client.1.smithi062.stdout:7/373: mkdir d0/d14/d19/d77 0 2024-02-22T15:50:40.028 INFO:tasks.workunit.client.1.smithi062.stdout:1/429: dread - d1/d6/d14/f6d zero size 2024-02-22T15:50:40.028 INFO:tasks.workunit.client.0.smithi019.stdout:4/613: fdatasync d2/db/f25 0 2024-02-22T15:50:40.029 INFO:tasks.workunit.client.1.smithi062.stdout:2/483: write d1/d28/d3f/d65/f7e [844846,28324] 0 2024-02-22T15:50:40.031 INFO:tasks.workunit.client.0.smithi019.stdout:4/614: symlink d2/db/d14/d23/d67/d8f/lc6 0 2024-02-22T15:50:40.034 INFO:tasks.workunit.client.0.smithi019.stdout:4/615: write d2/db/d14/f81 [113406,84740] 0 2024-02-22T15:50:40.034 INFO:tasks.workunit.client.1.smithi062.stdout:7/374: write d0/d14/d13/f4d [141503,97601] 0 2024-02-22T15:50:40.037 INFO:tasks.workunit.client.1.smithi062.stdout:2/484: creat d1/d28/d3f/f9f x:0 0 0 2024-02-22T15:50:40.038 INFO:tasks.workunit.client.0.smithi019.stdout:0/564: dwrite d4/d22/f51 [0,4194304] 0 2024-02-22T15:50:40.041 INFO:tasks.workunit.client.1.smithi062.stdout:1/430: mknod d1/c90 0 2024-02-22T15:50:40.042 INFO:tasks.workunit.client.0.smithi019.stdout:7/904: sync 2024-02-22T15:50:40.042 INFO:tasks.workunit.client.0.smithi019.stdout:7/905: chown d9/d4c/ddf 5544 1 2024-02-22T15:50:40.045 INFO:tasks.workunit.client.0.smithi019.stdout:0/565: dread d4/db/d76/d77/faa [0,4194304] 0 2024-02-22T15:50:40.046 INFO:tasks.workunit.client.0.smithi019.stdout:0/566: dread - d4/d23/d7d/fbf zero size 2024-02-22T15:50:40.046 INFO:tasks.workunit.client.1.smithi062.stdout:5/304: write d0/d3/f14 [3888689,64085] 0 2024-02-22T15:50:40.046 INFO:tasks.workunit.client.0.smithi019.stdout:4/616: link d2/db/d14/d23/fc3 d2/db/d14/fc7 0 2024-02-22T15:50:40.047 INFO:tasks.workunit.client.1.smithi062.stdout:6/513: dwrite d5/df/d40/d44/d4a/d41/f72 [0,4194304] 0 2024-02-22T15:50:40.047 INFO:tasks.workunit.client.0.smithi019.stdout:7/906: rmdir d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108 39 2024-02-22T15:50:40.049 INFO:tasks.workunit.client.0.smithi019.stdout:0/567: write d4/db/d1b/d40/d44/d75/f82 [1116265,16350] 0 2024-02-22T15:50:40.051 INFO:tasks.workunit.client.1.smithi062.stdout:1/431: creat d1/d6/d7/f91 x:0 0 0 2024-02-22T15:50:40.051 INFO:tasks.workunit.client.1.smithi062.stdout:8/411: write d8/f24 [4288716,109242] 0 2024-02-22T15:50:40.051 INFO:tasks.workunit.client.1.smithi062.stdout:7/375: rename d0/d14/d19/c21 to d0/d6/d2e/d2f/c78 0 2024-02-22T15:50:40.055 INFO:tasks.workunit.client.0.smithi019.stdout:7/907: unlink d9/d4c/d5b/f96 0 2024-02-22T15:50:40.055 INFO:tasks.workunit.client.0.smithi019.stdout:4/617: mknod d2/db/d14/d23/d21/d35/cc8 0 2024-02-22T15:50:40.056 INFO:tasks.workunit.client.1.smithi062.stdout:6/514: chown d5/df/l14 48 1 2024-02-22T15:50:40.056 INFO:tasks.workunit.client.1.smithi062.stdout:9/404: dread d4/d28/f38 [0,4194304] 0 2024-02-22T15:50:40.056 INFO:tasks.workunit.client.1.smithi062.stdout:9/405: readlink d4/d1c/l1f 0 2024-02-22T15:50:40.057 INFO:tasks.workunit.client.0.smithi019.stdout:0/568: unlink d4/d23/d7d/fbf 0 2024-02-22T15:50:40.059 INFO:tasks.workunit.client.0.smithi019.stdout:0/569: fdatasync d4/db/d1b/f1c 0 2024-02-22T15:50:40.059 INFO:tasks.workunit.client.0.smithi019.stdout:4/618: truncate d2/db/d14/d23/d4b/f82 309755 0 2024-02-22T15:50:40.060 INFO:tasks.workunit.client.0.smithi019.stdout:0/570: rename d4/db/d1b to d4/db/d1b/d63/db0/da1/dbb/dc/dc8 22 2024-02-22T15:50:40.060 INFO:tasks.workunit.client.0.smithi019.stdout:4/619: chown d2/db/d14/d61/l89 170913 1 2024-02-22T15:50:40.061 INFO:tasks.workunit.client.0.smithi019.stdout:7/908: unlink d9/d26/d77/lbe 0 2024-02-22T15:50:40.061 INFO:tasks.workunit.client.1.smithi062.stdout:7/376: readlink d0/l71 0 2024-02-22T15:50:40.062 INFO:tasks.workunit.client.0.smithi019.stdout:7/909: read d9/d38/d41/fb7 [1275118,111355] 0 2024-02-22T15:50:40.063 INFO:tasks.workunit.client.1.smithi062.stdout:6/515: mknod d5/df/d32/ca3 0 2024-02-22T15:50:40.063 INFO:tasks.workunit.client.1.smithi062.stdout:5/305: mkdir d0/d3/d2f/d38/d3f/d5f 0 2024-02-22T15:50:40.063 INFO:tasks.workunit.client.1.smithi062.stdout:0/529: dwrite d1/d13/f31 [0,4194304] 0 2024-02-22T15:50:40.064 INFO:tasks.workunit.client.0.smithi019.stdout:0/571: chown d4/db/d76/c81 23 1 2024-02-22T15:50:40.064 INFO:tasks.workunit.client.0.smithi019.stdout:7/910: read - d9/d26/f101 zero size 2024-02-22T15:50:40.065 INFO:tasks.workunit.client.0.smithi019.stdout:4/620: mkdir d2/db/d14/d23/d9e/dac/dc2/dc9 0 2024-02-22T15:50:40.067 INFO:tasks.workunit.client.1.smithi062.stdout:9/406: write d4/f29 [1785515,24648] 0 2024-02-22T15:50:40.068 INFO:tasks.workunit.client.1.smithi062.stdout:8/412: link d8/cc d8/c7f 0 2024-02-22T15:50:40.069 INFO:tasks.workunit.client.0.smithi019.stdout:4/621: symlink d2/db/d14/d23/d33/db1/lca 0 2024-02-22T15:50:40.069 INFO:tasks.workunit.client.0.smithi019.stdout:7/911: symlink d9/d3a/l12b 0 2024-02-22T15:50:40.071 INFO:tasks.workunit.client.1.smithi062.stdout:5/306: mkdir d0/d3/da/d13/d23/d4e/d60 0 2024-02-22T15:50:40.072 INFO:tasks.workunit.client.0.smithi019.stdout:4/622: stat d2/db/d14/d23/d21/l2e 0 2024-02-22T15:50:40.073 INFO:tasks.workunit.client.1.smithi062.stdout:2/485: dwrite d1/d8/f1a [0,4194304] 0 2024-02-22T15:50:40.074 INFO:tasks.workunit.client.0.smithi019.stdout:0/572: dwrite d4/db/d1b/d63/f70 [0,4194304] 0 2024-02-22T15:50:40.075 INFO:tasks.workunit.client.0.smithi019.stdout:7/912: getdents d9/d38/df3 0 2024-02-22T15:50:40.076 INFO:tasks.workunit.client.1.smithi062.stdout:8/413: write d8/f20 [4805480,6796] 0 2024-02-22T15:50:40.077 INFO:tasks.workunit.client.0.smithi019.stdout:0/573: write d4/d22/d2c/f57 [5036951,8413] 0 2024-02-22T15:50:40.081 INFO:tasks.workunit.client.1.smithi062.stdout:2/486: write d1/d8/f3d [5714,168] 0 2024-02-22T15:50:40.082 INFO:tasks.workunit.client.1.smithi062.stdout:7/377: dread d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:40.083 INFO:tasks.workunit.client.1.smithi062.stdout:5/307: chown d0/d3/da/d13/d57 1264 1 2024-02-22T15:50:40.084 INFO:tasks.workunit.client.1.smithi062.stdout:1/432: dwrite d1/d6/d10/f8f [0,4194304] 0 2024-02-22T15:50:40.087 INFO:tasks.workunit.client.1.smithi062.stdout:0/530: rename d1/d3/d4/d1a/f72 to d1/d3/d4/d1a/d76/d8c/d2c/d87/fb9 0 2024-02-22T15:50:40.088 INFO:tasks.workunit.client.1.smithi062.stdout:6/516: truncate d5/df/d40/d44/f6b 930135 0 2024-02-22T15:50:40.088 INFO:tasks.workunit.client.0.smithi019.stdout:7/913: dread d9/d38/d41/d45/db0/df7/f100 [0,4194304] 0 2024-02-22T15:50:40.089 INFO:tasks.workunit.client.1.smithi062.stdout:6/517: dread - d5/df/d40/d44/d4a/d41/f5f zero size 2024-02-22T15:50:40.090 INFO:tasks.workunit.client.1.smithi062.stdout:2/487: unlink d1/d47/c3b 0 2024-02-22T15:50:40.090 INFO:tasks.workunit.client.1.smithi062.stdout:1/433: write d1/d6/d7/d4a/d65/d83/f8b [740362,21220] 0 2024-02-22T15:50:40.092 INFO:tasks.workunit.client.1.smithi062.stdout:8/414: mknod d8/d9/d44/c80 0 2024-02-22T15:50:40.093 INFO:tasks.workunit.client.0.smithi019.stdout:7/914: getdents d9/d4c/d5b 0 2024-02-22T15:50:40.101 INFO:tasks.workunit.client.1.smithi062.stdout:2/488: mknod d1/d47/d32/d52/ca0 0 2024-02-22T15:50:40.101 INFO:tasks.workunit.client.1.smithi062.stdout:6/518: creat d5/dd/d33/d66/fa4 x:0 0 0 2024-02-22T15:50:40.102 INFO:tasks.workunit.client.1.smithi062.stdout:1/434: rename d1/d6/d7/d2b/f5b to d1/d6/d7/d2b/d47/f92 0 2024-02-22T15:50:40.104 INFO:tasks.workunit.client.1.smithi062.stdout:8/415: dwrite d8/f29 [0,4194304] 0 2024-02-22T15:50:40.104 INFO:tasks.workunit.client.1.smithi062.stdout:5/308: rmdir d0/d3/d2f/d38/d3f/d56 0 2024-02-22T15:50:40.107 INFO:tasks.workunit.client.1.smithi062.stdout:2/489: rmdir d1/d28 39 2024-02-22T15:50:40.109 INFO:tasks.workunit.client.1.smithi062.stdout:6/519: mknod d5/df/d40/d44/d4a/d41/ca5 0 2024-02-22T15:50:40.109 INFO:tasks.workunit.client.1.smithi062.stdout:8/416: dread - d8/f7a zero size 2024-02-22T15:50:40.111 INFO:tasks.workunit.client.1.smithi062.stdout:0/531: dwrite d1/d21/d85/d8e/d97/fb7 [0,4194304] 0 2024-02-22T15:50:40.114 INFO:tasks.workunit.client.1.smithi062.stdout:1/435: rename d1/d6/d10/c43 to d1/d6/d7/d4a/c93 0 2024-02-22T15:50:40.114 INFO:tasks.workunit.client.0.smithi019.stdout:0/574: write d4/db/d1b/d63/db0/da1/dbb/d52/f58 [3868660,8421] 0 2024-02-22T15:50:40.115 INFO:tasks.workunit.client.1.smithi062.stdout:8/417: dread d8/f3a [0,4194304] 0 2024-02-22T15:50:40.116 INFO:tasks.workunit.client.0.smithi019.stdout:0/575: stat d4/d32/f46 0 2024-02-22T15:50:40.116 INFO:tasks.workunit.client.1.smithi062.stdout:5/309: mknod d0/d3/da/d13/d23/d4e/c61 0 2024-02-22T15:50:40.116 INFO:tasks.workunit.client.0.smithi019.stdout:0/576: stat d4/f5c 0 2024-02-22T15:50:40.118 INFO:tasks.workunit.client.1.smithi062.stdout:0/532: truncate d1/d3/d4/d1e/d74/f70 829308 0 2024-02-22T15:50:40.118 INFO:tasks.workunit.client.1.smithi062.stdout:9/407: truncate d4/f26 642301 0 2024-02-22T15:50:40.118 INFO:tasks.workunit.client.0.smithi019.stdout:7/915: dwrite d9/d26/f31 [0,4194304] 0 2024-02-22T15:50:40.119 INFO:tasks.workunit.client.1.smithi062.stdout:1/436: mkdir d1/d6/d7/d2b/d94 0 2024-02-22T15:50:40.122 INFO:tasks.workunit.client.1.smithi062.stdout:0/533: write d1/d3/f39 [2037729,91304] 0 2024-02-22T15:50:40.124 INFO:tasks.workunit.client.1.smithi062.stdout:5/310: mknod d0/c62 0 2024-02-22T15:50:40.125 INFO:tasks.workunit.client.1.smithi062.stdout:8/418: write d8/d28/f35 [241888,126350] 0 2024-02-22T15:50:40.125 INFO:tasks.workunit.client.0.smithi019.stdout:7/916: creat d9/d38/d41/d45/db0/df7/dfb/f12c x:0 0 0 2024-02-22T15:50:40.125 INFO:tasks.workunit.client.0.smithi019.stdout:0/577: truncate d4/db/d1b/d63/db0/da1/dbb/f39 4777855 0 2024-02-22T15:50:40.128 INFO:tasks.workunit.client.1.smithi062.stdout:8/419: dread - d8/d9/f5f zero size 2024-02-22T15:50:40.129 INFO:tasks.workunit.client.0.smithi019.stdout:7/917: link d9/d4c/f104 d9/d38/d41/f12d 0 2024-02-22T15:50:40.130 INFO:tasks.workunit.client.1.smithi062.stdout:5/311: write d0/d3/f14 [4114337,125936] 0 2024-02-22T15:50:40.131 INFO:tasks.workunit.client.0.smithi019.stdout:0/578: link d4/f5c d4/d22/d2c/d68/d6d/fc9 0 2024-02-22T15:50:40.131 INFO:tasks.workunit.client.1.smithi062.stdout:2/490: creat d1/d47/d32/fa1 x:0 0 0 2024-02-22T15:50:40.131 INFO:tasks.workunit.client.1.smithi062.stdout:9/408: mknod d4/d15/d5f/d6a/c71 0 2024-02-22T15:50:40.133 INFO:tasks.workunit.client.1.smithi062.stdout:0/534: creat d1/d21/d85/d8d/fba x:0 0 0 2024-02-22T15:50:40.133 INFO:tasks.workunit.client.1.smithi062.stdout:9/409: write f2 [1854714,55752] 0 2024-02-22T15:50:40.134 INFO:tasks.workunit.client.0.smithi019.stdout:0/579: write d4/db/d1b/d63/db0/da1/dbb/f39 [877707,60394] 0 2024-02-22T15:50:40.134 INFO:tasks.workunit.client.1.smithi062.stdout:7/378: dwrite d0/d14/d19/d30/d37/f6b [0,4194304] 0 2024-02-22T15:50:40.138 INFO:tasks.workunit.client.1.smithi062.stdout:8/420: write d8/d9/d44/d4d/d51/f68 [2791962,67905] 0 2024-02-22T15:50:40.139 INFO:tasks.workunit.client.0.smithi019.stdout:7/918: dwrite d9/d38/df3/f113 [0,4194304] 0 2024-02-22T15:50:40.140 INFO:tasks.workunit.client.0.smithi019.stdout:0/580: rename d4/d9/f11 to d4/db/d76/d77/fca 0 2024-02-22T15:50:40.142 INFO:tasks.workunit.client.1.smithi062.stdout:0/535: unlink d1/d3/d4/d1a/c93 0 2024-02-22T15:50:40.142 INFO:tasks.workunit.client.1.smithi062.stdout:8/421: chown d8/d9/f37 16054 1 2024-02-22T15:50:40.142 INFO:tasks.workunit.client.1.smithi062.stdout:9/410: truncate d4/d15/d31/f60 986649 0 2024-02-22T15:50:40.142 INFO:tasks.workunit.client.1.smithi062.stdout:9/411: dread - d4/f4f zero size 2024-02-22T15:50:40.145 INFO:tasks.workunit.client.1.smithi062.stdout:2/491: mknod d1/d1e/d38/d55/ca2 0 2024-02-22T15:50:40.146 INFO:tasks.workunit.client.1.smithi062.stdout:5/312: dread d0/f39 [0,4194304] 0 2024-02-22T15:50:40.148 INFO:tasks.workunit.client.1.smithi062.stdout:2/492: write d1/d8/f3d [1674204,94287] 0 2024-02-22T15:50:40.150 INFO:tasks.workunit.client.1.smithi062.stdout:7/379: symlink d0/d4f/l79 0 2024-02-22T15:50:40.152 INFO:tasks.workunit.client.0.smithi019.stdout:7/919: dwrite d9/d4c/d5b/f65 [4194304,4194304] 0 2024-02-22T15:50:40.152 INFO:tasks.workunit.client.1.smithi062.stdout:0/536: mknod d1/d21/d85/cbb 0 2024-02-22T15:50:40.155 INFO:tasks.workunit.client.1.smithi062.stdout:8/422: symlink d8/l81 0 2024-02-22T15:50:40.157 INFO:tasks.workunit.client.0.smithi019.stdout:0/581: dwrite d4/db/d1b/d63/db0/da1/dbb/d52/f62 [0,4194304] 0 2024-02-22T15:50:40.158 INFO:tasks.workunit.client.1.smithi062.stdout:2/493: symlink d1/d47/d32/d52/d80/la3 0 2024-02-22T15:50:40.159 INFO:tasks.workunit.client.1.smithi062.stdout:6/520: dread d5/f7 [4194304,4194304] 0 2024-02-22T15:50:40.160 INFO:tasks.workunit.client.1.smithi062.stdout:0/537: dread - d1/d3/d4/d1a/d76/d8c/d2c/d87/fa2 zero size 2024-02-22T15:50:40.160 INFO:tasks.workunit.client.0.smithi019.stdout:7/920: rename d9/f116 to d9/d26/dc4/f12e 0 2024-02-22T15:50:40.162 INFO:tasks.workunit.client.0.smithi019.stdout:7/921: read d9/d4c/ddf/f109 [72645,36133] 0 2024-02-22T15:50:40.163 INFO:tasks.workunit.client.1.smithi062.stdout:9/412: dwrite d4/d15/d3c/f3e [0,4194304] 0 2024-02-22T15:50:40.163 INFO:tasks.workunit.client.1.smithi062.stdout:6/521: fdatasync d5/df/d40/d44/d76/f8b 0 2024-02-22T15:50:40.166 INFO:tasks.workunit.client.0.smithi019.stdout:7/922: rename d9/d26/f101 to d9/d38/d41/d45/db0/df7/dfb/de3/f12f 0 2024-02-22T15:50:40.167 INFO:tasks.workunit.client.0.smithi019.stdout:7/923: chown d9/c2a 823929800 1 2024-02-22T15:50:40.169 INFO:tasks.workunit.client.1.smithi062.stdout:5/313: link d0/f39 d0/d3/da/d13/d23/d4e/d60/f63 0 2024-02-22T15:50:40.169 INFO:tasks.workunit.client.1.smithi062.stdout:8/423: creat d8/d9/d44/d4d/f82 x:0 0 0 2024-02-22T15:50:40.169 INFO:tasks.workunit.client.0.smithi019.stdout:7/924: truncate d9/d38/d41/d45/d92/f11e 1247591 0 2024-02-22T15:50:40.171 INFO:tasks.workunit.client.1.smithi062.stdout:2/494: rename d1/d1e/d38/d55/l79 to d1/d47/d32/d52/d80/la4 0 2024-02-22T15:50:40.172 INFO:tasks.workunit.client.1.smithi062.stdout:8/424: stat d8/f22 0 2024-02-22T15:50:40.172 INFO:tasks.workunit.client.1.smithi062.stdout:7/380: dwrite d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:40.175 INFO:tasks.workunit.client.1.smithi062.stdout:9/413: truncate d4/d1c/f1e 282562 0 2024-02-22T15:50:40.175 INFO:tasks.workunit.client.1.smithi062.stdout:5/314: mknod d0/d3/da/d13/d23/c64 0 2024-02-22T15:50:40.175 INFO:tasks.workunit.client.1.smithi062.stdout:9/414: chown d4/d1c 141135576 1 2024-02-22T15:50:40.178 INFO:tasks.workunit.client.1.smithi062.stdout:2/495: write d1/d28/f5c [2529670,102022] 0 2024-02-22T15:50:40.178 INFO:tasks.workunit.client.1.smithi062.stdout:7/381: write d0/f10 [703766,76416] 0 2024-02-22T15:50:40.181 INFO:tasks.workunit.client.1.smithi062.stdout:5/315: getdents d0/d3/d2f/d38/d3f/d5f 0 2024-02-22T15:50:40.185 INFO:tasks.workunit.client.1.smithi062.stdout:8/425: dwrite d8/d9/f18 [0,4194304] 0 2024-02-22T15:50:40.186 INFO:tasks.workunit.client.1.smithi062.stdout:7/382: dread d0/d14/d13/f2a [0,4194304] 0 2024-02-22T15:50:40.191 INFO:tasks.workunit.client.1.smithi062.stdout:7/383: mknod d0/d6/d2e/d4a/d5a/d5e/c7a 0 2024-02-22T15:50:40.193 INFO:tasks.workunit.client.1.smithi062.stdout:5/316: dwrite d0/f45 [0,4194304] 0 2024-02-22T15:50:40.195 INFO:tasks.workunit.client.1.smithi062.stdout:7/384: unlink d0/d14/f25 0 2024-02-22T15:50:40.197 INFO:tasks.workunit.client.1.smithi062.stdout:5/317: creat d0/d3/da/d32/d42/f65 x:0 0 0 2024-02-22T15:50:40.197 INFO:tasks.workunit.client.1.smithi062.stdout:7/385: readlink d0/d14/d28/l75 0 2024-02-22T15:50:40.199 INFO:tasks.workunit.client.1.smithi062.stdout:7/386: rmdir d0/d6/d5b/d60 39 2024-02-22T15:50:40.200 INFO:tasks.workunit.client.1.smithi062.stdout:5/318: truncate d0/d3/d2f/d38/f41 610010 0 2024-02-22T15:50:40.201 INFO:tasks.workunit.client.1.smithi062.stdout:5/319: chown d0/d3/d2f/d38/f41 3455648 1 2024-02-22T15:50:40.202 INFO:tasks.workunit.client.0.smithi019.stdout:7/925: rename d9/d38/d41/d45/db0/df7/dfb/de3/deb/d108 to d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130 0 2024-02-22T15:50:40.207 INFO:tasks.workunit.client.1.smithi062.stdout:7/387: dwrite d0/d6/d2e/d4a/f1f [0,4194304] 0 2024-02-22T15:50:40.208 INFO:tasks.workunit.client.1.smithi062.stdout:5/320: readlink d0/d3/da/d13/l29 0 2024-02-22T15:50:40.209 INFO:tasks.workunit.client.0.smithi019.stdout:7/926: dwrite d9/d38/d41/d45/d92/ffa [0,4194304] 0 2024-02-22T15:50:40.210 INFO:tasks.workunit.client.1.smithi062.stdout:5/321: write d0/d3/da/d13/f3d [129244,121146] 0 2024-02-22T15:50:40.212 INFO:tasks.workunit.client.0.smithi019.stdout:7/927: chown d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130 17127932 1 2024-02-22T15:50:40.213 INFO:tasks.workunit.client.1.smithi062.stdout:8/426: fdatasync d8/d9/f18 0 2024-02-22T15:50:40.214 INFO:tasks.workunit.client.1.smithi062.stdout:7/388: creat d0/d14/d19/f7b x:0 0 0 2024-02-22T15:50:40.215 INFO:tasks.workunit.client.1.smithi062.stdout:0/538: truncate d1/d3/de/ff 6232411 0 2024-02-22T15:50:40.219 INFO:tasks.workunit.client.1.smithi062.stdout:5/322: mkdir d0/d3/d2f/d38/d3f/d66 0 2024-02-22T15:50:40.220 INFO:tasks.workunit.client.1.smithi062.stdout:0/539: write d1/d3/d4/d1a/d76/d8c/f4c [1582660,59576] 0 2024-02-22T15:50:40.221 INFO:tasks.workunit.client.1.smithi062.stdout:8/427: chown d8/d9/c25 0 1 2024-02-22T15:50:40.222 INFO:tasks.workunit.client.0.smithi019.stdout:7/928: rename d9/d4c/d5b/ff2 to d9/d3a/f131 0 2024-02-22T15:50:40.223 INFO:tasks.workunit.client.1.smithi062.stdout:5/323: fdatasync d0/d3/d30/f4a 0 2024-02-22T15:50:40.224 INFO:tasks.workunit.client.1.smithi062.stdout:0/540: creat d1/d21/d85/d8e/d97/fbc x:0 0 0 2024-02-22T15:50:40.224 INFO:tasks.workunit.client.1.smithi062.stdout:7/389: chown d0/d14/f20 37250699 1 2024-02-22T15:50:40.227 INFO:tasks.workunit.client.1.smithi062.stdout:7/390: write d0/f10 [2753208,69198] 0 2024-02-22T15:50:40.228 INFO:tasks.workunit.client.1.smithi062.stdout:0/541: symlink d1/d21/d83/lbd 0 2024-02-22T15:50:40.229 INFO:tasks.workunit.client.1.smithi062.stdout:8/428: write d8/f53 [2212296,131023] 0 2024-02-22T15:50:40.230 INFO:tasks.workunit.client.0.smithi019.stdout:7/929: dwrite d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f124 [0,4194304] 0 2024-02-22T15:50:40.231 INFO:tasks.workunit.client.1.smithi062.stdout:5/324: rmdir d0/d3/da/d13/d23/d4e 39 2024-02-22T15:50:40.232 INFO:tasks.workunit.client.1.smithi062.stdout:2/496: dwrite d1/d47/f88 [0,4194304] 0 2024-02-22T15:50:40.232 INFO:tasks.workunit.client.1.smithi062.stdout:0/542: read - d1/d38/f98 zero size 2024-02-22T15:50:40.233 INFO:tasks.workunit.client.0.smithi019.stdout:7/930: read d9/d26/f29 [530361,87706] 0 2024-02-22T15:50:40.234 INFO:tasks.workunit.client.1.smithi062.stdout:8/429: dread d8/f3a [0,4194304] 0 2024-02-22T15:50:40.237 INFO:tasks.workunit.client.0.smithi019.stdout:7/931: symlink d9/d38/d41/d45/l132 0 2024-02-22T15:50:40.239 INFO:tasks.workunit.client.0.smithi019.stdout:7/932: fdatasync d9/f16 0 2024-02-22T15:50:40.244 INFO:tasks.workunit.client.0.smithi019.stdout:7/933: mkdir d9/d38/d41/d45/db0/df7/dfb/d12a/d5d/d133 0 2024-02-22T15:50:40.245 INFO:tasks.workunit.client.1.smithi062.stdout:2/497: unlink d1/d1e/d38/f5f 0 2024-02-22T15:50:40.245 INFO:tasks.workunit.client.1.smithi062.stdout:2/498: chown d1/d1e/f94 0 1 2024-02-22T15:50:40.245 INFO:tasks.workunit.client.1.smithi062.stdout:5/325: mkdir d0/d3/d2f/d59/d67 0 2024-02-22T15:50:40.246 INFO:tasks.workunit.client.1.smithi062.stdout:8/430: chown d8/d9/l66 73702 1 2024-02-22T15:50:40.246 INFO:tasks.workunit.client.1.smithi062.stdout:5/326: fdatasync d0/d3/da/d13/d23/f49 0 2024-02-22T15:50:40.248 INFO:tasks.workunit.client.0.smithi019.stdout:2/573: sync 2024-02-22T15:50:40.248 INFO:tasks.workunit.client.0.smithi019.stdout:1/658: sync 2024-02-22T15:50:40.248 INFO:tasks.workunit.client.1.smithi062.stdout:7/391: dread d0/d6/f52 [0,4194304] 0 2024-02-22T15:50:40.249 INFO:tasks.workunit.client.1.smithi062.stdout:8/431: mknod d8/d28/c83 0 2024-02-22T15:50:40.249 INFO:tasks.workunit.client.0.smithi019.stdout:1/659: dread - d8/d15/d5b/fb2 zero size 2024-02-22T15:50:40.249 INFO:tasks.workunit.client.1.smithi062.stdout:5/327: chown d0/d3/d30/f4a 60 1 2024-02-22T15:50:40.249 INFO:tasks.workunit.client.1.smithi062.stdout:2/499: dwrite d1/d1e/d38/f54 [0,4194304] 0 2024-02-22T15:50:40.251 INFO:tasks.workunit.client.1.smithi062.stdout:8/432: write d8/d9/f5f [484381,24488] 0 2024-02-22T15:50:40.257 INFO:tasks.workunit.client.1.smithi062.stdout:8/433: dread d8/f3a [0,4194304] 0 2024-02-22T15:50:40.260 INFO:tasks.workunit.client.0.smithi019.stdout:1/660: dwrite d8/d18/d85/f98 [4194304,4194304] 0 2024-02-22T15:50:40.263 INFO:tasks.workunit.client.0.smithi019.stdout:1/661: dread - d8/d18/fda zero size 2024-02-22T15:50:40.264 INFO:tasks.workunit.client.0.smithi019.stdout:2/574: dwrite d1/d17/f90 [0,4194304] 0 2024-02-22T15:50:40.264 INFO:tasks.workunit.client.1.smithi062.stdout:8/434: mknod d8/d9/d44/d4d/d51/d55/c84 0 2024-02-22T15:50:40.264 INFO:tasks.workunit.client.1.smithi062.stdout:2/500: dwrite d1/d8/d62/f9c [0,4194304] 0 2024-02-22T15:50:40.266 INFO:tasks.workunit.client.1.smithi062.stdout:2/501: dread - d1/d8/d8f/f95 zero size 2024-02-22T15:50:40.267 INFO:tasks.workunit.client.0.smithi019.stdout:2/575: creat d1/de/d9a/fd7 x:0 0 0 2024-02-22T15:50:40.267 INFO:tasks.workunit.client.0.smithi019.stdout:7/934: dwrite d9/d38/d41/d45/d61/d6a/f6d [0,4194304] 0 2024-02-22T15:50:40.270 INFO:tasks.workunit.client.0.smithi019.stdout:7/935: fsync d9/d38/d41/d45/d92/f11e 0 2024-02-22T15:50:40.271 INFO:tasks.workunit.client.0.smithi019.stdout:2/576: mknod d1/d46/cd8 0 2024-02-22T15:50:40.275 INFO:tasks.workunit.client.0.smithi019.stdout:2/577: creat d1/d17/d38/fd9 x:0 0 0 2024-02-22T15:50:40.277 INFO:tasks.workunit.client.1.smithi062.stdout:5/328: write d0/d3/ff [323322,63762] 0 2024-02-22T15:50:40.277 INFO:tasks.workunit.client.0.smithi019.stdout:2/578: fsync d1/d17/d21/d51/fcc 0 2024-02-22T15:50:40.280 INFO:tasks.workunit.client.0.smithi019.stdout:7/936: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f134 x:0 0 0 2024-02-22T15:50:40.282 INFO:tasks.workunit.client.0.smithi019.stdout:2/579: write d1/db4/fca [160022,2967] 0 2024-02-22T15:50:40.283 INFO:tasks.workunit.client.0.smithi019.stdout:2/580: dread - d1/d17/fb7 zero size 2024-02-22T15:50:40.285 INFO:tasks.workunit.client.1.smithi062.stdout:8/435: link d8/d26/c50 d8/d9/c85 0 2024-02-22T15:50:40.285 INFO:tasks.workunit.client.1.smithi062.stdout:2/502: mknod d1/d8/ca5 0 2024-02-22T15:50:40.286 INFO:tasks.workunit.client.0.smithi019.stdout:7/937: dread d9/d26/f31 [0,4194304] 0 2024-02-22T15:50:40.288 INFO:tasks.workunit.client.1.smithi062.stdout:7/392: dwrite d0/d14/f29 [0,4194304] 0 2024-02-22T15:50:40.288 INFO:tasks.workunit.client.0.smithi019.stdout:7/938: fsync d9/d26/dc4/f12e 0 2024-02-22T15:50:40.289 INFO:tasks.workunit.client.1.smithi062.stdout:2/503: truncate d1/d47/f48 429345 0 2024-02-22T15:50:40.290 INFO:tasks.workunit.client.1.smithi062.stdout:5/329: truncate d0/d3/da/d13/d23/d4e/d5a/f5e 756401 0 2024-02-22T15:50:40.290 INFO:tasks.workunit.client.1.smithi062.stdout:8/436: dread d8/d9/f71 [0,4194304] 0 2024-02-22T15:50:40.291 INFO:tasks.workunit.client.1.smithi062.stdout:2/504: write d1/d28/f5c [454815,22585] 0 2024-02-22T15:50:40.292 INFO:tasks.workunit.client.0.smithi019.stdout:7/939: fsync d9/d38/f52 0 2024-02-22T15:50:40.293 INFO:tasks.workunit.client.0.smithi019.stdout:2/581: dwrite d1/db4/fca [0,4194304] 0 2024-02-22T15:50:40.294 INFO:tasks.workunit.client.1.smithi062.stdout:2/505: mkdir d1/d8/d62/d73/da6 0 2024-02-22T15:50:40.296 INFO:tasks.workunit.client.1.smithi062.stdout:8/437: mkdir d8/d86 0 2024-02-22T15:50:40.298 INFO:tasks.workunit.client.1.smithi062.stdout:2/506: symlink d1/d47/d39/d5d/la7 0 2024-02-22T15:50:40.299 INFO:tasks.workunit.client.1.smithi062.stdout:5/330: creat d0/d3/da/f68 x:0 0 0 2024-02-22T15:50:40.302 INFO:tasks.workunit.client.0.smithi019.stdout:2/582: link c0 d1/d17/d1c/d7f/cda 0 2024-02-22T15:50:40.303 INFO:tasks.workunit.client.0.smithi019.stdout:2/583: chown d1/d17/d21/d71/ca6 1025853889 1 2024-02-22T15:50:40.303 INFO:tasks.workunit.client.1.smithi062.stdout:5/331: rmdir d0/d3/da/d13/d23 39 2024-02-22T15:50:40.304 INFO:tasks.workunit.client.1.smithi062.stdout:5/332: readlink d0/d3/ld 0 2024-02-22T15:50:40.305 INFO:tasks.workunit.client.1.smithi062.stdout:8/438: dwrite d8/f29 [0,4194304] 0 2024-02-22T15:50:40.305 INFO:tasks.workunit.client.0.smithi019.stdout:2/584: mkdir d1/d46/ddb 0 2024-02-22T15:50:40.306 INFO:tasks.workunit.client.0.smithi019.stdout:5/647: sync 2024-02-22T15:50:40.306 INFO:tasks.workunit.client.0.smithi019.stdout:6/781: sync 2024-02-22T15:50:40.306 INFO:tasks.workunit.client.0.smithi019.stdout:3/494: sync 2024-02-22T15:50:40.306 INFO:tasks.workunit.client.0.smithi019.stdout:4/623: sync 2024-02-22T15:50:40.306 INFO:tasks.workunit.client.0.smithi019.stdout:8/617: sync 2024-02-22T15:50:40.306 INFO:tasks.workunit.client.0.smithi019.stdout:9/616: sync 2024-02-22T15:50:40.308 INFO:tasks.workunit.client.0.smithi019.stdout:5/648: readlink d8/dd/d19/l26 0 2024-02-22T15:50:40.309 INFO:tasks.workunit.client.1.smithi062.stdout:5/333: rename d0/d3/d2f/d38 to d0/d3/d2f/d59/d67/d69 0 2024-02-22T15:50:40.310 INFO:tasks.workunit.client.0.smithi019.stdout:4/624: chown d2/c8d 18 1 2024-02-22T15:50:40.312 INFO:tasks.workunit.client.0.smithi019.stdout:3/495: creat d9/d14/f9b x:0 0 0 2024-02-22T15:50:40.313 INFO:tasks.workunit.client.0.smithi019.stdout:6/782: creat d6/d31/d69/ffa x:0 0 0 2024-02-22T15:50:40.314 INFO:tasks.workunit.client.0.smithi019.stdout:7/940: write d9/d4c/f104 [581425,108273] 0 2024-02-22T15:50:40.315 INFO:tasks.workunit.client.0.smithi019.stdout:3/496: chown d9/d26/d2d/c78 324 1 2024-02-22T15:50:40.315 INFO:tasks.workunit.client.1.smithi062.stdout:8/439: mkdir d8/d42/d87 0 2024-02-22T15:50:40.319 INFO:tasks.workunit.client.0.smithi019.stdout:1/662: dwrite d8/d15/d84/fa5 [0,4194304] 0 2024-02-22T15:50:40.320 INFO:tasks.workunit.client.0.smithi019.stdout:3/497: write d9/d11/f41 [1187036,129675] 0 2024-02-22T15:50:40.321 INFO:tasks.workunit.client.0.smithi019.stdout:4/625: creat d2/d3f/d48/d76/fcb x:0 0 0 2024-02-22T15:50:40.321 INFO:tasks.workunit.client.0.smithi019.stdout:2/585: mknod d1/d17/d21/d71/cdc 0 2024-02-22T15:50:40.322 INFO:tasks.workunit.client.0.smithi019.stdout:9/617: symlink d2/d7/ldb 0 2024-02-22T15:50:40.323 INFO:tasks.workunit.client.1.smithi062.stdout:2/507: dwrite d1/d28/d3f/d7d/f93 [0,4194304] 0 2024-02-22T15:50:40.326 INFO:tasks.workunit.client.0.smithi019.stdout:8/618: symlink d9/d14/d6f/d7e/d94/d9c/lcd 0 2024-02-22T15:50:40.326 INFO:tasks.workunit.client.0.smithi019.stdout:4/626: chown d2/db/d14/d61/l89 60287 1 2024-02-22T15:50:40.326 INFO:tasks.workunit.client.0.smithi019.stdout:2/586: chown d1/d46/d9d/l9f 16648773 1 2024-02-22T15:50:40.327 INFO:tasks.workunit.client.0.smithi019.stdout:6/783: rename d6/d4b/d6d/da3/ddb/ddc/lf8 to d6/d4b/d6d/d7b/d78/lfb 0 2024-02-22T15:50:40.328 INFO:tasks.workunit.client.1.smithi062.stdout:5/334: dwrite d0/d3/d2f/d59/d67/d69/f41 [0,4194304] 0 2024-02-22T15:50:40.328 INFO:tasks.workunit.client.1.smithi062.stdout:2/508: truncate d1/d8/fa 4911991 0 2024-02-22T15:50:40.328 INFO:tasks.workunit.client.1.smithi062.stdout:7/393: dwrite d0/d6/d2e/d55/f69 [0,4194304] 0 2024-02-22T15:50:40.329 INFO:tasks.workunit.client.0.smithi019.stdout:3/498: stat c5 0 2024-02-22T15:50:40.330 INFO:tasks.workunit.client.1.smithi062.stdout:8/440: link d8/cc d8/d26/c88 0 2024-02-22T15:50:40.330 INFO:tasks.workunit.client.1.smithi062.stdout:7/394: write d0/d6/d2e/d55/f69 [235063,82066] 0 2024-02-22T15:50:40.332 INFO:tasks.workunit.client.0.smithi019.stdout:5/649: creat d8/dd/d19/d3a/d46/fd5 x:0 0 0 2024-02-22T15:50:40.332 INFO:tasks.workunit.client.1.smithi062.stdout:7/395: dread - d0/d6/d2e/d35/f70 zero size 2024-02-22T15:50:40.334 INFO:tasks.workunit.client.1.smithi062.stdout:4/463: sync 2024-02-22T15:50:40.336 INFO:tasks.workunit.client.0.smithi019.stdout:7/941: dwrite d9/d38/d41/d45/d61/fb1 [0,4194304] 0 2024-02-22T15:50:40.337 INFO:tasks.workunit.client.1.smithi062.stdout:2/509: creat d1/d47/d32/d52/d80/fa8 x:0 0 0 2024-02-22T15:50:40.338 INFO:tasks.workunit.client.1.smithi062.stdout:2/510: dread d1/f82 [0,4194304] 0 2024-02-22T15:50:40.339 INFO:tasks.workunit.client.1.smithi062.stdout:2/511: readlink l0 0 2024-02-22T15:50:40.340 INFO:tasks.workunit.client.0.smithi019.stdout:9/618: mknod d2/d7/dc/d27/d31/cdc 0 2024-02-22T15:50:40.341 INFO:tasks.workunit.client.0.smithi019.stdout:7/942: fdatasync d9/f1b 0 2024-02-22T15:50:40.344 INFO:tasks.workunit.client.0.smithi019.stdout:3/499: chown d9/d22/d27/d6d/f82 2593 1 2024-02-22T15:50:40.345 INFO:tasks.workunit.client.1.smithi062.stdout:7/396: mkdir d0/d4f/d7c 0 2024-02-22T15:50:40.349 INFO:tasks.workunit.client.1.smithi062.stdout:7/397: truncate d0/f10 5493551 0 2024-02-22T15:50:40.350 INFO:tasks.workunit.client.1.smithi062.stdout:8/441: truncate d8/d9/f18 3193625 0 2024-02-22T15:50:40.351 INFO:tasks.workunit.client.1.smithi062.stdout:5/335: symlink d0/d3/da/d13/d23/d4e/d60/l6a 0 2024-02-22T15:50:40.352 INFO:tasks.workunit.client.0.smithi019.stdout:3/500: fdatasync d9/d22/d27/f46 0 2024-02-22T15:50:40.353 INFO:tasks.workunit.client.0.smithi019.stdout:5/650: symlink d8/dd/d19/d3a/d46/ld6 0 2024-02-22T15:50:40.353 INFO:tasks.workunit.client.1.smithi062.stdout:2/512: symlink d1/d8/d62/d73/da6/la9 0 2024-02-22T15:50:40.354 INFO:tasks.workunit.client.1.smithi062.stdout:5/336: chown d0/c4 3397684 1 2024-02-22T15:50:40.355 INFO:tasks.workunit.client.0.smithi019.stdout:2/587: dwrite d1/d17/fbd [0,4194304] 0 2024-02-22T15:50:40.359 INFO:tasks.workunit.client.1.smithi062.stdout:7/398: rename d0/d6/d2e/f61 to d0/d6/f7d 0 2024-02-22T15:50:40.360 INFO:tasks.workunit.client.0.smithi019.stdout:9/619: rename d2/d1f/d2e/d48/d36/fa4 to d2/d1f/d2e/d48/d45/d5a/fdd 0 2024-02-22T15:50:40.362 INFO:tasks.workunit.client.0.smithi019.stdout:3/501: creat d9/d26/d32/f9c x:0 0 0 2024-02-22T15:50:40.363 INFO:tasks.workunit.client.0.smithi019.stdout:2/588: dread d1/f14 [0,4194304] 0 2024-02-22T15:50:40.363 INFO:tasks.workunit.client.1.smithi062.stdout:8/442: dwrite d8/f40 [0,4194304] 0 2024-02-22T15:50:40.363 INFO:tasks.workunit.client.1.smithi062.stdout:5/337: mknod d0/d3/da/d32/d4f/c6b 0 2024-02-22T15:50:40.364 INFO:tasks.workunit.client.1.smithi062.stdout:2/513: fsync d1/d47/d32/d52/d80/f9b 0 2024-02-22T15:50:40.365 INFO:tasks.workunit.client.1.smithi062.stdout:7/399: mknod d0/d6/d2e/d35/c7e 0 2024-02-22T15:50:40.366 INFO:tasks.workunit.client.1.smithi062.stdout:8/443: creat d8/d9/d2c/d74/f89 x:0 0 0 2024-02-22T15:50:40.368 INFO:tasks.workunit.client.1.smithi062.stdout:2/514: unlink d1/d1e/d38/d55/f5e 0 2024-02-22T15:50:40.369 INFO:tasks.workunit.client.0.smithi019.stdout:9/620: write d2/d7/dc/d27/dbc/fd8 [237969,4596] 0 2024-02-22T15:50:40.370 INFO:tasks.workunit.client.0.smithi019.stdout:5/651: rename d8/dd/d19/d77/fd0 to d8/dd/d19/d3a/d46/d7e/dbe/fd7 0 2024-02-22T15:50:40.370 INFO:tasks.workunit.client.1.smithi062.stdout:8/444: mkdir d8/d9/d44/d4d/d8a 0 2024-02-22T15:50:40.370 INFO:tasks.workunit.client.1.smithi062.stdout:8/445: stat d8/d28/c6f 0 2024-02-22T15:50:40.373 INFO:tasks.workunit.client.1.smithi062.stdout:8/446: write d8/d9/d44/d4d/d51/f68 [3612971,85374] 0 2024-02-22T15:50:40.376 INFO:tasks.workunit.client.0.smithi019.stdout:3/502: rename d9/d26/d3f/d59/f74 to d9/d22/f9d 0 2024-02-22T15:50:40.376 INFO:tasks.workunit.client.1.smithi062.stdout:5/338: truncate d0/d3/da/d13/d23/f3b 118501 0 2024-02-22T15:50:40.376 INFO:tasks.workunit.client.1.smithi062.stdout:2/515: creat d1/d28/d3f/d7d/faa x:0 0 0 2024-02-22T15:50:40.377 INFO:tasks.workunit.client.0.smithi019.stdout:5/652: creat d8/dd/d19/d38/d47/fd8 x:0 0 0 2024-02-22T15:50:40.377 INFO:tasks.workunit.client.0.smithi019.stdout:9/621: read d2/d1f/d2e/d48/d36/f43 [44668,25127] 0 2024-02-22T15:50:40.378 INFO:tasks.workunit.client.0.smithi019.stdout:5/653: dread - d8/db1/fb7 zero size 2024-02-22T15:50:40.378 INFO:tasks.workunit.client.1.smithi062.stdout:7/400: link d0/d14/d13/f2a d0/d14/d19/d30/f7f 0 2024-02-22T15:50:40.380 INFO:tasks.workunit.client.1.smithi062.stdout:8/447: symlink d8/d9/d2c/d49/l8b 0 2024-02-22T15:50:40.381 INFO:tasks.workunit.client.0.smithi019.stdout:9/622: creat d2/d1f/db5/fde x:0 0 0 2024-02-22T15:50:40.381 INFO:tasks.workunit.client.0.smithi019.stdout:4/627: dread d2/fa [4194304,4194304] 0 2024-02-22T15:50:40.382 INFO:tasks.workunit.client.1.smithi062.stdout:7/401: mkdir d0/d6/d2e/d55/d80 0 2024-02-22T15:50:40.382 INFO:tasks.workunit.client.1.smithi062.stdout:8/448: chown d8/d9/d44/d4d/d54/f76 12 1 2024-02-22T15:50:40.382 INFO:tasks.workunit.client.1.smithi062.stdout:2/516: fsync d1/d47/d32/d52/d80/f9b 0 2024-02-22T15:50:40.384 INFO:tasks.workunit.client.0.smithi019.stdout:6/784: dread d6/d4b/d6d/d7b/d78/fc4 [0,4194304] 0 2024-02-22T15:50:40.384 INFO:tasks.workunit.client.1.smithi062.stdout:5/339: unlink d0/d3/da/f27 0 2024-02-22T15:50:40.385 INFO:tasks.workunit.client.0.smithi019.stdout:5/654: mkdir d8/dd/d19/d77/dd2/dd9 0 2024-02-22T15:50:40.386 INFO:tasks.workunit.client.1.smithi062.stdout:7/402: write d0/d14/f38 [1104913,6550] 0 2024-02-22T15:50:40.387 INFO:tasks.workunit.client.0.smithi019.stdout:1/663: dread d8/fd [0,4194304] 0 2024-02-22T15:50:40.387 INFO:tasks.workunit.client.1.smithi062.stdout:2/517: write d1/d1e/d38/f4f [373017,124219] 0 2024-02-22T15:50:40.387 INFO:tasks.workunit.client.0.smithi019.stdout:6/785: truncate d6/d4b/d6d/da3/ff0 146018 0 2024-02-22T15:50:40.388 INFO:tasks.workunit.client.0.smithi019.stdout:5/655: chown d8/dd/d19/d3a/d46/f61 315782627 1 2024-02-22T15:50:40.390 INFO:tasks.workunit.client.0.smithi019.stdout:8/619: write d9/d14/d17/d5d/f68 [2731948,73394] 0 2024-02-22T15:50:40.396 INFO:tasks.workunit.client.0.smithi019.stdout:7/943: dread d9/d4c/d5b/f64 [0,4194304] 0 2024-02-22T15:50:40.397 INFO:tasks.workunit.client.0.smithi019.stdout:1/664: creat d8/fe1 x:0 0 0 2024-02-22T15:50:40.398 INFO:tasks.workunit.client.1.smithi062.stdout:8/449: dwrite d8/d9/d44/d4d/d51/f68 [0,4194304] 0 2024-02-22T15:50:40.398 INFO:tasks.workunit.client.1.smithi062.stdout:7/403: unlink d0/d14/d19/d30/d37/l39 0 2024-02-22T15:50:40.400 INFO:tasks.workunit.client.0.smithi019.stdout:7/944: read d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f124 [1788024,83806] 0 2024-02-22T15:50:40.400 INFO:tasks.workunit.client.0.smithi019.stdout:3/503: write d9/d22/d27/f47 [559870,108791] 0 2024-02-22T15:50:40.400 INFO:tasks.workunit.client.0.smithi019.stdout:1/665: dread - d8/d18/fda zero size 2024-02-22T15:50:40.402 INFO:tasks.workunit.client.1.smithi062.stdout:4/464: dread d4/d2a/d2d/f85 [0,4194304] 0 2024-02-22T15:50:40.403 INFO:tasks.workunit.client.0.smithi019.stdout:5/656: getdents d8/dd/d19/d3a/dc7 0 2024-02-22T15:50:40.403 INFO:tasks.workunit.client.0.smithi019.stdout:7/945: readlink d9/d26/d77/l128 0 2024-02-22T15:50:40.404 INFO:tasks.workunit.client.0.smithi019.stdout:3/504: symlink d9/d22/d27/l9e 0 2024-02-22T15:50:40.405 INFO:tasks.workunit.client.1.smithi062.stdout:5/340: dwrite d0/d3/d30/f47 [0,4194304] 0 2024-02-22T15:50:40.406 INFO:tasks.workunit.client.0.smithi019.stdout:5/657: mknod d8/d11/d33/cda 0 2024-02-22T15:50:40.407 INFO:tasks.workunit.client.0.smithi019.stdout:8/620: dread d9/d14/f34 [0,4194304] 0 2024-02-22T15:50:40.407 INFO:tasks.workunit.client.1.smithi062.stdout:2/518: dread d1/d1e/d38/f54 [0,4194304] 0 2024-02-22T15:50:40.408 INFO:tasks.workunit.client.0.smithi019.stdout:1/666: symlink d8/d15/d5b/le2 0 2024-02-22T15:50:40.414 INFO:tasks.workunit.client.1.smithi062.stdout:4/465: read d4/d1c/f3d [294138,114369] 0 2024-02-22T15:50:40.414 INFO:tasks.workunit.client.1.smithi062.stdout:7/404: dwrite d0/d14/d13/f48 [0,4194304] 0 2024-02-22T15:50:40.418 INFO:tasks.workunit.client.1.smithi062.stdout:2/519: mknod d1/d47/d32/d52/cab 0 2024-02-22T15:50:40.419 INFO:tasks.workunit.client.0.smithi019.stdout:3/505: mknod d9/d14/d19/c9f 0 2024-02-22T15:50:40.420 INFO:tasks.workunit.client.1.smithi062.stdout:2/520: fsync d1/d8/f15 0 2024-02-22T15:50:40.421 INFO:tasks.workunit.client.1.smithi062.stdout:7/405: chown d0/d6/d2e/d2f/l68 2 1 2024-02-22T15:50:40.421 INFO:tasks.workunit.client.0.smithi019.stdout:6/786: truncate d6/d58/d6a/f85 3080234 0 2024-02-22T15:50:40.422 INFO:tasks.workunit.client.1.smithi062.stdout:5/341: dwrite d0/f26 [4194304,4194304] 0 2024-02-22T15:50:40.424 INFO:tasks.workunit.client.0.smithi019.stdout:9/623: dread d2/fb [4194304,4194304] 0 2024-02-22T15:50:40.427 INFO:tasks.workunit.client.1.smithi062.stdout:5/342: write d0/f7 [992258,6683] 0 2024-02-22T15:50:40.429 INFO:tasks.workunit.client.0.smithi019.stdout:9/624: read - d2/d1f/d2e/d48/d45/d5a/daf/fbd zero size 2024-02-22T15:50:40.429 INFO:tasks.workunit.client.0.smithi019.stdout:1/667: dwrite d8/d15/d34/fc8 [0,4194304] 0 2024-02-22T15:50:40.434 INFO:tasks.workunit.client.0.smithi019.stdout:5/658: dwrite d8/dd/d19/d38/d54/f6b [0,4194304] 0 2024-02-22T15:50:40.438 INFO:tasks.workunit.client.1.smithi062.stdout:5/343: write d0/d3/f10 [4358823,64953] 0 2024-02-22T15:50:40.438 INFO:tasks.workunit.client.1.smithi062.stdout:2/521: creat d1/d47/d39/fac x:0 0 0 2024-02-22T15:50:40.438 INFO:tasks.workunit.client.1.smithi062.stdout:7/406: creat d0/d6/d2e/d55/f81 x:0 0 0 2024-02-22T15:50:40.441 INFO:tasks.workunit.client.0.smithi019.stdout:7/946: rmdir d9/d38/d41/d45/d92/d122 0 2024-02-22T15:50:40.442 INFO:tasks.workunit.client.0.smithi019.stdout:3/506: mkdir d9/d26/d2d/d53/da0 0 2024-02-22T15:50:40.442 INFO:tasks.workunit.client.1.smithi062.stdout:7/407: dread - d0/d6/d2e/d35/f70 zero size 2024-02-22T15:50:40.442 INFO:tasks.workunit.client.1.smithi062.stdout:4/466: dwrite d4/d1c/f3d [0,4194304] 0 2024-02-22T15:50:40.442 INFO:tasks.workunit.client.1.smithi062.stdout:2/522: write d1/d8/d8f/f96 [408763,52473] 0 2024-02-22T15:50:40.445 INFO:tasks.workunit.client.1.smithi062.stdout:7/408: chown d0/d14/d28/l3c 89609885 1 2024-02-22T15:50:40.447 INFO:tasks.workunit.client.0.smithi019.stdout:1/668: mkdir d8/d18/d64/d7f/de3 0 2024-02-22T15:50:40.447 INFO:tasks.workunit.client.1.smithi062.stdout:8/450: write d8/f12 [675046,123924] 0 2024-02-22T15:50:40.449 INFO:tasks.workunit.client.0.smithi019.stdout:5/659: mkdir d8/dd/d19/d38/d47/ddb 0 2024-02-22T15:50:40.449 INFO:tasks.workunit.client.0.smithi019.stdout:1/669: chown d8/d18/d64/f67 4 1 2024-02-22T15:50:40.450 INFO:tasks.workunit.client.1.smithi062.stdout:5/344: dwrite d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:40.450 INFO:tasks.workunit.client.1.smithi062.stdout:2/523: symlink d1/d47/lad 0 2024-02-22T15:50:40.450 INFO:tasks.workunit.client.0.smithi019.stdout:7/947: mknod d9/d38/d41/d45/db0/df7/dfb/de3/c135 0 2024-02-22T15:50:40.451 INFO:tasks.workunit.client.1.smithi062.stdout:2/524: stat d1/cd 0 2024-02-22T15:50:40.451 INFO:tasks.workunit.client.1.smithi062.stdout:4/467: truncate d4/d2a/d2d/d2e/d6a/d8a/f9a 186361 0 2024-02-22T15:50:40.452 INFO:tasks.workunit.client.0.smithi019.stdout:3/507: symlink d9/d26/d3f/d59/la1 0 2024-02-22T15:50:40.452 INFO:tasks.workunit.client.1.smithi062.stdout:7/409: creat d0/d6/d2e/d2f/f82 x:0 0 0 2024-02-22T15:50:40.453 INFO:tasks.workunit.client.0.smithi019.stdout:9/625: mkdir d2/d1f/d2e/d57/d60/d8c/ddf 0 2024-02-22T15:50:40.455 INFO:tasks.workunit.client.0.smithi019.stdout:7/948: write d9/d38/d41/d45/db0/f121 [879479,118046] 0 2024-02-22T15:50:40.456 INFO:tasks.workunit.client.1.smithi062.stdout:8/451: truncate d8/f39 1032876 0 2024-02-22T15:50:40.457 INFO:tasks.workunit.client.1.smithi062.stdout:7/410: write d0/f10 [398004,84776] 0 2024-02-22T15:50:40.457 INFO:tasks.workunit.client.0.smithi019.stdout:1/670: rename d8/d15/f5c to d8/d15/da4/fe4 0 2024-02-22T15:50:40.458 INFO:tasks.workunit.client.1.smithi062.stdout:2/525: chown d1/d47/d39/d5d/f63 3077 1 2024-02-22T15:50:40.458 INFO:tasks.workunit.client.1.smithi062.stdout:5/345: link d0/c4 d0/d3/da/d13/d23/d31/c6c 0 2024-02-22T15:50:40.462 INFO:tasks.workunit.client.1.smithi062.stdout:8/452: write d8/d9/f5f [743602,93654] 0 2024-02-22T15:50:40.463 INFO:tasks.workunit.client.1.smithi062.stdout:2/526: symlink d1/d1e/lae 0 2024-02-22T15:50:40.464 INFO:tasks.workunit.client.0.smithi019.stdout:7/949: mkdir d9/d38/d41/d45/db0/df7/dfb/de3/d107/d136 0 2024-02-22T15:50:40.465 INFO:tasks.workunit.client.0.smithi019.stdout:9/626: creat d2/d1f/d2e/d57/d60/d8c/fe0 x:0 0 0 2024-02-22T15:50:40.466 INFO:tasks.workunit.client.0.smithi019.stdout:1/671: rmdir d8/d15/d5b/dbc/dd6 0 2024-02-22T15:50:40.468 INFO:tasks.workunit.client.0.smithi019.stdout:7/950: symlink d9/d4c/ddf/d105/l137 0 2024-02-22T15:50:40.469 INFO:tasks.workunit.client.1.smithi062.stdout:7/411: link d0/d14/c2b d0/d6/d2e/d4a/c83 0 2024-02-22T15:50:40.469 INFO:tasks.workunit.client.1.smithi062.stdout:5/346: write d0/d3/d2f/d59/d67/d69/f41 [1249470,92875] 0 2024-02-22T15:50:40.471 INFO:tasks.workunit.client.0.smithi019.stdout:1/672: mknod d8/d18/d64/d7f/d94/ce5 0 2024-02-22T15:50:40.472 INFO:tasks.workunit.client.1.smithi062.stdout:8/453: symlink d8/d42/d87/l8c 0 2024-02-22T15:50:40.474 INFO:tasks.workunit.client.1.smithi062.stdout:7/412: symlink d0/d4f/l84 0 2024-02-22T15:50:40.475 INFO:tasks.workunit.client.1.smithi062.stdout:4/468: dread d4/d2a/d2d/d2e/f84 [0,4194304] 0 2024-02-22T15:50:40.477 INFO:tasks.workunit.client.0.smithi019.stdout:6/787: fsync d6/d58/d6a/f85 0 2024-02-22T15:50:40.477 INFO:tasks.workunit.client.0.smithi019.stdout:7/951: dread d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f124 [0,4194304] 0 2024-02-22T15:50:40.483 INFO:tasks.workunit.client.1.smithi062.stdout:8/454: creat d8/d9/d2c/d49/d5e/f8d x:0 0 0 2024-02-22T15:50:40.483 INFO:tasks.workunit.client.1.smithi062.stdout:8/455: readlink d8/d9/l15 0 2024-02-22T15:50:40.484 INFO:tasks.workunit.client.0.smithi019.stdout:6/788: creat d6/d31/d43/de7/ffc x:0 0 0 2024-02-22T15:50:40.484 INFO:tasks.workunit.client.0.smithi019.stdout:6/789: fsync d6/f86 0 2024-02-22T15:50:40.485 INFO:tasks.workunit.client.1.smithi062.stdout:2/527: dwrite d1/d1e/d38/d76/f99 [0,4194304] 0 2024-02-22T15:50:40.485 INFO:tasks.workunit.client.1.smithi062.stdout:4/469: creat d4/d1c/d30/d37/fa3 x:0 0 0 2024-02-22T15:50:40.486 INFO:tasks.workunit.client.1.smithi062.stdout:7/413: mknod d0/d4f/d7c/c85 0 2024-02-22T15:50:40.488 INFO:tasks.workunit.client.1.smithi062.stdout:8/456: creat d8/d28/d4e/f8e x:0 0 0 2024-02-22T15:50:40.488 INFO:tasks.workunit.client.0.smithi019.stdout:6/790: link d6/d4b/c91 d6/d58/cfd 0 2024-02-22T15:50:40.490 INFO:tasks.workunit.client.1.smithi062.stdout:4/470: write d4/d2a/d2d/f85 [4124477,130523] 0 2024-02-22T15:50:40.492 INFO:tasks.workunit.client.1.smithi062.stdout:8/457: dread d8/f39 [0,4194304] 0 2024-02-22T15:50:40.495 INFO:tasks.workunit.client.1.smithi062.stdout:5/347: truncate d0/d3/f14 2831390 0 2024-02-22T15:50:40.495 INFO:tasks.workunit.client.0.smithi019.stdout:0/582: sync 2024-02-22T15:50:40.496 INFO:tasks.workunit.client.1.smithi062.stdout:2/528: creat d1/d1e/faf x:0 0 0 2024-02-22T15:50:40.497 INFO:tasks.workunit.client.0.smithi019.stdout:6/791: dwrite d6/d4b/d8d/f95 [0,4194304] 0 2024-02-22T15:50:40.497 INFO:tasks.workunit.client.1.smithi062.stdout:2/529: stat d1/d1e/d38/d76/f92 0 2024-02-22T15:50:40.498 INFO:tasks.workunit.client.1.smithi062.stdout:7/414: link d0/d14/d19/f1a d0/d6/d2e/f86 0 2024-02-22T15:50:40.500 INFO:tasks.workunit.client.0.smithi019.stdout:9/627: truncate d2/f6 11986854 0 2024-02-22T15:50:40.500 INFO:tasks.workunit.client.1.smithi062.stdout:4/471: rename d4/d8f to d4/d2a/d2d/d2e/d6a/d9e/da4 0 2024-02-22T15:50:40.501 INFO:tasks.workunit.client.1.smithi062.stdout:5/348: chown d0/d3/da/d13/d23/f3b 234831285 1 2024-02-22T15:50:40.502 INFO:tasks.workunit.client.0.smithi019.stdout:0/583: link f2 d4/db/d1b/d63/db0/da1/dbb/d7/fcb 0 2024-02-22T15:50:40.502 INFO:tasks.workunit.client.1.smithi062.stdout:2/530: creat d1/d47/d32/d52/d80/fb0 x:0 0 0 2024-02-22T15:50:40.503 INFO:tasks.workunit.client.1.smithi062.stdout:7/415: creat d0/d6/d2e/d55/f87 x:0 0 0 2024-02-22T15:50:40.504 INFO:tasks.workunit.client.0.smithi019.stdout:9/628: stat d2/d7/dc/d27/d31 0 2024-02-22T15:50:40.504 INFO:tasks.workunit.client.0.smithi019.stdout:6/792: truncate d6/d4b/d6d/da3/ddb/fe8 557315 0 2024-02-22T15:50:40.505 INFO:tasks.workunit.client.1.smithi062.stdout:8/458: rename d8/d9/d44/d4d/d54 to d8/d42/d8f 0 2024-02-22T15:50:40.506 INFO:tasks.workunit.client.1.smithi062.stdout:8/459: readlink d8/d42/l5d 0 2024-02-22T15:50:40.507 INFO:tasks.workunit.client.1.smithi062.stdout:4/472: rmdir d4/d1c/d2c/d6b/d70 39 2024-02-22T15:50:40.507 INFO:tasks.workunit.client.0.smithi019.stdout:9/629: fsync d2/d7/dc/f24 0 2024-02-22T15:50:40.509 INFO:tasks.workunit.client.1.smithi062.stdout:7/416: mkdir d0/d6/d2e/d2f/d88 0 2024-02-22T15:50:40.511 INFO:tasks.workunit.client.1.smithi062.stdout:4/473: chown d4/d1c/d30/d3b/f67 677134 1 2024-02-22T15:50:40.512 INFO:tasks.workunit.client.0.smithi019.stdout:6/793: creat d6/d4b/d6d/d7b/d7f/ffe x:0 0 0 2024-02-22T15:50:40.513 INFO:tasks.workunit.client.1.smithi062.stdout:8/460: creat d8/d9/d2c/d49/f90 x:0 0 0 2024-02-22T15:50:40.513 INFO:tasks.workunit.client.1.smithi062.stdout:5/349: dread d0/d3/da/d13/d23/d31/f43 [8388608,4194304] 0 2024-02-22T15:50:40.518 INFO:tasks.workunit.client.1.smithi062.stdout:8/461: rmdir d8/d28/d4e 39 2024-02-22T15:50:40.518 INFO:tasks.workunit.client.0.smithi019.stdout:0/584: dwrite d4/db/d49/d65/fc7 [0,4194304] 0 2024-02-22T15:50:40.519 INFO:tasks.workunit.client.1.smithi062.stdout:8/462: write d8/d9/f16 [54280,41464] 0 2024-02-22T15:50:40.520 INFO:tasks.workunit.client.0.smithi019.stdout:1/673: dwrite d8/d15/d34/f65 [0,4194304] 0 2024-02-22T15:50:40.520 INFO:tasks.workunit.client.1.smithi062.stdout:5/350: mkdir d0/d3/da/d13/d23/d4e/d6d 0 2024-02-22T15:50:40.521 INFO:tasks.workunit.client.0.smithi019.stdout:6/794: creat d6/d4b/d6d/da3/fff x:0 0 0 2024-02-22T15:50:40.521 INFO:tasks.workunit.client.0.smithi019.stdout:7/952: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:40.523 INFO:tasks.workunit.client.1.smithi062.stdout:8/463: creat d8/d42/d8f/f91 x:0 0 0 2024-02-22T15:50:40.529 INFO:tasks.workunit.client.1.smithi062.stdout:8/464: unlink l5 0 2024-02-22T15:50:40.529 INFO:tasks.workunit.client.0.smithi019.stdout:1/674: stat d8/d2a/db3/fb1 0 2024-02-22T15:50:40.531 INFO:tasks.workunit.client.0.smithi019.stdout:6/795: mkdir d6/d4b/d6d/d7b/d78/d100 0 2024-02-22T15:50:40.532 INFO:tasks.workunit.client.1.smithi062.stdout:2/531: write d1/d8/fa [4146802,81519] 0 2024-02-22T15:50:40.533 INFO:tasks.workunit.client.1.smithi062.stdout:8/465: creat d8/d9/d44/d4d/d51/d55/f92 x:0 0 0 2024-02-22T15:50:40.534 INFO:tasks.workunit.client.1.smithi062.stdout:2/532: truncate d1/d47/f83 1438595 0 2024-02-22T15:50:40.535 INFO:tasks.workunit.client.0.smithi019.stdout:1/675: rmdir d8/d18/d85 39 2024-02-22T15:50:40.535 INFO:tasks.workunit.client.0.smithi019.stdout:7/953: dwrite d9/d4c/f68 [0,4194304] 0 2024-02-22T15:50:40.536 INFO:tasks.workunit.client.0.smithi019.stdout:1/676: stat d8/f14 0 2024-02-22T15:50:40.537 INFO:tasks.workunit.client.1.smithi062.stdout:2/533: rmdir d1/d47 39 2024-02-22T15:50:40.538 INFO:tasks.workunit.client.0.smithi019.stdout:6/796: dread d6/d4b/d6d/da3/fe4 [0,4194304] 0 2024-02-22T15:50:40.538 INFO:tasks.workunit.client.0.smithi019.stdout:0/585: dread d4/d32/f9c [0,4194304] 0 2024-02-22T15:50:40.538 INFO:tasks.workunit.client.1.smithi062.stdout:7/417: truncate d0/d6/d2e/d55/f69 368472 0 2024-02-22T15:50:40.539 INFO:tasks.workunit.client.1.smithi062.stdout:5/351: link d0/d3/da/d13/d23/d4e/d60/l6a d0/d3/d2f/d59/d67/d69/d3f/d66/l6e 0 2024-02-22T15:50:40.539 INFO:tasks.workunit.client.0.smithi019.stdout:1/677: mknod d8/d44/d5d/ce6 0 2024-02-22T15:50:40.539 INFO:tasks.workunit.client.0.smithi019.stdout:7/954: chown d9/d38/d41/d45/fe7 11454405 1 2024-02-22T15:50:40.539 INFO:tasks.workunit.client.0.smithi019.stdout:0/586: dread - d4/d23/d7d/f85 zero size 2024-02-22T15:50:40.540 INFO:tasks.workunit.client.1.smithi062.stdout:8/466: symlink d8/d9/d44/d4d/d51/d55/l93 0 2024-02-22T15:50:40.542 INFO:tasks.workunit.client.0.smithi019.stdout:6/797: rename d6/d4b/d8d/cbe to d6/d4b/d6d/d7b/d78/d100/c101 0 2024-02-22T15:50:40.543 INFO:tasks.workunit.client.0.smithi019.stdout:6/798: fsync d6/d4b/d6d/da3/ff0 0 2024-02-22T15:50:40.543 INFO:tasks.workunit.client.0.smithi019.stdout:0/587: unlink d4/db/d76/c79 0 2024-02-22T15:50:40.543 INFO:tasks.workunit.client.1.smithi062.stdout:5/352: mknod d0/d3/da/d13/d23/d4e/d60/c6f 0 2024-02-22T15:50:40.546 INFO:tasks.workunit.client.0.smithi019.stdout:7/955: creat d9/d38/d41/d45/db0/df7/dfb/d12a/d5d/d133/f138 x:0 0 0 2024-02-22T15:50:40.546 INFO:tasks.workunit.client.0.smithi019.stdout:0/588: fsync d4/d9/f26 0 2024-02-22T15:50:40.547 INFO:tasks.workunit.client.1.smithi062.stdout:5/353: mknod d0/d3/d2f/d59/d67/c70 0 2024-02-22T15:50:40.548 INFO:tasks.workunit.client.0.smithi019.stdout:6/799: mknod d6/d31/d69/c102 0 2024-02-22T15:50:40.550 INFO:tasks.workunit.client.1.smithi062.stdout:8/467: dwrite d8/d9/d2c/d49/f90 [0,4194304] 0 2024-02-22T15:50:40.551 INFO:tasks.workunit.client.0.smithi019.stdout:7/956: rename d9/f23 to d9/d38/d41/d45/db0/df7/dfb/de3/f139 0 2024-02-22T15:50:40.554 INFO:tasks.workunit.client.1.smithi062.stdout:7/418: read d0/d14/d19/d30/d37/f53 [14313,120027] 0 2024-02-22T15:50:40.554 INFO:tasks.workunit.client.1.smithi062.stdout:2/534: dwrite d1/d28/d3f/f59 [0,4194304] 0 2024-02-22T15:50:40.555 INFO:tasks.workunit.client.0.smithi019.stdout:6/800: rename d6/d31/d43/f5f to d6/d4b/f103 0 2024-02-22T15:50:40.557 INFO:tasks.workunit.client.1.smithi062.stdout:2/535: chown d1/d1e/f9a 105625549 1 2024-02-22T15:50:40.557 INFO:tasks.workunit.client.0.smithi019.stdout:7/957: dread d9/d38/d41/d45/db0/f121 [0,4194304] 0 2024-02-22T15:50:40.560 INFO:tasks.workunit.client.1.smithi062.stdout:5/354: dwrite d0/d3/d30/f46 [0,4194304] 0 2024-02-22T15:50:40.562 INFO:tasks.workunit.client.0.smithi019.stdout:6/801: rename d6/d4b/d6d/d7b/lcb to d6/d4b/d71/l104 0 2024-02-22T15:50:40.562 INFO:tasks.workunit.client.1.smithi062.stdout:2/536: chown d1/d47/f48 7 1 2024-02-22T15:50:40.564 INFO:tasks.workunit.client.0.smithi019.stdout:6/802: fdatasync d6/d3e/f50 0 2024-02-22T15:50:40.565 INFO:tasks.workunit.client.1.smithi062.stdout:2/537: creat d1/d8/d62/d73/da6/fb1 x:0 0 0 2024-02-22T15:50:40.566 INFO:tasks.workunit.client.1.smithi062.stdout:2/538: chown d1/d8/fa 40349691 1 2024-02-22T15:50:40.567 INFO:tasks.workunit.client.0.smithi019.stdout:0/589: write d4/db/d1b/f2f [2001618,30639] 0 2024-02-22T15:50:40.568 INFO:tasks.workunit.client.0.smithi019.stdout:6/803: mkdir d6/d4b/d6d/dda/d105 0 2024-02-22T15:50:40.569 INFO:tasks.workunit.client.0.smithi019.stdout:0/590: chown d4/db/d49/cb3 12200 1 2024-02-22T15:50:40.571 INFO:tasks.workunit.client.1.smithi062.stdout:8/468: truncate d8/d9/f16 3687194 0 2024-02-22T15:50:40.571 INFO:tasks.workunit.client.1.smithi062.stdout:3/393: sync 2024-02-22T15:50:40.572 INFO:tasks.workunit.client.0.smithi019.stdout:6/804: stat d6/d4b/d6d/d7b/d7f/ffe 0 2024-02-22T15:50:40.573 INFO:tasks.workunit.client.0.smithi019.stdout:6/805: chown d6/ld 23 1 2024-02-22T15:50:40.577 INFO:tasks.workunit.client.1.smithi062.stdout:5/355: getdents d0/d3/da/d13/d23/d4e 0 2024-02-22T15:50:40.578 INFO:tasks.workunit.client.0.smithi019.stdout:0/591: dwrite d4/db/f43 [0,4194304] 0 2024-02-22T15:50:40.578 INFO:tasks.workunit.client.0.smithi019.stdout:6/806: getdents d6/d31/d69/d8e/dcf 0 2024-02-22T15:50:40.579 INFO:tasks.workunit.client.0.smithi019.stdout:6/807: readlink d6/d4b/d71/dc2/le0 0 2024-02-22T15:50:40.579 INFO:tasks.workunit.client.0.smithi019.stdout:0/592: readlink d4/db/d1b/d63/l91 0 2024-02-22T15:50:40.581 INFO:tasks.workunit.client.1.smithi062.stdout:5/356: mkdir d0/d3/d30/d71 0 2024-02-22T15:50:40.581 INFO:tasks.workunit.client.1.smithi062.stdout:2/539: symlink d1/d28/d3f/d65/lb2 0 2024-02-22T15:50:40.581 INFO:tasks.workunit.client.1.smithi062.stdout:3/394: getdents d5/d16 0 2024-02-22T15:50:40.583 INFO:tasks.workunit.client.0.smithi019.stdout:0/593: mkdir d4/d22/dcc 0 2024-02-22T15:50:40.589 INFO:tasks.workunit.client.0.smithi019.stdout:6/808: dwrite d6/d3e/fb7 [0,4194304] 0 2024-02-22T15:50:40.591 INFO:tasks.workunit.client.0.smithi019.stdout:0/594: link d4/d67/c9e d4/db/d24/d80/daf/ccd 0 2024-02-22T15:50:40.592 INFO:tasks.workunit.client.0.smithi019.stdout:6/809: creat d6/d31/d43/f106 x:0 0 0 2024-02-22T15:50:40.611 INFO:tasks.workunit.client.1.smithi062.stdout:5/357: mkdir d0/d3/da/d32/d4f/d72 0 2024-02-22T15:50:40.611 INFO:tasks.workunit.client.1.smithi062.stdout:3/395: rename d5 to d5/d16/d21/d32/d6a/d76 22 2024-02-22T15:50:40.612 INFO:tasks.workunit.client.1.smithi062.stdout:5/358: chown d0/d3/ff 3459414 1 2024-02-22T15:50:40.613 INFO:tasks.workunit.client.1.smithi062.stdout:2/540: rename d1/d47/d32/d52/d80/la3 to d1/d47/d39/d5d/lb3 0 2024-02-22T15:50:40.615 INFO:tasks.workunit.client.1.smithi062.stdout:3/396: symlink d5/d16/l77 0 2024-02-22T15:50:40.616 INFO:tasks.workunit.client.1.smithi062.stdout:5/359: mkdir d0/d3/d2f/d59/d67/d69/d3f/d73 0 2024-02-22T15:50:40.617 INFO:tasks.workunit.client.1.smithi062.stdout:2/541: mkdir d1/db4 0 2024-02-22T15:50:40.619 INFO:tasks.workunit.client.1.smithi062.stdout:2/542: stat d1/d1e/faf 0 2024-02-22T15:50:40.621 INFO:tasks.workunit.client.1.smithi062.stdout:2/543: chown d1/cd 46 1 2024-02-22T15:50:40.622 INFO:tasks.workunit.client.1.smithi062.stdout:2/544: dread - d1/d47/f72 zero size 2024-02-22T15:50:40.623 INFO:tasks.workunit.client.1.smithi062.stdout:3/397: dwrite d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:40.624 INFO:tasks.workunit.client.1.smithi062.stdout:5/360: dwrite d0/f45 [0,4194304] 0 2024-02-22T15:50:40.625 INFO:tasks.workunit.client.1.smithi062.stdout:2/545: write d1/d28/d3f/d7d/faa [246107,62343] 0 2024-02-22T15:50:40.633 INFO:tasks.workunit.client.1.smithi062.stdout:2/546: rename d1/d47/l86 to d1/d8/df/lb5 0 2024-02-22T15:50:40.634 INFO:tasks.workunit.client.1.smithi062.stdout:2/547: chown d1/d47/d32/d52/d80/fa8 7582339 1 2024-02-22T15:50:40.638 INFO:tasks.workunit.client.1.smithi062.stdout:2/548: fsync d1/d8/f15 0 2024-02-22T15:50:40.638 INFO:tasks.workunit.client.1.smithi062.stdout:5/361: truncate d0/d3/da/d13/f3d 2417455 0 2024-02-22T15:50:40.643 INFO:tasks.workunit.client.1.smithi062.stdout:8/469: dwrite d8/f3a [0,4194304] 0 2024-02-22T15:50:40.645 INFO:tasks.workunit.client.1.smithi062.stdout:2/549: mknod d1/d47/cb6 0 2024-02-22T15:50:40.647 INFO:tasks.workunit.client.1.smithi062.stdout:5/362: truncate d0/d3/da/d13/d23/f3b 615030 0 2024-02-22T15:50:40.652 INFO:tasks.workunit.client.0.smithi019.stdout:6/810: dwrite d6/d4b/f103 [0,4194304] 0 2024-02-22T15:50:40.655 INFO:tasks.workunit.client.1.smithi062.stdout:8/470: getdents d8/d26 0 2024-02-22T15:50:40.656 INFO:tasks.workunit.client.1.smithi062.stdout:2/550: link d1/d8/c98 d1/d8/cb7 0 2024-02-22T15:50:40.659 INFO:tasks.workunit.client.1.smithi062.stdout:5/363: dwrite d0/d3/da/f53 [0,4194304] 0 2024-02-22T15:50:40.662 INFO:tasks.workunit.client.0.smithi019.stdout:0/595: dread d4/db/d1b/f2f [0,4194304] 0 2024-02-22T15:50:40.663 INFO:tasks.workunit.client.0.smithi019.stdout:6/811: dwrite d6/f28 [0,4194304] 0 2024-02-22T15:50:40.663 INFO:tasks.workunit.client.1.smithi062.stdout:3/398: dwrite d5/d16/d21/d32/f4c [0,4194304] 0 2024-02-22T15:50:40.665 INFO:tasks.workunit.client.0.smithi019.stdout:6/812: write d6/d4b/d6d/d7b/d78/f7e [4192990,55830] 0 2024-02-22T15:50:40.665 INFO:tasks.workunit.client.1.smithi062.stdout:5/364: mknod d0/d3/c74 0 2024-02-22T15:50:40.668 INFO:tasks.workunit.client.0.smithi019.stdout:0/596: unlink d4/db/d1b/d63/db0/da1/dbb/d52/f71 0 2024-02-22T15:50:40.669 INFO:tasks.workunit.client.0.smithi019.stdout:6/813: stat d6/d31/f36 0 2024-02-22T15:50:40.670 INFO:tasks.workunit.client.1.smithi062.stdout:2/551: getdents d1/d1e/d38 0 2024-02-22T15:50:40.670 INFO:tasks.workunit.client.0.smithi019.stdout:0/597: fsync d4/db/d1b/f1c 0 2024-02-22T15:50:40.671 INFO:tasks.workunit.client.1.smithi062.stdout:2/552: stat d1/d1e/d38/d76 0 2024-02-22T15:50:40.672 INFO:tasks.workunit.client.0.smithi019.stdout:0/598: rmdir d4/db 39 2024-02-22T15:50:40.672 INFO:tasks.workunit.client.1.smithi062.stdout:8/471: dwrite d8/d42/d8f/f91 [0,4194304] 0 2024-02-22T15:50:40.673 INFO:tasks.workunit.client.1.smithi062.stdout:3/399: getdents d5/d16/d21/d36/d4a 0 2024-02-22T15:50:40.675 INFO:tasks.workunit.client.1.smithi062.stdout:8/472: stat d8/d9/d44/d4d/d51/d55/f6d 0 2024-02-22T15:50:40.678 INFO:tasks.workunit.client.1.smithi062.stdout:5/365: rename d0/d3/da/d13/l4d to d0/d3/da/d32/l75 0 2024-02-22T15:50:40.681 INFO:tasks.workunit.client.1.smithi062.stdout:8/473: mkdir d8/d9/d2c/d49/d94 0 2024-02-22T15:50:40.683 INFO:tasks.workunit.client.0.smithi019.stdout:6/814: dwrite d6/d31/d43/fdf [0,4194304] 0 2024-02-22T15:50:40.683 INFO:tasks.workunit.client.1.smithi062.stdout:2/553: write d1/d28/d3f/f46 [2758771,109675] 0 2024-02-22T15:50:40.684 INFO:tasks.workunit.client.1.smithi062.stdout:3/400: rename d5/d16/f44 to d5/d16/d21/d32/d6a/f78 0 2024-02-22T15:50:40.686 INFO:tasks.workunit.client.0.smithi019.stdout:6/815: stat d6/d4b/d71/d88/lea 0 2024-02-22T15:50:40.690 INFO:tasks.workunit.client.0.smithi019.stdout:6/816: symlink d6/d31/d43/l107 0 2024-02-22T15:50:40.690 INFO:tasks.workunit.client.1.smithi062.stdout:2/554: readlink d1/d1e/lae 0 2024-02-22T15:50:40.691 INFO:tasks.workunit.client.1.smithi062.stdout:5/366: dread d0/d3/d30/f47 [0,4194304] 0 2024-02-22T15:50:40.691 INFO:tasks.workunit.client.0.smithi019.stdout:6/817: fdatasync d6/f1c 0 2024-02-22T15:50:40.691 INFO:tasks.workunit.client.1.smithi062.stdout:8/474: dread d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:40.693 INFO:tasks.workunit.client.1.smithi062.stdout:2/555: rename d1/d28/f5c to d1/fb8 0 2024-02-22T15:50:40.694 INFO:tasks.workunit.client.0.smithi019.stdout:0/599: mknod d4/db/d1b/cce 0 2024-02-22T15:50:40.697 INFO:tasks.workunit.client.1.smithi062.stdout:8/475: getdents d8/d28/d70 0 2024-02-22T15:50:40.698 INFO:tasks.workunit.client.1.smithi062.stdout:5/367: dread d0/d3/da/d13/d23/f3b [0,4194304] 0 2024-02-22T15:50:40.699 INFO:tasks.workunit.client.0.smithi019.stdout:6/818: unlink d6/d3e/l6e 0 2024-02-22T15:50:40.700 INFO:tasks.workunit.client.1.smithi062.stdout:8/476: stat d8/d42/l5d 0 2024-02-22T15:50:40.701 INFO:tasks.workunit.client.0.smithi019.stdout:0/600: mknod d4/db/d49/dc2/dc1/ccf 0 2024-02-22T15:50:40.702 INFO:tasks.workunit.client.1.smithi062.stdout:8/477: write d8/f29 [2704018,89715] 0 2024-02-22T15:50:40.702 INFO:tasks.workunit.client.0.smithi019.stdout:6/819: creat d6/d31/d43/de7/f108 x:0 0 0 2024-02-22T15:50:40.702 INFO:tasks.workunit.client.0.smithi019.stdout:0/601: chown d4/db/d1b/d40/c4d 1 1 2024-02-22T15:50:40.704 INFO:tasks.workunit.client.0.smithi019.stdout:0/602: fdatasync d4/d22/f5f 0 2024-02-22T15:50:40.704 INFO:tasks.workunit.client.1.smithi062.stdout:2/556: dwrite d1/d1e/d38/f74 [0,4194304] 0 2024-02-22T15:50:40.706 INFO:tasks.workunit.client.1.smithi062.stdout:2/557: chown d1/d1e/f66 2260995 1 2024-02-22T15:50:40.707 INFO:tasks.workunit.client.0.smithi019.stdout:6/820: mknod d6/d4b/d6d/da3/ddb/ddc/c109 0 2024-02-22T15:50:40.710 INFO:tasks.workunit.client.0.smithi019.stdout:0/603: dread d4/d22/fa9 [0,4194304] 0 2024-02-22T15:50:40.711 INFO:tasks.workunit.client.1.smithi062.stdout:2/558: fsync d1/f16 0 2024-02-22T15:50:40.713 INFO:tasks.workunit.client.1.smithi062.stdout:8/478: dwrite d8/d9/f2a [0,4194304] 0 2024-02-22T15:50:40.714 INFO:tasks.workunit.client.1.smithi062.stdout:2/559: dread - d1/d8/df/f78 zero size 2024-02-22T15:50:40.715 INFO:tasks.workunit.client.1.smithi062.stdout:2/560: chown d1/db4 0 1 2024-02-22T15:50:40.715 INFO:tasks.workunit.client.1.smithi062.stdout:8/479: chown d8/d28/d4e/f8e 7 1 2024-02-22T15:50:40.716 INFO:tasks.workunit.client.0.smithi019.stdout:0/604: creat d4/db/d1b/d40/d8c/fd0 x:0 0 0 2024-02-22T15:50:40.717 INFO:tasks.workunit.client.0.smithi019.stdout:6/821: dwrite d6/d4b/d6d/d7b/f62 [0,4194304] 0 2024-02-22T15:50:40.717 INFO:tasks.workunit.client.1.smithi062.stdout:2/561: read d1/d1e/f66 [1296983,73302] 0 2024-02-22T15:50:40.720 INFO:tasks.workunit.client.0.smithi019.stdout:0/605: write d4/d22/d2c/d68/fa6 [1523227,17917] 0 2024-02-22T15:50:40.720 INFO:tasks.workunit.client.1.smithi062.stdout:3/401: creat d5/d16/d21/f79 x:0 0 0 2024-02-22T15:50:40.721 INFO:tasks.workunit.client.0.smithi019.stdout:6/822: readlink d6/d31/la4 0 2024-02-22T15:50:40.722 INFO:tasks.workunit.client.1.smithi062.stdout:5/368: truncate d0/d3/da/f53 3611598 0 2024-02-22T15:50:40.722 INFO:tasks.workunit.client.0.smithi019.stdout:6/823: chown d6/d4b/d71/d88/dc0/lf3 4824015 1 2024-02-22T15:50:40.723 INFO:tasks.workunit.client.1.smithi062.stdout:5/369: readlink d0/d3/da/l20 0 2024-02-22T15:50:40.724 INFO:tasks.workunit.client.0.smithi019.stdout:6/824: chown d6/f44 53992533 1 2024-02-22T15:50:40.725 INFO:tasks.workunit.client.0.smithi019.stdout:6/825: write d6/d4b/d71/d88/dc0/ff9 [1033391,38680] 0 2024-02-22T15:50:40.727 INFO:tasks.workunit.client.0.smithi019.stdout:0/606: link d4/d22/fa9 d4/d22/dcc/fd1 0 2024-02-22T15:50:40.728 INFO:tasks.workunit.client.1.smithi062.stdout:5/370: truncate d0/d3/d30/f4a 1787458 0 2024-02-22T15:50:40.729 INFO:tasks.workunit.client.0.smithi019.stdout:0/607: chown d4/db/l84 133 1 2024-02-22T15:50:40.729 INFO:tasks.workunit.client.0.smithi019.stdout:6/826: mkdir d6/d4b/d6d/dda/d105/d10a 0 2024-02-22T15:50:40.730 INFO:tasks.workunit.client.1.smithi062.stdout:3/402: mknod d5/d16/d21/d32/d75/c7a 0 2024-02-22T15:50:40.730 INFO:tasks.workunit.client.1.smithi062.stdout:8/480: link d8/d9/d44/d4d/d51/d55/f77 d8/d9/d44/d4d/d51/f95 0 2024-02-22T15:50:40.730 INFO:tasks.workunit.client.1.smithi062.stdout:2/562: dwrite d1/d47/d39/f4b [0,4194304] 0 2024-02-22T15:50:40.733 INFO:tasks.workunit.client.1.smithi062.stdout:2/563: stat d1/d8/d62/l8c 0 2024-02-22T15:50:40.734 INFO:tasks.workunit.client.1.smithi062.stdout:6/522: sync 2024-02-22T15:50:40.734 INFO:tasks.workunit.client.1.smithi062.stdout:1/437: sync 2024-02-22T15:50:40.735 INFO:tasks.workunit.client.0.smithi019.stdout:6/827: rename d6/d31/f6b to d6/d4b/d6d/da3/ddb/f10b 0 2024-02-22T15:50:40.736 INFO:tasks.workunit.client.1.smithi062.stdout:9/415: sync 2024-02-22T15:50:40.738 INFO:tasks.workunit.client.0.smithi019.stdout:6/828: mknod d6/d4b/d6d/d7b/d78/c10c 0 2024-02-22T15:50:40.738 INFO:tasks.workunit.client.0.smithi019.stdout:2/589: sync 2024-02-22T15:50:40.738 INFO:tasks.workunit.client.0.smithi019.stdout:6/829: dread - d6/d31/d69/ffa zero size 2024-02-22T15:50:40.741 INFO:tasks.workunit.client.1.smithi062.stdout:9/416: dread - d4/d28/d50/f55 zero size 2024-02-22T15:50:40.742 INFO:tasks.workunit.client.0.smithi019.stdout:2/590: getdents d1/d17 0 2024-02-22T15:50:40.743 INFO:tasks.workunit.client.1.smithi062.stdout:2/564: write d1/d8/f15 [224625,95957] 0 2024-02-22T15:50:40.744 INFO:tasks.workunit.client.0.smithi019.stdout:2/591: mkdir d1/d17/d1c/d66/db9/ddd 0 2024-02-22T15:50:40.745 INFO:tasks.workunit.client.1.smithi062.stdout:5/371: dwrite d0/d3/da/d13/f2a [0,4194304] 0 2024-02-22T15:50:40.746 INFO:tasks.workunit.client.1.smithi062.stdout:6/523: chown d5/df/d40/d44/d4a/f28 24 1 2024-02-22T15:50:40.746 INFO:tasks.workunit.client.1.smithi062.stdout:1/438: write d1/d6/d36/d37/f8e [119543,37055] 0 2024-02-22T15:50:40.752 INFO:tasks.workunit.client.1.smithi062.stdout:2/565: chown d1/d1e/d38/f74 391465 1 2024-02-22T15:50:40.752 INFO:tasks.workunit.client.1.smithi062.stdout:9/417: creat d4/d28/f72 x:0 0 0 2024-02-22T15:50:40.752 INFO:tasks.workunit.client.1.smithi062.stdout:1/439: readlink d1/d6/de/d9/l41 0 2024-02-22T15:50:40.752 INFO:tasks.workunit.client.0.smithi019.stdout:0/608: dwrite d4/f92 [0,4194304] 0 2024-02-22T15:50:40.753 INFO:tasks.workunit.client.1.smithi062.stdout:5/372: mknod d0/d3/da/d13/c76 0 2024-02-22T15:50:40.754 INFO:tasks.workunit.client.1.smithi062.stdout:3/403: dwrite d5/d16/d21/d32/f4c [0,4194304] 0 2024-02-22T15:50:40.756 INFO:tasks.workunit.client.1.smithi062.stdout:8/481: dwrite d8/f12 [0,4194304] 0 2024-02-22T15:50:40.757 INFO:tasks.workunit.client.1.smithi062.stdout:1/440: fsync d1/d6/d36/d37/f86 0 2024-02-22T15:50:40.757 INFO:tasks.workunit.client.1.smithi062.stdout:6/524: creat d5/df/d40/d44/d4a/d41/d85/d9d/fa6 x:0 0 0 2024-02-22T15:50:40.758 INFO:tasks.workunit.client.1.smithi062.stdout:5/373: unlink d0/c2c 0 2024-02-22T15:50:40.761 INFO:tasks.workunit.client.1.smithi062.stdout:6/525: chown d5/df/d40/l9e 18710057 1 2024-02-22T15:50:40.765 INFO:tasks.workunit.client.1.smithi062.stdout:9/418: mkdir d4/d35/d3f/d57/d73 0 2024-02-22T15:50:40.767 INFO:tasks.workunit.client.1.smithi062.stdout:1/441: mkdir d1/d6/d7/d2b/d56/d95 0 2024-02-22T15:50:40.774 INFO:tasks.workunit.client.0.smithi019.stdout:2/592: write d1/d17/d21/f36 [2027769,651] 0 2024-02-22T15:50:40.775 INFO:tasks.workunit.client.1.smithi062.stdout:9/419: mknod d4/d35/d3f/d57/c74 0 2024-02-22T15:50:40.776 INFO:tasks.workunit.client.0.smithi019.stdout:2/593: dread - d1/d46/fab zero size 2024-02-22T15:50:40.776 INFO:tasks.workunit.client.1.smithi062.stdout:1/442: mkdir d1/d6/d7/d2b/d96 0 2024-02-22T15:50:40.779 INFO:tasks.workunit.client.1.smithi062.stdout:1/443: chown d1/d6/d7/d2b/d56/d95 121832219 1 2024-02-22T15:50:40.780 INFO:tasks.workunit.client.0.smithi019.stdout:0/609: dwrite d4/db/d76/d77/fca [0,4194304] 0 2024-02-22T15:50:40.782 INFO:tasks.workunit.client.1.smithi062.stdout:1/444: write d1/d6/de/f17 [1311387,62935] 0 2024-02-22T15:50:40.784 INFO:tasks.workunit.client.0.smithi019.stdout:0/610: chown d4/db/d1b/d40/d8c/fd0 16670716 1 2024-02-22T15:50:40.787 INFO:tasks.workunit.client.1.smithi062.stdout:6/526: link d5/df/d40/d44/d4a/d41/d85/d9d/l9b d5/dd/d33/d66/la7 0 2024-02-22T15:50:40.788 INFO:tasks.workunit.client.0.smithi019.stdout:2/594: dwrite d1/d17/fb2 [0,4194304] 0 2024-02-22T15:50:40.788 INFO:tasks.workunit.client.0.smithi019.stdout:0/611: rmdir d4/db/d49/d65 39 2024-02-22T15:50:40.788 INFO:tasks.workunit.client.1.smithi062.stdout:3/404: getdents d5/d16/d21/d32 0 2024-02-22T15:50:40.789 INFO:tasks.workunit.client.1.smithi062.stdout:5/374: getdents d0/d3/da/d13/d23/d4e/d5a 0 2024-02-22T15:50:40.790 INFO:tasks.workunit.client.1.smithi062.stdout:3/405: fsync d5/d26/f31 0 2024-02-22T15:50:40.795 INFO:tasks.workunit.client.1.smithi062.stdout:9/420: dwrite d4/d1c/f1d [0,4194304] 0 2024-02-22T15:50:40.797 INFO:tasks.workunit.client.1.smithi062.stdout:9/421: chown d4/d15/d3c/f59 3349 1 2024-02-22T15:50:40.798 INFO:tasks.workunit.client.0.smithi019.stdout:2/595: dwrite d1/d17/d38/fd9 [0,4194304] 0 2024-02-22T15:50:40.801 INFO:tasks.workunit.client.1.smithi062.stdout:5/375: creat d0/d3/d30/d71/f77 x:0 0 0 2024-02-22T15:50:40.801 INFO:tasks.workunit.client.1.smithi062.stdout:6/527: write d5/df/d40/d44/d4a/f28 [3300168,52671] 0 2024-02-22T15:50:40.802 INFO:tasks.workunit.client.0.smithi019.stdout:2/596: mknod d1/d46/cde 0 2024-02-22T15:50:40.809 INFO:tasks.workunit.client.1.smithi062.stdout:8/482: dread d8/f24 [0,4194304] 0 2024-02-22T15:50:40.810 INFO:tasks.workunit.client.0.smithi019.stdout:2/597: dwrite d1/d17/f47 [0,4194304] 0 2024-02-22T15:50:40.811 INFO:tasks.workunit.client.1.smithi062.stdout:9/422: rename d4/d35/d3f/d57 to d4/d15/d5f/d75 0 2024-02-22T15:50:40.814 INFO:tasks.workunit.client.0.smithi019.stdout:2/598: rename d1/d17/d21/d71/f2f to d1/d17/d1c/d7f/fdf 0 2024-02-22T15:50:40.815 INFO:tasks.workunit.client.1.smithi062.stdout:1/445: dwrite d1/d6/d7/d2b/d47/d63/f68 [0,4194304] 0 2024-02-22T15:50:40.816 INFO:tasks.workunit.client.0.smithi019.stdout:0/612: truncate f1 321186 0 2024-02-22T15:50:40.817 INFO:tasks.workunit.client.1.smithi062.stdout:0/543: sync 2024-02-22T15:50:40.817 INFO:tasks.workunit.client.0.smithi019.stdout:2/599: truncate d1/d17/d21/d51/f72 208404 0 2024-02-22T15:50:40.818 INFO:tasks.workunit.client.1.smithi062.stdout:2/566: dwrite d1/d47/d32/d52/d80/f9b [0,4194304] 0 2024-02-22T15:50:40.821 INFO:tasks.workunit.client.1.smithi062.stdout:0/544: write d1/d21/d85/fa3 [861353,96503] 0 2024-02-22T15:50:40.824 INFO:tasks.workunit.client.1.smithi062.stdout:5/376: chown d0/d3/da/d13/d23/d4e/f5d 159502 1 2024-02-22T15:50:40.827 INFO:tasks.workunit.client.0.smithi019.stdout:0/613: dwrite d4/db/d76/d77/fca [4194304,4194304] 0 2024-02-22T15:50:40.828 INFO:tasks.workunit.client.1.smithi062.stdout:3/406: rename d5/d16/d21/d55/d5e to d5/d16/d21/d32/d6a/d7b 0 2024-02-22T15:50:40.829 INFO:tasks.workunit.client.1.smithi062.stdout:0/545: read d1/dd/f6b [3740251,57749] 0 2024-02-22T15:50:40.830 INFO:tasks.workunit.client.1.smithi062.stdout:1/446: symlink d1/d6/d7/d4a/d65/l97 0 2024-02-22T15:50:40.830 INFO:tasks.workunit.client.1.smithi062.stdout:3/407: chown d5/d1a/f25 23 1 2024-02-22T15:50:40.831 INFO:tasks.workunit.client.1.smithi062.stdout:2/567: mknod d1/d47/d32/d52/cb9 0 2024-02-22T15:50:40.832 INFO:tasks.workunit.client.1.smithi062.stdout:1/447: write d1/d6/f1a [1821839,105800] 0 2024-02-22T15:50:40.834 INFO:tasks.workunit.client.0.smithi019.stdout:0/614: symlink d4/d67/ld2 0 2024-02-22T15:50:40.836 INFO:tasks.workunit.client.0.smithi019.stdout:2/600: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:40.838 INFO:tasks.workunit.client.0.smithi019.stdout:2/601: fdatasync d1/d17/fbd 0 2024-02-22T15:50:40.840 INFO:tasks.workunit.client.1.smithi062.stdout:6/528: creat d5/dd/d33/d66/fa8 x:0 0 0 2024-02-22T15:50:40.840 INFO:tasks.workunit.client.1.smithi062.stdout:1/448: truncate d1/d66/f75 575494 0 2024-02-22T15:50:40.842 INFO:tasks.workunit.client.0.smithi019.stdout:0/615: rename d4/db/d1b/d63/db0/da1/dbb/d7/c3a to d4/db/d24/cd3 0 2024-02-22T15:50:40.846 INFO:tasks.workunit.client.0.smithi019.stdout:2/602: rename d1/d17/d1c/d7f/ld4 to d1/d17/d21/le0 0 2024-02-22T15:50:40.847 INFO:tasks.workunit.client.0.smithi019.stdout:2/603: chown d1/d17/d21/d71/c81 16181 1 2024-02-22T15:50:40.849 INFO:tasks.workunit.client.1.smithi062.stdout:2/568: dwrite d1/f5 [0,4194304] 0 2024-02-22T15:50:40.849 INFO:tasks.workunit.client.1.smithi062.stdout:2/569: stat d1/d1e/d38/f54 0 2024-02-22T15:50:40.849 INFO:tasks.workunit.client.0.smithi019.stdout:2/604: mkdir d1/d17/d21/d71/de1 0 2024-02-22T15:50:40.850 INFO:tasks.workunit.client.0.smithi019.stdout:2/605: chown d1/da4 299 1 2024-02-22T15:50:40.850 INFO:tasks.workunit.client.1.smithi062.stdout:4/474: sync 2024-02-22T15:50:40.850 INFO:tasks.workunit.client.1.smithi062.stdout:7/419: sync 2024-02-22T15:50:40.851 INFO:tasks.workunit.client.1.smithi062.stdout:4/475: readlink d4/d1c/d2c/la1 0 2024-02-22T15:50:40.856 INFO:tasks.workunit.client.1.smithi062.stdout:9/423: link d4/d15/l3d d4/d28/d50/l76 0 2024-02-22T15:50:40.856 INFO:tasks.workunit.client.1.smithi062.stdout:0/546: readlink d1/d3/d4/d1a/d76/d8c/d2c/lac 0 2024-02-22T15:50:40.859 INFO:tasks.workunit.client.1.smithi062.stdout:5/377: rmdir d0/d3/d2f/d59/d67/d69/d3f/d73 0 2024-02-22T15:50:40.860 INFO:tasks.workunit.client.1.smithi062.stdout:8/483: write d8/d26/f47 [522853,76190] 0 2024-02-22T15:50:40.860 INFO:tasks.workunit.client.1.smithi062.stdout:1/449: unlink d1/c6b 0 2024-02-22T15:50:40.863 INFO:tasks.workunit.client.1.smithi062.stdout:0/547: creat d1/d21/d83/fbe x:0 0 0 2024-02-22T15:50:40.867 INFO:tasks.workunit.client.1.smithi062.stdout:1/450: fdatasync d1/d6/d7/f28 0 2024-02-22T15:50:40.867 INFO:tasks.workunit.client.1.smithi062.stdout:5/378: mkdir d0/d3/da/d13/d23/d78 0 2024-02-22T15:50:40.873 INFO:tasks.workunit.client.1.smithi062.stdout:4/476: dwrite d4/df/f14 [0,4194304] 0 2024-02-22T15:50:40.873 INFO:tasks.workunit.client.1.smithi062.stdout:0/548: truncate d1/d3/d4/d1e/f79 962849 0 2024-02-22T15:50:40.874 INFO:tasks.workunit.client.0.smithi019.stdout:0/616: write d4/db/d1b/d40/d44/d75/db7/f90 [1132740,9946] 0 2024-02-22T15:50:40.874 INFO:tasks.workunit.client.1.smithi062.stdout:8/484: write d8/d9/d44/f45 [879374,97465] 0 2024-02-22T15:50:40.875 INFO:tasks.workunit.client.1.smithi062.stdout:1/451: write d1/d6/d14/f6d [634920,123869] 0 2024-02-22T15:50:40.876 INFO:tasks.workunit.client.1.smithi062.stdout:4/477: fdatasync d4/f53 0 2024-02-22T15:50:40.878 INFO:tasks.workunit.client.0.smithi019.stdout:0/617: creat d4/d9/da7/db2/fd4 x:0 0 0 2024-02-22T15:50:40.879 INFO:tasks.workunit.client.1.smithi062.stdout:6/529: link d5/f1a d5/dd/d33/fa9 0 2024-02-22T15:50:40.881 INFO:tasks.workunit.client.0.smithi019.stdout:0/618: write d4/db/d1b/d40/d44/d75/f82 [887588,46096] 0 2024-02-22T15:50:40.883 INFO:tasks.workunit.client.1.smithi062.stdout:6/530: readlink d5/df/d40/d44/d4a/l62 0 2024-02-22T15:50:40.885 INFO:tasks.workunit.client.1.smithi062.stdout:9/424: dwrite d4/fc [8388608,4194304] 0 2024-02-22T15:50:40.886 INFO:tasks.workunit.client.0.smithi019.stdout:0/619: creat d4/d22/d2c/d68/d6d/fd5 x:0 0 0 2024-02-22T15:50:40.886 INFO:tasks.workunit.client.0.smithi019.stdout:2/606: dread d1/d17/d21/d51/fa1 [0,4194304] 0 2024-02-22T15:50:40.887 INFO:tasks.workunit.client.1.smithi062.stdout:3/408: truncate d5/d16/d21/d32/f4c 2109062 0 2024-02-22T15:50:40.888 INFO:tasks.workunit.client.0.smithi019.stdout:2/607: write d1/f14 [3650245,32321] 0 2024-02-22T15:50:40.890 INFO:tasks.workunit.client.1.smithi062.stdout:0/549: symlink d1/d3/d4/d1a/d76/d8c/d92/lbf 0 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:8/621: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:5/660: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:3/508: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:9/630: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:4/628: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:7/958: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.0.smithi019.stdout:1/678: sync 2024-02-22T15:50:40.891 INFO:tasks.workunit.client.1.smithi062.stdout:8/485: creat d8/d86/f96 x:0 0 0 2024-02-22T15:50:40.892 INFO:tasks.workunit.client.0.smithi019.stdout:0/620: getdents d4/db/d24/d80/daf 0 2024-02-22T15:50:40.894 INFO:tasks.workunit.client.1.smithi062.stdout:9/425: mkdir d4/d15/d5f/d6a/d77 0 2024-02-22T15:50:40.894 INFO:tasks.workunit.client.1.smithi062.stdout:0/550: chown d1/d3/d4/d1a/l27 0 1 2024-02-22T15:50:40.895 INFO:tasks.workunit.client.1.smithi062.stdout:4/478: dwrite d4/fd [0,4194304] 0 2024-02-22T15:50:40.895 INFO:tasks.workunit.client.1.smithi062.stdout:8/486: dread - d8/d9/d2c/d49/d5e/f6a zero size 2024-02-22T15:50:40.896 INFO:tasks.workunit.client.0.smithi019.stdout:3/509: write d9/d14/f1b [3497052,115140] 0 2024-02-22T15:50:40.900 INFO:tasks.workunit.client.0.smithi019.stdout:0/621: truncate d4/db/d24/d80/fa0 625444 0 2024-02-22T15:50:40.901 INFO:tasks.workunit.client.1.smithi062.stdout:1/452: getdents d1/d6/d7/d2b/d56 0 2024-02-22T15:50:40.901 INFO:tasks.workunit.client.0.smithi019.stdout:4/629: symlink d2/db/d14/d23/d21/d35/lcc 0 2024-02-22T15:50:40.904 INFO:tasks.workunit.client.0.smithi019.stdout:0/622: chown d4/db/d1b/d63/db0/da1/dbb/fe 1 1 2024-02-22T15:50:40.905 INFO:tasks.workunit.client.0.smithi019.stdout:2/608: write d1/d17/fb2 [4606068,63038] 0 2024-02-22T15:50:40.906 INFO:tasks.workunit.client.1.smithi062.stdout:3/409: creat d5/d26/d2d/d3f/f7c x:0 0 0 2024-02-22T15:50:40.908 INFO:tasks.workunit.client.0.smithi019.stdout:7/959: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f13a x:0 0 0 2024-02-22T15:50:40.909 INFO:tasks.workunit.client.1.smithi062.stdout:0/551: chown d1/d3/d4/d1a/d1d/fa7 2 1 2024-02-22T15:50:40.909 INFO:tasks.workunit.client.0.smithi019.stdout:7/960: chown d9/d26/dc4/dd9 431381535 1 2024-02-22T15:50:40.910 INFO:tasks.workunit.client.1.smithi062.stdout:6/531: link d5/df/d40/d44/d76/d56/f61 d5/df/d40/d44/d4a/faa 0 2024-02-22T15:50:40.910 INFO:tasks.workunit.client.0.smithi019.stdout:4/630: truncate d2/f32 54957 0 2024-02-22T15:50:40.910 INFO:tasks.workunit.client.1.smithi062.stdout:9/426: fdatasync d4/d15/d5f/d75/f68 0 2024-02-22T15:50:40.911 INFO:tasks.workunit.client.1.smithi062.stdout:0/552: readlink d1/d3/d4/d1a/l27 0 2024-02-22T15:50:40.911 INFO:tasks.workunit.client.0.smithi019.stdout:8/622: getdents d9/d14/d6f/d7e/d94/db8 0 2024-02-22T15:50:40.913 INFO:tasks.workunit.client.0.smithi019.stdout:9/631: dwrite d2/d7/dc/fb7 [0,4194304] 0 2024-02-22T15:50:40.914 INFO:tasks.workunit.client.1.smithi062.stdout:8/487: rename d8/d9/d2c/d49/d5e/f6a to d8/d28/d70/f97 0 2024-02-22T15:50:40.914 INFO:tasks.workunit.client.0.smithi019.stdout:5/661: truncate d8/fe 1426414 0 2024-02-22T15:50:40.915 INFO:tasks.workunit.client.1.smithi062.stdout:8/488: write d8/d9/d2c/d74/f89 [857049,29685] 0 2024-02-22T15:50:40.916 INFO:tasks.workunit.client.0.smithi019.stdout:7/961: write d9/d26/f87 [2680438,23591] 0 2024-02-22T15:50:40.917 INFO:tasks.workunit.client.0.smithi019.stdout:5/662: chown d8/d11/f1c 916 1 2024-02-22T15:50:40.918 INFO:tasks.workunit.client.0.smithi019.stdout:0/623: creat d4/db/d49/dc2/dc1/fd6 x:0 0 0 2024-02-22T15:50:40.920 INFO:tasks.workunit.client.1.smithi062.stdout:3/410: mknod d5/d16/d21/d32/d6a/d7b/c7d 0 2024-02-22T15:50:40.921 INFO:tasks.workunit.client.1.smithi062.stdout:9/427: mknod d4/d15/d5f/d6a/c78 0 2024-02-22T15:50:40.921 INFO:tasks.workunit.client.1.smithi062.stdout:6/532: mknod d5/df/d40/d44/d4a/d41/d43/cab 0 2024-02-22T15:50:40.922 INFO:tasks.workunit.client.1.smithi062.stdout:8/489: dread d8/f12 [0,4194304] 0 2024-02-22T15:50:40.924 INFO:tasks.workunit.client.0.smithi019.stdout:4/631: write d2/db/d14/d23/d21/f2c [1152963,100152] 0 2024-02-22T15:50:40.925 INFO:tasks.workunit.client.1.smithi062.stdout:0/553: mknod d1/cc0 0 2024-02-22T15:50:40.927 INFO:tasks.workunit.client.0.smithi019.stdout:8/623: link d9/d14/d17/fbe d9/d14/fce 0 2024-02-22T15:50:40.929 INFO:tasks.workunit.client.0.smithi019.stdout:7/962: creat d9/d38/d41/d45/d61/d6a/f13b x:0 0 0 2024-02-22T15:50:40.931 INFO:tasks.workunit.client.0.smithi019.stdout:4/632: chown d2/db/d54/d8c/fa4 126132 1 2024-02-22T15:50:40.933 INFO:tasks.workunit.client.1.smithi062.stdout:6/533: mkdir d5/df/d40/dac 0 2024-02-22T15:50:40.933 INFO:tasks.workunit.client.1.smithi062.stdout:1/453: truncate d1/d6/d7/f29 1562183 0 2024-02-22T15:50:40.934 INFO:tasks.workunit.client.1.smithi062.stdout:8/490: symlink d8/d86/l98 0 2024-02-22T15:50:40.935 INFO:tasks.workunit.client.1.smithi062.stdout:5/379: dread d0/d3/d2f/d59/d67/d69/d3f/f51 [0,4194304] 0 2024-02-22T15:50:40.935 INFO:tasks.workunit.client.0.smithi019.stdout:2/609: dwrite d1/d17/f47 [4194304,4194304] 0 2024-02-22T15:50:40.936 INFO:tasks.workunit.client.1.smithi062.stdout:8/491: fdatasync d8/d9/d48/f57 0 2024-02-22T15:50:40.936 INFO:tasks.workunit.client.0.smithi019.stdout:8/624: write d9/d14/fac [1024381,73671] 0 2024-02-22T15:50:40.938 INFO:tasks.workunit.client.1.smithi062.stdout:5/380: chown d0/d3/da/d13/d23/d31/f43 1117 1 2024-02-22T15:50:40.940 INFO:tasks.workunit.client.1.smithi062.stdout:3/411: rename d5/f41 to d5/d16/d56/f7e 0 2024-02-22T15:50:40.940 INFO:tasks.workunit.client.1.smithi062.stdout:9/428: mknod d4/c79 0 2024-02-22T15:50:40.941 INFO:tasks.workunit.client.0.smithi019.stdout:0/624: rename d4/db/d1b/d63/db0/da1/dbb/dc/l34 to d4/db/d1b/d40/d44/d75/ld7 0 2024-02-22T15:50:40.941 INFO:tasks.workunit.client.1.smithi062.stdout:0/554: mkdir d1/d3/d4/d1a/d76/d8c/d5d/dc1 0 2024-02-22T15:50:40.942 INFO:tasks.workunit.client.1.smithi062.stdout:9/429: stat d4/d28/d50/f55 0 2024-02-22T15:50:40.942 INFO:tasks.workunit.client.0.smithi019.stdout:4/633: chown d2/db/d14/d23/f28 0 1 2024-02-22T15:50:40.943 INFO:tasks.workunit.client.0.smithi019.stdout:3/510: dread f3 [8388608,4194304] 0 2024-02-22T15:50:40.943 INFO:tasks.workunit.client.1.smithi062.stdout:5/381: rmdir d0/d3/da/d13/d23/d4e/d5a 39 2024-02-22T15:50:40.945 INFO:tasks.workunit.client.1.smithi062.stdout:1/454: rename d1/d6/d7/d2b/d47/d24/f78 to d1/d6/d7/f98 0 2024-02-22T15:50:40.946 INFO:tasks.workunit.client.1.smithi062.stdout:9/430: dread d4/d1c/f58 [0,4194304] 0 2024-02-22T15:50:40.946 INFO:tasks.workunit.client.0.smithi019.stdout:0/625: write d4/d22/d2c/d68/fa6 [2711624,114798] 0 2024-02-22T15:50:40.946 INFO:tasks.workunit.client.0.smithi019.stdout:2/610: fsync d1/d46/f98 0 2024-02-22T15:50:40.947 INFO:tasks.workunit.client.1.smithi062.stdout:5/382: chown d0/d3/d2f/d59/d67/d69/f41 7148743 1 2024-02-22T15:50:40.947 INFO:tasks.workunit.client.1.smithi062.stdout:8/492: fdatasync d8/d9/f18 0 2024-02-22T15:50:40.947 INFO:tasks.workunit.client.1.smithi062.stdout:3/412: mknod d5/d26/d2d/d3f/c7f 0 2024-02-22T15:50:40.949 INFO:tasks.workunit.client.0.smithi019.stdout:7/963: dwrite d9/d38/df3/f113 [4194304,4194304] 0 2024-02-22T15:50:40.949 INFO:tasks.workunit.client.0.smithi019.stdout:8/625: creat d9/d14/d6f/d7e/d94/d9c/fcf x:0 0 0 2024-02-22T15:50:40.949 INFO:tasks.workunit.client.1.smithi062.stdout:9/431: write d4/d35/f49 [947329,110014] 0 2024-02-22T15:50:40.950 INFO:tasks.workunit.client.1.smithi062.stdout:1/455: unlink d1/d6/l15 0 2024-02-22T15:50:40.952 INFO:tasks.workunit.client.1.smithi062.stdout:8/493: symlink d8/d42/d8f/l99 0 2024-02-22T15:50:40.952 INFO:tasks.workunit.client.0.smithi019.stdout:3/511: mknod d9/d26/d2d/d39/ca2 0 2024-02-22T15:50:40.956 INFO:tasks.workunit.client.0.smithi019.stdout:9/632: dread d2/d7/f44 [0,4194304] 0 2024-02-22T15:50:40.958 INFO:tasks.workunit.client.0.smithi019.stdout:4/634: rename d2/db/d14/d23/l1c to d2/db/d54/d8c/lcd 0 2024-02-22T15:50:40.958 INFO:tasks.workunit.client.0.smithi019.stdout:4/635: write d2/db/fa3 [692004,40841] 0 2024-02-22T15:50:40.959 INFO:tasks.workunit.client.1.smithi062.stdout:8/494: truncate d8/d9/d44/d4d/d51/d55/f92 455349 0 2024-02-22T15:50:40.960 INFO:tasks.workunit.client.1.smithi062.stdout:5/383: mkdir d0/d3/da/d13/d57/d79 0 2024-02-22T15:50:40.960 INFO:tasks.workunit.client.0.smithi019.stdout:8/626: mkdir d9/d14/d6f/d7e/d94/daa/dd0 0 2024-02-22T15:50:40.962 INFO:tasks.workunit.client.0.smithi019.stdout:3/512: mkdir d9/d22/da3 0 2024-02-22T15:50:40.963 INFO:tasks.workunit.client.0.smithi019.stdout:7/964: rmdir d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded 39 2024-02-22T15:50:40.964 INFO:tasks.workunit.client.1.smithi062.stdout:0/555: rename d1/d13/f31 to d1/fc2 0 2024-02-22T15:50:40.965 INFO:tasks.workunit.client.0.smithi019.stdout:7/965: truncate d9/d38/d41/d45/db0/df7/f100 719205 0 2024-02-22T15:50:40.965 INFO:tasks.workunit.client.0.smithi019.stdout:0/626: symlink d4/db/d1b/d63/db0/da1/dbb/ld8 0 2024-02-22T15:50:40.968 INFO:tasks.workunit.client.0.smithi019.stdout:9/633: creat d2/d7/dc/d27/d31/d79/fe1 x:0 0 0 2024-02-22T15:50:40.971 INFO:tasks.workunit.client.0.smithi019.stdout:3/513: mknod d9/d22/d49/ca4 0 2024-02-22T15:50:40.971 INFO:tasks.workunit.client.1.smithi062.stdout:0/556: readlink d1/d3/d4/d1a/d76/l7a 0 2024-02-22T15:50:40.974 INFO:tasks.workunit.client.0.smithi019.stdout:3/514: write d9/d11/f8a [658634,88715] 0 2024-02-22T15:50:40.974 INFO:tasks.workunit.client.1.smithi062.stdout:3/413: rename d5/cf to d5/c80 0 2024-02-22T15:50:40.975 INFO:tasks.workunit.client.0.smithi019.stdout:7/966: dwrite d9/d4c/d5b/f65 [0,4194304] 0 2024-02-22T15:50:40.976 INFO:tasks.workunit.client.0.smithi019.stdout:9/634: symlink d2/d7/dc/d27/dbc/dcc/le2 0 2024-02-22T15:50:40.977 INFO:tasks.workunit.client.1.smithi062.stdout:5/384: symlink d0/d3/da/d13/d23/d78/l7a 0 2024-02-22T15:50:40.979 INFO:tasks.workunit.client.0.smithi019.stdout:0/627: link d4/db/d76/d77/faa d4/d22/d3d/fd9 0 2024-02-22T15:50:40.980 INFO:tasks.workunit.client.0.smithi019.stdout:7/967: write d9/d26/d77/f95 [383463,46615] 0 2024-02-22T15:50:40.981 INFO:tasks.workunit.client.0.smithi019.stdout:2/611: dwrite d1/f1f [0,4194304] 0 2024-02-22T15:50:40.982 INFO:tasks.workunit.client.0.smithi019.stdout:8/627: rename d9/d14/d1f/d20/l57 to d9/d14/d17/d3d/ld1 0 2024-02-22T15:50:40.984 INFO:tasks.workunit.client.0.smithi019.stdout:4/636: link d2/db/d14/d23/d67/d8f/lbd d2/d3f/lce 0 2024-02-22T15:50:40.984 INFO:tasks.workunit.client.1.smithi062.stdout:6/534: dwrite d5/dd/d33/f48 [0,4194304] 0 2024-02-22T15:50:40.985 INFO:tasks.workunit.client.0.smithi019.stdout:0/628: readlink d4/db/d1b/d63/db0/da1/dbb/ld8 0 2024-02-22T15:50:40.986 INFO:tasks.workunit.client.1.smithi062.stdout:0/557: dwrite d1/d21/d83/fbe [0,4194304] 0 2024-02-22T15:50:40.990 INFO:tasks.workunit.client.1.smithi062.stdout:0/558: chown d1/d21/d85/d8e/d97/fb7 286861 1 2024-02-22T15:50:40.991 INFO:tasks.workunit.client.0.smithi019.stdout:2/612: write d1/d17/fb1 [804459,50493] 0 2024-02-22T15:50:40.991 INFO:tasks.workunit.client.0.smithi019.stdout:4/637: write d2/db/d92/f91 [617326,93042] 0 2024-02-22T15:50:40.994 INFO:tasks.workunit.client.1.smithi062.stdout:3/414: mknod d5/d16/d21/d36/d5a/c81 0 2024-02-22T15:50:40.995 INFO:tasks.workunit.client.0.smithi019.stdout:7/968: mkdir d9/d38/d41/d45/d61/d6a/d13c 0 2024-02-22T15:50:40.996 INFO:tasks.workunit.client.0.smithi019.stdout:8/628: symlink d9/d14/d17/d3d/d83/ld2 0 2024-02-22T15:50:40.997 INFO:tasks.workunit.client.1.smithi062.stdout:3/415: read - d5/d16/d21/f60 zero size 2024-02-22T15:50:40.998 INFO:tasks.workunit.client.0.smithi019.stdout:3/515: rmdir d9/d22/da3 0 2024-02-22T15:50:40.998 INFO:tasks.workunit.client.0.smithi019.stdout:0/629: mkdir d4/db/d1b/d40/d8c/dda 0 2024-02-22T15:50:41.000 INFO:tasks.workunit.client.0.smithi019.stdout:0/630: chown d4/d9/da7/db2 720700 1 2024-02-22T15:50:41.001 INFO:tasks.workunit.client.1.smithi062.stdout:6/535: creat d5/df/d40/d44/d4a/fad x:0 0 0 2024-02-22T15:50:41.002 INFO:tasks.workunit.client.0.smithi019.stdout:9/635: dwrite d2/d1f/d2e/d48/d45/fd3 [0,4194304] 0 2024-02-22T15:50:41.002 INFO:tasks.workunit.client.0.smithi019.stdout:0/631: write f2 [8782206,96611] 0 2024-02-22T15:50:41.003 INFO:tasks.workunit.client.1.smithi062.stdout:0/559: dwrite d1/d21/f48 [0,4194304] 0 2024-02-22T15:50:41.005 INFO:tasks.workunit.client.0.smithi019.stdout:0/632: readlink d4/d67/l7c 0 2024-02-22T15:50:41.005 INFO:tasks.workunit.client.1.smithi062.stdout:6/536: fdatasync d5/df/d40/d44/d4a/d41/d43/f58 0 2024-02-22T15:50:41.006 INFO:tasks.workunit.client.0.smithi019.stdout:0/633: write d4/db/d49/dc2/dc1/fd6 [438667,99121] 0 2024-02-22T15:50:41.007 INFO:tasks.workunit.client.1.smithi062.stdout:5/385: mknod d0/d3/da/d13/d57/d79/c7b 0 2024-02-22T15:50:41.007 INFO:tasks.workunit.client.1.smithi062.stdout:6/537: readlink d5/df/d40/d44/d4a/l64 0 2024-02-22T15:50:41.009 INFO:tasks.workunit.client.0.smithi019.stdout:9/636: write d2/d1f/d2e/f42 [2625637,123637] 0 2024-02-22T15:50:41.009 INFO:tasks.workunit.client.0.smithi019.stdout:0/634: fdatasync d4/d22/d2c/d68/fa6 0 2024-02-22T15:50:41.011 INFO:tasks.workunit.client.1.smithi062.stdout:9/432: truncate d4/d35/f37 2349878 0 2024-02-22T15:50:41.012 INFO:tasks.workunit.client.0.smithi019.stdout:9/637: chown d2/d1f/d2e/d57/fc7 90334 1 2024-02-22T15:50:41.014 INFO:tasks.workunit.client.0.smithi019.stdout:3/516: link d9/d26/d3f/l63 d9/d22/d27/d6d/la5 0 2024-02-22T15:50:41.014 INFO:tasks.workunit.client.0.smithi019.stdout:2/613: dread d1/d17/d21/f87 [0,4194304] 0 2024-02-22T15:50:41.015 INFO:tasks.workunit.client.1.smithi062.stdout:3/416: mknod d5/d26/c82 0 2024-02-22T15:50:41.016 INFO:tasks.workunit.client.1.smithi062.stdout:1/456: truncate d1/d6/d7/d2b/d49/f6a 3709745 0 2024-02-22T15:50:41.017 INFO:tasks.workunit.client.1.smithi062.stdout:5/386: mkdir d0/d3/d30/d71/d7c 0 2024-02-22T15:50:41.017 INFO:tasks.workunit.client.1.smithi062.stdout:9/433: rmdir d4/d15/d5f 39 2024-02-22T15:50:41.018 INFO:tasks.workunit.client.0.smithi019.stdout:9/638: readlink d2/d1f/d2e/d48/d36/lad 0 2024-02-22T15:50:41.018 INFO:tasks.workunit.client.1.smithi062.stdout:9/434: chown f2 76 1 2024-02-22T15:50:41.020 INFO:tasks.workunit.client.1.smithi062.stdout:6/538: mkdir d5/df/d40/d44/d76/dae 0 2024-02-22T15:50:41.020 INFO:tasks.workunit.client.0.smithi019.stdout:3/517: dread d9/d26/d3f/f4a [0,4194304] 0 2024-02-22T15:50:41.021 INFO:tasks.workunit.client.1.smithi062.stdout:0/560: fdatasync d1/d3/de/f3a 0 2024-02-22T15:50:41.022 INFO:tasks.workunit.client.1.smithi062.stdout:0/561: chown d1/d3/d4/d1e/d74/d68 0 1 2024-02-22T15:50:41.022 INFO:tasks.workunit.client.0.smithi019.stdout:9/639: write d2/d1f/d2e/d57/d60/d8c/d66/fba [2893681,48025] 0 2024-02-22T15:50:41.022 INFO:tasks.workunit.client.1.smithi062.stdout:3/417: symlink d5/d16/d21/d36/l83 0 2024-02-22T15:50:41.023 INFO:tasks.workunit.client.0.smithi019.stdout:3/518: symlink d9/d26/d2d/d53/da0/la6 0 2024-02-22T15:50:41.024 INFO:tasks.workunit.client.1.smithi062.stdout:1/457: mknod d1/d6/d7/c99 0 2024-02-22T15:50:41.024 INFO:tasks.workunit.client.0.smithi019.stdout:0/635: dread d4/db/d1b/d40/d44/d75/f82 [0,4194304] 0 2024-02-22T15:50:41.025 INFO:tasks.workunit.client.0.smithi019.stdout:9/640: creat d2/d7/dc/dd6/fe3 x:0 0 0 2024-02-22T15:50:41.026 INFO:tasks.workunit.client.1.smithi062.stdout:5/387: creat d0/d3/da/d13/d23/d78/f7d x:0 0 0 2024-02-22T15:50:41.026 INFO:tasks.workunit.client.0.smithi019.stdout:3/519: mkdir d9/d65/da7 0 2024-02-22T15:50:41.026 INFO:tasks.workunit.client.1.smithi062.stdout:1/458: chown d1/d6/d7/d2b/d47/d63/f68 115725722 1 2024-02-22T15:50:41.027 INFO:tasks.workunit.client.1.smithi062.stdout:9/435: creat d4/d28/d50/f7a x:0 0 0 2024-02-22T15:50:41.028 INFO:tasks.workunit.client.1.smithi062.stdout:3/418: mkdir d5/d16/d21/d84 0 2024-02-22T15:50:41.029 INFO:tasks.workunit.client.1.smithi062.stdout:6/539: mknod d5/d8a/caf 0 2024-02-22T15:50:41.031 INFO:tasks.workunit.client.0.smithi019.stdout:0/636: link d4/d22/l64 d4/d22/ldb 0 2024-02-22T15:50:41.032 INFO:tasks.workunit.client.0.smithi019.stdout:9/641: dread d2/d7/dc/fb7 [0,4194304] 0 2024-02-22T15:50:41.033 INFO:tasks.workunit.client.1.smithi062.stdout:1/459: chown d1/d6/d36/d37/l39 9203830 1 2024-02-22T15:50:41.034 INFO:tasks.workunit.client.0.smithi019.stdout:3/520: dread d9/d26/d2d/f4b [0,4194304] 0 2024-02-22T15:50:41.034 INFO:tasks.workunit.client.0.smithi019.stdout:2/614: dread d1/d17/fae [0,4194304] 0 2024-02-22T15:50:41.034 INFO:tasks.workunit.client.1.smithi062.stdout:3/419: mkdir d5/d16/d21/d36/d5a/d85 0 2024-02-22T15:50:41.035 INFO:tasks.workunit.client.0.smithi019.stdout:8/629: readlink d9/d14/d17/d3d/ld1 0 2024-02-22T15:50:41.036 INFO:tasks.workunit.client.1.smithi062.stdout:3/420: write f3 [3749070,59418] 0 2024-02-22T15:50:41.037 INFO:tasks.workunit.client.1.smithi062.stdout:9/436: chown d4/d15/l3d 46627 1 2024-02-22T15:50:41.040 INFO:tasks.workunit.client.0.smithi019.stdout:9/642: rename d2/d1f/d2e/d57/d60/d8c/ddf to d2/d7/d52/dc0/de4 0 2024-02-22T15:50:41.040 INFO:tasks.workunit.client.1.smithi062.stdout:0/562: dwrite d1/d21/d83/fbe [0,4194304] 0 2024-02-22T15:50:41.041 INFO:tasks.workunit.client.0.smithi019.stdout:8/630: chown d9/d14/d17/l55 71415085 1 2024-02-22T15:50:41.043 INFO:tasks.workunit.client.1.smithi062.stdout:5/388: dread d0/d3/da/f53 [0,4194304] 0 2024-02-22T15:50:41.045 INFO:tasks.workunit.client.0.smithi019.stdout:9/643: symlink d2/d7/dc/d27/dbc/le5 0 2024-02-22T15:50:41.046 INFO:tasks.workunit.client.1.smithi062.stdout:5/389: dread - d0/d3/d2f/d59/d67/d69/d3f/f4b zero size 2024-02-22T15:50:41.047 INFO:tasks.workunit.client.0.smithi019.stdout:8/631: rename d9/d14/d17/d3d/d3f/d7d to d9/d14/d6f/d7e/d94/d9a/dd3 0 2024-02-22T15:50:41.052 INFO:tasks.workunit.client.0.smithi019.stdout:4/638: truncate d2/db/d14/d23/f65 1232975 0 2024-02-22T15:50:41.055 INFO:tasks.workunit.client.1.smithi062.stdout:1/460: creat d1/d6/de/f9a x:0 0 0 2024-02-22T15:50:41.055 INFO:tasks.workunit.client.1.smithi062.stdout:2/570: sync 2024-02-22T15:50:41.055 INFO:tasks.workunit.client.1.smithi062.stdout:9/437: creat d4/d15/d5f/f7b x:0 0 0 2024-02-22T15:50:41.055 INFO:tasks.workunit.client.1.smithi062.stdout:5/390: rmdir d0/d3/d2f/d59/d67/d69 39 2024-02-22T15:50:41.055 INFO:tasks.workunit.client.1.smithi062.stdout:2/571: chown d1/d47/d32/d52/d80 128227 1 2024-02-22T15:50:41.057 INFO:tasks.workunit.client.0.smithi019.stdout:7/969: dwrite d9/f19 [4194304,4194304] 0 2024-02-22T15:50:41.058 INFO:tasks.workunit.client.0.smithi019.stdout:2/615: write d1/de/f4e [4282410,52581] 0 2024-02-22T15:50:41.059 INFO:tasks.workunit.client.0.smithi019.stdout:0/637: dread d4/db/d1b/f1f [0,4194304] 0 2024-02-22T15:50:41.060 INFO:tasks.workunit.client.0.smithi019.stdout:4/639: mknod d2/db/d14/d23/d21/d35/ccf 0 2024-02-22T15:50:41.060 INFO:tasks.workunit.client.1.smithi062.stdout:6/540: dwrite d5/df/f17 [0,4194304] 0 2024-02-22T15:50:41.060 INFO:tasks.workunit.client.1.smithi062.stdout:0/563: rename d1/d21 to d1/d3/d4/d1a/d76/d8c/d5d/dc3 0 2024-02-22T15:50:41.063 INFO:tasks.workunit.client.0.smithi019.stdout:2/616: fdatasync d1/de/f5a 0 2024-02-22T15:50:41.064 INFO:tasks.workunit.client.1.smithi062.stdout:2/572: chown d1/l34 3649 1 2024-02-22T15:50:41.064 INFO:tasks.workunit.client.1.smithi062.stdout:5/391: chown d0/d3/da/d13/l29 1637 1 2024-02-22T15:50:41.066 INFO:tasks.workunit.client.0.smithi019.stdout:9/644: write d2/d1f/d2e/d57/d60/f92 [32642,47727] 0 2024-02-22T15:50:41.067 INFO:tasks.workunit.client.1.smithi062.stdout:3/421: rename d5/d16/d21/d36/d4a/c59 to d5/d26/c86 0 2024-02-22T15:50:41.069 INFO:tasks.workunit.client.1.smithi062.stdout:6/541: mknod d5/df/d32/d47/cb0 0 2024-02-22T15:50:41.069 INFO:tasks.workunit.client.0.smithi019.stdout:4/640: write d2/db/d92/f91 [17466,66583] 0 2024-02-22T15:50:41.070 INFO:tasks.workunit.client.0.smithi019.stdout:7/970: creat d9/d38/d41/d45/db0/df7/dfb/d12a/f13d x:0 0 0 2024-02-22T15:50:41.070 INFO:tasks.workunit.client.0.smithi019.stdout:0/638: write d4/f56 [3814325,79754] 0 2024-02-22T15:50:41.071 INFO:tasks.workunit.client.1.smithi062.stdout:3/422: write d5/d26/f31 [5126071,13881] 0 2024-02-22T15:50:41.073 INFO:tasks.workunit.client.1.smithi062.stdout:6/542: chown d5/df/f30 129 1 2024-02-22T15:50:41.074 INFO:tasks.workunit.client.1.smithi062.stdout:0/564: chown d1/d3/d4/d1a/c6e 761515813 1 2024-02-22T15:50:41.074 INFO:tasks.workunit.client.1.smithi062.stdout:1/461: dwrite d1/d6/d7/d2b/d49/f51 [0,4194304] 0 2024-02-22T15:50:41.076 INFO:tasks.workunit.client.0.smithi019.stdout:6/830: sync 2024-02-22T15:50:41.077 INFO:tasks.workunit.client.0.smithi019.stdout:8/632: dread d9/d14/d1f/f21 [0,4194304] 0 2024-02-22T15:50:41.077 INFO:tasks.workunit.client.1.smithi062.stdout:0/565: read - d1/d3/d4/d1a/d76/fb1 zero size 2024-02-22T15:50:41.077 INFO:tasks.workunit.client.1.smithi062.stdout:6/543: readlink d5/df/d40/d44/d78/l3a 0 2024-02-22T15:50:41.078 INFO:tasks.workunit.client.0.smithi019.stdout:9/645: rmdir d2/d7/d52/dc0 39 2024-02-22T15:50:41.079 INFO:tasks.workunit.client.1.smithi062.stdout:5/392: creat d0/d3/d2f/d59/d67/f7e x:0 0 0 2024-02-22T15:50:41.079 INFO:tasks.workunit.client.0.smithi019.stdout:7/971: dread - d9/d38/d41/d45/d61/d6a/f13b zero size 2024-02-22T15:50:41.080 INFO:tasks.workunit.client.0.smithi019.stdout:4/641: mknod d2/db/d14/d23/d21/d42/cd0 0 2024-02-22T15:50:41.080 INFO:tasks.workunit.client.1.smithi062.stdout:6/544: mknod d5/df/d40/d44/d76/cb1 0 2024-02-22T15:50:41.082 INFO:tasks.workunit.client.1.smithi062.stdout:5/393: chown d0/d3/d2f/d59/d67/d69/d3f/f5c 5452 1 2024-02-22T15:50:41.082 INFO:tasks.workunit.client.0.smithi019.stdout:9/646: rmdir d2/dc8 39 2024-02-22T15:50:41.082 INFO:tasks.workunit.client.1.smithi062.stdout:5/394: chown d0/d3/da/c1c 0 1 2024-02-22T15:50:41.084 INFO:tasks.workunit.client.1.smithi062.stdout:5/395: rename d0/d3/da to d0/d3/da/d32/d4f/d72/d7f 22 2024-02-22T15:50:41.085 INFO:tasks.workunit.client.0.smithi019.stdout:6/831: chown d6/fe9 1652524346 1 2024-02-22T15:50:41.085 INFO:tasks.workunit.client.0.smithi019.stdout:2/617: dwrite d1/d17/d21/f87 [0,4194304] 0 2024-02-22T15:50:41.087 INFO:tasks.workunit.client.0.smithi019.stdout:0/639: truncate d4/db/d1b/d63/db0/da1/dbb/fe 8480607 0 2024-02-22T15:50:41.090 INFO:tasks.workunit.client.1.smithi062.stdout:5/396: unlink d0/d3/d2f/d59/d67/f7e 0 2024-02-22T15:50:41.091 INFO:tasks.workunit.client.1.smithi062.stdout:5/397: creat d0/d3/da/d13/d23/d4e/d6d/f80 x:0 0 0 2024-02-22T15:50:41.100 INFO:tasks.workunit.client.1.smithi062.stdout:2/573: truncate d1/d47/d39/d5d/f63 148501 0 2024-02-22T15:50:41.102 INFO:tasks.workunit.client.1.smithi062.stdout:6/545: write d5/df/d40/d44/d4a/faa [2233064,5348] 0 2024-02-22T15:50:41.104 INFO:tasks.workunit.client.1.smithi062.stdout:5/398: dwrite d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:41.105 INFO:tasks.workunit.client.1.smithi062.stdout:2/574: mkdir d1/d47/d32/d52/dba 0 2024-02-22T15:50:41.107 INFO:tasks.workunit.client.1.smithi062.stdout:9/438: dwrite d4/d15/f44 [0,4194304] 0 2024-02-22T15:50:41.109 INFO:tasks.workunit.client.1.smithi062.stdout:6/546: write d5/df/d40/d44/d78/f5c [82353,6340] 0 2024-02-22T15:50:41.110 INFO:tasks.workunit.client.1.smithi062.stdout:5/399: unlink d0/d3/c74 0 2024-02-22T15:50:41.116 INFO:tasks.workunit.client.1.smithi062.stdout:6/547: creat d5/df/d40/fb2 x:0 0 0 2024-02-22T15:50:41.118 INFO:tasks.workunit.client.1.smithi062.stdout:9/439: creat d4/d15/d3c/f7c x:0 0 0 2024-02-22T15:50:41.120 INFO:tasks.workunit.client.1.smithi062.stdout:1/462: dwrite d1/d6/d7/d2b/d49/f6a [0,4194304] 0 2024-02-22T15:50:41.121 INFO:tasks.workunit.client.1.smithi062.stdout:9/440: stat d4/d1c/f1d 0 2024-02-22T15:50:41.123 INFO:tasks.workunit.client.1.smithi062.stdout:0/566: dread d1/d3/d4/d1e/fb2 [0,4194304] 0 2024-02-22T15:50:41.124 INFO:tasks.workunit.client.1.smithi062.stdout:6/548: fsync d5/f1a 0 2024-02-22T15:50:41.124 INFO:tasks.workunit.client.1.smithi062.stdout:2/575: dwrite d1/d8/d62/f9c [0,4194304] 0 2024-02-22T15:50:41.136 INFO:tasks.workunit.client.1.smithi062.stdout:6/549: mkdir d5/df/d40/db3 0 2024-02-22T15:50:41.139 INFO:tasks.workunit.client.1.smithi062.stdout:1/463: creat d1/d6/d7/d2b/d56/d95/f9b x:0 0 0 2024-02-22T15:50:41.142 INFO:tasks.workunit.client.1.smithi062.stdout:1/464: write d1/d6/d10/f8f [837787,9940] 0 2024-02-22T15:50:41.143 INFO:tasks.workunit.client.1.smithi062.stdout:0/567: dwrite d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8d/fba [0,4194304] 0 2024-02-22T15:50:41.144 INFO:tasks.workunit.client.1.smithi062.stdout:6/550: stat d5/df/d40/d44/d4a/l62 0 2024-02-22T15:50:41.153 INFO:tasks.workunit.client.1.smithi062.stdout:6/551: getdents d5/df/d40/d44/d4a 0 2024-02-22T15:50:41.156 INFO:tasks.workunit.client.1.smithi062.stdout:0/568: truncate d1/d3/f39 4812482 0 2024-02-22T15:50:41.166 INFO:tasks.workunit.client.1.smithi062.stdout:6/552: symlink d5/df/d40/d44/d78/lb4 0 2024-02-22T15:50:41.166 INFO:tasks.workunit.client.1.smithi062.stdout:5/400: dwrite d0/d3/d2f/f44 [0,4194304] 0 2024-02-22T15:50:41.167 INFO:tasks.workunit.client.1.smithi062.stdout:1/465: truncate d1/d6/d7/d2b/d47/d24/f2a 1291414 0 2024-02-22T15:50:41.169 INFO:tasks.workunit.client.1.smithi062.stdout:2/576: dread d1/d8/f1a [0,4194304] 0 2024-02-22T15:50:41.170 INFO:tasks.workunit.client.1.smithi062.stdout:9/441: dread d4/d28/f38 [4194304,4194304] 0 2024-02-22T15:50:41.171 INFO:tasks.workunit.client.1.smithi062.stdout:2/577: write d1/d28/d3f/f46 [1795186,106982] 0 2024-02-22T15:50:41.173 INFO:tasks.workunit.client.1.smithi062.stdout:9/442: chown d4/d35/f36 1780235 1 2024-02-22T15:50:41.174 INFO:tasks.workunit.client.1.smithi062.stdout:6/553: symlink d5/df/d40/d44/d76/d56/lb5 0 2024-02-22T15:50:41.182 INFO:tasks.workunit.client.1.smithi062.stdout:0/569: link d1/d3/d4/d1a/d76/d8c/d5d/f37 d1/fc4 0 2024-02-22T15:50:41.185 INFO:tasks.workunit.client.1.smithi062.stdout:1/466: dwrite d1/d6/d7/f7a [4194304,4194304] 0 2024-02-22T15:50:41.186 INFO:tasks.workunit.client.0.smithi019.stdout:9/647: mknod d2/d7/dc/d27/dbc/ce6 0 2024-02-22T15:50:41.187 INFO:tasks.workunit.client.1.smithi062.stdout:2/578: mkdir d1/d28/d3f/dbb 0 2024-02-22T15:50:41.187 INFO:tasks.workunit.client.0.smithi019.stdout:4/642: creat d2/db/fd1 x:0 0 0 2024-02-22T15:50:41.187 INFO:tasks.workunit.client.1.smithi062.stdout:6/554: mkdir d5/df/d40/dac/db6 0 2024-02-22T15:50:41.188 INFO:tasks.workunit.client.1.smithi062.stdout:2/579: chown d1/d28/d3f/dbb 904894583 1 2024-02-22T15:50:41.188 INFO:tasks.workunit.client.0.smithi019.stdout:6/832: fdatasync d6/d4b/d6d/d7b/d78/fc4 0 2024-02-22T15:50:41.188 INFO:tasks.workunit.client.0.smithi019.stdout:4/643: fdatasync d2/db/d14/d23/d4b/f82 0 2024-02-22T15:50:41.189 INFO:tasks.workunit.client.1.smithi062.stdout:9/443: dwrite d4/f8 [0,4194304] 0 2024-02-22T15:50:41.189 INFO:tasks.workunit.client.1.smithi062.stdout:2/580: readlink d1/l34 0 2024-02-22T15:50:41.190 INFO:tasks.workunit.client.1.smithi062.stdout:6/555: dread - d5/df/d32/d47/fa2 zero size 2024-02-22T15:50:41.192 INFO:tasks.workunit.client.1.smithi062.stdout:6/556: readlink d5/df/d40/d44/d4a/l62 0 2024-02-22T15:50:41.199 INFO:tasks.workunit.client.1.smithi062.stdout:5/401: getdents d0/d3/d2f 0 2024-02-22T15:50:41.202 INFO:tasks.workunit.client.1.smithi062.stdout:0/570: symlink d1/d3/d4/d1a/d76/da5/lc5 0 2024-02-22T15:50:41.209 INFO:tasks.workunit.client.1.smithi062.stdout:7/420: sync 2024-02-22T15:50:41.209 INFO:tasks.workunit.client.1.smithi062.stdout:5/402: dread d0/d3/da/d13/d23/d4e/d60/f63 [0,4194304] 0 2024-02-22T15:50:41.209 INFO:tasks.workunit.client.1.smithi062.stdout:6/557: creat d5/dd/d33/fb7 x:0 0 0 2024-02-22T15:50:41.212 INFO:tasks.workunit.client.1.smithi062.stdout:1/467: getdents d1/d6/d7 0 2024-02-22T15:50:41.214 INFO:tasks.workunit.client.1.smithi062.stdout:2/581: write d1/d47/d39/d5d/f63 [413756,43402] 0 2024-02-22T15:50:41.237 INFO:tasks.workunit.client.1.smithi062.stdout:0/571: dwrite d1/d3/d4/f99 [0,4194304] 0 2024-02-22T15:50:41.241 INFO:tasks.workunit.client.1.smithi062.stdout:1/468: fdatasync d1/d6/d36/f5d 0 2024-02-22T15:50:41.243 INFO:tasks.workunit.client.1.smithi062.stdout:7/421: getdents d0/d6/d2e/d4a/d5a 0 2024-02-22T15:50:41.250 INFO:tasks.workunit.client.1.smithi062.stdout:9/444: dwrite d4/d1c/f1e [0,4194304] 0 2024-02-22T15:50:41.250 INFO:tasks.workunit.client.1.smithi062.stdout:1/469: read d1/d6/de/f31 [3243239,27702] 0 2024-02-22T15:50:41.251 INFO:tasks.workunit.client.1.smithi062.stdout:7/422: write d0/d6/d2e/d55/f81 [557779,81279] 0 2024-02-22T15:50:41.256 INFO:tasks.workunit.client.0.smithi019.stdout:2/618: rename d1/d17/f90 to d1/d46/d9d/fe2 0 2024-02-22T15:50:41.259 INFO:tasks.workunit.client.1.smithi062.stdout:5/403: dwrite d0/d3/da/fb [0,4194304] 0 2024-02-22T15:50:41.259 INFO:tasks.workunit.client.1.smithi062.stdout:7/423: truncate d0/d6/d2e/d2f/f82 533583 0 2024-02-22T15:50:41.259 INFO:tasks.workunit.client.0.smithi019.stdout:1/679: sync 2024-02-22T15:50:41.260 INFO:tasks.workunit.client.1.smithi062.stdout:1/470: symlink d1/d6/d7/d8a/l9c 0 2024-02-22T15:50:41.260 INFO:tasks.workunit.client.0.smithi019.stdout:1/680: write d8/d18/d72/fd3 [650194,75660] 0 2024-02-22T15:50:41.261 INFO:tasks.workunit.client.1.smithi062.stdout:2/582: truncate d1/d47/f2b 1442503 0 2024-02-22T15:50:41.262 INFO:tasks.workunit.client.1.smithi062.stdout:0/572: dread d1/d3/d4/d1a/d76/d8c/d5d/dc3/d83/fbe [0,4194304] 0 2024-02-22T15:50:41.262 INFO:tasks.workunit.client.1.smithi062.stdout:5/404: unlink d0/d3/da/d13/c3c 0 2024-02-22T15:50:41.263 INFO:tasks.workunit.client.0.smithi019.stdout:6/833: creat d6/d4b/d6d/dda/f10d x:0 0 0 2024-02-22T15:50:41.265 INFO:tasks.workunit.client.1.smithi062.stdout:2/583: write d1/d1e/f9a [84254,60557] 0 2024-02-22T15:50:41.265 INFO:tasks.workunit.client.0.smithi019.stdout:4/644: creat d2/db/d14/d23/d9e/dac/dc2/fd2 x:0 0 0 2024-02-22T15:50:41.266 INFO:tasks.workunit.client.0.smithi019.stdout:0/640: creat d4/fdc x:0 0 0 2024-02-22T15:50:41.267 INFO:tasks.workunit.client.1.smithi062.stdout:6/558: dread d5/df/d32/f3d [0,4194304] 0 2024-02-22T15:50:41.268 INFO:tasks.workunit.client.1.smithi062.stdout:9/445: creat d4/f7d x:0 0 0 2024-02-22T15:50:41.268 INFO:tasks.workunit.client.0.smithi019.stdout:1/681: rmdir d8/d18/d64/d7f/d94 39 2024-02-22T15:50:41.268 INFO:tasks.workunit.client.0.smithi019.stdout:0/641: chown d4/db/d49/dc2/dc1 0 1 2024-02-22T15:50:41.271 INFO:tasks.workunit.client.0.smithi019.stdout:4/645: unlink d2/cb5 0 2024-02-22T15:50:41.273 INFO:tasks.workunit.client.1.smithi062.stdout:5/405: mkdir d0/d3/da/d13/d23/d78/d81 0 2024-02-22T15:50:41.274 INFO:tasks.workunit.client.0.smithi019.stdout:1/682: creat d8/d15/d34/d9e/fe7 x:0 0 0 2024-02-22T15:50:41.276 INFO:tasks.workunit.client.0.smithi019.stdout:6/834: symlink d6/d31/d69/d8e/l10e 0 2024-02-22T15:50:41.277 INFO:tasks.workunit.client.1.smithi062.stdout:0/573: write d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/d97/fb7 [2268883,42035] 0 2024-02-22T15:50:41.277 INFO:tasks.workunit.client.1.smithi062.stdout:9/446: creat d4/d1c/f7e x:0 0 0 2024-02-22T15:50:41.277 INFO:tasks.workunit.client.1.smithi062.stdout:1/471: write d1/d6/d7/d2b/d47/f92 [772292,46945] 0 2024-02-22T15:50:41.277 INFO:tasks.workunit.client.1.smithi062.stdout:0/574: readlink d1/dd/l7f 0 2024-02-22T15:50:41.278 INFO:tasks.workunit.client.0.smithi019.stdout:0/642: rmdir d4/db/d1b/d63/db0/da1 39 2024-02-22T15:50:41.280 INFO:tasks.workunit.client.1.smithi062.stdout:5/406: chown d0/d3/da/d13/d23/d78 700 1 2024-02-22T15:50:41.282 INFO:tasks.workunit.client.0.smithi019.stdout:1/683: symlink d8/d15/d84/le8 0 2024-02-22T15:50:41.283 INFO:tasks.workunit.client.0.smithi019.stdout:2/619: chown d1/d17/d21/d71/d60 20422 1 2024-02-22T15:50:41.283 INFO:tasks.workunit.client.1.smithi062.stdout:6/559: truncate d5/f1e 516884 0 2024-02-22T15:50:41.285 INFO:tasks.workunit.client.0.smithi019.stdout:6/835: symlink d6/d31/d43/de7/l10f 0 2024-02-22T15:50:41.285 INFO:tasks.workunit.client.0.smithi019.stdout:6/836: readlink d6/l53 0 2024-02-22T15:50:41.285 INFO:tasks.workunit.client.0.smithi019.stdout:4/646: symlink d2/ld3 0 2024-02-22T15:50:41.286 INFO:tasks.workunit.client.1.smithi062.stdout:9/447: write d4/d15/d5e/d63/f64 [3138641,84951] 0 2024-02-22T15:50:41.287 INFO:tasks.workunit.client.0.smithi019.stdout:9/648: getdents d2/dc8 0 2024-02-22T15:50:41.289 INFO:tasks.workunit.client.1.smithi062.stdout:0/575: creat d1/d3/d4/d1e/fc6 x:0 0 0 2024-02-22T15:50:41.289 INFO:tasks.workunit.client.0.smithi019.stdout:9/649: chown d2/d1f/d2e/d57 393 1 2024-02-22T15:50:41.290 INFO:tasks.workunit.client.0.smithi019.stdout:6/837: creat d6/d4b/d6d/d7b/d78/f110 x:0 0 0 2024-02-22T15:50:41.290 INFO:tasks.workunit.client.1.smithi062.stdout:2/584: dwrite d1/d1e/faf [0,4194304] 0 2024-02-22T15:50:41.292 INFO:tasks.workunit.client.0.smithi019.stdout:0/643: truncate d4/d22/dcc/fd1 1535905 0 2024-02-22T15:50:41.292 INFO:tasks.workunit.client.0.smithi019.stdout:4/647: write d2/db/d44/f55 [1512716,91726] 0 2024-02-22T15:50:41.296 INFO:tasks.workunit.client.1.smithi062.stdout:2/585: chown d1/d8/df/f31 18734 1 2024-02-22T15:50:41.297 INFO:tasks.workunit.client.0.smithi019.stdout:0/644: chown d4/db/d1b/c4e 14545637 1 2024-02-22T15:50:41.298 INFO:tasks.workunit.client.0.smithi019.stdout:2/620: dwrite d1/de/d9a/fd7 [0,4194304] 0 2024-02-22T15:50:41.298 INFO:tasks.workunit.client.0.smithi019.stdout:6/838: mkdir d6/d4b/d8d/d111 0 2024-02-22T15:50:41.300 INFO:tasks.workunit.client.1.smithi062.stdout:6/560: rmdir d5 39 2024-02-22T15:50:41.300 INFO:tasks.workunit.client.0.smithi019.stdout:4/648: write d2/db/d14/d23/d9e/dac/dc2/fd2 [982480,45210] 0 2024-02-22T15:50:41.301 INFO:tasks.workunit.client.1.smithi062.stdout:0/576: creat d1/d3/d4/d1e/d74/d68/fc7 x:0 0 0 2024-02-22T15:50:41.301 INFO:tasks.workunit.client.1.smithi062.stdout:5/407: symlink d0/d3/da/d32/d4f/d72/l82 0 2024-02-22T15:50:41.303 INFO:tasks.workunit.client.1.smithi062.stdout:2/586: mkdir d1/d47/d32/d52/dbc 0 2024-02-22T15:50:41.304 INFO:tasks.workunit.client.1.smithi062.stdout:9/448: dwrite d4/fc [0,4194304] 0 2024-02-22T15:50:41.305 INFO:tasks.workunit.client.0.smithi019.stdout:0/645: write d4/d22/f51 [4387380,76819] 0 2024-02-22T15:50:41.305 INFO:tasks.workunit.client.0.smithi019.stdout:9/650: mknod d2/d1f/d2e/d57/d60/d8c/ce7 0 2024-02-22T15:50:41.307 INFO:tasks.workunit.client.0.smithi019.stdout:9/651: readlink d2/d7/dc/d27/dbc/l18 0 2024-02-22T15:50:41.309 INFO:tasks.workunit.client.1.smithi062.stdout:5/408: fdatasync d0/d3/da/d13/d23/f3b 0 2024-02-22T15:50:41.310 INFO:tasks.workunit.client.1.smithi062.stdout:0/577: stat d1/d3/d4/d1a/d76/d8c/d2c/d84/f26 0 2024-02-22T15:50:41.310 INFO:tasks.workunit.client.0.smithi019.stdout:6/839: unlink d6/d4b/d6d/d7b/d7f/ca1 0 2024-02-22T15:50:41.311 INFO:tasks.workunit.client.0.smithi019.stdout:2/621: dread d1/f14 [0,4194304] 0 2024-02-22T15:50:41.313 INFO:tasks.workunit.client.0.smithi019.stdout:0/646: creat d4/d22/d2c/fdd x:0 0 0 2024-02-22T15:50:41.315 INFO:tasks.workunit.client.1.smithi062.stdout:2/587: rename d1/d28/d3f to d1/d8/d62/d73/dbd 0 2024-02-22T15:50:41.316 INFO:tasks.workunit.client.0.smithi019.stdout:9/652: fsync d2/d7/dc/d27/d68/f73 0 2024-02-22T15:50:41.316 INFO:tasks.workunit.client.0.smithi019.stdout:2/622: dread - d1/d46/d9d/fb5 zero size 2024-02-22T15:50:41.316 INFO:tasks.workunit.client.1.smithi062.stdout:0/578: write d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/d97/faa [736274,448] 0 2024-02-22T15:50:41.316 INFO:tasks.workunit.client.0.smithi019.stdout:6/840: fsync d6/d4b/d6d/d7b/d7f/fb8 0 2024-02-22T15:50:41.318 INFO:tasks.workunit.client.1.smithi062.stdout:5/409: creat d0/d3/f83 x:0 0 0 2024-02-22T15:50:41.324 INFO:tasks.workunit.client.1.smithi062.stdout:2/588: truncate d1/d8/f6e 1150232 0 2024-02-22T15:50:41.324 INFO:tasks.workunit.client.1.smithi062.stdout:6/561: dwrite d5/df/d32/f9f [0,4194304] 0 2024-02-22T15:50:41.325 INFO:tasks.workunit.client.0.smithi019.stdout:1/684: dwrite d8/d15/f17 [0,4194304] 0 2024-02-22T15:50:41.325 INFO:tasks.workunit.client.0.smithi019.stdout:2/623: rmdir d1/d17/d1c/d7f 39 2024-02-22T15:50:41.326 INFO:tasks.workunit.client.1.smithi062.stdout:6/562: stat d5/dd/f20 0 2024-02-22T15:50:41.327 INFO:tasks.workunit.client.1.smithi062.stdout:1/472: dread d1/d6/d10/f8f [0,4194304] 0 2024-02-22T15:50:41.328 INFO:tasks.workunit.client.1.smithi062.stdout:5/410: symlink d0/d3/d2f/d59/d67/d69/l84 0 2024-02-22T15:50:41.330 INFO:tasks.workunit.client.1.smithi062.stdout:6/563: dread - d5/df/d40/f8e zero size 2024-02-22T15:50:41.330 INFO:tasks.workunit.client.0.smithi019.stdout:1/685: write d8/d15/d84/fd5 [659170,43427] 0 2024-02-22T15:50:41.332 INFO:tasks.workunit.client.1.smithi062.stdout:0/579: rename d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/cad to d1/d3/d4/d1e/cc8 0 2024-02-22T15:50:41.332 INFO:tasks.workunit.client.1.smithi062.stdout:6/564: write d5/df/d40/d44/f95 [903707,98417] 0 2024-02-22T15:50:41.332 INFO:tasks.workunit.client.1.smithi062.stdout:1/473: mknod d1/d6/d7/d2b/d56/c9d 0 2024-02-22T15:50:41.333 INFO:tasks.workunit.client.0.smithi019.stdout:6/841: rename d6/d4b/d71/dc2/ccc to d6/d31/c112 0 2024-02-22T15:50:41.336 INFO:tasks.workunit.client.0.smithi019.stdout:2/624: truncate d1/d17/d21/daf/fd6 383300 0 2024-02-22T15:50:41.338 INFO:tasks.workunit.client.1.smithi062.stdout:2/589: rename d1/d8/df/c6f to d1/d47/d39/d5d/cbe 0 2024-02-22T15:50:41.338 INFO:tasks.workunit.client.0.smithi019.stdout:0/647: dwrite d4/db/d49/d65/fc7 [0,4194304] 0 2024-02-22T15:50:41.338 INFO:tasks.workunit.client.0.smithi019.stdout:1/686: write d8/d2a/d75/d81/f96 [504297,13858] 0 2024-02-22T15:50:41.339 INFO:tasks.workunit.client.1.smithi062.stdout:9/449: write d4/d1c/f58 [304925,42994] 0 2024-02-22T15:50:41.341 INFO:tasks.workunit.client.1.smithi062.stdout:0/580: chown d1/d3/d4/d1a/d1d/d3b/c8a 64048855 1 2024-02-22T15:50:41.341 INFO:tasks.workunit.client.0.smithi019.stdout:0/648: read d4/db/d1b/d40/d44/d75/f82 [541118,80858] 0 2024-02-22T15:50:41.344 INFO:tasks.workunit.client.1.smithi062.stdout:9/450: mknod d4/d15/d5f/d6a/c7f 0 2024-02-22T15:50:41.345 INFO:tasks.workunit.client.0.smithi019.stdout:1/687: creat d8/d2a/d75/da0/fe9 x:0 0 0 2024-02-22T15:50:41.345 INFO:tasks.workunit.client.1.smithi062.stdout:0/581: chown d1/d3/d4/d1a/d1d/d3b/c7e 2137182503 1 2024-02-22T15:50:41.346 INFO:tasks.workunit.client.1.smithi062.stdout:9/451: mknod d4/d1c/c80 0 2024-02-22T15:50:41.346 INFO:tasks.workunit.client.0.smithi019.stdout:2/625: creat d1/d17/d21/d71/de1/fe3 x:0 0 0 2024-02-22T15:50:41.346 INFO:tasks.workunit.client.1.smithi062.stdout:1/474: write d1/d6/d7/f98 [63162,125533] 0 2024-02-22T15:50:41.347 INFO:tasks.workunit.client.0.smithi019.stdout:0/649: mknod d4/db/d76/d77/cde 0 2024-02-22T15:50:41.347 INFO:tasks.workunit.client.1.smithi062.stdout:0/582: chown d1/d3/d5f/f57 16256 1 2024-02-22T15:50:41.349 INFO:tasks.workunit.client.0.smithi019.stdout:1/688: symlink d8/d18/d64/d7f/d94/lea 0 2024-02-22T15:50:41.350 INFO:tasks.workunit.client.0.smithi019.stdout:2/626: creat d1/d17/d1c/d7f/dcd/fe4 x:0 0 0 2024-02-22T15:50:41.352 INFO:tasks.workunit.client.0.smithi019.stdout:0/650: mknod d4/db/d76/dae/cdf 0 2024-02-22T15:50:41.353 INFO:tasks.workunit.client.0.smithi019.stdout:1/689: symlink d8/d15/leb 0 2024-02-22T15:50:41.356 INFO:tasks.workunit.client.1.smithi062.stdout:9/452: rename d4/d15/d5f/f6f to d4/d15/d5f/d6a/f81 0 2024-02-22T15:50:41.356 INFO:tasks.workunit.client.0.smithi019.stdout:1/690: dread - d8/d2a/f5f zero size 2024-02-22T15:50:41.356 INFO:tasks.workunit.client.1.smithi062.stdout:1/475: creat d1/d6/d7/d2b/d96/f9e x:0 0 0 2024-02-22T15:50:41.357 INFO:tasks.workunit.client.1.smithi062.stdout:0/583: creat d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/fc9 x:0 0 0 2024-02-22T15:50:41.360 INFO:tasks.workunit.client.0.smithi019.stdout:1/691: creat d8/d2a/d75/d81/d8e/fec x:0 0 0 2024-02-22T15:50:41.360 INFO:tasks.workunit.client.1.smithi062.stdout:1/476: creat d1/d6/d7/d2b/d47/d24/f9f x:0 0 0 2024-02-22T15:50:41.362 INFO:tasks.workunit.client.1.smithi062.stdout:9/453: rename d4/d35/d3f/l51 to d4/d15/d5e/l82 0 2024-02-22T15:50:41.363 INFO:tasks.workunit.client.1.smithi062.stdout:1/477: symlink d1/d6/d7/d2b/d47/d63/la0 0 2024-02-22T15:50:41.364 INFO:tasks.workunit.client.1.smithi062.stdout:2/590: dread d1/f50 [0,4194304] 0 2024-02-22T15:50:41.365 INFO:tasks.workunit.client.0.smithi019.stdout:9/653: truncate d2/d1f/d2e/d48/d45/d5a/fdd 2556025 0 2024-02-22T15:50:41.365 INFO:tasks.workunit.client.0.smithi019.stdout:2/627: dwrite d1/d17/fb2 [0,4194304] 0 2024-02-22T15:50:41.367 INFO:tasks.workunit.client.1.smithi062.stdout:9/454: unlink f2 0 2024-02-22T15:50:41.368 INFO:tasks.workunit.client.0.smithi019.stdout:0/651: dwrite d4/db/d1b/d63/db0/da1/dbb/d7/fcb [4194304,4194304] 0 2024-02-22T15:50:41.369 INFO:tasks.workunit.client.1.smithi062.stdout:0/584: getdents d1/d13 0 2024-02-22T15:50:41.370 INFO:tasks.workunit.client.1.smithi062.stdout:1/478: creat d1/d6/d36/d76/fa1 x:0 0 0 2024-02-22T15:50:41.370 INFO:tasks.workunit.client.1.smithi062.stdout:0/585: write d1/dd/f36 [2074191,78629] 0 2024-02-22T15:50:41.374 INFO:tasks.workunit.client.0.smithi019.stdout:6/842: rename d6/d4b/da9/d67/f84 to d6/d4b/d6d/d7b/d78/f113 0 2024-02-22T15:50:41.375 INFO:tasks.workunit.client.1.smithi062.stdout:2/591: dwrite d1/d8/d62/d73/f75 [0,4194304] 0 2024-02-22T15:50:41.375 INFO:tasks.workunit.client.1.smithi062.stdout:9/455: mkdir d4/d1c/d83 0 2024-02-22T15:50:41.376 INFO:tasks.workunit.client.0.smithi019.stdout:0/652: creat d4/db/d1b/d63/db0/da1/dbb/d7/fe0 x:0 0 0 2024-02-22T15:50:41.379 INFO:tasks.workunit.client.0.smithi019.stdout:1/692: link d8/d18/d64/f83 d8/d18/d64/d7f/de3/fed 0 2024-02-22T15:50:41.379 INFO:tasks.workunit.client.1.smithi062.stdout:9/456: dread - d4/d15/d5f/d6a/f81 zero size 2024-02-22T15:50:41.380 INFO:tasks.workunit.client.0.smithi019.stdout:1/693: chown d8/d18/d64/d7f/de3 1535521 1 2024-02-22T15:50:41.381 INFO:tasks.workunit.client.1.smithi062.stdout:2/592: read d1/fb8 [1520968,67773] 0 2024-02-22T15:50:41.381 INFO:tasks.workunit.client.1.smithi062.stdout:1/479: dwrite d1/f32 [0,4194304] 0 2024-02-22T15:50:41.384 INFO:tasks.workunit.client.0.smithi019.stdout:6/843: read d6/d4b/d6d/d7b/d7f/f89 [165420,14547] 0 2024-02-22T15:50:41.387 INFO:tasks.workunit.client.0.smithi019.stdout:6/844: rmdir d6/d4b/d71 39 2024-02-22T15:50:41.391 INFO:tasks.workunit.client.1.smithi062.stdout:9/457: rename d4/d15/d3c to d4/d15/d5f/d75/d84 0 2024-02-22T15:50:41.392 INFO:tasks.workunit.client.0.smithi019.stdout:1/694: dwrite d8/d15/d34/f39 [4194304,4194304] 0 2024-02-22T15:50:41.392 INFO:tasks.workunit.client.0.smithi019.stdout:2/628: write d1/d46/f98 [307962,88450] 0 2024-02-22T15:50:41.393 INFO:tasks.workunit.client.1.smithi062.stdout:1/480: mknod d1/d6/d7/d8a/ca2 0 2024-02-22T15:50:41.394 INFO:tasks.workunit.client.0.smithi019.stdout:2/629: chown d1/d17/d21/daf 209 1 2024-02-22T15:50:41.395 INFO:tasks.workunit.client.1.smithi062.stdout:2/593: dread d1/f50 [0,4194304] 0 2024-02-22T15:50:41.399 INFO:tasks.workunit.client.1.smithi062.stdout:1/481: mknod d1/d6/ca3 0 2024-02-22T15:50:41.400 INFO:tasks.workunit.client.0.smithi019.stdout:9/654: dread d2/d1f/d2e/d48/d45/d5a/fa0 [0,4194304] 0 2024-02-22T15:50:41.400 INFO:tasks.workunit.client.0.smithi019.stdout:1/695: mknod d8/d2a/d75/d81/d8e/cee 0 2024-02-22T15:50:41.400 INFO:tasks.workunit.client.0.smithi019.stdout:6/845: rename d6/d4b/da9/d67/c7c to d6/d4b/d71/dc2/c114 0 2024-02-22T15:50:41.401 INFO:tasks.workunit.client.0.smithi019.stdout:2/630: unlink d1/d17/d21/f36 0 2024-02-22T15:50:41.401 INFO:tasks.workunit.client.1.smithi062.stdout:0/586: dwrite d1/f12 [0,4194304] 0 2024-02-22T15:50:41.402 INFO:tasks.workunit.client.0.smithi019.stdout:9/655: fdatasync d2/d7/dc/d27/dbc/f3f 0 2024-02-22T15:50:41.403 INFO:tasks.workunit.client.1.smithi062.stdout:9/458: rmdir d4/d15/d5f/d75/d73 0 2024-02-22T15:50:41.403 INFO:tasks.workunit.client.1.smithi062.stdout:1/482: mkdir d1/d6/d36/d37/da4 0 2024-02-22T15:50:41.404 INFO:tasks.workunit.client.1.smithi062.stdout:0/587: readlink d1/d3/d4/d1a/d76/d8c/d92/lbf 0 2024-02-22T15:50:41.404 INFO:tasks.workunit.client.0.smithi019.stdout:9/656: chown d2/d1f/d2e/d57/d60/d8c/d66/fba 325142 1 2024-02-22T15:50:41.404 INFO:tasks.workunit.client.0.smithi019.stdout:2/631: rmdir d1/d17/d21/d71/d60 39 2024-02-22T15:50:41.407 INFO:tasks.workunit.client.1.smithi062.stdout:9/459: creat d4/d28/d50/d5a/f85 x:0 0 0 2024-02-22T15:50:41.407 INFO:tasks.workunit.client.0.smithi019.stdout:2/632: fsync d1/de/f16 0 2024-02-22T15:50:41.408 INFO:tasks.workunit.client.1.smithi062.stdout:0/588: fsync d1/d3/d4/d1a/d76/d8c/d5d/dc3/d83/fbe 0 2024-02-22T15:50:41.408 INFO:tasks.workunit.client.0.smithi019.stdout:6/846: truncate d6/f20 103944 0 2024-02-22T15:50:41.409 INFO:tasks.workunit.client.0.smithi019.stdout:9/657: symlink d2/d7/dc/d27/d68/dca/le8 0 2024-02-22T15:50:41.410 INFO:tasks.workunit.client.1.smithi062.stdout:9/460: read - d4/d15/d31/f4e zero size 2024-02-22T15:50:41.410 INFO:tasks.workunit.client.0.smithi019.stdout:6/847: write d6/fc [3364210,130629] 0 2024-02-22T15:50:41.412 INFO:tasks.workunit.client.0.smithi019.stdout:2/633: creat d1/d17/d1c/d66/db9/dd0/fe5 x:0 0 0 2024-02-22T15:50:41.413 INFO:tasks.workunit.client.1.smithi062.stdout:1/483: rmdir d1/d6/d36/d37/da4 0 2024-02-22T15:50:41.413 INFO:tasks.workunit.client.1.smithi062.stdout:9/461: creat d4/d15/d5e/d63/f86 x:0 0 0 2024-02-22T15:50:41.414 INFO:tasks.workunit.client.0.smithi019.stdout:6/848: write d6/d4b/d6d/d7b/d78/f113 [3333354,70635] 0 2024-02-22T15:50:41.415 INFO:tasks.workunit.client.0.smithi019.stdout:5/663: sync 2024-02-22T15:50:41.417 INFO:tasks.workunit.client.0.smithi019.stdout:9/658: truncate d2/d7/dc/f29 244398 0 2024-02-22T15:50:41.417 INFO:tasks.workunit.client.0.smithi019.stdout:6/849: write d6/d4b/fe6 [1908309,130593] 0 2024-02-22T15:50:41.419 INFO:tasks.workunit.client.0.smithi019.stdout:6/850: fdatasync d6/d4b/d6d/d7b/d7f/ffe 0 2024-02-22T15:50:41.419 INFO:tasks.workunit.client.0.smithi019.stdout:9/659: chown d2/dc8/ca5 13 1 2024-02-22T15:50:41.422 INFO:tasks.workunit.client.1.smithi062.stdout:9/462: dwrite d4/d28/d50/f55 [0,4194304] 0 2024-02-22T15:50:41.442 INFO:tasks.workunit.client.1.smithi062.stdout:4/479: sync 2024-02-22T15:50:41.442 INFO:tasks.workunit.client.1.smithi062.stdout:8/495: sync 2024-02-22T15:50:41.444 INFO:tasks.workunit.client.1.smithi062.stdout:1/484: dread d1/d6/d7/d2b/d47/f92 [0,4194304] 0 2024-02-22T15:50:41.445 INFO:tasks.workunit.client.1.smithi062.stdout:8/496: unlink d8/d9/d44/d4d/d51/d55/f75 0 2024-02-22T15:50:41.445 INFO:tasks.workunit.client.1.smithi062.stdout:8/497: stat d8/d9/f1f 0 2024-02-22T15:50:41.450 INFO:tasks.workunit.client.0.smithi019.stdout:2/634: dread d1/d46/f5e [4194304,4194304] 0 2024-02-22T15:50:41.451 INFO:tasks.workunit.client.1.smithi062.stdout:8/498: getdents d8/d9/d48 0 2024-02-22T15:50:41.452 INFO:tasks.workunit.client.1.smithi062.stdout:1/485: truncate d1/d6/d7/f54 3146532 0 2024-02-22T15:50:41.452 INFO:tasks.workunit.client.0.smithi019.stdout:9/660: creat d2/d1f/d2e/d57/d60/fe9 x:0 0 0 2024-02-22T15:50:41.454 INFO:tasks.workunit.client.0.smithi019.stdout:9/661: chown d2/d1f/d2e/d48/d36/lad 1234 1 2024-02-22T15:50:41.454 INFO:tasks.workunit.client.0.smithi019.stdout:6/851: creat d6/d31/d69/d8e/dcf/f115 x:0 0 0 2024-02-22T15:50:41.455 INFO:tasks.workunit.client.0.smithi019.stdout:6/852: readlink d6/d4b/d71/lae 0 2024-02-22T15:50:41.456 INFO:tasks.workunit.client.0.smithi019.stdout:2/635: read - d1/d17/fb7 zero size 2024-02-22T15:50:41.456 INFO:tasks.workunit.client.0.smithi019.stdout:5/664: creat d8/d11/fdc x:0 0 0 2024-02-22T15:50:41.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:41 smithi062 ceph-mon[48130]: pgmap v36: 65 pgs: 65 active+clean; 2.1 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 72 MiB/s rd, 292 MiB/s wr, 456 op/s 2024-02-22T15:50:41.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:41 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:41.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:41 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:41.458 INFO:tasks.workunit.client.0.smithi019.stdout:6/853: mkdir d6/d4b/d116 0 2024-02-22T15:50:41.459 INFO:tasks.workunit.client.1.smithi062.stdout:1/486: write d1/d6/de/f31 [5094745,112485] 0 2024-02-22T15:50:41.459 INFO:tasks.workunit.client.1.smithi062.stdout:8/499: dwrite d8/d9/d44/f5c [0,4194304] 0 2024-02-22T15:50:41.459 INFO:tasks.workunit.client.0.smithi019.stdout:2/636: chown d1/d17/d21/d71/l2c 188809480 1 2024-02-22T15:50:41.461 INFO:tasks.workunit.client.0.smithi019.stdout:2/637: fsync d1/de/d9a/fcb 0 2024-02-22T15:50:41.461 INFO:tasks.workunit.client.0.smithi019.stdout:6/854: mkdir d6/d4b/d6d/da3/d117 0 2024-02-22T15:50:41.462 INFO:tasks.workunit.client.0.smithi019.stdout:2/638: stat d1/d17/d21/d71/dd1 0 2024-02-22T15:50:41.463 INFO:tasks.workunit.client.1.smithi062.stdout:8/500: write d8/d9/d44/f45 [489134,59353] 0 2024-02-22T15:50:41.463 INFO:tasks.workunit.client.0.smithi019.stdout:5/665: creat d8/dd/d19/d77/d7a/fdd x:0 0 0 2024-02-22T15:50:41.464 INFO:tasks.workunit.client.0.smithi019.stdout:6/855: write d6/d31/d69/d8e/dcf/fec [1348133,81484] 0 2024-02-22T15:50:41.465 INFO:tasks.workunit.client.0.smithi019.stdout:2/639: creat d1/da4/fe6 x:0 0 0 2024-02-22T15:50:41.467 INFO:tasks.workunit.client.0.smithi019.stdout:5/666: mkdir d8/dd/d19/d38/d47/ddb/dde 0 2024-02-22T15:50:41.469 INFO:tasks.workunit.client.0.smithi019.stdout:6/856: read d6/d4b/d6d/d7b/d78/f113 [798307,71825] 0 2024-02-22T15:50:41.470 INFO:tasks.workunit.client.0.smithi019.stdout:2/640: write d1/d17/d38/fce [563229,104686] 0 2024-02-22T15:50:41.472 INFO:tasks.workunit.client.0.smithi019.stdout:6/857: readlink d6/d31/d43/l77 0 2024-02-22T15:50:41.472 INFO:tasks.workunit.client.0.smithi019.stdout:5/667: symlink d8/dd/d19/d3a/d46/ldf 0 2024-02-22T15:50:41.475 INFO:tasks.workunit.client.0.smithi019.stdout:6/858: unlink d6/d31/d48/l61 0 2024-02-22T15:50:41.475 INFO:tasks.workunit.client.1.smithi062.stdout:8/501: write d8/d9/d2c/d49/d5e/f7d [797461,80373] 0 2024-02-22T15:50:41.477 INFO:tasks.workunit.client.0.smithi019.stdout:2/641: creat d1/d17/d1c/fe7 x:0 0 0 2024-02-22T15:50:41.478 INFO:tasks.workunit.client.1.smithi062.stdout:8/502: creat d8/d42/d8f/f9a x:0 0 0 2024-02-22T15:50:41.479 INFO:tasks.workunit.client.0.smithi019.stdout:6/859: mknod d6/d4b/c118 0 2024-02-22T15:50:41.479 INFO:tasks.workunit.client.1.smithi062.stdout:8/503: chown d8/d28/d4e/f8e 679638064 1 2024-02-22T15:50:41.481 INFO:tasks.workunit.client.0.smithi019.stdout:6/860: creat d6/d4b/d71/f119 x:0 0 0 2024-02-22T15:50:41.483 INFO:tasks.workunit.client.0.smithi019.stdout:2/642: getdents d1/de 0 2024-02-22T15:50:41.483 INFO:tasks.workunit.client.1.smithi062.stdout:8/504: rename d8/d9/d2c/d49/d5e/f8d to d8/d9/d44/f9b 0 2024-02-22T15:50:41.488 INFO:tasks.workunit.client.1.smithi062.stdout:8/505: link d8/d9/d2c/d49/d5e/c73 d8/d42/d87/c9c 0 2024-02-22T15:50:41.490 INFO:tasks.workunit.client.1.smithi062.stdout:8/506: write d8/d9/f16 [1705285,33389] 0 2024-02-22T15:50:41.491 INFO:tasks.workunit.client.1.smithi062.stdout:8/507: truncate d8/d42/d8f/f9a 531848 0 2024-02-22T15:50:41.492 INFO:tasks.workunit.client.1.smithi062.stdout:8/508: stat d8/d9/d2c/d49/l8b 0 2024-02-22T15:50:41.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:41 smithi019 ceph-mon[67418]: pgmap v36: 65 pgs: 65 active+clean; 2.1 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 72 MiB/s rd, 292 MiB/s wr, 456 op/s 2024-02-22T15:50:41.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:41 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:41.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:41 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:41.565 INFO:tasks.workunit.client.0.smithi019.stdout:9/662: dread d2/d7/dc/f24 [4194304,4194304] 0 2024-02-22T15:50:41.566 INFO:tasks.workunit.client.0.smithi019.stdout:9/663: write d2/dc8/fa2 [321053,24898] 0 2024-02-22T15:50:41.568 INFO:tasks.workunit.client.0.smithi019.stdout:9/664: rmdir d2/d7/dc/d97 39 2024-02-22T15:50:41.571 INFO:tasks.workunit.client.1.smithi062.stdout:8/509: dwrite d8/f40 [4194304,4194304] 0 2024-02-22T15:50:41.573 INFO:tasks.workunit.client.1.smithi062.stdout:8/510: write d8/d9/f16 [121687,95860] 0 2024-02-22T15:50:41.573 INFO:tasks.workunit.client.0.smithi019.stdout:5/668: truncate d8/dd/d19/d3a/d46/f7b 595945 0 2024-02-22T15:50:41.575 INFO:tasks.workunit.client.1.smithi062.stdout:8/511: rename d8/d9/d48 to d8/d9/d48/d9d 22 2024-02-22T15:50:41.578 INFO:tasks.workunit.client.0.smithi019.stdout:5/669: creat d8/dd/d19/d38/d54/fe0 x:0 0 0 2024-02-22T15:50:41.582 INFO:tasks.workunit.client.0.smithi019.stdout:9/665: dwrite d2/d1f/d2e/d48/d45/d5a/fa0 [0,4194304] 0 2024-02-22T15:50:41.586 INFO:tasks.workunit.client.0.smithi019.stdout:9/666: creat d2/dc8/fea x:0 0 0 2024-02-22T15:50:41.587 INFO:tasks.workunit.client.0.smithi019.stdout:6/861: truncate d6/d31/f35 6410632 0 2024-02-22T15:50:41.587 INFO:tasks.workunit.client.0.smithi019.stdout:9/667: chown d2/c65 6291602 1 2024-02-22T15:50:41.587 INFO:tasks.workunit.client.0.smithi019.stdout:2/643: write d1/d17/d21/f6f [5206,114821] 0 2024-02-22T15:50:41.588 INFO:tasks.workunit.client.1.smithi062.stdout:8/512: dwrite d8/f12 [4194304,4194304] 0 2024-02-22T15:50:41.590 INFO:tasks.workunit.client.0.smithi019.stdout:9/668: fdatasync d2/d1f/d2e/d57/d60/fe9 0 2024-02-22T15:50:41.591 INFO:tasks.workunit.client.1.smithi062.stdout:3/423: sync 2024-02-22T15:50:41.592 INFO:tasks.workunit.client.1.smithi062.stdout:8/513: readlink d8/d9/d2c/l65 0 2024-02-22T15:50:41.592 INFO:tasks.workunit.client.0.smithi019.stdout:2/644: creat d1/d86/fe8 x:0 0 0 2024-02-22T15:50:41.593 INFO:tasks.workunit.client.0.smithi019.stdout:6/862: truncate d6/d4b/d6d/da3/ddb/fdd 811250 0 2024-02-22T15:50:41.594 INFO:tasks.workunit.client.0.smithi019.stdout:9/669: fdatasync d2/d7/dc/d27/d31/d79/fb2 0 2024-02-22T15:50:41.595 INFO:tasks.workunit.client.1.smithi062.stdout:3/424: truncate d5/d26/f6c 4743690 0 2024-02-22T15:50:41.599 INFO:tasks.workunit.client.1.smithi062.stdout:3/425: mknod d5/d26/d2d/d3f/c87 0 2024-02-22T15:50:41.603 INFO:tasks.workunit.client.0.smithi019.stdout:2/645: creat d1/db4/fe9 x:0 0 0 2024-02-22T15:50:41.603 INFO:tasks.workunit.client.0.smithi019.stdout:2/646: read - d1/fc4 zero size 2024-02-22T15:50:41.603 INFO:tasks.workunit.client.0.smithi019.stdout:6/863: chown d6/d4b/da9/d67/d8a/l9d 176 1 2024-02-22T15:50:41.603 INFO:tasks.workunit.client.0.smithi019.stdout:5/670: dwrite d8/dd/d19/f29 [0,4194304] 0 2024-02-22T15:50:41.603 INFO:tasks.workunit.client.0.smithi019.stdout:8/633: sync 2024-02-22T15:50:41.604 INFO:tasks.workunit.client.1.smithi062.stdout:3/426: mkdir d5/d16/d88 0 2024-02-22T15:50:41.607 INFO:tasks.workunit.client.0.smithi019.stdout:5/671: rename d8/dd/d19/d77/f91 to d8/d11/d33/fe1 0 2024-02-22T15:50:41.609 INFO:tasks.workunit.client.0.smithi019.stdout:5/672: write d8/f9b [118372,75172] 0 2024-02-22T15:50:41.610 INFO:tasks.workunit.client.1.smithi062.stdout:8/514: truncate d8/d9/d44/d4d/d51/f68 1382242 0 2024-02-22T15:50:41.610 INFO:tasks.workunit.client.1.smithi062.stdout:3/427: dwrite f3 [0,4194304] 0 2024-02-22T15:50:41.611 INFO:tasks.workunit.client.0.smithi019.stdout:8/634: mkdir d9/d14/d17/d3d/d3f/dd4 0 2024-02-22T15:50:41.613 INFO:tasks.workunit.client.0.smithi019.stdout:3/521: sync 2024-02-22T15:50:41.613 INFO:tasks.workunit.client.0.smithi019.stdout:7/972: sync 2024-02-22T15:50:41.614 INFO:tasks.workunit.client.0.smithi019.stdout:9/670: mknod d2/d1f/d2e/d57/d60/d8c/ceb 0 2024-02-22T15:50:41.615 INFO:tasks.workunit.client.0.smithi019.stdout:5/673: creat d8/d11/d33/fe2 x:0 0 0 2024-02-22T15:50:41.618 INFO:tasks.workunit.client.0.smithi019.stdout:9/671: write d2/d1f/d2e/d48/fc4 [616255,125652] 0 2024-02-22T15:50:41.618 INFO:tasks.workunit.client.0.smithi019.stdout:2/647: link d1/de/d15/fb3 d1/d17/d4b/fea 0 2024-02-22T15:50:41.619 INFO:tasks.workunit.client.0.smithi019.stdout:6/864: dwrite d6/d31/d43/fb0 [0,4194304] 0 2024-02-22T15:50:41.619 INFO:tasks.workunit.client.1.smithi062.stdout:8/515: read d8/d42/d8f/f9a [8892,116602] 0 2024-02-22T15:50:41.621 INFO:tasks.workunit.client.0.smithi019.stdout:7/973: creat d9/d79/f13e x:0 0 0 2024-02-22T15:50:41.623 INFO:tasks.workunit.client.1.smithi062.stdout:3/428: truncate d5/d16/d21/f60 26533 0 2024-02-22T15:50:41.623 INFO:tasks.workunit.client.0.smithi019.stdout:3/522: dread d9/d26/d3f/d59/d66/f69 [0,4194304] 0 2024-02-22T15:50:41.624 INFO:tasks.workunit.client.1.smithi062.stdout:8/516: creat d8/d26/f9e x:0 0 0 2024-02-22T15:50:41.625 INFO:tasks.workunit.client.0.smithi019.stdout:2/648: symlink d1/db4/leb 0 2024-02-22T15:50:41.626 INFO:tasks.workunit.client.0.smithi019.stdout:5/674: mknod d8/dd/d19/d3a/d46/ce3 0 2024-02-22T15:50:41.628 INFO:tasks.workunit.client.0.smithi019.stdout:7/974: rmdir d9/d38/d41/d45/d92 39 2024-02-22T15:50:41.628 INFO:tasks.workunit.client.1.smithi062.stdout:3/429: readlink d5/l11 0 2024-02-22T15:50:41.630 INFO:tasks.workunit.client.0.smithi019.stdout:6/865: creat d6/d4b/d8d/d111/f11a x:0 0 0 2024-02-22T15:50:41.632 INFO:tasks.workunit.client.1.smithi062.stdout:3/430: creat d5/d16/d88/f89 x:0 0 0 2024-02-22T15:50:41.632 INFO:tasks.workunit.client.0.smithi019.stdout:2/649: mknod d1/d17/d1c/d7f/cec 0 2024-02-22T15:50:41.632 INFO:tasks.workunit.client.0.smithi019.stdout:2/650: dread - d1/d17/d4b/f7b zero size 2024-02-22T15:50:41.633 INFO:tasks.workunit.client.1.smithi062.stdout:3/431: write d5/d26/f6c [3237669,38615] 0 2024-02-22T15:50:41.634 INFO:tasks.workunit.client.0.smithi019.stdout:2/651: dread - d1/d17/d1c/d7f/dcd/fe4 zero size 2024-02-22T15:50:41.635 INFO:tasks.workunit.client.1.smithi062.stdout:8/517: dwrite d8/f29 [0,4194304] 0 2024-02-22T15:50:41.636 INFO:tasks.workunit.client.0.smithi019.stdout:2/652: write d1/d46/fab [777360,15473] 0 2024-02-22T15:50:41.636 INFO:tasks.workunit.client.1.smithi062.stdout:8/518: readlink d8/d26/l36 0 2024-02-22T15:50:41.637 INFO:tasks.workunit.client.1.smithi062.stdout:8/519: chown d8/d9/d44/d4d 855567471 1 2024-02-22T15:50:41.638 INFO:tasks.workunit.client.0.smithi019.stdout:9/672: rename d2/d7/dc/d27/d31 to d2/dec 0 2024-02-22T15:50:41.639 INFO:tasks.workunit.client.0.smithi019.stdout:9/673: chown d2/d1f/f47 246 1 2024-02-22T15:50:41.640 INFO:tasks.workunit.client.0.smithi019.stdout:7/975: mkdir d9/d38/d41/d45/d92/d13f 0 2024-02-22T15:50:41.642 INFO:tasks.workunit.client.1.smithi062.stdout:3/432: rename d5/d26/d2d/f47 to d5/f8a 0 2024-02-22T15:50:41.643 INFO:tasks.workunit.client.0.smithi019.stdout:5/675: creat d8/d11/fe4 x:0 0 0 2024-02-22T15:50:41.644 INFO:tasks.workunit.client.1.smithi062.stdout:3/433: creat d5/d16/d21/d32/f8b x:0 0 0 2024-02-22T15:50:41.645 INFO:tasks.workunit.client.0.smithi019.stdout:9/674: symlink d2/d7/dc/d27/d68/dca/led 0 2024-02-22T15:50:41.645 INFO:tasks.workunit.client.0.smithi019.stdout:7/976: truncate d9/d26/f87 3000158 0 2024-02-22T15:50:41.646 INFO:tasks.workunit.client.1.smithi062.stdout:8/520: link d8/d9/d44/d4d/d51/d55/f6d d8/d9/d44/d4d/f9f 0 2024-02-22T15:50:41.646 INFO:tasks.workunit.client.1.smithi062.stdout:8/521: write d8/d9/d2c/f78 [356972,9962] 0 2024-02-22T15:50:41.647 INFO:tasks.workunit.client.1.smithi062.stdout:3/434: creat d5/d16/d21/d36/d5a/f8c x:0 0 0 2024-02-22T15:50:41.647 INFO:tasks.workunit.client.1.smithi062.stdout:8/522: readlink d8/d26/l33 0 2024-02-22T15:50:41.648 INFO:tasks.workunit.client.0.smithi019.stdout:2/653: dwrite d1/d17/d4b/f4a [0,4194304] 0 2024-02-22T15:50:41.648 INFO:tasks.workunit.client.0.smithi019.stdout:8/635: write d9/d14/d6f/d7e/d94/d9c/fa3 [1400018,34922] 0 2024-02-22T15:50:41.648 INFO:tasks.workunit.client.0.smithi019.stdout:5/676: chown d8/d11/f87 83 1 2024-02-22T15:50:41.649 INFO:tasks.workunit.client.1.smithi062.stdout:8/523: write d8/d9/f71 [209681,114598] 0 2024-02-22T15:50:41.650 INFO:tasks.workunit.client.0.smithi019.stdout:9/675: mkdir d2/d7/dee 0 2024-02-22T15:50:41.651 INFO:tasks.workunit.client.1.smithi062.stdout:8/524: write d8/d9/fe [1453934,36276] 0 2024-02-22T15:50:41.652 INFO:tasks.workunit.client.0.smithi019.stdout:7/977: write d9/ff1 [2836304,71805] 0 2024-02-22T15:50:41.655 INFO:tasks.workunit.client.0.smithi019.stdout:3/523: write d9/d26/f35 [3739259,105347] 0 2024-02-22T15:50:41.656 INFO:tasks.workunit.client.1.smithi062.stdout:8/525: creat d8/d9/d2c/d74/fa0 x:0 0 0 2024-02-22T15:50:41.657 INFO:tasks.workunit.client.1.smithi062.stdout:3/435: dwrite d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:41.657 INFO:tasks.workunit.client.0.smithi019.stdout:5/677: mknod d8/dd/d19/d38/d47/ddb/dde/ce5 0 2024-02-22T15:50:41.659 INFO:tasks.workunit.client.0.smithi019.stdout:9/676: write d2/d7/dc/d27/dbc/f3f [3168000,9017] 0 2024-02-22T15:50:41.660 INFO:tasks.workunit.client.0.smithi019.stdout:7/978: chown d9/d38/d41/d45/d61/d6a/d13c 2 1 2024-02-22T15:50:41.661 INFO:tasks.workunit.client.0.smithi019.stdout:8/636: dread d9/d14/d6f/d7e/d94/db8/fa1 [0,4194304] 0 2024-02-22T15:50:41.663 INFO:tasks.workunit.client.0.smithi019.stdout:3/524: mknod d9/d22/d27/d6d/d72/ca8 0 2024-02-22T15:50:41.663 INFO:tasks.workunit.client.0.smithi019.stdout:7/979: stat d9/d26/dc4 0 2024-02-22T15:50:41.665 INFO:tasks.workunit.client.1.smithi062.stdout:8/526: mkdir d8/d9/d2c/d49/d94/da1 0 2024-02-22T15:50:41.668 INFO:tasks.workunit.client.0.smithi019.stdout:7/980: symlink d9/d38/d41/d45/db0/df7/dfb/l140 0 2024-02-22T15:50:41.669 INFO:tasks.workunit.client.1.smithi062.stdout:8/527: creat d8/d9/d2c/d49/d94/fa2 x:0 0 0 2024-02-22T15:50:41.670 INFO:tasks.workunit.client.0.smithi019.stdout:3/525: rename d9/d22/d49/l92 to d9/d26/la9 0 2024-02-22T15:50:41.671 INFO:tasks.workunit.client.1.smithi062.stdout:8/528: fsync d8/d9/d2c/f30 0 2024-02-22T15:50:41.672 INFO:tasks.workunit.client.0.smithi019.stdout:3/526: stat d9/d22/d58 0 2024-02-22T15:50:41.672 INFO:tasks.workunit.client.0.smithi019.stdout:9/677: dwrite d2/d7/dc/d27/dbc/f3f [0,4194304] 0 2024-02-22T15:50:41.674 INFO:tasks.workunit.client.0.smithi019.stdout:2/654: dread d1/d17/d1c/f34 [0,4194304] 0 2024-02-22T15:50:41.674 INFO:tasks.workunit.client.1.smithi062.stdout:8/529: mknod d8/d9/d2c/d49/d94/da1/ca3 0 2024-02-22T15:50:41.675 INFO:tasks.workunit.client.0.smithi019.stdout:7/981: chown d9/d38/d41/l99 772 1 2024-02-22T15:50:41.675 INFO:tasks.workunit.client.0.smithi019.stdout:9/678: chown d2/d1f/d2e/d57/cc2 0 1 2024-02-22T15:50:41.676 INFO:tasks.workunit.client.1.smithi062.stdout:8/530: readlink d8/d9/d44/d4d/d51/d55/l93 0 2024-02-22T15:50:41.676 INFO:tasks.workunit.client.0.smithi019.stdout:2/655: chown d1/d17/d1c/d66/db9/dd0/fe5 104 1 2024-02-22T15:50:41.678 INFO:tasks.workunit.client.0.smithi019.stdout:2/656: dread - d1/fbe zero size 2024-02-22T15:50:41.678 INFO:tasks.workunit.client.0.smithi019.stdout:8/637: write f6 [5384053,68518] 0 2024-02-22T15:50:41.679 INFO:tasks.workunit.client.1.smithi062.stdout:8/531: getdents d8/d9/d44/d58 0 2024-02-22T15:50:41.682 INFO:tasks.workunit.client.1.smithi062.stdout:3/436: dwrite d5/f6 [0,4194304] 0 2024-02-22T15:50:41.684 INFO:tasks.workunit.client.0.smithi019.stdout:9/679: creat d2/d1f/d2e/d57/d60/d8c/d66/fef x:0 0 0 2024-02-22T15:50:41.684 INFO:tasks.workunit.client.1.smithi062.stdout:8/532: rename d8/f7a to d8/d9/d48/fa4 0 2024-02-22T15:50:41.686 INFO:tasks.workunit.client.0.smithi019.stdout:2/657: creat d1/d17/d21/d51/fed x:0 0 0 2024-02-22T15:50:41.686 INFO:tasks.workunit.client.1.smithi062.stdout:3/437: write d5/d16/f19 [2736869,18304] 0 2024-02-22T15:50:41.687 INFO:tasks.workunit.client.1.smithi062.stdout:3/438: stat d5/d16/d21/f29 0 2024-02-22T15:50:41.687 INFO:tasks.workunit.client.0.smithi019.stdout:3/527: dwrite d9/d26/d3f/d5c/f6e [0,4194304] 0 2024-02-22T15:50:41.690 INFO:tasks.workunit.client.0.smithi019.stdout:8/638: creat d9/d14/d6f/d7e/d94/daa/fd5 x:0 0 0 2024-02-22T15:50:41.690 INFO:tasks.workunit.client.0.smithi019.stdout:7/982: write d9/d4c/ddf/d105/f10b [5140372,7494] 0 2024-02-22T15:50:41.695 INFO:tasks.workunit.client.0.smithi019.stdout:9/680: write d2/f4f [2469734,29672] 0 2024-02-22T15:50:41.697 INFO:tasks.workunit.client.0.smithi019.stdout:2/658: mkdir d1/d17/d1c/d66/db9/dd0/dee 0 2024-02-22T15:50:41.697 INFO:tasks.workunit.client.0.smithi019.stdout:5/678: dwrite d8/d11/f65 [0,4194304] 0 2024-02-22T15:50:41.699 INFO:tasks.workunit.client.0.smithi019.stdout:8/639: stat d9/d14/d6f/d7e/d94/d9c/fcf 0 2024-02-22T15:50:41.707 INFO:tasks.workunit.client.1.smithi062.stdout:3/439: write d5/d1a/f25 [2090769,80257] 0 2024-02-22T15:50:41.707 INFO:tasks.workunit.client.0.smithi019.stdout:3/528: dwrite d9/d26/d3f/f8c [0,4194304] 0 2024-02-22T15:50:41.710 INFO:tasks.workunit.client.0.smithi019.stdout:5/679: link d8/dd/d19/d3a/d46/f53 d8/dd/d19/d38/d54/db0/fe6 0 2024-02-22T15:50:41.713 INFO:tasks.workunit.client.1.smithi062.stdout:3/440: rename d5/d26/c82 to d5/d16/c8d 0 2024-02-22T15:50:41.717 INFO:tasks.workunit.client.0.smithi019.stdout:2/659: dwrite d1/d17/fb2 [0,4194304] 0 2024-02-22T15:50:41.718 INFO:tasks.workunit.client.0.smithi019.stdout:3/529: rmdir d9/d22 39 2024-02-22T15:50:41.719 INFO:tasks.workunit.client.1.smithi062.stdout:8/533: dread d8/d9/d2c/d49/f90 [0,4194304] 0 2024-02-22T15:50:41.720 INFO:tasks.workunit.client.0.smithi019.stdout:7/983: rename d9/d26/d77/f126 to d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/d11b/f141 0 2024-02-22T15:50:41.723 INFO:tasks.workunit.client.1.smithi062.stdout:8/534: read d8/d9/d2c/d49/d5e/f7d [2960280,7342] 0 2024-02-22T15:50:41.727 INFO:tasks.workunit.client.0.smithi019.stdout:7/984: creat d9/d4c/d5b/f142 x:0 0 0 2024-02-22T15:50:41.729 INFO:tasks.workunit.client.0.smithi019.stdout:7/985: read - d9/d38/d41/d45/f112 zero size 2024-02-22T15:50:41.730 INFO:tasks.workunit.client.1.smithi062.stdout:8/535: dwrite d8/d9/d2c/d49/f6b [0,4194304] 0 2024-02-22T15:50:41.730 INFO:tasks.workunit.client.0.smithi019.stdout:2/660: rename d1/d17/d21/daf to d1/def 0 2024-02-22T15:50:41.731 INFO:tasks.workunit.client.0.smithi019.stdout:3/530: mknod d9/d22/caa 0 2024-02-22T15:50:41.732 INFO:tasks.workunit.client.0.smithi019.stdout:2/661: creat d1/def/ff0 x:0 0 0 2024-02-22T15:50:41.733 INFO:tasks.workunit.client.0.smithi019.stdout:8/640: dread d9/d14/d17/d3d/d83/fa6 [0,4194304] 0 2024-02-22T15:50:41.734 INFO:tasks.workunit.client.0.smithi019.stdout:2/662: dread - d1/d17/d21/d51/fc2 zero size 2024-02-22T15:50:41.735 INFO:tasks.workunit.client.0.smithi019.stdout:4/649: sync 2024-02-22T15:50:41.739 INFO:tasks.workunit.client.0.smithi019.stdout:9/681: write d2/f6 [3889135,126349] 0 2024-02-22T15:50:41.740 INFO:tasks.workunit.client.1.smithi062.stdout:8/536: dwrite d8/d9/d2c/d49/f90 [4194304,4194304] 0 2024-02-22T15:50:41.740 INFO:tasks.workunit.client.0.smithi019.stdout:8/641: dread - d9/d14/d1f/d5c/fa2 zero size 2024-02-22T15:50:41.741 INFO:tasks.workunit.client.0.smithi019.stdout:2/663: creat d1/d17/d21/d71/ff1 x:0 0 0 2024-02-22T15:50:41.741 INFO:tasks.workunit.client.0.smithi019.stdout:9/682: chown d2/c90 7 1 2024-02-22T15:50:41.741 INFO:tasks.workunit.client.1.smithi062.stdout:8/537: readlink d8/d9/l66 0 2024-02-22T15:50:41.744 INFO:tasks.workunit.client.1.smithi062.stdout:8/538: symlink d8/d9/d44/d4d/d51/la5 0 2024-02-22T15:50:41.745 INFO:tasks.workunit.client.0.smithi019.stdout:9/683: mkdir d2/d1f/d2e/d57/df0 0 2024-02-22T15:50:41.746 INFO:tasks.workunit.client.1.smithi062.stdout:8/539: dread - d8/d9/d44/d4d/f69 zero size 2024-02-22T15:50:41.747 INFO:tasks.workunit.client.0.smithi019.stdout:4/650: rename d2/c13 to d2/db/d14/d23/d9e/dac/dc2/cd4 0 2024-02-22T15:50:41.748 INFO:tasks.workunit.client.0.smithi019.stdout:8/642: symlink d9/d14/d17/d3d/d83/ld6 0 2024-02-22T15:50:41.748 INFO:tasks.workunit.client.1.smithi062.stdout:8/540: getdents d8/d9/d44/d4d 0 2024-02-22T15:50:41.749 INFO:tasks.workunit.client.1.smithi062.stdout:8/541: readlink d8/d42/d8f/l67 0 2024-02-22T15:50:41.750 INFO:tasks.workunit.client.1.smithi062.stdout:8/542: creat d8/d86/fa6 x:0 0 0 2024-02-22T15:50:41.752 INFO:tasks.workunit.client.0.smithi019.stdout:9/684: creat d2/d1f/d2e/d48/d36/ff1 x:0 0 0 2024-02-22T15:50:41.755 INFO:tasks.workunit.client.0.smithi019.stdout:8/643: dread d9/d14/d17/d3d/d83/fa6 [0,4194304] 0 2024-02-22T15:50:41.756 INFO:tasks.workunit.client.0.smithi019.stdout:9/685: creat d2/d7/d52/ff2 x:0 0 0 2024-02-22T15:50:41.758 INFO:tasks.workunit.client.0.smithi019.stdout:8/644: symlink d9/d14/d17/d3d/d3f/dd4/ld7 0 2024-02-22T15:50:41.760 INFO:tasks.workunit.client.0.smithi019.stdout:8/645: mkdir d9/d14/d17/d1c/dd8 0 2024-02-22T15:50:41.761 INFO:tasks.workunit.client.1.smithi062.stdout:7/424: sync 2024-02-22T15:50:41.764 INFO:tasks.workunit.client.0.smithi019.stdout:4/651: dread d2/fa [4194304,4194304] 0 2024-02-22T15:50:41.766 INFO:tasks.workunit.client.0.smithi019.stdout:4/652: rmdir d2/db/d44 39 2024-02-22T15:50:41.768 INFO:tasks.workunit.client.0.smithi019.stdout:4/653: readlink d2/db/d14/d23/d21/d35/l37 0 2024-02-22T15:50:41.771 INFO:tasks.workunit.client.1.smithi062.stdout:8/543: dwrite d8/d9/d2c/d49/d94/fa2 [0,4194304] 0 2024-02-22T15:50:41.773 INFO:tasks.workunit.client.1.smithi062.stdout:8/544: truncate d8/d42/d8f/f91 4642524 0 2024-02-22T15:50:41.773 INFO:tasks.workunit.client.0.smithi019.stdout:2/664: dread d1/d17/d21/d71/f37 [0,4194304] 0 2024-02-22T15:50:41.774 INFO:tasks.workunit.client.0.smithi019.stdout:2/665: fsync d1/def/fd6 0 2024-02-22T15:50:41.775 INFO:tasks.workunit.client.1.smithi062.stdout:8/545: creat d8/d42/d87/fa7 x:0 0 0 2024-02-22T15:50:41.777 INFO:tasks.workunit.client.1.smithi062.stdout:8/546: symlink d8/d9/d2c/d49/d94/da1/la8 0 2024-02-22T15:50:41.778 INFO:tasks.workunit.client.0.smithi019.stdout:2/666: dread - d1/de/d15/fb3 zero size 2024-02-22T15:50:41.778 INFO:tasks.workunit.client.1.smithi062.stdout:8/547: creat d8/d86/fa9 x:0 0 0 2024-02-22T15:50:41.779 INFO:tasks.workunit.client.1.smithi062.stdout:8/548: chown d8/d28/d70/f97 111 1 2024-02-22T15:50:41.781 INFO:tasks.workunit.client.1.smithi062.stdout:8/549: creat d8/d28/d4e/faa x:0 0 0 2024-02-22T15:50:41.783 INFO:tasks.workunit.client.1.smithi062.stdout:8/550: mkdir d8/d9/d2c/d49/dab 0 2024-02-22T15:50:41.787 INFO:tasks.workunit.client.0.smithi019.stdout:2/667: dread d1/d17/d21/d51/f6e [0,4194304] 0 2024-02-22T15:50:41.787 INFO:tasks.workunit.client.1.smithi062.stdout:8/551: link d8/d26/c31 d8/d9/d44/d4d/d8a/cac 0 2024-02-22T15:50:41.788 INFO:tasks.workunit.client.0.smithi019.stdout:7/986: dread d9/d26/f2b [0,4194304] 0 2024-02-22T15:50:41.788 INFO:tasks.workunit.client.0.smithi019.stdout:5/680: dwrite d8/dd/d19/d77/d7a/f97 [0,4194304] 0 2024-02-22T15:50:41.795 INFO:tasks.workunit.client.0.smithi019.stdout:3/531: dwrite d9/d26/d3f/f4a [0,4194304] 0 2024-02-22T15:50:41.796 INFO:tasks.workunit.client.0.smithi019.stdout:2/668: chown d1/d17/d1c/d7f/cda 1144 1 2024-02-22T15:50:41.798 INFO:tasks.workunit.client.0.smithi019.stdout:5/681: creat d8/dd/d19/d77/d7a/d9e/fe7 x:0 0 0 2024-02-22T15:50:41.798 INFO:tasks.workunit.client.0.smithi019.stdout:8/646: dwrite d9/d14/d1f/d20/f61 [0,4194304] 0 2024-02-22T15:50:41.798 INFO:tasks.workunit.client.0.smithi019.stdout:4/654: dread d2/db/d14/d23/d21/f9b [0,4194304] 0 2024-02-22T15:50:41.801 INFO:tasks.workunit.client.0.smithi019.stdout:3/532: readlink d9/d22/d27/d6d/d72/l8e 0 2024-02-22T15:50:41.804 INFO:tasks.workunit.client.0.smithi019.stdout:7/987: mknod d9/d38/d41/d45/c143 0 2024-02-22T15:50:41.806 INFO:tasks.workunit.client.0.smithi019.stdout:3/533: read d9/d26/d32/f43 [2226708,123931] 0 2024-02-22T15:50:41.806 INFO:tasks.workunit.client.0.smithi019.stdout:2/669: write d1/d86/f9e [1276304,34504] 0 2024-02-22T15:50:41.808 INFO:tasks.workunit.client.0.smithi019.stdout:3/534: read - d9/f6a zero size 2024-02-22T15:50:41.811 INFO:tasks.workunit.client.0.smithi019.stdout:8/647: mknod d9/d14/d17/d3d/d3f/dd4/cd9 0 2024-02-22T15:50:41.812 INFO:tasks.workunit.client.0.smithi019.stdout:5/682: rmdir d8/dd/d19/d77/d7a/d9e 39 2024-02-22T15:50:41.812 INFO:tasks.workunit.client.0.smithi019.stdout:2/670: symlink d1/d17/d1c/d7f/lf2 0 2024-02-22T15:50:41.812 INFO:tasks.workunit.client.0.smithi019.stdout:2/671: write d1/d17/d21/d71/de1/fe3 [970497,31061] 0 2024-02-22T15:50:41.814 INFO:tasks.workunit.client.0.smithi019.stdout:4/655: mkdir d2/db/d14/d23/d21/d42/dc4/dd5 0 2024-02-22T15:50:41.815 INFO:tasks.workunit.client.0.smithi019.stdout:3/535: mkdir d9/d65/da7/dab 0 2024-02-22T15:50:41.815 INFO:tasks.workunit.client.0.smithi019.stdout:5/683: mkdir d8/dd/d19/d3a/d46/d7e/dbe/de8 0 2024-02-22T15:50:41.817 INFO:tasks.workunit.client.0.smithi019.stdout:2/672: symlink d1/d17/d21/d71/de1/lf3 0 2024-02-22T15:50:41.821 INFO:tasks.workunit.client.0.smithi019.stdout:4/656: creat d2/db/d54/d8c/fd6 x:0 0 0 2024-02-22T15:50:41.822 INFO:tasks.workunit.client.0.smithi019.stdout:2/673: stat d1/d17/d67/c93 0 2024-02-22T15:50:41.823 INFO:tasks.workunit.client.0.smithi019.stdout:7/988: link d9/d38/d41/d45/db0/df7/dfb/d12a/d5d/d8f/le8 d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/l144 0 2024-02-22T15:50:41.825 INFO:tasks.workunit.client.0.smithi019.stdout:5/684: dwrite d8/dd/d8e/d2c/f31 [0,4194304] 0 2024-02-22T15:50:41.825 INFO:tasks.workunit.client.0.smithi019.stdout:2/674: write d1/de/f5a [4664975,47133] 0 2024-02-22T15:50:41.827 INFO:tasks.workunit.client.0.smithi019.stdout:4/657: fdatasync d2/db/d14/d23/f36 0 2024-02-22T15:50:41.827 INFO:tasks.workunit.client.0.smithi019.stdout:7/989: write d9/d4c/ddf/d105/f10b [1227174,120235] 0 2024-02-22T15:50:41.833 INFO:tasks.workunit.client.0.smithi019.stdout:8/648: dread d9/d14/f38 [0,4194304] 0 2024-02-22T15:50:41.833 INFO:tasks.workunit.client.0.smithi019.stdout:4/658: fsync d2/db/d44/f55 0 2024-02-22T15:50:41.834 INFO:tasks.workunit.client.0.smithi019.stdout:4/659: chown d2/db/cf 342713 1 2024-02-22T15:50:41.835 INFO:tasks.workunit.client.0.smithi019.stdout:2/675: mkdir d1/d17/d1c/d66/db9/dd0/df4 0 2024-02-22T15:50:41.837 INFO:tasks.workunit.client.0.smithi019.stdout:2/676: write d1/d17/d21/d51/f7c [428607,64591] 0 2024-02-22T15:50:41.838 INFO:tasks.workunit.client.0.smithi019.stdout:7/990: creat d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/d11b/f145 x:0 0 0 2024-02-22T15:50:41.842 INFO:tasks.workunit.client.0.smithi019.stdout:4/660: truncate d2/db/d14/d23/f2f 1834944 0 2024-02-22T15:50:41.845 INFO:tasks.workunit.client.0.smithi019.stdout:2/677: read d1/d17/fbd [662517,53480] 0 2024-02-22T15:50:41.847 INFO:tasks.workunit.client.0.smithi019.stdout:3/536: dwrite d9/d11/f1e [0,4194304] 0 2024-02-22T15:50:41.849 INFO:tasks.workunit.client.0.smithi019.stdout:8/649: rename d9/d14/f3e to d9/d14/d1f/fda 0 2024-02-22T15:50:41.851 INFO:tasks.workunit.client.0.smithi019.stdout:4/661: fdatasync d2/f32 0 2024-02-22T15:50:41.854 INFO:tasks.workunit.client.0.smithi019.stdout:7/991: truncate d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f134 156309 0 2024-02-22T15:50:41.854 INFO:tasks.workunit.client.0.smithi019.stdout:5/685: dread d8/dd/d19/d3a/f3f [0,4194304] 0 2024-02-22T15:50:41.855 INFO:tasks.workunit.client.0.smithi019.stdout:5/686: chown d8/db1/fb7 158748090 1 2024-02-22T15:50:41.855 INFO:tasks.workunit.client.0.smithi019.stdout:2/678: rename d1/d17/d21/d71/de1/lf3 to d1/d17/d21/d71/de1/lf5 0 2024-02-22T15:50:41.863 INFO:tasks.workunit.client.0.smithi019.stdout:4/662: mkdir d2/db/d14/d23/d21/dbf/dd7 0 2024-02-22T15:50:41.868 INFO:tasks.workunit.client.0.smithi019.stdout:3/537: truncate d9/d22/d27/f47 500037 0 2024-02-22T15:50:41.871 INFO:tasks.workunit.client.0.smithi019.stdout:5/687: getdents d8/dd/d19/d77/dd2 0 2024-02-22T15:50:41.873 INFO:tasks.workunit.client.0.smithi019.stdout:3/538: rename d9/d22/d27/l54 to d9/d26/d2d/d53/da0/lac 0 2024-02-22T15:50:41.877 INFO:tasks.workunit.client.0.smithi019.stdout:7/992: dwrite d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/d130/f124 [4194304,4194304] 0 2024-02-22T15:50:41.881 INFO:tasks.workunit.client.0.smithi019.stdout:3/539: mknod d9/d26/d3f/d59/d66/cad 0 2024-02-22T15:50:41.882 INFO:tasks.workunit.client.0.smithi019.stdout:2/679: dwrite d1/d17/f30 [0,4194304] 0 2024-02-22T15:50:41.884 INFO:tasks.workunit.client.0.smithi019.stdout:4/663: dread d2/db/d14/d23/d21/fb3 [0,4194304] 0 2024-02-22T15:50:41.887 INFO:tasks.workunit.client.0.smithi019.stdout:3/540: read d9/d14/f86 [225759,120127] 0 2024-02-22T15:50:41.889 INFO:tasks.workunit.client.0.smithi019.stdout:2/680: symlink d1/d46/lf6 0 2024-02-22T15:50:41.890 INFO:tasks.workunit.client.0.smithi019.stdout:4/664: dread d2/d3f/d48/d76/fa8 [0,4194304] 0 2024-02-22T15:50:41.893 INFO:tasks.workunit.client.0.smithi019.stdout:7/993: read d9/d38/d41/d45/db0/df7/dfb/d12a/d5d/f72 [11925,75353] 0 2024-02-22T15:50:41.893 INFO:tasks.workunit.client.0.smithi019.stdout:3/541: rename d9/d22/d27/d6d/d72 to d9/d26/d3f/d59/dae 0 2024-02-22T15:50:41.896 INFO:tasks.workunit.client.0.smithi019.stdout:4/665: creat d2/d3f/d48/d76/fd8 x:0 0 0 2024-02-22T15:50:41.898 INFO:tasks.workunit.client.0.smithi019.stdout:5/688: dread d8/d11/f93 [0,4194304] 0 2024-02-22T15:50:41.905 INFO:tasks.workunit.client.0.smithi019.stdout:5/689: rename f5 to d8/dd/d19/d3a/d46/d7e/fe9 0 2024-02-22T15:50:41.906 INFO:tasks.workunit.client.0.smithi019.stdout:4/666: dwrite d2/d3f/d48/d76/fd8 [0,4194304] 0 2024-02-22T15:50:41.911 INFO:tasks.workunit.client.0.smithi019.stdout:5/690: fsync d8/dd/d19/d3a/d46/f53 0 2024-02-22T15:50:41.911 INFO:tasks.workunit.client.1.smithi062.stdout:9/463: sync 2024-02-22T15:50:41.912 INFO:tasks.workunit.client.1.smithi062.stdout:5/411: sync 2024-02-22T15:50:41.912 INFO:tasks.workunit.client.1.smithi062.stdout:2/594: sync 2024-02-22T15:50:41.912 INFO:tasks.workunit.client.1.smithi062.stdout:6/565: sync 2024-02-22T15:50:41.912 INFO:tasks.workunit.client.1.smithi062.stdout:4/480: sync 2024-02-22T15:50:41.912 INFO:tasks.workunit.client.1.smithi062.stdout:0/589: sync 2024-02-22T15:50:41.914 INFO:tasks.workunit.client.0.smithi019.stdout:4/667: read d2/db/d14/d23/f36 [1864543,78772] 0 2024-02-22T15:50:41.914 INFO:tasks.workunit.client.1.smithi062.stdout:6/566: dread - d5/df/d32/d8f/f91 zero size 2024-02-22T15:50:41.916 INFO:tasks.workunit.client.0.smithi019.stdout:2/681: dwrite d1/de/d15/f8c [0,4194304] 0 2024-02-22T15:50:41.916 INFO:tasks.workunit.client.1.smithi062.stdout:0/590: dread - d1/d38/f98 zero size 2024-02-22T15:50:41.916 INFO:tasks.workunit.client.1.smithi062.stdout:2/595: chown d1/d8/d62 181074 1 2024-02-22T15:50:41.916 INFO:tasks.workunit.client.1.smithi062.stdout:4/481: mkdir d4/d1c/d30/da5 0 2024-02-22T15:50:41.917 INFO:tasks.workunit.client.1.smithi062.stdout:9/464: readlink d4/d15/d31/l61 0 2024-02-22T15:50:41.917 INFO:tasks.workunit.client.0.smithi019.stdout:7/994: dread d9/d38/d41/d45/db0/ff5 [0,4194304] 0 2024-02-22T15:50:41.917 INFO:tasks.workunit.client.1.smithi062.stdout:5/412: chown d0/d3/da/d13/d23/d78 52 1 2024-02-22T15:50:41.918 INFO:tasks.workunit.client.0.smithi019.stdout:5/691: getdents d8/dd/d19/d38/d54/db0 0 2024-02-22T15:50:41.920 INFO:tasks.workunit.client.0.smithi019.stdout:4/668: symlink d2/db/d14/d23/d9e/dac/ld9 0 2024-02-22T15:50:41.920 INFO:tasks.workunit.client.1.smithi062.stdout:2/596: rename d1/d47/d32/d52/dbc to d1/d47/d32/d52/dbc/dbf 22 2024-02-22T15:50:41.921 INFO:tasks.workunit.client.1.smithi062.stdout:5/413: chown d0/d3/da/d13/d23/d78/l7a 0 1 2024-02-22T15:50:41.922 INFO:tasks.workunit.client.0.smithi019.stdout:2/682: fsync d1/d17/d1c/d66/f68 0 2024-02-22T15:50:41.923 INFO:tasks.workunit.client.1.smithi062.stdout:9/465: truncate d4/d15/d5f/d75/d84/f5b 625067 0 2024-02-22T15:50:41.923 INFO:tasks.workunit.client.1.smithi062.stdout:4/482: unlink d4/d2a/d2d/d2e/f69 0 2024-02-22T15:50:41.924 INFO:tasks.workunit.client.0.smithi019.stdout:4/669: write d2/db/d14/d23/f4d [3932874,58049] 0 2024-02-22T15:50:41.925 INFO:tasks.workunit.client.0.smithi019.stdout:7/995: creat d9/d38/d41/d45/db0/df7/dfb/d12a/f146 x:0 0 0 2024-02-22T15:50:41.926 INFO:tasks.workunit.client.0.smithi019.stdout:5/692: creat d8/dd/d19/d3a/d46/d7e/dbe/dc3/fea x:0 0 0 2024-02-22T15:50:41.928 INFO:tasks.workunit.client.0.smithi019.stdout:5/693: dread - d8/d11/fa8 zero size 2024-02-22T15:50:41.928 INFO:tasks.workunit.client.1.smithi062.stdout:0/591: dread d1/d3/d4/d1e/f79 [0,4194304] 0 2024-02-22T15:50:41.929 INFO:tasks.workunit.client.0.smithi019.stdout:3/542: dread d9/d11/f41 [0,4194304] 0 2024-02-22T15:50:41.929 INFO:tasks.workunit.client.1.smithi062.stdout:2/597: rename d1/d8/df/lb5 to d1/d8/d62/d73/dbd/d65/lc0 0 2024-02-22T15:50:41.929 INFO:tasks.workunit.client.0.smithi019.stdout:2/683: link d1/d17/d1c/fe7 d1/d17/d1c/d66/db9/dd0/ff7 0 2024-02-22T15:50:41.930 INFO:tasks.workunit.client.0.smithi019.stdout:3/543: chown d9/d26/d32 63181357 1 2024-02-22T15:50:41.931 INFO:tasks.workunit.client.1.smithi062.stdout:4/483: rmdir d4/d1c/d30/d3b/d49/d4b 0 2024-02-22T15:50:41.931 INFO:tasks.workunit.client.0.smithi019.stdout:4/670: creat d2/fda x:0 0 0 2024-02-22T15:50:41.932 INFO:tasks.workunit.client.0.smithi019.stdout:3/544: chown d9/d65 25984818 1 2024-02-22T15:50:41.932 INFO:tasks.workunit.client.0.smithi019.stdout:7/996: unlink d9/d38/d41/d45/db0/df7/dfb/de3/deb/ded/f111 0 2024-02-22T15:50:41.933 INFO:tasks.workunit.client.0.smithi019.stdout:7/997: chown c5 37475 1 2024-02-22T15:50:41.933 INFO:tasks.workunit.client.1.smithi062.stdout:9/466: rename d4/d15/d5f/d6a/d77 to d4/d1c/d83/d87 0 2024-02-22T15:50:41.935 INFO:tasks.workunit.client.0.smithi019.stdout:3/545: write d9/d26/d3f/d59/d66/f7f [862058,128073] 0 2024-02-22T15:50:41.935 INFO:tasks.workunit.client.0.smithi019.stdout:2/684: creat d1/d17/d21/d71/dd1/ff8 x:0 0 0 2024-02-22T15:50:41.935 INFO:tasks.workunit.client.1.smithi062.stdout:2/598: unlink d1/d8/d62/d73/c91 0 2024-02-22T15:50:41.937 INFO:tasks.workunit.client.1.smithi062.stdout:6/567: write d5/df/d32/f34 [1344482,9598] 0 2024-02-22T15:50:41.938 INFO:tasks.workunit.client.1.smithi062.stdout:4/484: dread d4/d1c/d2c/d6b/f7e [0,4194304] 0 2024-02-22T15:50:41.939 INFO:tasks.workunit.client.0.smithi019.stdout:5/694: rename d8/dd/d19/d77/d7a/d9e to d8/dd/d19/d38/d47/ddb/dde/deb 0 2024-02-22T15:50:41.940 INFO:tasks.workunit.client.1.smithi062.stdout:5/414: rename d0/d3/c22 to d0/d3/da/d13/d23/d4e/d6d/c85 0 2024-02-22T15:50:41.941 INFO:tasks.workunit.client.1.smithi062.stdout:0/592: dwrite d1/d3/d4/d1a/d76/d8c/d5d/dc3/f49 [0,4194304] 0 2024-02-22T15:50:41.941 INFO:tasks.workunit.client.0.smithi019.stdout:3/546: fsync f6 0 2024-02-22T15:50:41.942 INFO:tasks.workunit.client.0.smithi019.stdout:4/671: dwrite d2/db/d92/f91 [0,4194304] 0 2024-02-22T15:50:41.943 INFO:tasks.workunit.client.1.smithi062.stdout:9/467: creat d4/d15/d5f/d75/d84/f88 x:0 0 0 2024-02-22T15:50:41.944 INFO:tasks.workunit.client.0.smithi019.stdout:2/685: write d1/d17/d21/d71/d60/fba [2414568,10438] 0 2024-02-22T15:50:41.944 INFO:tasks.workunit.client.1.smithi062.stdout:2/599: mkdir d1/d1e/d38/d55/dc1 0 2024-02-22T15:50:41.944 INFO:tasks.workunit.client.0.smithi019.stdout:5/695: mkdir d8/dd/d19/d77/dd2/dec 0 2024-02-22T15:50:41.945 INFO:tasks.workunit.client.0.smithi019.stdout:3/547: chown d9/d26/d3f/f4a 1279 1 2024-02-22T15:50:41.946 INFO:tasks.workunit.client.1.smithi062.stdout:1/487: sync 2024-02-22T15:50:41.946 INFO:tasks.workunit.client.1.smithi062.stdout:3/441: sync 2024-02-22T15:50:41.946 INFO:tasks.workunit.client.1.smithi062.stdout:7/425: sync 2024-02-22T15:50:41.946 INFO:tasks.workunit.client.1.smithi062.stdout:8/552: sync 2024-02-22T15:50:41.949 INFO:tasks.workunit.client.0.smithi019.stdout:7/998: dwrite d9/d3a/f6b [0,4194304] 0 2024-02-22T15:50:41.949 INFO:tasks.workunit.client.1.smithi062.stdout:4/485: creat d4/d13/fa6 x:0 0 0 2024-02-22T15:50:41.956 INFO:tasks.workunit.client.0.smithi019.stdout:3/548: mknod d9/d26/d2d/d53/da0/caf 0 2024-02-22T15:50:41.957 INFO:tasks.workunit.client.1.smithi062.stdout:8/553: symlink d8/d9/d2c/lad 0 2024-02-22T15:50:41.957 INFO:tasks.workunit.client.1.smithi062.stdout:5/415: dread d0/d3/da/d13/f3d [0,4194304] 0 2024-02-22T15:50:41.959 INFO:tasks.workunit.client.0.smithi019.stdout:7/999: stat d9/d38/d41/d45/db0/df7/dfb/f12c 0 2024-02-22T15:50:41.960 INFO:tasks.workunit.client.1.smithi062.stdout:4/486: readlink d4/d34/l96 0 2024-02-22T15:50:41.961 INFO:tasks.workunit.client.0.smithi019.stdout:0/653: sync 2024-02-22T15:50:41.962 INFO:tasks.workunit.client.1.smithi062.stdout:0/593: stat d1/d3/d4/d1a/d76/d8c/d5d/f37 0 2024-02-22T15:50:41.964 INFO:tasks.workunit.client.0.smithi019.stdout:1/696: sync 2024-02-22T15:50:41.967 INFO:tasks.workunit.client.0.smithi019.stdout:4/672: dwrite d2/db/d14/d61/f73 [0,4194304] 0 2024-02-22T15:50:41.967 INFO:tasks.workunit.client.0.smithi019.stdout:0/654: write d4/f8e [786915,25414] 0 2024-02-22T15:50:41.968 INFO:tasks.workunit.client.1.smithi062.stdout:9/468: dwrite d4/d28/d50/d5a/f85 [0,4194304] 0 2024-02-22T15:50:41.969 INFO:tasks.workunit.client.1.smithi062.stdout:2/600: dwrite d1/d8/f1a [0,4194304] 0 2024-02-22T15:50:41.969 INFO:tasks.workunit.client.0.smithi019.stdout:5/696: dwrite d8/dd/d19/d3a/d46/fd5 [0,4194304] 0 2024-02-22T15:50:41.970 INFO:tasks.workunit.client.0.smithi019.stdout:2/686: dwrite d1/d17/d21/f6f [0,4194304] 0 2024-02-22T15:50:41.971 INFO:tasks.workunit.client.1.smithi062.stdout:5/416: mknod d0/d3/da/d32/d42/c86 0 2024-02-22T15:50:41.973 INFO:tasks.workunit.client.1.smithi062.stdout:0/594: mknod d1/d3/d4/d1a/d76/da5/cca 0 2024-02-22T15:50:41.980 INFO:tasks.workunit.client.0.smithi019.stdout:1/697: mkdir d8/d2a/d75/def 0 2024-02-22T15:50:41.981 INFO:tasks.workunit.client.1.smithi062.stdout:9/469: fsync d4/d35/f37 0 2024-02-22T15:50:41.983 INFO:tasks.workunit.client.0.smithi019.stdout:0/655: mkdir d4/db/d1b/d63/db0/da1/dbb/da3/de1 0 2024-02-22T15:50:41.983 INFO:tasks.workunit.client.1.smithi062.stdout:2/601: creat d1/d28/fc2 x:0 0 0 2024-02-22T15:50:41.983 INFO:tasks.workunit.client.1.smithi062.stdout:6/568: dread f0 [0,4194304] 0 2024-02-22T15:50:41.984 INFO:tasks.workunit.client.1.smithi062.stdout:5/417: unlink d0/d3/da/d32/d42/c86 0 2024-02-22T15:50:41.985 INFO:tasks.workunit.client.0.smithi019.stdout:5/697: creat d8/d11/fed x:0 0 0 2024-02-22T15:50:41.985 INFO:tasks.workunit.client.0.smithi019.stdout:4/673: truncate d2/db/d14/d23/d21/f3a 4874997 0 2024-02-22T15:50:41.986 INFO:tasks.workunit.client.0.smithi019.stdout:1/698: write d8/d18/d64/f66 [2408573,118457] 0 2024-02-22T15:50:41.989 INFO:tasks.workunit.client.1.smithi062.stdout:6/569: read d5/df/d40/d44/f6b [915016,112610] 0 2024-02-22T15:50:41.990 INFO:tasks.workunit.client.0.smithi019.stdout:2/687: getdents d1/d17/d1c 0 2024-02-22T15:50:41.990 INFO:tasks.workunit.client.0.smithi019.stdout:0/656: mknod d4/db/d76/dc6/ce2 0 2024-02-22T15:50:41.994 INFO:tasks.workunit.client.0.smithi019.stdout:3/549: dwrite d9/d26/d3f/d59/d66/f69 [0,4194304] 0 2024-02-22T15:50:41.994 INFO:tasks.workunit.client.0.smithi019.stdout:1/699: mknod d8/d44/d5d/cf0 0 2024-02-22T15:50:41.995 INFO:tasks.workunit.client.0.smithi019.stdout:5/698: mknod d8/dd/d8e/d2c/cee 0 2024-02-22T15:50:41.995 INFO:tasks.workunit.client.0.smithi019.stdout:0/657: fsync d4/db/d1b/d40/f5d 0 2024-02-22T15:50:41.997 INFO:tasks.workunit.client.1.smithi062.stdout:6/570: dwrite d5/df/d40/d44/f7c [0,4194304] 0 2024-02-22T15:50:41.997 INFO:tasks.workunit.client.0.smithi019.stdout:0/658: readlink d4/db/d24/l8f 0 2024-02-22T15:50:41.997 INFO:tasks.workunit.client.0.smithi019.stdout:2/688: creat d1/d17/d21/d71/de1/ff9 x:0 0 0 2024-02-22T15:50:41.999 INFO:tasks.workunit.client.1.smithi062.stdout:6/571: stat d5/df/d40/dac/db6 0 2024-02-22T15:50:41.999 INFO:tasks.workunit.client.1.smithi062.stdout:9/470: creat d4/d15/d5f/d75/f89 x:0 0 0 2024-02-22T15:50:42.000 INFO:tasks.workunit.client.0.smithi019.stdout:0/659: write d4/d22/f51 [5444880,126361] 0 2024-02-22T15:50:42.000 INFO:tasks.workunit.client.0.smithi019.stdout:2/689: write d1/d17/d1c/d66/db9/dd0/fe5 [177339,55192] 0 2024-02-22T15:50:42.000 INFO:tasks.workunit.client.0.smithi019.stdout:1/700: symlink d8/d18/d72/lf1 0 2024-02-22T15:50:42.001 INFO:tasks.workunit.client.1.smithi062.stdout:6/572: chown d5/df/f26 1 1 2024-02-22T15:50:42.002 INFO:tasks.workunit.client.0.smithi019.stdout:3/550: creat d9/d22/d27/fb0 x:0 0 0 2024-02-22T15:50:42.003 INFO:tasks.workunit.client.0.smithi019.stdout:5/699: mkdir d8/dd/d19/d77/dd2/def 0 2024-02-22T15:50:42.005 INFO:tasks.workunit.client.0.smithi019.stdout:2/690: mkdir d1/d17/d67/dfa 0 2024-02-22T15:50:42.005 INFO:tasks.workunit.client.1.smithi062.stdout:9/471: write d4/d1c/f6b [515125,12301] 0 2024-02-22T15:50:42.009 INFO:tasks.workunit.client.0.smithi019.stdout:0/660: creat d4/d22/d2c/d68/d6d/fe3 x:0 0 0 2024-02-22T15:50:42.009 INFO:tasks.workunit.client.0.smithi019.stdout:1/701: write d8/d15/f69 [3411297,72829] 0 2024-02-22T15:50:42.009 INFO:tasks.workunit.client.0.smithi019.stdout:3/551: fsync d9/d22/d27/f47 0 2024-02-22T15:50:42.011 INFO:tasks.workunit.client.0.smithi019.stdout:3/552: write d9/d26/d76/f77 [2558558,39904] 0 2024-02-22T15:50:42.012 INFO:tasks.workunit.client.1.smithi062.stdout:9/472: rename d4/f29 to d4/d35/f8a 0 2024-02-22T15:50:42.012 INFO:tasks.workunit.client.1.smithi062.stdout:2/602: getdents d1 0 2024-02-22T15:50:42.013 INFO:tasks.workunit.client.0.smithi019.stdout:5/700: creat d8/dd/d19/d3a/d46/d7e/dbe/de8/ff0 x:0 0 0 2024-02-22T15:50:42.014 INFO:tasks.workunit.client.1.smithi062.stdout:3/442: truncate d5/d16/f19 3016310 0 2024-02-22T15:50:42.014 INFO:tasks.workunit.client.0.smithi019.stdout:2/691: mknod d1/d17/d67/dfa/cfb 0 2024-02-22T15:50:42.016 INFO:tasks.workunit.client.0.smithi019.stdout:0/661: creat d4/d22/dcc/fe4 x:0 0 0 2024-02-22T15:50:42.017 INFO:tasks.workunit.client.1.smithi062.stdout:1/488: dread d1/d6/d36/f5d [0,4194304] 0 2024-02-22T15:50:42.017 INFO:tasks.workunit.client.1.smithi062.stdout:3/443: write d5/d1a/f25 [4036191,68771] 0 2024-02-22T15:50:42.019 INFO:tasks.workunit.client.0.smithi019.stdout:1/702: fdatasync d8/d18/d85/f8a 0 2024-02-22T15:50:42.020 INFO:tasks.workunit.client.1.smithi062.stdout:4/487: truncate d4/d1c/d2c/d6b/d70/f76 2322747 0 2024-02-22T15:50:42.021 INFO:tasks.workunit.client.1.smithi062.stdout:4/488: truncate d4/d13/fa6 436528 0 2024-02-22T15:50:42.021 INFO:tasks.workunit.client.1.smithi062.stdout:9/473: creat d4/d15/d5f/d75/d84/f8b x:0 0 0 2024-02-22T15:50:42.022 INFO:tasks.workunit.client.1.smithi062.stdout:6/573: link d5/df/d32/c4c d5/dd/cb8 0 2024-02-22T15:50:42.024 INFO:tasks.workunit.client.0.smithi019.stdout:3/553: mknod d9/d26/d3f/cb1 0 2024-02-22T15:50:42.025 INFO:tasks.workunit.client.0.smithi019.stdout:5/701: symlink d8/dd/d19/d77/dd2/def/lf1 0 2024-02-22T15:50:42.025 INFO:tasks.workunit.client.1.smithi062.stdout:6/574: fsync d5/df/f30 0 2024-02-22T15:50:42.026 INFO:tasks.workunit.client.0.smithi019.stdout:2/692: creat d1/d17/d1c/d66/db9/dd0/dee/ffc x:0 0 0 2024-02-22T15:50:42.027 INFO:tasks.workunit.client.1.smithi062.stdout:9/474: creat d4/d35/d3f/f8c x:0 0 0 2024-02-22T15:50:42.027 INFO:tasks.workunit.client.0.smithi019.stdout:1/703: rmdir d8/d2a/d75/da0 39 2024-02-22T15:50:42.028 INFO:tasks.workunit.client.0.smithi019.stdout:0/662: dwrite d4/db/d1b/d40/d44/d75/db7/f90 [0,4194304] 0 2024-02-22T15:50:42.029 INFO:tasks.workunit.client.1.smithi062.stdout:3/444: rename d5/d16/d88/f89 to d5/d16/d21/d32/d6a/f8e 0 2024-02-22T15:50:42.029 INFO:tasks.workunit.client.0.smithi019.stdout:3/554: write d9/d11/f4d [2027897,41829] 0 2024-02-22T15:50:42.031 INFO:tasks.workunit.client.0.smithi019.stdout:2/693: chown d1/de/lbb 106592 1 2024-02-22T15:50:42.033 INFO:tasks.workunit.client.0.smithi019.stdout:4/674: dwrite d2/f32 [0,4194304] 0 2024-02-22T15:50:42.034 INFO:tasks.workunit.client.1.smithi062.stdout:0/595: dread d1/d3/d4/d1a/d76/d8c/d5d/f37 [0,4194304] 0 2024-02-22T15:50:42.034 INFO:tasks.workunit.client.1.smithi062.stdout:8/554: dread d8/d9/fe [0,4194304] 0 2024-02-22T15:50:42.034 INFO:tasks.workunit.client.1.smithi062.stdout:2/603: getdents d1/d8/d62/d73/dbd 0 2024-02-22T15:50:42.039 INFO:tasks.workunit.client.0.smithi019.stdout:1/704: symlink d8/d2a/d75/d81/d8e/lf2 0 2024-02-22T15:50:42.042 INFO:tasks.workunit.client.1.smithi062.stdout:0/596: unlink d1/d13/d43/fa8 0 2024-02-22T15:50:42.043 INFO:tasks.workunit.client.1.smithi062.stdout:3/445: creat d5/d26/d2d/f8f x:0 0 0 2024-02-22T15:50:42.046 INFO:tasks.workunit.client.1.smithi062.stdout:6/575: rename d5/df/d40/d44/d4a/d41/d85/d9d to d5/dd/d33/d66/db9 0 2024-02-22T15:50:42.049 INFO:tasks.workunit.client.0.smithi019.stdout:5/702: dwrite d8/dd/d19/d3a/d46/d7e/fe9 [0,4194304] 0 2024-02-22T15:50:42.051 INFO:tasks.workunit.client.1.smithi062.stdout:1/489: write d1/d6/d7/d2b/d47/d63/f87 [970675,106345] 0 2024-02-22T15:50:42.052 INFO:tasks.workunit.client.0.smithi019.stdout:0/663: dwrite d4/db/d1b/d63/db0/da1/dbb/fba [0,4194304] 0 2024-02-22T15:50:42.052 INFO:tasks.workunit.client.0.smithi019.stdout:4/675: rename d2/db/d14/d23/d21/d35/d3d/c7b to d2/db/d14/d23/d21/dbf/cdb 0 2024-02-22T15:50:42.052 INFO:tasks.workunit.client.1.smithi062.stdout:8/555: fdatasync d8/d9/d44/d4d/f9f 0 2024-02-22T15:50:42.053 INFO:tasks.workunit.client.1.smithi062.stdout:8/556: readlink l7 0 2024-02-22T15:50:42.054 INFO:tasks.workunit.client.0.smithi019.stdout:3/555: link d9/d22/d27/l9e d9/d22/d27/lb2 0 2024-02-22T15:50:42.055 INFO:tasks.workunit.client.1.smithi062.stdout:8/557: chown d8/d26 237 1 2024-02-22T15:50:42.056 INFO:tasks.workunit.client.0.smithi019.stdout:2/694: getdents d1/def 0 2024-02-22T15:50:42.057 INFO:tasks.workunit.client.0.smithi019.stdout:5/703: creat d8/dd/d19/d77/dd2/dec/ff2 x:0 0 0 2024-02-22T15:50:42.058 INFO:tasks.workunit.client.0.smithi019.stdout:0/664: mkdir d4/d67/de5 0 2024-02-22T15:50:42.061 INFO:tasks.workunit.client.0.smithi019.stdout:1/705: dwrite d8/d18/d64/f66 [0,4194304] 0 2024-02-22T15:50:42.061 INFO:tasks.workunit.client.0.smithi019.stdout:4/676: truncate d2/db/d44/f55 1789245 0 2024-02-22T15:50:42.062 INFO:tasks.workunit.client.1.smithi062.stdout:6/576: symlink d5/df/d32/lba 0 2024-02-22T15:50:42.062 INFO:tasks.workunit.client.0.smithi019.stdout:3/556: creat d9/d26/d3f/d59/d66/fb3 x:0 0 0 2024-02-22T15:50:42.063 INFO:tasks.workunit.client.1.smithi062.stdout:4/489: dread d4/d1c/d30/f32 [0,4194304] 0 2024-02-22T15:50:42.063 INFO:tasks.workunit.client.1.smithi062.stdout:2/604: symlink d1/d8/d62/d73/dbd/dbb/lc3 0 2024-02-22T15:50:42.064 INFO:tasks.workunit.client.1.smithi062.stdout:9/475: write d4/d35/f37 [3037206,40361] 0 2024-02-22T15:50:42.064 INFO:tasks.workunit.client.0.smithi019.stdout:5/704: chown d8/d11/d33/f74 14942259 1 2024-02-22T15:50:42.064 INFO:tasks.workunit.client.1.smithi062.stdout:8/558: mknod d8/d9/d44/cae 0 2024-02-22T15:50:42.067 INFO:tasks.workunit.client.1.smithi062.stdout:0/597: dwrite d1/d3/d4/d1a/d76/d8c/f88 [0,4194304] 0 2024-02-22T15:50:42.067 INFO:tasks.workunit.client.0.smithi019.stdout:4/677: chown d2/db/d44/c72 249354353 1 2024-02-22T15:50:42.067 INFO:tasks.workunit.client.0.smithi019.stdout:0/665: readlink d4/db/d1b/d63/db0/da1/dbb/d52/la4 0 2024-02-22T15:50:42.068 INFO:tasks.workunit.client.0.smithi019.stdout:2/695: rename d1/d17/c5b to d1/d17/d4b/cfd 0 2024-02-22T15:50:42.069 INFO:tasks.workunit.client.1.smithi062.stdout:2/605: stat d1/d8/df/f78 0 2024-02-22T15:50:42.069 INFO:tasks.workunit.client.0.smithi019.stdout:5/705: mkdir d8/dd/d19/d38/d47/df3 0 2024-02-22T15:50:42.069 INFO:tasks.workunit.client.1.smithi062.stdout:6/577: chown d5/df/d40/d44/d78/lb4 43209699 1 2024-02-22T15:50:42.070 INFO:tasks.workunit.client.1.smithi062.stdout:9/476: fdatasync d4/d1c/f1d 0 2024-02-22T15:50:42.071 INFO:tasks.workunit.client.1.smithi062.stdout:3/446: dwrite d5/d1a/f1e [0,4194304] 0 2024-02-22T15:50:42.076 INFO:tasks.workunit.client.1.smithi062.stdout:0/598: unlink d1/d3/l19 0 2024-02-22T15:50:42.076 INFO:tasks.workunit.client.1.smithi062.stdout:6/578: readlink d5/df/d40/d44/d78/l7b 0 2024-02-22T15:50:42.077 INFO:tasks.workunit.client.1.smithi062.stdout:0/599: readlink d1/d13/l4b 0 2024-02-22T15:50:42.079 INFO:tasks.workunit.client.0.smithi019.stdout:3/557: dwrite d9/d26/d32/f9c [0,4194304] 0 2024-02-22T15:50:42.079 INFO:tasks.workunit.client.0.smithi019.stdout:4/678: rename d2/db/d14/f27 to d2/db/fdc 0 2024-02-22T15:50:42.079 INFO:tasks.workunit.client.1.smithi062.stdout:4/490: mkdir d4/d1c/d30/da7 0 2024-02-22T15:50:42.081 INFO:tasks.workunit.client.1.smithi062.stdout:1/490: rename d1/d6/d7/d2b/d47/d24/f2a to d1/d6/d7/d8a/fa5 0 2024-02-22T15:50:42.082 INFO:tasks.workunit.client.1.smithi062.stdout:3/447: creat d5/d26/d2d/f90 x:0 0 0 2024-02-22T15:50:42.082 INFO:tasks.workunit.client.1.smithi062.stdout:8/559: symlink d8/d9/d44/d4d/laf 0 2024-02-22T15:50:42.084 INFO:tasks.workunit.client.1.smithi062.stdout:9/477: mknod d4/c8d 0 2024-02-22T15:50:42.085 INFO:tasks.workunit.client.1.smithi062.stdout:0/600: fsync d1/d3/d4/d1a/d76/d8c/d2c/d87/fb9 0 2024-02-22T15:50:42.087 INFO:tasks.workunit.client.1.smithi062.stdout:8/560: dread - d8/d28/d70/f97 zero size 2024-02-22T15:50:42.087 INFO:tasks.workunit.client.1.smithi062.stdout:2/606: dread d1/d8/d62/d73/f75 [0,4194304] 0 2024-02-22T15:50:42.088 INFO:tasks.workunit.client.1.smithi062.stdout:6/579: truncate d5/df/d32/d8f/f91 953694 0 2024-02-22T15:50:42.090 INFO:tasks.workunit.client.1.smithi062.stdout:3/448: readlink d5/l45 0 2024-02-22T15:50:42.091 INFO:tasks.workunit.client.0.smithi019.stdout:1/706: write d8/d15/d34/fa3 [801265,62011] 0 2024-02-22T15:50:42.093 INFO:tasks.workunit.client.0.smithi019.stdout:1/707: write d8/d15/d34/d9e/fe7 [565223,112057] 0 2024-02-22T15:50:42.097 INFO:tasks.workunit.client.0.smithi019.stdout:1/708: rename d8/d15/d5b/dbc/c8f to d8/d18/dd2/cf3 0 2024-02-22T15:50:42.099 INFO:tasks.workunit.client.1.smithi062.stdout:4/491: rename d4/d2a/d2d/l80 to d4/d2a/d2d/d46/la8 0 2024-02-22T15:50:42.100 INFO:tasks.workunit.client.0.smithi019.stdout:1/709: creat d8/d18/dd2/ff4 x:0 0 0 2024-02-22T15:50:42.100 INFO:tasks.workunit.client.1.smithi062.stdout:6/580: write d5/df/d40/d44/d4a/fad [1000474,115068] 0 2024-02-22T15:50:42.101 INFO:tasks.workunit.client.0.smithi019.stdout:0/666: dread d4/db/d76/d77/faa [0,4194304] 0 2024-02-22T15:50:42.101 INFO:tasks.workunit.client.1.smithi062.stdout:1/491: creat d1/d6/d7/d2b/d47/d63/fa6 x:0 0 0 2024-02-22T15:50:42.101 INFO:tasks.workunit.client.1.smithi062.stdout:2/607: unlink d1/d1e/lae 0 2024-02-22T15:50:42.105 INFO:tasks.workunit.client.0.smithi019.stdout:0/667: creat d4/d32/fe6 x:0 0 0 2024-02-22T15:50:42.106 INFO:tasks.workunit.client.0.smithi019.stdout:5/706: write d8/d11/d33/f76 [254400,75852] 0 2024-02-22T15:50:42.107 INFO:tasks.workunit.client.1.smithi062.stdout:4/492: dread - d4/d1c/d2c/d6b/d70/f93 zero size 2024-02-22T15:50:42.108 INFO:tasks.workunit.client.1.smithi062.stdout:3/449: write d5/d16/d21/d36/d4a/f4f [3661453,11079] 0 2024-02-22T15:50:42.110 INFO:tasks.workunit.client.1.smithi062.stdout:8/561: truncate d8/d42/f7c 624528 0 2024-02-22T15:50:42.110 INFO:tasks.workunit.client.0.smithi019.stdout:3/558: truncate d9/d11/f8a 419437 0 2024-02-22T15:50:42.111 INFO:tasks.workunit.client.1.smithi062.stdout:4/493: chown d4/d2a/l5d 1 1 2024-02-22T15:50:42.112 INFO:tasks.workunit.client.1.smithi062.stdout:0/601: dwrite d1/d3/d4/d1a/d76/fb1 [0,4194304] 0 2024-02-22T15:50:42.112 INFO:tasks.workunit.client.0.smithi019.stdout:5/707: dread - d8/dd/d19/d3a/d46/d7e/dbe/dc3/fea zero size 2024-02-22T15:50:42.112 INFO:tasks.workunit.client.0.smithi019.stdout:1/710: dwrite d8/d18/dd2/ff4 [0,4194304] 0 2024-02-22T15:50:42.113 INFO:tasks.workunit.client.1.smithi062.stdout:1/492: creat d1/d6/d7/d2b/d47/d7e/fa7 x:0 0 0 2024-02-22T15:50:42.113 INFO:tasks.workunit.client.0.smithi019.stdout:0/668: truncate d4/d22/d2c/d68/f6a 4823348 0 2024-02-22T15:50:42.113 INFO:tasks.workunit.client.0.smithi019.stdout:5/708: dread - d8/d11/fa8 zero size 2024-02-22T15:50:42.114 INFO:tasks.workunit.client.0.smithi019.stdout:1/711: chown d8/d15/d34/d9e/fe7 299 1 2024-02-22T15:50:42.119 INFO:tasks.workunit.client.0.smithi019.stdout:1/712: fdatasync d8/d15/f69 0 2024-02-22T15:50:42.120 INFO:tasks.workunit.client.1.smithi062.stdout:4/494: mknod d4/d2a/d2d/d64/ca9 0 2024-02-22T15:50:42.121 INFO:tasks.workunit.client.0.smithi019.stdout:3/559: link d9/d14/f95 d9/d11/fb4 0 2024-02-22T15:50:42.122 INFO:tasks.workunit.client.1.smithi062.stdout:4/495: stat l0 0 2024-02-22T15:50:42.122 INFO:tasks.workunit.client.0.smithi019.stdout:2/696: write d1/d17/d21/d71/f37 [1559312,28547] 0 2024-02-22T15:50:42.125 INFO:tasks.workunit.client.0.smithi019.stdout:1/713: truncate d8/d15/d5b/fb2 197998 0 2024-02-22T15:50:42.125 INFO:tasks.workunit.client.0.smithi019.stdout:3/560: unlink d9/d11/f1e 0 2024-02-22T15:50:42.126 INFO:tasks.workunit.client.1.smithi062.stdout:8/562: creat d8/d9/d44/d4d/fb0 x:0 0 0 2024-02-22T15:50:42.126 INFO:tasks.workunit.client.1.smithi062.stdout:6/581: truncate d5/df/f26 2888095 0 2024-02-22T15:50:42.127 INFO:tasks.workunit.client.1.smithi062.stdout:9/478: dread d4/d35/f8a [0,4194304] 0 2024-02-22T15:50:42.128 INFO:tasks.workunit.client.0.smithi019.stdout:5/709: dread d8/dd/d8e/d2c/f63 [0,4194304] 0 2024-02-22T15:50:42.129 INFO:tasks.workunit.client.0.smithi019.stdout:2/697: read d1/d17/d4b/f92 [242541,72854] 0 2024-02-22T15:50:42.131 INFO:tasks.workunit.client.1.smithi062.stdout:8/563: truncate d8/d26/f47 1224542 0 2024-02-22T15:50:42.132 INFO:tasks.workunit.client.1.smithi062.stdout:1/493: rename d1/d6/d7/c13 to d1/d6/d7/d2b/d47/ca8 0 2024-02-22T15:50:42.133 INFO:tasks.workunit.client.0.smithi019.stdout:2/698: stat d1/d17/d1c/c5f 0 2024-02-22T15:50:42.133 INFO:tasks.workunit.client.1.smithi062.stdout:2/608: dwrite d1/d47/d32/d52/d80/f9b [0,4194304] 0 2024-02-22T15:50:42.134 INFO:tasks.workunit.client.1.smithi062.stdout:3/450: dwrite d5/d1a/d3c/f54 [4194304,4194304] 0 2024-02-22T15:50:42.134 INFO:tasks.workunit.client.0.smithi019.stdout:2/699: mkdir d1/d17/d1c/d66/db9/dfe 0 2024-02-22T15:50:42.135 INFO:tasks.workunit.client.1.smithi062.stdout:0/602: mknod d1/d3/d4/d1a/d76/d8c/d5d/dc3/ccb 0 2024-02-22T15:50:42.136 INFO:tasks.workunit.client.1.smithi062.stdout:9/479: truncate d4/d28/d50/f52 827846 0 2024-02-22T15:50:42.137 INFO:tasks.workunit.client.1.smithi062.stdout:8/564: write d8/d26/f47 [1991274,122114] 0 2024-02-22T15:50:42.139 INFO:tasks.workunit.client.0.smithi019.stdout:1/714: write d8/d18/d64/f95 [199344,30566] 0 2024-02-22T15:50:42.141 INFO:tasks.workunit.client.1.smithi062.stdout:9/480: chown d4/c8d 2684 1 2024-02-22T15:50:42.142 INFO:tasks.workunit.client.0.smithi019.stdout:2/700: symlink d1/d17/d67/dfa/lff 0 2024-02-22T15:50:42.143 INFO:tasks.workunit.client.1.smithi062.stdout:1/494: mkdir d1/d6/d7/d2b/d49/da9 0 2024-02-22T15:50:42.145 INFO:tasks.workunit.client.0.smithi019.stdout:2/701: dread d1/d17/d4b/f92 [0,4194304] 0 2024-02-22T15:50:42.147 INFO:tasks.workunit.client.1.smithi062.stdout:3/451: creat d5/d16/d88/f91 x:0 0 0 2024-02-22T15:50:42.148 INFO:tasks.workunit.client.0.smithi019.stdout:2/702: fdatasync d1/d46/f5e 0 2024-02-22T15:50:42.149 INFO:tasks.workunit.client.1.smithi062.stdout:2/609: mkdir d1/d8/d62/d73/dbd/dbb/dc4 0 2024-02-22T15:50:42.151 INFO:tasks.workunit.client.1.smithi062.stdout:9/481: mknod d4/d15/d5f/d6a/c8e 0 2024-02-22T15:50:42.151 INFO:tasks.workunit.client.1.smithi062.stdout:2/610: fsync d1/d8/d62/d73/dbd/f59 0 2024-02-22T15:50:42.151 INFO:tasks.workunit.client.0.smithi019.stdout:2/703: rmdir d1/d17/d21/d71/dd1 39 2024-02-22T15:50:42.153 INFO:tasks.workunit.client.1.smithi062.stdout:1/495: unlink d1/d6/d36/l7b 0 2024-02-22T15:50:42.154 INFO:tasks.workunit.client.0.smithi019.stdout:0/669: dread d4/db/d1b/d63/db0/da1/dbb/d52/f62 [0,4194304] 0 2024-02-22T15:50:42.155 INFO:tasks.workunit.client.1.smithi062.stdout:3/452: readlink d5/d16/l2c 0 2024-02-22T15:50:42.155 INFO:tasks.workunit.client.0.smithi019.stdout:2/704: creat d1/d17/d67/f100 x:0 0 0 2024-02-22T15:50:42.156 INFO:tasks.workunit.client.1.smithi062.stdout:9/482: creat d4/d15/d31/f8f x:0 0 0 2024-02-22T15:50:42.156 INFO:tasks.workunit.client.0.smithi019.stdout:3/561: dread d9/d22/d27/d6d/f82 [0,4194304] 0 2024-02-22T15:50:42.156 INFO:tasks.workunit.client.1.smithi062.stdout:3/453: write d5/d26/d2d/f90 [264273,118682] 0 2024-02-22T15:50:42.157 INFO:tasks.workunit.client.1.smithi062.stdout:2/611: mkdir d1/d1e/d38/d76/dc5 0 2024-02-22T15:50:42.159 INFO:tasks.workunit.client.1.smithi062.stdout:1/496: read d1/d6/d7/f7a [3992468,45040] 0 2024-02-22T15:50:42.159 INFO:tasks.workunit.client.1.smithi062.stdout:9/483: dread - d4/d15/d5e/d63/f86 zero size 2024-02-22T15:50:42.159 INFO:tasks.workunit.client.0.smithi019.stdout:3/562: creat d9/d26/fb5 x:0 0 0 2024-02-22T15:50:42.160 INFO:tasks.workunit.client.0.smithi019.stdout:0/670: dread d4/f8e [0,4194304] 0 2024-02-22T15:50:42.161 INFO:tasks.workunit.client.0.smithi019.stdout:1/715: write d8/d18/d64/d7f/de3/fed [1923771,31688] 0 2024-02-22T15:50:42.161 INFO:tasks.workunit.client.0.smithi019.stdout:2/705: dread d1/d17/d21/f41 [0,4194304] 0 2024-02-22T15:50:42.162 INFO:tasks.workunit.client.1.smithi062.stdout:9/484: mknod d4/d1c/d83/d87/c90 0 2024-02-22T15:50:42.162 INFO:tasks.workunit.client.0.smithi019.stdout:3/563: creat d9/d22/d27/fb6 x:0 0 0 2024-02-22T15:50:42.163 INFO:tasks.workunit.client.0.smithi019.stdout:1/716: write d8/f88 [1330060,128971] 0 2024-02-22T15:50:42.167 INFO:tasks.workunit.client.1.smithi062.stdout:1/497: rename d1/f2e to d1/d6/d7/d4a/d65/d83/faa 0 2024-02-22T15:50:42.169 INFO:tasks.workunit.client.1.smithi062.stdout:6/582: write d5/df/d40/d44/d4a/f2d [518496,109895] 0 2024-02-22T15:50:42.170 INFO:tasks.workunit.client.1.smithi062.stdout:1/498: chown d1/d6/d36/l73 1387719 1 2024-02-22T15:50:42.170 INFO:tasks.workunit.client.1.smithi062.stdout:3/454: dwrite d5/d16/d21/f2b [4194304,4194304] 0 2024-02-22T15:50:42.170 INFO:tasks.workunit.client.0.smithi019.stdout:3/564: rename d9/d22/d27/l34 to d9/d16/lb7 0 2024-02-22T15:50:42.170 INFO:tasks.workunit.client.1.smithi062.stdout:9/485: symlink d4/l91 0 2024-02-22T15:50:42.171 INFO:tasks.workunit.client.0.smithi019.stdout:0/671: dwrite d4/db/d1b/d40/d44/d75/db7/f90 [0,4194304] 0 2024-02-22T15:50:42.172 INFO:tasks.workunit.client.1.smithi062.stdout:1/499: dread - d1/d6/de/f72 zero size 2024-02-22T15:50:42.173 INFO:tasks.workunit.client.0.smithi019.stdout:1/717: mkdir d8/d15/df5 0 2024-02-22T15:50:42.174 INFO:tasks.workunit.client.1.smithi062.stdout:1/500: readlink d1/d6/de/l46 0 2024-02-22T15:50:42.174 INFO:tasks.workunit.client.1.smithi062.stdout:3/455: dread - d5/d16/d21/d32/f51 zero size 2024-02-22T15:50:42.175 INFO:tasks.workunit.client.1.smithi062.stdout:9/486: mkdir d4/d35/d3f/d92 0 2024-02-22T15:50:42.177 INFO:tasks.workunit.client.0.smithi019.stdout:3/565: truncate d9/d14/f3c 1021670 0 2024-02-22T15:50:42.177 INFO:tasks.workunit.client.1.smithi062.stdout:1/501: symlink d1/d6/d7/lab 0 2024-02-22T15:50:42.178 INFO:tasks.workunit.client.0.smithi019.stdout:2/706: link d1/d17/d1c/cbf d1/d17/d4b/c101 0 2024-02-22T15:50:42.180 INFO:tasks.workunit.client.0.smithi019.stdout:1/718: dread - d8/d2a/d75/da0/fe9 zero size 2024-02-22T15:50:42.182 INFO:tasks.workunit.client.1.smithi062.stdout:6/583: creat d5/fbb x:0 0 0 2024-02-22T15:50:42.182 INFO:tasks.workunit.client.0.smithi019.stdout:0/672: dwrite d4/d9/f26 [4194304,4194304] 0 2024-02-22T15:50:42.183 INFO:tasks.workunit.client.1.smithi062.stdout:3/456: read d5/d16/f48 [1216100,97908] 0 2024-02-22T15:50:42.185 INFO:tasks.workunit.client.0.smithi019.stdout:2/707: symlink d1/d17/d1c/d66/l102 0 2024-02-22T15:50:42.185 INFO:tasks.workunit.client.1.smithi062.stdout:4/496: dread d4/f45 [0,4194304] 0 2024-02-22T15:50:42.187 INFO:tasks.workunit.client.1.smithi062.stdout:9/487: truncate d4/d1c/f1d 3353405 0 2024-02-22T15:50:42.187 INFO:tasks.workunit.client.0.smithi019.stdout:1/719: creat d8/d18/d72/ff6 x:0 0 0 2024-02-22T15:50:42.188 INFO:tasks.workunit.client.1.smithi062.stdout:0/603: dread d1/d3/d4/d1a/d76/d8c/f64 [0,4194304] 0 2024-02-22T15:50:42.188 INFO:tasks.workunit.client.0.smithi019.stdout:0/673: mkdir d4/db/d1b/d63/db0/da1/dbb/de7 0 2024-02-22T15:50:42.191 INFO:tasks.workunit.client.1.smithi062.stdout:3/457: mknod d5/d16/d21/d32/d75/c92 0 2024-02-22T15:50:42.193 INFO:tasks.workunit.client.0.smithi019.stdout:6/866: sync 2024-02-22T15:50:42.193 INFO:tasks.workunit.client.0.smithi019.stdout:2/708: creat d1/de/f103 x:0 0 0 2024-02-22T15:50:42.196 INFO:tasks.workunit.client.0.smithi019.stdout:1/720: fsync d8/d15/fa6 0 2024-02-22T15:50:42.196 INFO:tasks.workunit.client.0.smithi019.stdout:0/674: dwrite d4/d23/f4a [0,4194304] 0 2024-02-22T15:50:42.197 INFO:tasks.workunit.client.0.smithi019.stdout:6/867: rmdir d6/d4b/d6d/d7b 39 2024-02-22T15:50:42.198 INFO:tasks.workunit.client.1.smithi062.stdout:0/604: mknod d1/d3/d4/ccc 0 2024-02-22T15:50:42.201 INFO:tasks.workunit.client.1.smithi062.stdout:1/502: dwrite d1/d6/d7/d2b/d96/f9e [0,4194304] 0 2024-02-22T15:50:42.202 INFO:tasks.workunit.client.1.smithi062.stdout:9/488: mknod d4/c93 0 2024-02-22T15:50:42.202 INFO:tasks.workunit.client.0.smithi019.stdout:0/675: mkdir d4/d22/d3d/de8 0 2024-02-22T15:50:42.203 INFO:tasks.workunit.client.1.smithi062.stdout:1/503: write d1/d6/de/f42 [3109156,100583] 0 2024-02-22T15:50:42.204 INFO:tasks.workunit.client.1.smithi062.stdout:4/497: creat d4/d1c/faa x:0 0 0 2024-02-22T15:50:42.204 INFO:tasks.workunit.client.1.smithi062.stdout:9/489: chown d4/d35/d3f/c21 17 1 2024-02-22T15:50:42.206 INFO:tasks.workunit.client.1.smithi062.stdout:3/458: creat d5/d16/d21/d36/d5a/d85/f93 x:0 0 0 2024-02-22T15:50:42.207 INFO:tasks.workunit.client.0.smithi019.stdout:6/868: getdents d6/d4b/d71/d88/ded 0 2024-02-22T15:50:42.207 INFO:tasks.workunit.client.0.smithi019.stdout:2/709: read d1/d17/fae [482286,38728] 0 2024-02-22T15:50:42.207 INFO:tasks.workunit.client.0.smithi019.stdout:0/676: symlink d4/le9 0 2024-02-22T15:50:42.208 INFO:tasks.workunit.client.1.smithi062.stdout:0/605: mknod d1/d3/d4/d1a/d76/d8c/d5d/dc3/ccd 0 2024-02-22T15:50:42.208 INFO:tasks.workunit.client.1.smithi062.stdout:3/459: fdatasync d5/d26/f38 0 2024-02-22T15:50:42.209 INFO:tasks.workunit.client.0.smithi019.stdout:3/566: write d9/d22/f4f [830091,59861] 0 2024-02-22T15:50:42.210 INFO:tasks.workunit.client.0.smithi019.stdout:1/721: dwrite d8/d18/d64/f67 [4194304,4194304] 0 2024-02-22T15:50:42.211 INFO:tasks.workunit.client.1.smithi062.stdout:1/504: unlink d1/d6/d14/f45 0 2024-02-22T15:50:42.212 INFO:tasks.workunit.client.1.smithi062.stdout:4/498: mknod d4/d1c/d30/da7/cab 0 2024-02-22T15:50:42.214 INFO:tasks.workunit.client.1.smithi062.stdout:9/490: rename d4/d15/f17 to d4/d15/d31/f94 0 2024-02-22T15:50:42.215 INFO:tasks.workunit.client.0.smithi019.stdout:2/710: write d1/d17/d21/d51/fcc [874038,21030] 0 2024-02-22T15:50:42.215 INFO:tasks.workunit.client.0.smithi019.stdout:3/567: mknod d9/d65/da7/cb8 0 2024-02-22T15:50:42.215 INFO:tasks.workunit.client.1.smithi062.stdout:4/499: fsync d4/d1c/f23 0 2024-02-22T15:50:42.215 INFO:tasks.workunit.client.1.smithi062.stdout:9/491: fdatasync d4/d15/d5f/f7b 0 2024-02-22T15:50:42.217 INFO:tasks.workunit.client.0.smithi019.stdout:1/722: symlink d8/d2a/d97/lf7 0 2024-02-22T15:50:42.220 INFO:tasks.workunit.client.0.smithi019.stdout:6/869: getdents d6/d31/d69 0 2024-02-22T15:50:42.221 INFO:tasks.workunit.client.1.smithi062.stdout:3/460: rename d5/d16/l77 to d5/d16/d21/d36/d5a/d85/l94 0 2024-02-22T15:50:42.222 INFO:tasks.workunit.client.0.smithi019.stdout:0/677: rename d4/db/d1b/d63/db0/da1/dbb/dc/d99 to d4/db/d1b/dea 0 2024-02-22T15:50:42.223 INFO:tasks.workunit.client.1.smithi062.stdout:0/606: truncate d1/d3/d4/d1a/d76/d8c/d5d/f37 4014298 0 2024-02-22T15:50:42.223 INFO:tasks.workunit.client.1.smithi062.stdout:9/492: symlink d4/d28/d50/d5a/l95 0 2024-02-22T15:50:42.224 INFO:tasks.workunit.client.0.smithi019.stdout:3/568: getdents d9/d22 0 2024-02-22T15:50:42.224 INFO:tasks.workunit.client.1.smithi062.stdout:0/607: write d1/d3/d5f/f57 [3265747,42051] 0 2024-02-22T15:50:42.225 INFO:tasks.workunit.client.1.smithi062.stdout:9/493: chown d4/f10 6 1 2024-02-22T15:50:42.227 INFO:tasks.workunit.client.0.smithi019.stdout:1/723: rename d8/d15/d84 to d8/d2a/d75/def/df8 0 2024-02-22T15:50:42.228 INFO:tasks.workunit.client.1.smithi062.stdout:4/500: mkdir d4/d1c/dac 0 2024-02-22T15:50:42.230 INFO:tasks.workunit.client.1.smithi062.stdout:1/505: getdents d1/d6/d7/d4a 0 2024-02-22T15:50:42.231 INFO:tasks.workunit.client.1.smithi062.stdout:9/494: mknod d4/d28/c96 0 2024-02-22T15:50:42.231 INFO:tasks.workunit.client.0.smithi019.stdout:1/724: write d8/fdf [149497,47928] 0 2024-02-22T15:50:42.231 INFO:tasks.workunit.client.0.smithi019.stdout:3/569: symlink d9/lb9 0 2024-02-22T15:50:42.231 INFO:tasks.workunit.client.1.smithi062.stdout:0/608: rmdir d1/d3/d4/d1a/d76/d8c/d92 39 2024-02-22T15:50:42.232 INFO:tasks.workunit.client.0.smithi019.stdout:6/870: dwrite d6/d4b/d6d/d7b/f60 [0,4194304] 0 2024-02-22T15:50:42.232 INFO:tasks.workunit.client.0.smithi019.stdout:0/678: link d4/db/d1b/cce d4/db/d76/ceb 0 2024-02-22T15:50:42.233 INFO:tasks.workunit.client.1.smithi062.stdout:4/501: read d4/d1c/f3d [1976540,20661] 0 2024-02-22T15:50:42.235 INFO:tasks.workunit.client.0.smithi019.stdout:0/679: chown d4/d22/f51 5607606 1 2024-02-22T15:50:42.235 INFO:tasks.workunit.client.0.smithi019.stdout:1/725: rmdir d8/d2a/d97 39 2024-02-22T15:50:42.236 INFO:tasks.workunit.client.1.smithi062.stdout:6/584: dread d5/df/d40/d44/d76/f6a [0,4194304] 0 2024-02-22T15:50:42.236 INFO:tasks.workunit.client.0.smithi019.stdout:3/570: truncate d9/d22/f9d 5999067 0 2024-02-22T15:50:42.237 INFO:tasks.workunit.client.1.smithi062.stdout:9/495: chown d4/d15/d5f/d75/f68 409164 1 2024-02-22T15:50:42.238 INFO:tasks.workunit.client.0.smithi019.stdout:6/871: write d6/d31/f36 [4126351,48613] 0 2024-02-22T15:50:42.239 INFO:tasks.workunit.client.0.smithi019.stdout:0/680: symlink d4/d9/da7/db2/lec 0 2024-02-22T15:50:42.240 INFO:tasks.workunit.client.1.smithi062.stdout:3/461: dwrite d5/f8 [0,4194304] 0 2024-02-22T15:50:42.240 INFO:tasks.workunit.client.1.smithi062.stdout:0/609: write d1/d3/d4/d1a/d76/d8c/d5d/dc3/f48 [2890379,58173] 0 2024-02-22T15:50:42.241 INFO:tasks.workunit.client.0.smithi019.stdout:1/726: mkdir d8/d18/d64/d7f/d94/df9 0 2024-02-22T15:50:42.242 INFO:tasks.workunit.client.1.smithi062.stdout:1/506: mknod d1/d6/d7/d2b/d47/d7e/cac 0 2024-02-22T15:50:42.245 INFO:tasks.workunit.client.1.smithi062.stdout:0/610: chown d1/d3/d4/d1e/d74/d68/fc7 284416411 1 2024-02-22T15:50:42.247 INFO:tasks.workunit.client.0.smithi019.stdout:6/872: dwrite d6/f44 [0,4194304] 0 2024-02-22T15:50:42.248 INFO:tasks.workunit.client.1.smithi062.stdout:9/496: mkdir d4/d15/d5f/d75/d97 0 2024-02-22T15:50:42.248 INFO:tasks.workunit.client.1.smithi062.stdout:3/462: creat d5/d16/d21/d36/f95 x:0 0 0 2024-02-22T15:50:42.250 INFO:tasks.workunit.client.0.smithi019.stdout:1/727: creat d8/d18/d64/d7f/de3/ffa x:0 0 0 2024-02-22T15:50:42.251 INFO:tasks.workunit.client.1.smithi062.stdout:6/585: mkdir d5/df/dbc 0 2024-02-22T15:50:42.253 INFO:tasks.workunit.client.1.smithi062.stdout:9/497: read d4/d15/d5f/d75/d84/f59 [315282,13345] 0 2024-02-22T15:50:42.253 INFO:tasks.workunit.client.1.smithi062.stdout:3/463: chown d5/d26/d2d/f8f 11 1 2024-02-22T15:50:42.253 INFO:tasks.workunit.client.1.smithi062.stdout:4/502: dwrite d4/f45 [4194304,4194304] 0 2024-02-22T15:50:42.257 INFO:tasks.workunit.client.1.smithi062.stdout:0/611: write d1/dd/f6b [1029674,4942] 0 2024-02-22T15:50:42.257 INFO:tasks.workunit.client.0.smithi019.stdout:6/873: rename d6/d4b/d71/d88/dc0 to d6/d4b/da9/d67/d11b 0 2024-02-22T15:50:42.258 INFO:tasks.workunit.client.1.smithi062.stdout:1/507: mkdir d1/d6/d7/d2b/d49/d74/dad 0 2024-02-22T15:50:42.261 INFO:tasks.workunit.client.1.smithi062.stdout:1/508: truncate d1/d6/de/f42 4880889 0 2024-02-22T15:50:42.262 INFO:tasks.workunit.client.0.smithi019.stdout:6/874: readlink d6/d4b/da9/d67/d8a/d90/lc6 0 2024-02-22T15:50:42.263 INFO:tasks.workunit.client.0.smithi019.stdout:6/875: readlink d6/d31/d69/d8e/l10e 0 2024-02-22T15:50:42.263 INFO:tasks.workunit.client.0.smithi019.stdout:0/681: dwrite d4/db/d1b/d63/db0/da1/dbb/fe [0,4194304] 0 2024-02-22T15:50:42.266 INFO:tasks.workunit.client.0.smithi019.stdout:0/682: write d4/d22/d2c/fac [1048330,34028] 0 2024-02-22T15:50:42.267 INFO:tasks.workunit.client.1.smithi062.stdout:6/586: symlink d5/df/d32/d47/lbd 0 2024-02-22T15:50:42.268 INFO:tasks.workunit.client.0.smithi019.stdout:0/683: dread - d4/d23/d7d/f85 zero size 2024-02-22T15:50:42.269 INFO:tasks.workunit.client.0.smithi019.stdout:0/684: write f2 [7646388,69920] 0 2024-02-22T15:50:42.269 INFO:tasks.workunit.client.0.smithi019.stdout:1/728: read d8/d15/d34/f4d [3833227,78154] 0 2024-02-22T15:50:42.270 INFO:tasks.workunit.client.0.smithi019.stdout:6/876: getdents d6/d4b/d6d/d7b/d7f 0 2024-02-22T15:50:42.273 INFO:tasks.workunit.client.1.smithi062.stdout:4/503: readlink d4/d13/d20/l31 0 2024-02-22T15:50:42.277 INFO:tasks.workunit.client.0.smithi019.stdout:6/877: dwrite d6/d58/f66 [0,4194304] 0 2024-02-22T15:50:42.279 INFO:tasks.workunit.client.1.smithi062.stdout:9/498: dwrite d4/d15/d5f/f7b [0,4194304] 0 2024-02-22T15:50:42.280 INFO:tasks.workunit.client.1.smithi062.stdout:3/464: dread f3 [0,4194304] 0 2024-02-22T15:50:42.280 INFO:tasks.workunit.client.0.smithi019.stdout:6/878: creat d6/d4b/d71/dc2/f11c x:0 0 0 2024-02-22T15:50:42.281 INFO:tasks.workunit.client.1.smithi062.stdout:1/509: dread d1/d6/d7/d2b/d49/f6a [0,4194304] 0 2024-02-22T15:50:42.283 INFO:tasks.workunit.client.0.smithi019.stdout:6/879: write d6/d3e/f42 [3103678,41450] 0 2024-02-22T15:50:42.286 INFO:tasks.workunit.client.0.smithi019.stdout:6/880: unlink d6/d31/f36 0 2024-02-22T15:50:42.292 INFO:tasks.workunit.client.0.smithi019.stdout:3/571: dread d9/d22/d27/f3d [0,4194304] 0 2024-02-22T15:50:42.299 INFO:tasks.workunit.client.1.smithi062.stdout:0/612: dwrite d1/d3/de/f3a [0,4194304] 0 2024-02-22T15:50:42.299 INFO:tasks.workunit.client.0.smithi019.stdout:6/881: dwrite d6/d31/d43/f8b [0,4194304] 0 2024-02-22T15:50:42.300 INFO:tasks.workunit.client.1.smithi062.stdout:0/613: readlink d1/d3/d4/d1a/d76/da5/lc5 0 2024-02-22T15:50:42.305 INFO:tasks.workunit.client.1.smithi062.stdout:9/499: write d4/d28/f42 [1108224,16708] 0 2024-02-22T15:50:42.305 INFO:tasks.workunit.client.1.smithi062.stdout:1/510: write d1/d6/d7/d8a/fa5 [673060,61940] 0 2024-02-22T15:50:42.307 INFO:tasks.workunit.client.0.smithi019.stdout:0/685: dread d4/d22/d2c/f9a [0,4194304] 0 2024-02-22T15:50:42.310 INFO:tasks.workunit.client.0.smithi019.stdout:6/882: creat d6/d4b/d6d/da3/d117/f11d x:0 0 0 2024-02-22T15:50:42.310 INFO:tasks.workunit.client.0.smithi019.stdout:1/729: dwrite d8/d18/f27 [0,4194304] 0 2024-02-22T15:50:42.313 INFO:tasks.workunit.client.1.smithi062.stdout:9/500: symlink d4/d1c/d83/l98 0 2024-02-22T15:50:42.313 INFO:tasks.workunit.client.1.smithi062.stdout:1/511: creat d1/d6/d7/d2b/fae x:0 0 0 2024-02-22T15:50:42.313 INFO:tasks.workunit.client.0.smithi019.stdout:0/686: mknod d4/d32/ced 0 2024-02-22T15:50:42.316 INFO:tasks.workunit.client.1.smithi062.stdout:9/501: write d4/d1c/f7e [949153,12843] 0 2024-02-22T15:50:42.316 INFO:tasks.workunit.client.1.smithi062.stdout:1/512: fsync d1/d6/f61 0 2024-02-22T15:50:42.318 INFO:tasks.workunit.client.0.smithi019.stdout:3/572: write d9/d14/f3c [1349354,49684] 0 2024-02-22T15:50:42.322 INFO:tasks.workunit.client.0.smithi019.stdout:3/573: mkdir d9/d22/d27/d96/dba 0 2024-02-22T15:50:42.325 INFO:tasks.workunit.client.0.smithi019.stdout:1/730: dwrite d8/d18/d72/ff6 [0,4194304] 0 2024-02-22T15:50:42.325 INFO:tasks.workunit.client.0.smithi019.stdout:0/687: read d4/db/d1b/d63/db0/da1/dbb/d52/f58 [1063425,73508] 0 2024-02-22T15:50:42.326 INFO:tasks.workunit.client.0.smithi019.stdout:0/688: dread - d4/fdc zero size 2024-02-22T15:50:42.326 INFO:tasks.workunit.client.1.smithi062.stdout:0/614: dwrite d1/d3/d4/d1e/f79 [0,4194304] 0 2024-02-22T15:50:42.328 INFO:tasks.workunit.client.0.smithi019.stdout:1/731: rmdir d8/d15/d34/d9e 39 2024-02-22T15:50:42.328 INFO:tasks.workunit.client.0.smithi019.stdout:0/689: chown d4/db/d1b/d40/d44/d75/db7 56675 1 2024-02-22T15:50:42.329 INFO:tasks.workunit.client.0.smithi019.stdout:3/574: rmdir d9/d11/d71 0 2024-02-22T15:50:42.331 INFO:tasks.workunit.client.0.smithi019.stdout:1/732: read d8/d44/d5d/fc4 [3055883,15332] 0 2024-02-22T15:50:42.335 INFO:tasks.workunit.client.0.smithi019.stdout:3/575: creat d9/d14/d19/fbb x:0 0 0 2024-02-22T15:50:42.340 INFO:tasks.workunit.client.1.smithi062.stdout:3/465: dread d5/f10 [0,4194304] 0 2024-02-22T15:50:42.340 INFO:tasks.workunit.client.0.smithi019.stdout:0/690: write d4/db/d1b/d63/db0/da1/dbb/d52/f58 [3213207,89557] 0 2024-02-22T15:50:42.340 INFO:tasks.workunit.client.0.smithi019.stdout:1/733: dread d8/d18/dd2/ff4 [0,4194304] 0 2024-02-22T15:50:42.340 INFO:tasks.workunit.client.0.smithi019.stdout:3/576: mknod d9/d22/cbc 0 2024-02-22T15:50:42.341 INFO:tasks.workunit.client.0.smithi019.stdout:0/691: chown d4/db/d1b/d40/d8c/c93 159773318 1 2024-02-22T15:50:42.341 INFO:tasks.workunit.client.1.smithi062.stdout:3/466: mkdir d5/d16/d21/d96 0 2024-02-22T15:50:42.341 INFO:tasks.workunit.client.0.smithi019.stdout:1/734: fsync d8/d44/d5d/fd1 0 2024-02-22T15:50:42.342 INFO:tasks.workunit.client.0.smithi019.stdout:1/735: chown d8/d23/c6e 32493020 1 2024-02-22T15:50:42.345 INFO:tasks.workunit.client.0.smithi019.stdout:1/736: rmdir d8/d44 39 2024-02-22T15:50:42.346 INFO:tasks.workunit.client.0.smithi019.stdout:8/650: sync 2024-02-22T15:50:42.346 INFO:tasks.workunit.client.0.smithi019.stdout:9/686: sync 2024-02-22T15:50:42.347 INFO:tasks.workunit.client.0.smithi019.stdout:0/692: getdents d4/d22/d3d 0 2024-02-22T15:50:42.348 INFO:tasks.workunit.client.0.smithi019.stdout:8/651: write d9/d14/d17/f70 [690563,34546] 0 2024-02-22T15:50:42.348 INFO:tasks.workunit.client.0.smithi019.stdout:1/737: fsync d8/d2a/f5e 0 2024-02-22T15:50:42.350 INFO:tasks.workunit.client.0.smithi019.stdout:9/687: mknod d2/d1f/d2e/d57/df0/cf3 0 2024-02-22T15:50:42.352 INFO:tasks.workunit.client.0.smithi019.stdout:8/652: creat d9/d14/d1f/d20/fdb x:0 0 0 2024-02-22T15:50:42.355 INFO:tasks.workunit.client.0.smithi019.stdout:8/653: unlink f5 0 2024-02-22T15:50:42.357 INFO:tasks.workunit.client.1.smithi062.stdout:9/502: dread d4/d35/d3f/f39 [0,4194304] 0 2024-02-22T15:50:42.357 INFO:tasks.workunit.client.0.smithi019.stdout:9/688: creat d2/d7/d52/ff4 x:0 0 0 2024-02-22T15:50:42.360 INFO:tasks.workunit.client.0.smithi019.stdout:0/693: dwrite d4/d22/d3d/fd9 [0,4194304] 0 2024-02-22T15:50:42.363 INFO:tasks.workunit.client.1.smithi062.stdout:3/467: truncate d5/d16/d21/f2b 828154 0 2024-02-22T15:50:42.364 INFO:tasks.workunit.client.0.smithi019.stdout:1/738: dwrite d8/d2a/d75/d81/d8e/fec [0,4194304] 0 2024-02-22T15:50:42.364 INFO:tasks.workunit.client.1.smithi062.stdout:9/503: write d4/f26 [1096866,12604] 0 2024-02-22T15:50:42.365 INFO:tasks.workunit.client.1.smithi062.stdout:1/513: dread d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:42.365 INFO:tasks.workunit.client.1.smithi062.stdout:0/615: dread d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/d97/fb7 [0,4194304] 0 2024-02-22T15:50:42.367 INFO:tasks.workunit.client.0.smithi019.stdout:8/654: rename d9/d14/d17/f70 to d9/d14/d6f/d7e/d94/d9c/fdc 0 2024-02-22T15:50:42.368 INFO:tasks.workunit.client.1.smithi062.stdout:9/504: mknod d4/d28/d50/d5a/c99 0 2024-02-22T15:50:42.368 INFO:tasks.workunit.client.0.smithi019.stdout:1/739: write d8/d2a/d75/def/df8/fa5 [2660624,115706] 0 2024-02-22T15:50:42.370 INFO:tasks.workunit.client.1.smithi062.stdout:9/505: dread - d4/d28/d50/f7a zero size 2024-02-22T15:50:42.372 INFO:tasks.workunit.client.0.smithi019.stdout:0/694: mkdir d4/db/d1b/d63/db0/da1/dbb/d52/dee 0 2024-02-22T15:50:42.374 INFO:tasks.workunit.client.1.smithi062.stdout:3/468: dwrite d5/d16/d21/d36/f95 [0,4194304] 0 2024-02-22T15:50:42.375 INFO:tasks.workunit.client.0.smithi019.stdout:8/655: write d9/d14/d17/d3d/fbb [238553,77580] 0 2024-02-22T15:50:42.375 INFO:tasks.workunit.client.0.smithi019.stdout:0/695: write d4/db/d49/dc2/dc1/fd6 [388560,70368] 0 2024-02-22T15:50:42.377 INFO:tasks.workunit.client.0.smithi019.stdout:0/696: chown d4/db/d24/d80/daf/cc3 0 1 2024-02-22T15:50:42.377 INFO:tasks.workunit.client.1.smithi062.stdout:1/514: symlink d1/d6/d7/d4a/d65/d83/laf 0 2024-02-22T15:50:42.378 INFO:tasks.workunit.client.1.smithi062.stdout:9/506: rename d4/d1c/f7e to d4/d15/d5f/d75/f9a 0 2024-02-22T15:50:42.378 INFO:tasks.workunit.client.0.smithi019.stdout:0/697: fdatasync d4/db/d1b/d63/db0/da1/dbb/d7/fe0 0 2024-02-22T15:50:42.380 INFO:tasks.workunit.client.1.smithi062.stdout:3/469: truncate d5/d16/d21/d36/d5a/d85/f93 300497 0 2024-02-22T15:50:42.380 INFO:tasks.workunit.client.1.smithi062.stdout:9/507: creat d4/d15/d5f/f9b x:0 0 0 2024-02-22T15:50:42.383 INFO:tasks.workunit.client.1.smithi062.stdout:3/470: mkdir d5/d16/d97 0 2024-02-22T15:50:42.383 INFO:tasks.workunit.client.1.smithi062.stdout:9/508: write d4/d15/d5f/d75/f89 [358367,82893] 0 2024-02-22T15:50:42.384 INFO:tasks.workunit.client.0.smithi019.stdout:9/689: link d2/d7/dc/d27/d68/dca/cd1 d2/d7/dc/d27/d68/cf5 0 2024-02-22T15:50:42.385 INFO:tasks.workunit.client.1.smithi062.stdout:3/471: read - d5/d26/d2d/d3f/f7c zero size 2024-02-22T15:50:42.386 INFO:tasks.workunit.client.0.smithi019.stdout:8/656: rmdir d9/d14/d17/d3d/d83 39 2024-02-22T15:50:42.386 INFO:tasks.workunit.client.1.smithi062.stdout:9/509: creat d4/d28/d50/d5a/f9c x:0 0 0 2024-02-22T15:50:42.387 INFO:tasks.workunit.client.0.smithi019.stdout:9/690: dread - d2/d1f/d2e/d48/d36/ff1 zero size 2024-02-22T15:50:42.388 INFO:tasks.workunit.client.0.smithi019.stdout:3/577: mknod d9/d22/d49/cbd 0 2024-02-22T15:50:42.389 INFO:tasks.workunit.client.0.smithi019.stdout:1/740: creat d8/d18/ffb x:0 0 0 2024-02-22T15:50:42.390 INFO:tasks.workunit.client.0.smithi019.stdout:8/657: fsync d9/d14/d17/f85 0 2024-02-22T15:50:42.391 INFO:tasks.workunit.client.0.smithi019.stdout:9/691: mknod d2/d1f/db5/cf6 0 2024-02-22T15:50:42.393 INFO:tasks.workunit.client.0.smithi019.stdout:0/698: creat d4/d67/de5/fef x:0 0 0 2024-02-22T15:50:42.393 INFO:tasks.workunit.client.0.smithi019.stdout:9/692: fsync d2/d1f/d2e/d57/d60/d8c/f6a 0 2024-02-22T15:50:42.394 INFO:tasks.workunit.client.1.smithi062.stdout:0/616: dwrite d1/d13/f65 [0,4194304] 0 2024-02-22T15:50:42.395 INFO:tasks.workunit.client.0.smithi019.stdout:3/578: write d9/d22/d27/fb0 [221121,73917] 0 2024-02-22T15:50:42.397 INFO:tasks.workunit.client.0.smithi019.stdout:3/579: chown d9/d22/d49/cbd 166978271 1 2024-02-22T15:50:42.398 INFO:tasks.workunit.client.0.smithi019.stdout:1/741: creat d8/d18/d64/d7f/ffc x:0 0 0 2024-02-22T15:50:42.399 INFO:tasks.workunit.client.0.smithi019.stdout:0/699: mknod d4/db/d24/d80/daf/cf0 0 2024-02-22T15:50:42.400 INFO:tasks.workunit.client.1.smithi062.stdout:9/510: dwrite d4/d35/f36 [4194304,4194304] 0 2024-02-22T15:50:42.400 INFO:tasks.workunit.client.0.smithi019.stdout:3/580: readlink d9/d26/d3f/d59/d66/l6f 0 2024-02-22T15:50:42.401 INFO:tasks.workunit.client.1.smithi062.stdout:0/617: mkdir d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8d/dce 0 2024-02-22T15:50:42.402 INFO:tasks.workunit.client.1.smithi062.stdout:9/511: write d4/d1c/f2c [4527336,20832] 0 2024-02-22T15:50:42.405 INFO:tasks.workunit.client.1.smithi062.stdout:3/472: truncate d5/d16/d21/f29 1235659 0 2024-02-22T15:50:42.405 INFO:tasks.workunit.client.1.smithi062.stdout:9/512: mknod d4/d28/c9d 0 2024-02-22T15:50:42.406 INFO:tasks.workunit.client.1.smithi062.stdout:0/618: mknod d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8d/ccf 0 2024-02-22T15:50:42.407 INFO:tasks.workunit.client.0.smithi019.stdout:3/581: fsync d9/d26/d2d/f4b 0 2024-02-22T15:50:42.410 INFO:tasks.workunit.client.0.smithi019.stdout:0/700: write d4/db/d1b/d40/d44/d75/f82 [662166,63719] 0 2024-02-22T15:50:42.413 INFO:tasks.workunit.client.1.smithi062.stdout:0/619: write d1/d3/d4/d1a/d76/d8c/d5d/dc3/f49 [4054113,21322] 0 2024-02-22T15:50:42.413 INFO:tasks.workunit.client.0.smithi019.stdout:9/693: creat d2/d7/d52/ff7 x:0 0 0 2024-02-22T15:50:42.415 INFO:tasks.workunit.client.1.smithi062.stdout:9/513: dread d4/d15/d31/f60 [0,4194304] 0 2024-02-22T15:50:42.417 INFO:tasks.workunit.client.0.smithi019.stdout:9/694: symlink d2/d1f/d2e/d57/lf8 0 2024-02-22T15:50:42.417 INFO:tasks.workunit.client.1.smithi062.stdout:9/514: chown d4/d1c/f1e 0 1 2024-02-22T15:50:42.418 INFO:tasks.workunit.client.0.smithi019.stdout:1/742: getdents d8/d18/d64/d7f/d94 0 2024-02-22T15:50:42.418 INFO:tasks.workunit.client.0.smithi019.stdout:3/582: dread d9/d22/f50 [0,4194304] 0 2024-02-22T15:50:42.418 INFO:tasks.workunit.client.1.smithi062.stdout:3/473: dwrite f3 [0,4194304] 0 2024-02-22T15:50:42.420 INFO:tasks.workunit.client.0.smithi019.stdout:0/701: truncate d4/db/d1b/f1c 3614073 0 2024-02-22T15:50:42.421 INFO:tasks.workunit.client.0.smithi019.stdout:3/583: mknod d9/d26/d32/cbe 0 2024-02-22T15:50:42.423 INFO:tasks.workunit.client.0.smithi019.stdout:9/695: symlink d2/d1f/d2e/d57/lf9 0 2024-02-22T15:50:42.423 INFO:tasks.workunit.client.1.smithi062.stdout:9/515: mknod d4/d28/d50/d5a/c9e 0 2024-02-22T15:50:42.424 INFO:tasks.workunit.client.1.smithi062.stdout:0/620: creat d1/d3/d4/d1a/d76/d8c/d2c/d87/fd0 x:0 0 0 2024-02-22T15:50:42.425 INFO:tasks.workunit.client.1.smithi062.stdout:3/474: symlink d5/d1a/d3b/l98 0 2024-02-22T15:50:42.428 INFO:tasks.workunit.client.1.smithi062.stdout:9/516: dread d4/d28/d50/f52 [0,4194304] 0 2024-02-22T15:50:42.430 INFO:tasks.workunit.client.1.smithi062.stdout:0/621: creat d1/d3/d4/d1a/d1d/d3b/fd1 x:0 0 0 2024-02-22T15:50:42.430 INFO:tasks.workunit.client.1.smithi062.stdout:9/517: write d4/d15/d5f/d75/f65 [265477,33479] 0 2024-02-22T15:50:42.432 INFO:tasks.workunit.client.1.smithi062.stdout:3/475: dread d5/d1a/f1e [0,4194304] 0 2024-02-22T15:50:42.434 INFO:tasks.workunit.client.0.smithi019.stdout:3/584: creat d9/d26/d32/fbf x:0 0 0 2024-02-22T15:50:42.434 INFO:tasks.workunit.client.0.smithi019.stdout:8/658: dwrite d9/d14/d17/d1c/f9d [0,4194304] 0 2024-02-22T15:50:42.434 INFO:tasks.workunit.client.0.smithi019.stdout:0/702: rmdir d4/db/d1b/d63/db0/da1/dbb/da3 39 2024-02-22T15:50:42.440 INFO:tasks.workunit.client.1.smithi062.stdout:3/476: dwrite f3 [0,4194304] 0 2024-02-22T15:50:42.442 INFO:tasks.workunit.client.1.smithi062.stdout:3/477: fsync d5/d26/f27 0 2024-02-22T15:50:42.443 INFO:tasks.workunit.client.1.smithi062.stdout:9/518: write d4/d15/d31/f94 [4646932,110779] 0 2024-02-22T15:50:42.446 INFO:tasks.workunit.client.0.smithi019.stdout:1/743: dwrite d8/d18/d64/fad [0,4194304] 0 2024-02-22T15:50:42.448 INFO:tasks.workunit.client.1.smithi062.stdout:9/519: dread d4/d15/d31/f60 [0,4194304] 0 2024-02-22T15:50:42.450 INFO:tasks.workunit.client.1.smithi062.stdout:3/478: dread d5/f6 [0,4194304] 0 2024-02-22T15:50:42.453 INFO:tasks.workunit.client.1.smithi062.stdout:3/479: symlink d5/d16/d21/d32/d6a/l99 0 2024-02-22T15:50:42.454 INFO:tasks.workunit.client.0.smithi019.stdout:8/659: symlink d9/d14/d1f/d20/ldd 0 2024-02-22T15:50:42.456 INFO:tasks.workunit.client.0.smithi019.stdout:1/744: dwrite d8/f88 [0,4194304] 0 2024-02-22T15:50:42.462 INFO:tasks.workunit.client.0.smithi019.stdout:4/679: sync 2024-02-22T15:50:42.463 INFO:tasks.workunit.client.0.smithi019.stdout:5/710: sync 2024-02-22T15:50:42.463 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:42 smithi062 ceph-mon[48130]: Reconfiguring node-exporter.smithi062 (dependencies changed)... 2024-02-22T15:50:42.463 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:42 smithi062 ceph-mon[48130]: Reconfiguring daemon node-exporter.smithi062 on smithi062 2024-02-22T15:50:42.463 INFO:tasks.workunit.client.1.smithi062.stdout:3/480: getdents d5/d16/d21/d32 0 2024-02-22T15:50:42.464 INFO:tasks.workunit.client.1.smithi062.stdout:3/481: fsync d5/d26/f6c 0 2024-02-22T15:50:42.464 INFO:tasks.workunit.client.1.smithi062.stdout:0/622: dread d1/d3/d4/d1a/d76/d8c/d2c/d84/f26 [0,4194304] 0 2024-02-22T15:50:42.464 INFO:tasks.workunit.client.0.smithi019.stdout:5/711: stat d8/dd/d19/d3a/d46/f61 0 2024-02-22T15:50:42.465 INFO:tasks.workunit.client.1.smithi062.stdout:3/482: creat d5/d16/d21/d84/f9a x:0 0 0 2024-02-22T15:50:42.467 INFO:tasks.workunit.client.1.smithi062.stdout:7/426: sync 2024-02-22T15:50:42.468 INFO:tasks.workunit.client.1.smithi062.stdout:5/418: sync 2024-02-22T15:50:42.470 INFO:tasks.workunit.client.1.smithi062.stdout:3/483: creat d5/d16/d21/d36/d5a/d85/f9b x:0 0 0 2024-02-22T15:50:42.477 INFO:tasks.workunit.client.1.smithi062.stdout:5/419: getdents d0/d3/da/d32 0 2024-02-22T15:50:42.478 INFO:tasks.workunit.client.1.smithi062.stdout:3/484: write d5/d16/d56/f7e [2087369,53369] 0 2024-02-22T15:50:42.481 INFO:tasks.workunit.client.1.smithi062.stdout:5/420: readlink d0/d3/d2f/d59/d67/d69/l84 0 2024-02-22T15:50:42.482 INFO:tasks.workunit.client.1.smithi062.stdout:5/421: stat d0/d3/f14 0 2024-02-22T15:50:42.490 INFO:tasks.workunit.client.1.smithi062.stdout:3/485: symlink d5/d1a/d3c/l9c 0 2024-02-22T15:50:42.491 INFO:tasks.workunit.client.1.smithi062.stdout:9/520: dread d4/f1b [0,4194304] 0 2024-02-22T15:50:42.493 INFO:tasks.workunit.client.0.smithi019.stdout:8/660: dread d9/d14/d17/f47 [4194304,4194304] 0 2024-02-22T15:50:42.494 INFO:tasks.workunit.client.1.smithi062.stdout:3/486: truncate d5/d16/d21/d32/f4c 1196135 0 2024-02-22T15:50:42.494 INFO:tasks.workunit.client.0.smithi019.stdout:9/696: symlink d2/d7/dee/lfa 0 2024-02-22T15:50:42.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:42 smithi019 ceph-mon[67418]: Reconfiguring node-exporter.smithi062 (dependencies changed)... 2024-02-22T15:50:42.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:42 smithi019 ceph-mon[67418]: Reconfiguring daemon node-exporter.smithi062 on smithi062 2024-02-22T15:50:42.495 INFO:tasks.workunit.client.0.smithi019.stdout:9/697: stat d2/f19 0 2024-02-22T15:50:42.498 INFO:tasks.workunit.client.0.smithi019.stdout:3/585: symlink d9/d11/lc0 0 2024-02-22T15:50:42.498 INFO:tasks.workunit.client.1.smithi062.stdout:0/623: dread d1/d38/f7b [0,4194304] 0 2024-02-22T15:50:42.499 INFO:tasks.workunit.client.1.smithi062.stdout:3/487: mknod d5/d1a/d3c/d62/c9d 0 2024-02-22T15:50:42.500 INFO:tasks.workunit.client.0.smithi019.stdout:0/703: rename d4/db/d24/cb9 to d4/d23/d7d/cf1 0 2024-02-22T15:50:42.502 INFO:tasks.workunit.client.0.smithi019.stdout:1/745: fdatasync d8/d15/fba 0 2024-02-22T15:50:42.504 INFO:tasks.workunit.client.0.smithi019.stdout:4/680: creat d2/db/d14/d23/d21/dbf/fdd x:0 0 0 2024-02-22T15:50:42.506 INFO:tasks.workunit.client.1.smithi062.stdout:5/422: dwrite d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:42.510 INFO:tasks.workunit.client.1.smithi062.stdout:9/521: dwrite d4/d28/f72 [0,4194304] 0 2024-02-22T15:50:42.515 INFO:tasks.workunit.client.1.smithi062.stdout:0/624: creat d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/fd2 x:0 0 0 2024-02-22T15:50:42.516 INFO:tasks.workunit.client.0.smithi019.stdout:4/681: dwrite d2/db/d14/d23/d4b/f82 [0,4194304] 0 2024-02-22T15:50:42.519 INFO:tasks.workunit.client.0.smithi019.stdout:0/704: dread d4/db/d1b/d63/db0/da1/dbb/d52/f58 [0,4194304] 0 2024-02-22T15:50:42.521 INFO:tasks.workunit.client.1.smithi062.stdout:0/625: write d1/d3/d4/d1e/fb2 [1572057,48108] 0 2024-02-22T15:50:42.527 INFO:tasks.workunit.client.1.smithi062.stdout:9/522: readlink d4/d15/l1a 0 2024-02-22T15:50:42.527 INFO:tasks.workunit.client.1.smithi062.stdout:5/423: dwrite d0/d3/da/d13/d23/d4e/f5d [0,4194304] 0 2024-02-22T15:50:42.531 INFO:tasks.workunit.client.0.smithi019.stdout:0/705: mknod d4/d67/de5/cf2 0 2024-02-22T15:50:42.531 INFO:tasks.workunit.client.0.smithi019.stdout:9/698: creat d2/d1f/d2e/d48/d36/ffb x:0 0 0 2024-02-22T15:50:42.533 INFO:tasks.workunit.client.0.smithi019.stdout:1/746: link d8/fb4 d8/d44/ffd 0 2024-02-22T15:50:42.535 INFO:tasks.workunit.client.1.smithi062.stdout:9/523: rename d4/d15/d5f/d75/c74 to d4/d15/d5f/d75/d97/c9f 0 2024-02-22T15:50:42.535 INFO:tasks.workunit.client.1.smithi062.stdout:9/524: rmdir d4/d15/d5f/d75/d84 39 2024-02-22T15:50:42.540 INFO:tasks.workunit.client.0.smithi019.stdout:9/699: dwrite d2/d1f/d2e/d57/d60/d8c/fc3 [0,4194304] 0 2024-02-22T15:50:42.558 INFO:tasks.workunit.client.1.smithi062.stdout:3/488: truncate d5/d16/d21/d32/f4c 1709019 0 2024-02-22T15:50:42.562 INFO:tasks.workunit.client.1.smithi062.stdout:3/489: mkdir d5/d16/d21/d32/d9e 0 2024-02-22T15:50:42.564 INFO:tasks.workunit.client.1.smithi062.stdout:5/424: dwrite d0/d3/da/d13/d23/d31/f34 [0,4194304] 0 2024-02-22T15:50:42.569 INFO:tasks.workunit.client.1.smithi062.stdout:3/490: symlink d5/d16/d21/d32/d6a/d7b/l9f 0 2024-02-22T15:50:42.575 INFO:tasks.workunit.client.1.smithi062.stdout:5/425: dread d0/d3/d30/f4a [0,4194304] 0 2024-02-22T15:50:42.579 INFO:tasks.workunit.client.1.smithi062.stdout:3/491: write d5/d16/f19 [672079,60525] 0 2024-02-22T15:50:42.579 INFO:tasks.workunit.client.1.smithi062.stdout:5/426: write d0/d3/da/d13/d23/d4e/d60/f63 [147463,98992] 0 2024-02-22T15:50:42.581 INFO:tasks.workunit.client.1.smithi062.stdout:3/492: creat d5/d16/d21/d55/fa0 x:0 0 0 2024-02-22T15:50:42.583 INFO:tasks.workunit.client.1.smithi062.stdout:3/493: creat d5/d26/d2d/fa1 x:0 0 0 2024-02-22T15:50:42.589 INFO:tasks.workunit.client.1.smithi062.stdout:6/587: sync 2024-02-22T15:50:42.589 INFO:tasks.workunit.client.1.smithi062.stdout:8/565: sync 2024-02-22T15:50:42.589 INFO:tasks.workunit.client.1.smithi062.stdout:4/504: sync 2024-02-22T15:50:42.589 INFO:tasks.workunit.client.1.smithi062.stdout:1/515: sync 2024-02-22T15:50:42.590 INFO:tasks.workunit.client.1.smithi062.stdout:2/612: sync 2024-02-22T15:50:42.593 INFO:tasks.workunit.client.1.smithi062.stdout:4/505: creat d4/d2a/d2d/d46/fad x:0 0 0 2024-02-22T15:50:42.595 INFO:tasks.workunit.client.1.smithi062.stdout:8/566: unlink d8/d26/c50 0 2024-02-22T15:50:42.599 INFO:tasks.workunit.client.1.smithi062.stdout:5/427: dread d0/d3/da/d13/d23/d31/f43 [4194304,4194304] 0 2024-02-22T15:50:42.602 INFO:tasks.workunit.client.1.smithi062.stdout:4/506: mkdir d4/d2a/d2d/d2e/d6a/d8a/dae 0 2024-02-22T15:50:42.605 INFO:tasks.workunit.client.1.smithi062.stdout:1/516: dwrite d1/d6/d7/d4a/f7f [0,4194304] 0 2024-02-22T15:50:42.606 INFO:tasks.workunit.client.1.smithi062.stdout:6/588: dwrite d5/df/d40/d44/d4a/faa [4194304,4194304] 0 2024-02-22T15:50:42.607 INFO:tasks.workunit.client.1.smithi062.stdout:2/613: truncate d1/f2c 1738932 0 2024-02-22T15:50:42.608 INFO:tasks.workunit.client.1.smithi062.stdout:1/517: readlink d1/l1e 0 2024-02-22T15:50:42.609 INFO:tasks.workunit.client.1.smithi062.stdout:4/507: mkdir d4/df/daf 0 2024-02-22T15:50:42.611 INFO:tasks.workunit.client.1.smithi062.stdout:5/428: write d0/d3/d2f/d59/d67/d69/f41 [1172621,17886] 0 2024-02-22T15:50:42.614 INFO:tasks.workunit.client.1.smithi062.stdout:1/518: dread - d1/d6/d7/d2b/d47/d63/fa6 zero size 2024-02-22T15:50:42.616 INFO:tasks.workunit.client.1.smithi062.stdout:5/429: read - d0/d3/da/d32/d42/f65 zero size 2024-02-22T15:50:42.617 INFO:tasks.workunit.client.0.smithi019.stdout:6/883: sync 2024-02-22T15:50:42.617 INFO:tasks.workunit.client.0.smithi019.stdout:2/711: sync 2024-02-22T15:50:42.617 INFO:tasks.workunit.client.1.smithi062.stdout:4/508: unlink d4/df/l4e 0 2024-02-22T15:50:42.617 INFO:tasks.workunit.client.1.smithi062.stdout:2/614: write d1/d8/d62/d73/dbd/f46 [3859242,43845] 0 2024-02-22T15:50:42.618 INFO:tasks.workunit.client.0.smithi019.stdout:6/884: readlink d6/l53 0 2024-02-22T15:50:42.624 INFO:tasks.workunit.client.1.smithi062.stdout:6/589: mknod d5/df/d40/dac/cbe 0 2024-02-22T15:50:42.624 INFO:tasks.workunit.client.1.smithi062.stdout:1/519: write d1/d6/de/f42 [2910631,72994] 0 2024-02-22T15:50:42.626 INFO:tasks.workunit.client.1.smithi062.stdout:8/567: dread d8/f29 [0,4194304] 0 2024-02-22T15:50:42.631 INFO:tasks.workunit.client.1.smithi062.stdout:1/520: dread d1/f44 [0,4194304] 0 2024-02-22T15:50:42.635 INFO:tasks.workunit.client.1.smithi062.stdout:2/615: fdatasync d1/fb8 0 2024-02-22T15:50:42.636 INFO:tasks.workunit.client.1.smithi062.stdout:4/509: write d4/d1c/f27 [1618988,13491] 0 2024-02-22T15:50:42.639 INFO:tasks.workunit.client.0.smithi019.stdout:6/885: dread d6/d4b/d6d/d7b/d7f/fb8 [0,4194304] 0 2024-02-22T15:50:42.644 INFO:tasks.workunit.client.0.smithi019.stdout:2/712: truncate d1/de/f1e 8916836 0 2024-02-22T15:50:42.645 INFO:tasks.workunit.client.1.smithi062.stdout:2/616: symlink d1/d8/d62/d73/lc6 0 2024-02-22T15:50:42.645 INFO:tasks.workunit.client.0.smithi019.stdout:2/713: dread - d1/d17/d1c/fe7 zero size 2024-02-22T15:50:42.649 INFO:tasks.workunit.client.1.smithi062.stdout:2/617: truncate d1/d28/fc2 743071 0 2024-02-22T15:50:42.649 INFO:tasks.workunit.client.0.smithi019.stdout:2/714: read d1/de/f4e [727389,4594] 0 2024-02-22T15:50:42.650 INFO:tasks.workunit.client.0.smithi019.stdout:2/715: chown d1/d17/d21/d51/fcc 27 1 2024-02-22T15:50:42.651 INFO:tasks.workunit.client.0.smithi019.stdout:2/716: readlink d1/d17/d21/l7a 0 2024-02-22T15:50:42.652 INFO:tasks.workunit.client.1.smithi062.stdout:1/521: unlink d1/d6/d7/d2b/d49/f57 0 2024-02-22T15:50:42.652 INFO:tasks.workunit.client.1.smithi062.stdout:8/568: chown d8/d9/d2c/d49/d5e/f7d 7 1 2024-02-22T15:50:42.653 INFO:tasks.workunit.client.1.smithi062.stdout:8/569: fsync d8/d26/f9e 0 2024-02-22T15:50:42.654 INFO:tasks.workunit.client.0.smithi019.stdout:2/717: write d1/d17/fb1 [1708509,53276] 0 2024-02-22T15:50:42.654 INFO:tasks.workunit.client.1.smithi062.stdout:4/510: link d4/df/f14 d4/d34/fb0 0 2024-02-22T15:50:42.655 INFO:tasks.workunit.client.1.smithi062.stdout:6/590: link d5/dd/d33/f97 d5/df/d40/d44/fbf 0 2024-02-22T15:50:42.662 INFO:tasks.workunit.client.0.smithi019.stdout:8/661: write d9/d14/d17/f45 [1829623,5738] 0 2024-02-22T15:50:42.664 INFO:tasks.workunit.client.0.smithi019.stdout:8/662: write d9/d88/f8b [721603,90003] 0 2024-02-22T15:50:42.664 INFO:tasks.workunit.client.1.smithi062.stdout:5/430: write d0/d3/da/f18 [1279396,70471] 0 2024-02-22T15:50:42.665 INFO:tasks.workunit.client.1.smithi062.stdout:8/570: dwrite d8/f53 [0,4194304] 0 2024-02-22T15:50:42.666 INFO:tasks.workunit.client.0.smithi019.stdout:8/663: chown d9/d14/d17/d3d/c49 5 1 2024-02-22T15:50:42.668 INFO:tasks.workunit.client.1.smithi062.stdout:1/522: dread d1/d6/de/f31 [0,4194304] 0 2024-02-22T15:50:42.672 INFO:tasks.workunit.client.1.smithi062.stdout:4/511: mkdir d4/d1c/d2c/d6b/d9b/db1 0 2024-02-22T15:50:42.673 INFO:tasks.workunit.client.1.smithi062.stdout:1/523: read - d1/d6/d36/d76/fa1 zero size 2024-02-22T15:50:42.673 INFO:tasks.workunit.client.0.smithi019.stdout:4/682: dwrite d2/db/d14/d23/d33/db1/f95 [0,4194304] 0 2024-02-22T15:50:42.676 INFO:tasks.workunit.client.0.smithi019.stdout:4/683: dread d2/db/d14/d23/d21/d42/fa9 [0,4194304] 0 2024-02-22T15:50:42.677 INFO:tasks.workunit.client.1.smithi062.stdout:2/618: rmdir d1/db4 0 2024-02-22T15:50:42.678 INFO:tasks.workunit.client.1.smithi062.stdout:5/431: mknod d0/d3/da/d13/d23/d4e/d60/c87 0 2024-02-22T15:50:42.679 INFO:tasks.workunit.client.0.smithi019.stdout:2/718: dread d1/de/d9a/fd7 [0,4194304] 0 2024-02-22T15:50:42.680 INFO:tasks.workunit.client.1.smithi062.stdout:4/512: read d4/d34/fb0 [67930,62650] 0 2024-02-22T15:50:42.681 INFO:tasks.workunit.client.0.smithi019.stdout:2/719: chown d1/d17/d67/f100 99 1 2024-02-22T15:50:42.681 INFO:tasks.workunit.client.0.smithi019.stdout:4/684: truncate d2/db/f25 613213 0 2024-02-22T15:50:42.683 INFO:tasks.workunit.client.0.smithi019.stdout:2/720: mknod d1/d17/d1c/d66/db9/dd0/c104 0 2024-02-22T15:50:42.687 INFO:tasks.workunit.client.0.smithi019.stdout:9/700: truncate d2/d1f/d2e/d57/d60/d8c/fc3 4155786 0 2024-02-22T15:50:42.687 INFO:tasks.workunit.client.0.smithi019.stdout:9/701: stat d2/l13 0 2024-02-22T15:50:42.687 INFO:tasks.workunit.client.1.smithi062.stdout:4/513: symlink d4/d1c/dac/lb2 0 2024-02-22T15:50:42.687 INFO:tasks.workunit.client.1.smithi062.stdout:1/524: dwrite d1/d6/d7/d2b/d47/d63/f87 [0,4194304] 0 2024-02-22T15:50:42.688 INFO:tasks.workunit.client.0.smithi019.stdout:4/685: link d2/db/d14/d23/d21/c9d d2/db/d14/d23/d21/dbf/dd7/cde 0 2024-02-22T15:50:42.689 INFO:tasks.workunit.client.0.smithi019.stdout:2/721: creat d1/d17/d21/f105 x:0 0 0 2024-02-22T15:50:42.691 INFO:tasks.workunit.client.0.smithi019.stdout:4/686: mknod d2/d3f/d6e/cdf 0 2024-02-22T15:50:42.692 INFO:tasks.workunit.client.1.smithi062.stdout:4/514: dread - d4/d2a/d2d/d2e/f77 zero size 2024-02-22T15:50:42.692 INFO:tasks.workunit.client.1.smithi062.stdout:8/571: readlink d8/d9/d44/d4d/d51/la5 0 2024-02-22T15:50:42.694 INFO:tasks.workunit.client.0.smithi019.stdout:9/702: mknod d2/d7/d52/dc0/dcd/cfc 0 2024-02-22T15:50:42.695 INFO:tasks.workunit.client.1.smithi062.stdout:6/591: dwrite d5/df/d40/f53 [0,4194304] 0 2024-02-22T15:50:42.695 INFO:tasks.workunit.client.1.smithi062.stdout:5/432: rename d0/d3/d2f/d59/d67/d69/f41 to d0/d3/da/d32/d33/f88 0 2024-02-22T15:50:42.696 INFO:tasks.workunit.client.0.smithi019.stdout:9/703: chown d2/d1f/db5/cf6 29481 1 2024-02-22T15:50:42.699 INFO:tasks.workunit.client.0.smithi019.stdout:4/687: dwrite d2/db/d14/d23/d21/dbf/fdd [0,4194304] 0 2024-02-22T15:50:42.701 INFO:tasks.workunit.client.0.smithi019.stdout:4/688: chown d2/d3f/d48/d76/fa8 175120372 1 2024-02-22T15:50:42.701 INFO:tasks.workunit.client.1.smithi062.stdout:2/619: fdatasync d1/d8/d62/d73/dbd/f9f 0 2024-02-22T15:50:42.702 INFO:tasks.workunit.client.1.smithi062.stdout:8/572: mknod d8/d28/d70/cb1 0 2024-02-22T15:50:42.702 INFO:tasks.workunit.client.1.smithi062.stdout:1/525: fdatasync d1/d6/d7/f29 0 2024-02-22T15:50:42.705 INFO:tasks.workunit.client.0.smithi019.stdout:4/689: creat d2/db/d14/d23/d21/d35/d3d/fe0 x:0 0 0 2024-02-22T15:50:42.707 INFO:tasks.workunit.client.0.smithi019.stdout:4/690: chown d2/db/d14/d23/d21/d42/l9a 6181 1 2024-02-22T15:50:42.711 INFO:tasks.workunit.client.1.smithi062.stdout:5/433: creat d0/d3/d2f/d59/d67/d69/d3f/d5f/f89 x:0 0 0 2024-02-22T15:50:42.711 INFO:tasks.workunit.client.1.smithi062.stdout:6/592: chown d5/df/d40/d44/f82 37627 1 2024-02-22T15:50:42.711 INFO:tasks.workunit.client.1.smithi062.stdout:1/526: fdatasync d1/f27 0 2024-02-22T15:50:42.712 INFO:tasks.workunit.client.1.smithi062.stdout:2/620: creat d1/d28/fc7 x:0 0 0 2024-02-22T15:50:42.714 INFO:tasks.workunit.client.0.smithi019.stdout:3/586: rename d9/d22/d27/d6d/f82 to d9/fc1 0 2024-02-22T15:50:42.715 INFO:tasks.workunit.client.1.smithi062.stdout:8/573: fsync d8/d9/f18 0 2024-02-22T15:50:42.716 INFO:tasks.workunit.client.1.smithi062.stdout:1/527: creat d1/d6/d7/d2b/d47/fb0 x:0 0 0 2024-02-22T15:50:42.717 INFO:tasks.workunit.client.0.smithi019.stdout:0/706: sync 2024-02-22T15:50:42.717 INFO:tasks.workunit.client.0.smithi019.stdout:5/712: sync 2024-02-22T15:50:42.718 INFO:tasks.workunit.client.1.smithi062.stdout:8/574: write d8/d9/d2c/d74/fa0 [702671,21099] 0 2024-02-22T15:50:42.718 INFO:tasks.workunit.client.1.smithi062.stdout:6/593: mkdir d5/df/d40/dac/db6/dc0 0 2024-02-22T15:50:42.719 INFO:tasks.workunit.client.0.smithi019.stdout:4/691: dwrite d2/db/d14/d23/d21/f2d [0,4194304] 0 2024-02-22T15:50:42.721 INFO:tasks.workunit.client.1.smithi062.stdout:6/594: dread - d5/df/d40/d44/d4a/d41/f4d zero size 2024-02-22T15:50:42.723 INFO:tasks.workunit.client.0.smithi019.stdout:3/587: symlink d9/d26/d32/lc2 0 2024-02-22T15:50:42.723 INFO:tasks.workunit.client.0.smithi019.stdout:6/886: rename d6/d31/d43/de7/f108 to d6/d31/d43/f11e 0 2024-02-22T15:50:42.724 INFO:tasks.workunit.client.1.smithi062.stdout:6/595: chown d5/df/d40/d44/d4a/d41/d85/d88/c96 938 1 2024-02-22T15:50:42.724 INFO:tasks.workunit.client.1.smithi062.stdout:8/575: dread d8/f53 [0,4194304] 0 2024-02-22T15:50:42.724 INFO:tasks.workunit.client.1.smithi062.stdout:2/621: rename d1/d8/d62/d73/dbd/f59 to d1/d47/d32/d52/dba/fc8 0 2024-02-22T15:50:42.724 INFO:tasks.workunit.client.0.smithi019.stdout:3/588: fsync d9/d26/f35 0 2024-02-22T15:50:42.728 INFO:tasks.workunit.client.0.smithi019.stdout:6/887: readlink d6/d4b/d71/dc2/ld0 0 2024-02-22T15:50:42.728 INFO:tasks.workunit.client.0.smithi019.stdout:6/888: chown d6/d31/d43/f46 5463 1 2024-02-22T15:50:42.728 INFO:tasks.workunit.client.0.smithi019.stdout:8/664: rename d9/d14/f34 to d9/d14/d6f/fde 0 2024-02-22T15:50:42.730 INFO:tasks.workunit.client.1.smithi062.stdout:6/596: mkdir d5/d6e/dc1 0 2024-02-22T15:50:42.730 INFO:tasks.workunit.client.1.smithi062.stdout:1/528: creat d1/d6/d7/d2b/d49/fb1 x:0 0 0 2024-02-22T15:50:42.732 INFO:tasks.workunit.client.1.smithi062.stdout:6/597: mknod d5/dd/d33/d66/cc2 0 2024-02-22T15:50:42.732 INFO:tasks.workunit.client.0.smithi019.stdout:2/722: rename d1/d17/d1c/d7f/dcd to d1/d17/d21/d71/dd1/d106 0 2024-02-22T15:50:42.733 INFO:tasks.workunit.client.1.smithi062.stdout:6/598: chown d5/df/d32 896541 1 2024-02-22T15:50:42.734 INFO:tasks.workunit.client.0.smithi019.stdout:5/713: mknod d8/dd/d19/d38/d47/df3/cf4 0 2024-02-22T15:50:42.734 INFO:tasks.workunit.client.1.smithi062.stdout:4/515: write d4/d34/fb0 [7488521,26281] 0 2024-02-22T15:50:42.734 INFO:tasks.workunit.client.1.smithi062.stdout:8/576: link d8/d9/d44/d4d/d51/d55/f77 d8/d9/d44/fb2 0 2024-02-22T15:50:42.734 INFO:tasks.workunit.client.0.smithi019.stdout:2/723: fdatasync d1/d86/f9e 0 2024-02-22T15:50:42.737 INFO:tasks.workunit.client.0.smithi019.stdout:6/889: dwrite d6/d31/d69/ffa [0,4194304] 0 2024-02-22T15:50:42.738 INFO:tasks.workunit.client.0.smithi019.stdout:2/724: chown d1/def/cb6 38118822 1 2024-02-22T15:50:42.739 INFO:tasks.workunit.client.1.smithi062.stdout:4/516: mknod d4/d1c/d30/d3b/d49/cb3 0 2024-02-22T15:50:42.741 INFO:tasks.workunit.client.1.smithi062.stdout:8/577: truncate d8/d9/d44/d4d/f9f 231743 0 2024-02-22T15:50:42.742 INFO:tasks.workunit.client.1.smithi062.stdout:8/578: fsync d8/f20 0 2024-02-22T15:50:42.742 INFO:tasks.workunit.client.1.smithi062.stdout:6/599: rename d5/df/d40/d44/d76/d56/c5d to d5/df/d40/dac/db6/dc0/cc3 0 2024-02-22T15:50:42.744 INFO:tasks.workunit.client.1.smithi062.stdout:4/517: rmdir d4/d2a/d2d/d2e 39 2024-02-22T15:50:42.747 INFO:tasks.workunit.client.1.smithi062.stdout:5/434: truncate d0/f26 1920268 0 2024-02-22T15:50:42.747 INFO:tasks.workunit.client.0.smithi019.stdout:9/704: dwrite d2/d1f/d2e/d48/d36/f71 [4194304,4194304] 0 2024-02-22T15:50:42.748 INFO:tasks.workunit.client.1.smithi062.stdout:5/435: readlink d0/l25 0 2024-02-22T15:50:42.748 INFO:tasks.workunit.client.0.smithi019.stdout:0/707: creat d4/db/d1b/d40/ff3 x:0 0 0 2024-02-22T15:50:42.750 INFO:tasks.workunit.client.1.smithi062.stdout:5/436: write d0/d3/d2f/f44 [4979072,75939] 0 2024-02-22T15:50:42.752 INFO:tasks.workunit.client.0.smithi019.stdout:4/692: write d2/db/f17 [5522493,82893] 0 2024-02-22T15:50:42.754 INFO:tasks.workunit.client.0.smithi019.stdout:4/693: chown d2/c4 2623 1 2024-02-22T15:50:42.756 INFO:tasks.workunit.client.1.smithi062.stdout:8/579: rename d8/d9/d48/f57 to d8/d9/d2c/d49/d94/fb3 0 2024-02-22T15:50:42.757 INFO:tasks.workunit.client.0.smithi019.stdout:9/705: dread d2/d7/d52/faa [0,4194304] 0 2024-02-22T15:50:42.757 INFO:tasks.workunit.client.0.smithi019.stdout:0/708: chown d4/db/d1b/d63/db0/da1/dbb/d52/c5a 1 1 2024-02-22T15:50:42.759 INFO:tasks.workunit.client.0.smithi019.stdout:5/714: rename d8/d11/l62 to d8/dd/lf5 0 2024-02-22T15:50:42.759 INFO:tasks.workunit.client.1.smithi062.stdout:1/529: truncate d1/d6/d7/d2b/d49/f51 172910 0 2024-02-22T15:50:42.760 INFO:tasks.workunit.client.0.smithi019.stdout:3/589: truncate d9/d26/d32/f9c 2806495 0 2024-02-22T15:50:42.761 INFO:tasks.workunit.client.1.smithi062.stdout:8/580: creat d8/d26/fb4 x:0 0 0 2024-02-22T15:50:42.762 INFO:tasks.workunit.client.0.smithi019.stdout:9/706: mknod d2/d1f/cfd 0 2024-02-22T15:50:42.765 INFO:tasks.workunit.client.1.smithi062.stdout:1/530: mkdir d1/d6/d7/d2b/d47/d63/db2 0 2024-02-22T15:50:42.765 INFO:tasks.workunit.client.0.smithi019.stdout:4/694: creat d2/db/d14/d23/d9e/dac/dc2/dc9/fe1 x:0 0 0 2024-02-22T15:50:42.767 INFO:tasks.workunit.client.0.smithi019.stdout:5/715: fsync d8/dd/d8e/f22 0 2024-02-22T15:50:42.769 INFO:tasks.workunit.client.1.smithi062.stdout:5/437: link d0/d3/d2f/d59/d67/d69/l4c d0/d3/da/d13/d23/d78/d81/l8a 0 2024-02-22T15:50:42.772 INFO:tasks.workunit.client.0.smithi019.stdout:6/890: rename d6/d4b/d6d/d7b/d78/d100 to d6/d4b/d6d/d11f 0 2024-02-22T15:50:42.773 INFO:tasks.workunit.client.1.smithi062.stdout:6/600: truncate d5/df/d40/d44/d78/f31 1398209 0 2024-02-22T15:50:42.774 INFO:tasks.workunit.client.0.smithi019.stdout:0/709: dwrite d4/db/f43 [0,4194304] 0 2024-02-22T15:50:42.774 INFO:tasks.workunit.client.0.smithi019.stdout:3/590: mknod d9/d22/d27/d96/dba/cc3 0 2024-02-22T15:50:42.775 INFO:tasks.workunit.client.1.smithi062.stdout:2/622: truncate d1/d8/d62/d73/dbd/d7d/faa 97062 0 2024-02-22T15:50:42.777 INFO:tasks.workunit.client.1.smithi062.stdout:6/601: chown d5/lc 0 1 2024-02-22T15:50:42.777 INFO:tasks.workunit.client.1.smithi062.stdout:4/518: dread f2 [0,4194304] 0 2024-02-22T15:50:42.778 INFO:tasks.workunit.client.1.smithi062.stdout:1/531: dwrite d1/d6/d7/f28 [0,4194304] 0 2024-02-22T15:50:42.778 INFO:tasks.workunit.client.1.smithi062.stdout:5/438: mknod d0/d3/d2f/d59/d67/d69/d3f/c8b 0 2024-02-22T15:50:42.778 INFO:tasks.workunit.client.0.smithi019.stdout:8/665: dwrite d9/f32 [4194304,4194304] 0 2024-02-22T15:50:42.780 INFO:tasks.workunit.client.0.smithi019.stdout:4/695: write d2/db/d14/d23/d67/f68 [1330010,52701] 0 2024-02-22T15:50:42.784 INFO:tasks.workunit.client.0.smithi019.stdout:2/725: dread d1/d17/d21/d51/fcc [0,4194304] 0 2024-02-22T15:50:42.784 INFO:tasks.workunit.client.0.smithi019.stdout:9/707: chown d2/d1f/d2e/d48/d45/d5a/f72 0 1 2024-02-22T15:50:42.786 INFO:tasks.workunit.client.1.smithi062.stdout:8/581: dwrite d8/f3a [0,4194304] 0 2024-02-22T15:50:42.786 INFO:tasks.workunit.client.1.smithi062.stdout:8/582: chown d8/d26/f9e 15256990 1 2024-02-22T15:50:42.786 INFO:tasks.workunit.client.1.smithi062.stdout:6/602: rmdir d5/dd/d33/d66 39 2024-02-22T15:50:42.788 INFO:tasks.workunit.client.1.smithi062.stdout:2/623: creat d1/d1e/d38/d55/fc9 x:0 0 0 2024-02-22T15:50:42.789 INFO:tasks.workunit.client.1.smithi062.stdout:4/519: mkdir d4/d2a/d2d/d2e/d6a/d9e/db4 0 2024-02-22T15:50:42.791 INFO:tasks.workunit.client.1.smithi062.stdout:8/583: mkdir d8/d26/db5 0 2024-02-22T15:50:42.792 INFO:tasks.workunit.client.0.smithi019.stdout:3/591: chown d9/d14/d19/fbb 386565721 1 2024-02-22T15:50:42.792 INFO:tasks.workunit.client.0.smithi019.stdout:4/696: stat d2/db/d14/d23/d33/la0 0 2024-02-22T15:50:42.795 INFO:tasks.workunit.client.1.smithi062.stdout:8/584: mkdir d8/d86/db6 0 2024-02-22T15:50:42.797 INFO:tasks.workunit.client.0.smithi019.stdout:2/726: creat d1/d17/d4b/f107 x:0 0 0 2024-02-22T15:50:42.798 INFO:tasks.workunit.client.0.smithi019.stdout:5/716: dwrite d8/dd/f5b [0,4194304] 0 2024-02-22T15:50:42.798 INFO:tasks.workunit.client.1.smithi062.stdout:5/439: getdents d0/d3/da/d32/d4f 0 2024-02-22T15:50:42.798 INFO:tasks.workunit.client.1.smithi062.stdout:6/603: mknod d5/df/d40/d44/d76/dae/cc4 0 2024-02-22T15:50:42.800 INFO:tasks.workunit.client.1.smithi062.stdout:8/585: write d8/d28/d4e/faa [919414,115799] 0 2024-02-22T15:50:42.801 INFO:tasks.workunit.client.0.smithi019.stdout:5/717: truncate d8/d11/fb8 457301 0 2024-02-22T15:50:42.801 INFO:tasks.workunit.client.0.smithi019.stdout:9/708: symlink d2/d7/dc/d27/dbc/lfe 0 2024-02-22T15:50:42.802 INFO:tasks.workunit.client.1.smithi062.stdout:2/624: dwrite d1/d8/df/f31 [0,4194304] 0 2024-02-22T15:50:42.805 INFO:tasks.workunit.client.1.smithi062.stdout:2/625: chown d1/d47/d39/fac 161643107 1 2024-02-22T15:50:42.808 INFO:tasks.workunit.client.0.smithi019.stdout:5/718: truncate d8/d11/f87 1104724 0 2024-02-22T15:50:42.808 INFO:tasks.workunit.client.1.smithi062.stdout:8/586: creat d8/d42/d8f/fb7 x:0 0 0 2024-02-22T15:50:42.809 INFO:tasks.workunit.client.1.smithi062.stdout:2/626: readlink d1/d8/d62/d73/dbd/d7d/l87 0 2024-02-22T15:50:42.809 INFO:tasks.workunit.client.0.smithi019.stdout:3/592: getdents d9/d26/d2d 0 2024-02-22T15:50:42.812 INFO:tasks.workunit.client.0.smithi019.stdout:0/710: write d4/db/d1b/f1f [4092931,116015] 0 2024-02-22T15:50:42.812 INFO:tasks.workunit.client.0.smithi019.stdout:4/697: getdents d2/db/d14/d23/d21 0 2024-02-22T15:50:42.813 INFO:tasks.workunit.client.1.smithi062.stdout:5/440: getdents d0/d3/da/d13/d23/d4e/d60 0 2024-02-22T15:50:42.814 INFO:tasks.workunit.client.0.smithi019.stdout:2/727: dwrite d1/def/fd6 [0,4194304] 0 2024-02-22T15:50:42.814 INFO:tasks.workunit.client.1.smithi062.stdout:6/604: dread d5/df/d40/d44/f7c [0,4194304] 0 2024-02-22T15:50:42.815 INFO:tasks.workunit.client.0.smithi019.stdout:1/747: sync 2024-02-22T15:50:42.817 INFO:tasks.workunit.client.1.smithi062.stdout:8/587: creat d8/d9/d2c/d49/fb8 x:0 0 0 2024-02-22T15:50:42.817 INFO:tasks.workunit.client.1.smithi062.stdout:5/441: chown d0/d3/da/d13/c76 15 1 2024-02-22T15:50:42.817 INFO:tasks.workunit.client.1.smithi062.stdout:2/627: mknod d1/d8/d62/d73/dbd/dbb/cca 0 2024-02-22T15:50:42.818 INFO:tasks.workunit.client.1.smithi062.stdout:6/605: readlink d5/df/d40/d44/d76/l25 0 2024-02-22T15:50:42.819 INFO:tasks.workunit.client.1.smithi062.stdout:8/588: readlink d8/l81 0 2024-02-22T15:50:42.821 INFO:tasks.workunit.client.0.smithi019.stdout:0/711: write d4/db/d1b/d40/d8c/fd0 [186142,123466] 0 2024-02-22T15:50:42.821 INFO:tasks.workunit.client.1.smithi062.stdout:6/606: creat d5/df/d32/fc5 x:0 0 0 2024-02-22T15:50:42.822 INFO:tasks.workunit.client.0.smithi019.stdout:3/593: symlink d9/d26/d76/lc4 0 2024-02-22T15:50:42.822 INFO:tasks.workunit.client.1.smithi062.stdout:8/589: dread - d8/d42/d8f/f76 zero size 2024-02-22T15:50:42.822 INFO:tasks.workunit.client.1.smithi062.stdout:2/628: mkdir d1/d8/d62/dcb 0 2024-02-22T15:50:42.823 INFO:tasks.workunit.client.0.smithi019.stdout:1/748: mknod d8/d18/d72/cfe 0 2024-02-22T15:50:42.823 INFO:tasks.workunit.client.0.smithi019.stdout:5/719: chown d8/dd/d19/d3a/d46/c88 0 1 2024-02-22T15:50:42.824 INFO:tasks.workunit.client.1.smithi062.stdout:5/442: mkdir d0/d3/da/d13/d23/d78/d8c 0 2024-02-22T15:50:42.825 INFO:tasks.workunit.client.1.smithi062.stdout:6/607: mkdir d5/df/d40/d44/d4a/d41/d43/dc6 0 2024-02-22T15:50:42.828 INFO:tasks.workunit.client.1.smithi062.stdout:8/590: creat d8/d26/fb9 x:0 0 0 2024-02-22T15:50:42.829 INFO:tasks.workunit.client.0.smithi019.stdout:9/709: dwrite d2/d1f/d2e/d57/d60/f92 [0,4194304] 0 2024-02-22T15:50:42.830 INFO:tasks.workunit.client.1.smithi062.stdout:2/629: creat d1/d8/d62/d73/dbd/d7d/d84/fcc x:0 0 0 2024-02-22T15:50:42.830 INFO:tasks.workunit.client.0.smithi019.stdout:4/698: creat d2/db/d54/d8c/dbb/fe2 x:0 0 0 2024-02-22T15:50:42.832 INFO:tasks.workunit.client.1.smithi062.stdout:5/443: dread - d0/d3/d2f/d59/d67/d69/d3f/d5f/f89 zero size 2024-02-22T15:50:42.832 INFO:tasks.workunit.client.1.smithi062.stdout:6/608: rmdir d5/df/d40/d44/d4a/d41/d85 39 2024-02-22T15:50:42.832 INFO:tasks.workunit.client.1.smithi062.stdout:4/520: dread d4/f3f [0,4194304] 0 2024-02-22T15:50:42.835 INFO:tasks.workunit.client.0.smithi019.stdout:3/594: mknod d9/d26/d32/cc5 0 2024-02-22T15:50:42.835 INFO:tasks.workunit.client.0.smithi019.stdout:2/728: rename d1/d17/d4b/l73 to d1/d17/d1c/d66/db9/dd0/dee/l108 0 2024-02-22T15:50:42.837 INFO:tasks.workunit.client.0.smithi019.stdout:3/595: readlink d9/d26/l28 0 2024-02-22T15:50:42.838 INFO:tasks.workunit.client.1.smithi062.stdout:8/591: unlink d8/d42/d8f/fb7 0 2024-02-22T15:50:42.840 INFO:tasks.workunit.client.0.smithi019.stdout:1/749: link d8/d18/d64/f95 d8/d15/da4/fff 0 2024-02-22T15:50:42.840 INFO:tasks.workunit.client.0.smithi019.stdout:6/891: dread d6/d4b/d6d/da3/ddb/f10b [0,4194304] 0 2024-02-22T15:50:42.841 INFO:tasks.workunit.client.1.smithi062.stdout:6/609: getdents d5/d6e/dc1 0 2024-02-22T15:50:42.844 INFO:tasks.workunit.client.0.smithi019.stdout:0/712: creat d4/db/d1b/d63/db0/da1/dbb/d52/dee/ff4 x:0 0 0 2024-02-22T15:50:42.844 INFO:tasks.workunit.client.0.smithi019.stdout:9/710: creat d2/dec/d79/fff x:0 0 0 2024-02-22T15:50:42.844 INFO:tasks.workunit.client.1.smithi062.stdout:4/521: dread d4/d1c/d30/f32 [0,4194304] 0 2024-02-22T15:50:42.845 INFO:tasks.workunit.client.0.smithi019.stdout:5/720: write d8/d11/f93 [3483822,125632] 0 2024-02-22T15:50:42.845 INFO:tasks.workunit.client.0.smithi019.stdout:2/729: fdatasync d1/d17/d21/d51/fa1 0 2024-02-22T15:50:42.846 INFO:tasks.workunit.client.1.smithi062.stdout:2/630: dwrite d1/d1e/f9a [0,4194304] 0 2024-02-22T15:50:42.848 INFO:tasks.workunit.client.0.smithi019.stdout:8/666: dread d9/d14/d1f/d6d/f8f [0,4194304] 0 2024-02-22T15:50:42.848 INFO:tasks.workunit.client.0.smithi019.stdout:6/892: mkdir d6/d31/d43/d120 0 2024-02-22T15:50:42.851 INFO:tasks.workunit.client.0.smithi019.stdout:3/596: mknod d9/d11/cc6 0 2024-02-22T15:50:42.853 INFO:tasks.workunit.client.0.smithi019.stdout:4/699: dwrite d2/db/d92/f91 [0,4194304] 0 2024-02-22T15:50:42.854 INFO:tasks.workunit.client.1.smithi062.stdout:6/610: link d5/df/d32/c6d d5/df/d40/d44/d4a/d41/d43/dc6/cc7 0 2024-02-22T15:50:42.854 INFO:tasks.workunit.client.1.smithi062.stdout:4/522: dread d4/f45 [4194304,4194304] 0 2024-02-22T15:50:42.857 INFO:tasks.workunit.client.1.smithi062.stdout:6/611: write d5/f6 [4711306,17084] 0 2024-02-22T15:50:42.857 INFO:tasks.workunit.client.1.smithi062.stdout:4/523: write d4/d1c/d2c/f87 [2831058,19382] 0 2024-02-22T15:50:42.861 INFO:tasks.workunit.client.0.smithi019.stdout:5/721: mkdir d8/dd/d19/d38/d47/ddb/dde/df6 0 2024-02-22T15:50:42.861 INFO:tasks.workunit.client.0.smithi019.stdout:1/750: write d8/d15/f68 [9076918,82980] 0 2024-02-22T15:50:42.862 INFO:tasks.workunit.client.1.smithi062.stdout:5/444: dread d0/d3/da/fb [0,4194304] 0 2024-02-22T15:50:42.863 INFO:tasks.workunit.client.1.smithi062.stdout:2/631: dwrite d1/d1e/d38/d76/f92 [0,4194304] 0 2024-02-22T15:50:42.864 INFO:tasks.workunit.client.0.smithi019.stdout:4/700: creat d2/db/d59/fe3 x:0 0 0 2024-02-22T15:50:42.868 INFO:tasks.workunit.client.1.smithi062.stdout:6/612: creat d5/dd/fc8 x:0 0 0 2024-02-22T15:50:42.868 INFO:tasks.workunit.client.1.smithi062.stdout:5/445: truncate d0/d3/d2f/d59/d67/d69/d3f/d5f/f89 752874 0 2024-02-22T15:50:42.870 INFO:tasks.workunit.client.0.smithi019.stdout:3/597: dread d9/d22/f50 [0,4194304] 0 2024-02-22T15:50:42.871 INFO:tasks.workunit.client.0.smithi019.stdout:1/751: dread - d8/d15/d5b/dbc/fe0 zero size 2024-02-22T15:50:42.871 INFO:tasks.workunit.client.0.smithi019.stdout:9/711: symlink d2/d1f/d2e/d57/d60/d8c/d66/l100 0 2024-02-22T15:50:42.871 INFO:tasks.workunit.client.1.smithi062.stdout:2/632: link d1/d8/f3d d1/d47/d39/fcd 0 2024-02-22T15:50:42.876 INFO:tasks.workunit.client.1.smithi062.stdout:5/446: dread d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:42.876 INFO:tasks.workunit.client.1.smithi062.stdout:6/613: link d5/df/d32/d47/lbd d5/df/d40/d44/d4a/d41/d43/dc6/lc9 0 2024-02-22T15:50:42.878 INFO:tasks.workunit.client.0.smithi019.stdout:8/667: link d9/d14/c6e d9/d14/d17/d3d/d83/cdf 0 2024-02-22T15:50:42.878 INFO:tasks.workunit.client.0.smithi019.stdout:9/712: truncate d2/d7/d52/faa 1883434 0 2024-02-22T15:50:42.878 INFO:tasks.workunit.client.0.smithi019.stdout:1/752: fdatasync d8/d15/d5b/dbc/fe0 0 2024-02-22T15:50:42.878 INFO:tasks.workunit.client.0.smithi019.stdout:0/713: truncate d4/db/d1b/d63/f70 3038163 0 2024-02-22T15:50:42.881 INFO:tasks.workunit.client.0.smithi019.stdout:5/722: getdents d8/dd/d19/d77 0 2024-02-22T15:50:42.881 INFO:tasks.workunit.client.1.smithi062.stdout:6/614: creat d5/df/d40/d44/d76/d56/fca x:0 0 0 2024-02-22T15:50:42.883 INFO:tasks.workunit.client.1.smithi062.stdout:8/592: dread d8/f24 [0,4194304] 0 2024-02-22T15:50:42.884 INFO:tasks.workunit.client.0.smithi019.stdout:8/668: truncate d9/d14/d6f/d7e/d94/daa/fb6 513974 0 2024-02-22T15:50:42.885 INFO:tasks.workunit.client.0.smithi019.stdout:3/598: dread d9/d14/f86 [0,4194304] 0 2024-02-22T15:50:42.885 INFO:tasks.workunit.client.1.smithi062.stdout:8/593: dread - d8/d26/fb9 zero size 2024-02-22T15:50:42.888 INFO:tasks.workunit.client.1.smithi062.stdout:6/615: unlink d5/dd/d33/d66/fa4 0 2024-02-22T15:50:42.888 INFO:tasks.workunit.client.1.smithi062.stdout:4/524: dread d4/d2a/d2d/d2e/f84 [0,4194304] 0 2024-02-22T15:50:42.889 INFO:tasks.workunit.client.0.smithi019.stdout:4/701: dwrite d2/db/d14/d23/d21/f9b [0,4194304] 0 2024-02-22T15:50:42.889 INFO:tasks.workunit.client.0.smithi019.stdout:2/730: truncate d1/d17/d1c/f34 4005038 0 2024-02-22T15:50:42.890 INFO:tasks.workunit.client.1.smithi062.stdout:8/594: mknod d8/d9/d44/d7e/cba 0 2024-02-22T15:50:42.892 INFO:tasks.workunit.client.1.smithi062.stdout:6/616: creat d5/df/d40/d44/d76/d56/fcb x:0 0 0 2024-02-22T15:50:42.894 INFO:tasks.workunit.client.0.smithi019.stdout:9/713: rename d2/d7/dc/c22 to d2/d1f/d2e/c101 0 2024-02-22T15:50:42.895 INFO:tasks.workunit.client.1.smithi062.stdout:2/633: dread d1/d8/f3d [0,4194304] 0 2024-02-22T15:50:42.895 INFO:tasks.workunit.client.0.smithi019.stdout:1/753: write d8/fb4 [4999465,69662] 0 2024-02-22T15:50:42.896 INFO:tasks.workunit.client.1.smithi062.stdout:2/634: truncate d1/d28/fc7 509466 0 2024-02-22T15:50:42.897 INFO:tasks.workunit.client.1.smithi062.stdout:6/617: dread - d5/df/d40/d44/d76/d56/f65 zero size 2024-02-22T15:50:42.898 INFO:tasks.workunit.client.0.smithi019.stdout:5/723: creat d8/dd/d19/d3a/ff7 x:0 0 0 2024-02-22T15:50:42.899 INFO:tasks.workunit.client.0.smithi019.stdout:2/731: unlink d1/d17/fb7 0 2024-02-22T15:50:42.900 INFO:tasks.workunit.client.1.smithi062.stdout:8/595: link d8/d28/c32 d8/d9/d44/d7e/cbb 0 2024-02-22T15:50:42.900 INFO:tasks.workunit.client.1.smithi062.stdout:4/525: link d4/d1c/d2c/d6b/d70/f76 d4/d2a/d2d/d64/fb5 0 2024-02-22T15:50:42.904 INFO:tasks.workunit.client.0.smithi019.stdout:9/714: read d2/d7/dc/d27/d68/f73 [405003,34058] 0 2024-02-22T15:50:42.904 INFO:tasks.workunit.client.1.smithi062.stdout:4/526: dread - d4/d1c/d30/d3b/d49/f95 zero size 2024-02-22T15:50:42.904 INFO:tasks.workunit.client.1.smithi062.stdout:8/596: rmdir d8/d9/d2c/d49/d94 39 2024-02-22T15:50:42.906 INFO:tasks.workunit.client.0.smithi019.stdout:5/724: getdents d8/dd/d19/d38/d47 0 2024-02-22T15:50:42.907 INFO:tasks.workunit.client.1.smithi062.stdout:6/618: dwrite d5/f1e [0,4194304] 0 2024-02-22T15:50:42.907 INFO:tasks.workunit.client.1.smithi062.stdout:4/527: fdatasync d4/f45 0 2024-02-22T15:50:42.907 INFO:tasks.workunit.client.0.smithi019.stdout:1/754: dwrite d8/f53 [0,4194304] 0 2024-02-22T15:50:42.910 INFO:tasks.workunit.client.0.smithi019.stdout:9/715: mknod d2/d1f/d2e/d57/c102 0 2024-02-22T15:50:42.914 INFO:tasks.workunit.client.0.smithi019.stdout:5/725: mknod d8/d11/d33/cf8 0 2024-02-22T15:50:42.917 INFO:tasks.workunit.client.1.smithi062.stdout:4/528: unlink f2 0 2024-02-22T15:50:42.917 INFO:tasks.workunit.client.1.smithi062.stdout:4/529: write d4/fd [2558626,22360] 0 2024-02-22T15:50:42.917 INFO:tasks.workunit.client.0.smithi019.stdout:2/732: dwrite d1/d46/fab [0,4194304] 0 2024-02-22T15:50:42.919 INFO:tasks.workunit.client.1.smithi062.stdout:6/619: creat d5/dd/d33/d66/fcc x:0 0 0 2024-02-22T15:50:42.920 INFO:tasks.workunit.client.1.smithi062.stdout:4/530: mkdir d4/d2a/d2d/d2e/d6a/d9e/da4/db6 0 2024-02-22T15:50:42.921 INFO:tasks.workunit.client.0.smithi019.stdout:9/716: unlink d2/d7/dee/lfa 0 2024-02-22T15:50:42.921 INFO:tasks.workunit.client.0.smithi019.stdout:5/726: chown d8/dd/d19/d38/d47/ddb/dde/deb/fd4 216 1 2024-02-22T15:50:42.923 INFO:tasks.workunit.client.1.smithi062.stdout:5/447: dread d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:42.923 INFO:tasks.workunit.client.0.smithi019.stdout:9/717: rename d2 to d2/d1f/d2e/d48/d45/d5a/daf/d103 22 2024-02-22T15:50:42.924 INFO:tasks.workunit.client.1.smithi062.stdout:5/448: chown d0/d3/da/d32/d4f/f50 43276 1 2024-02-22T15:50:42.925 INFO:tasks.workunit.client.1.smithi062.stdout:8/597: creat d8/d9/d2c/d49/d94/da1/fbc x:0 0 0 2024-02-22T15:50:42.925 INFO:tasks.workunit.client.0.smithi019.stdout:1/755: dwrite d8/fe1 [0,4194304] 0 2024-02-22T15:50:42.926 INFO:tasks.workunit.client.1.smithi062.stdout:4/531: readlink d4/d1c/d30/d3b/d49/l61 0 2024-02-22T15:50:42.927 INFO:tasks.workunit.client.1.smithi062.stdout:6/620: mkdir d5/df/dbc/dcd 0 2024-02-22T15:50:42.927 INFO:tasks.workunit.client.1.smithi062.stdout:8/598: fsync d8/d86/fa6 0 2024-02-22T15:50:42.929 INFO:tasks.workunit.client.0.smithi019.stdout:9/718: unlink d2/d1f/d2e/f42 0 2024-02-22T15:50:42.930 INFO:tasks.workunit.client.0.smithi019.stdout:1/756: write d8/d15/f69 [4001321,7679] 0 2024-02-22T15:50:42.930 INFO:tasks.workunit.client.1.smithi062.stdout:8/599: creat d8/d26/fbd x:0 0 0 2024-02-22T15:50:42.931 INFO:tasks.workunit.client.1.smithi062.stdout:6/621: write d5/df/d40/d44/d76/d56/fcb [447149,2112] 0 2024-02-22T15:50:42.935 INFO:tasks.workunit.client.0.smithi019.stdout:5/727: rename d8/dd/d8e/d2c/cee to d8/dd/d19/d3a/d46/cf9 0 2024-02-22T15:50:42.938 INFO:tasks.workunit.client.1.smithi062.stdout:5/449: dwrite d0/d3/da/f53 [0,4194304] 0 2024-02-22T15:50:42.938 INFO:tasks.workunit.client.0.smithi019.stdout:2/733: dwrite d1/d17/d1c/d66/db9/dd0/fe5 [0,4194304] 0 2024-02-22T15:50:42.939 INFO:tasks.workunit.client.1.smithi062.stdout:4/532: rename d4/d1c/d30/d3b/c4a to d4/d1c/d2c/d6b/cb7 0 2024-02-22T15:50:42.939 INFO:tasks.workunit.client.0.smithi019.stdout:1/757: rename d8/d23/c29 to d8/d23/d33/c100 0 2024-02-22T15:50:42.939 INFO:tasks.workunit.client.1.smithi062.stdout:4/533: readlink d4/df/l1e 0 2024-02-22T15:50:42.940 INFO:tasks.workunit.client.0.smithi019.stdout:5/728: fdatasync d8/f23 0 2024-02-22T15:50:42.943 INFO:tasks.workunit.client.1.smithi062.stdout:2/635: dread d1/fb8 [0,4194304] 0 2024-02-22T15:50:42.944 INFO:tasks.workunit.client.1.smithi062.stdout:4/534: rmdir d4/d2a/d2d/d2e/d6a/d8a 39 2024-02-22T15:50:42.945 INFO:tasks.workunit.client.0.smithi019.stdout:8/669: dread d9/d14/d17/d1c/f48 [4194304,4194304] 0 2024-02-22T15:50:42.945 INFO:tasks.workunit.client.1.smithi062.stdout:8/600: link d8/c7f d8/d9/d44/d4d/d51/cbe 0 2024-02-22T15:50:42.948 INFO:tasks.workunit.client.1.smithi062.stdout:6/622: getdents d5/df/d40/d44/d78 0 2024-02-22T15:50:42.949 INFO:tasks.workunit.client.0.smithi019.stdout:1/758: rmdir d8/d2a/d75/d81/d8e 39 2024-02-22T15:50:42.949 INFO:tasks.workunit.client.1.smithi062.stdout:8/601: creat d8/d9/d48/fbf x:0 0 0 2024-02-22T15:50:42.951 INFO:tasks.workunit.client.1.smithi062.stdout:6/623: dread - d5/df/d32/fc5 zero size 2024-02-22T15:50:42.952 INFO:tasks.workunit.client.0.smithi019.stdout:5/729: mknod d8/d11/cfa 0 2024-02-22T15:50:42.953 INFO:tasks.workunit.client.1.smithi062.stdout:5/450: write d0/d3/da/d13/f3d [1624213,32881] 0 2024-02-22T15:50:42.954 INFO:tasks.workunit.client.0.smithi019.stdout:2/734: rename d1/d17/d21/d71/de1 to d1/d17/d1c/d66/db9/ddd/d109 0 2024-02-22T15:50:42.954 INFO:tasks.workunit.client.0.smithi019.stdout:9/719: write d2/d1f/d2e/d48/d36/f43 [1006708,104060] 0 2024-02-22T15:50:42.956 INFO:tasks.workunit.client.1.smithi062.stdout:4/535: dwrite d4/d1c/d30/f65 [0,4194304] 0 2024-02-22T15:50:42.957 INFO:tasks.workunit.client.0.smithi019.stdout:5/730: symlink d8/dd/d19/d77/lfb 0 2024-02-22T15:50:42.958 INFO:tasks.workunit.client.1.smithi062.stdout:4/536: chown d4/d2a/d2d/d2e/d6a/l83 4183171 1 2024-02-22T15:50:42.962 INFO:tasks.workunit.client.0.smithi019.stdout:1/759: truncate d8/d18/d85/f8d 3608729 0 2024-02-22T15:50:42.967 INFO:tasks.workunit.client.0.smithi019.stdout:8/670: link d9/d14/d1f/l23 d9/d14/d17/d1c/le0 0 2024-02-22T15:50:42.969 INFO:tasks.workunit.client.0.smithi019.stdout:1/760: symlink d8/d15/d34/l101 0 2024-02-22T15:50:42.972 INFO:tasks.workunit.client.0.smithi019.stdout:8/671: link d9/d14/d17/d3d/d3f/dd4/ld7 d9/d14/d17/d3d/d3f/dd4/le1 0 2024-02-22T15:50:42.974 INFO:tasks.workunit.client.0.smithi019.stdout:8/672: creat d9/d14/d6f/d7e/d94/db8/fe2 x:0 0 0 2024-02-22T15:50:42.980 INFO:tasks.workunit.client.1.smithi062.stdout:8/602: dread d8/d9/d2c/d49/f6b [0,4194304] 0 2024-02-22T15:50:42.982 INFO:tasks.workunit.client.0.smithi019.stdout:8/673: dwrite d9/fc3 [0,4194304] 0 2024-02-22T15:50:42.983 INFO:tasks.workunit.client.1.smithi062.stdout:4/537: dread d4/d34/fb0 [0,4194304] 0 2024-02-22T15:50:42.983 INFO:tasks.workunit.client.0.smithi019.stdout:2/735: truncate d1/d17/fb2 2784037 0 2024-02-22T15:50:42.983 INFO:tasks.workunit.client.1.smithi062.stdout:4/538: dread - d4/d2a/d2d/d2e/f97 zero size 2024-02-22T15:50:42.984 INFO:tasks.workunit.client.0.smithi019.stdout:2/736: chown d1/d46/f95 61486024 1 2024-02-22T15:50:42.987 INFO:tasks.workunit.client.0.smithi019.stdout:1/761: write d8/d2a/db3/fb1 [32527,25901] 0 2024-02-22T15:50:42.991 INFO:tasks.workunit.client.0.smithi019.stdout:5/731: mkdir d8/dd/d19/d77/d7a/dfc 0 2024-02-22T15:50:42.994 INFO:tasks.workunit.client.0.smithi019.stdout:8/674: creat d9/d14/d6f/fe3 x:0 0 0 2024-02-22T15:50:42.995 INFO:tasks.workunit.client.0.smithi019.stdout:2/737: creat d1/d17/d21/d71/dd1/d106/f10a x:0 0 0 2024-02-22T15:50:42.995 INFO:tasks.workunit.client.0.smithi019.stdout:5/732: truncate d8/dd/d19/d38/d47/ddb/dde/deb/fd4 304855 0 2024-02-22T15:50:42.997 INFO:tasks.workunit.client.0.smithi019.stdout:9/720: dread d2/f6f [0,4194304] 0 2024-02-22T15:50:43.006 INFO:tasks.workunit.client.0.smithi019.stdout:9/721: fdatasync d2/d7/d52/ff2 0 2024-02-22T15:50:43.006 INFO:tasks.workunit.client.0.smithi019.stdout:8/675: creat d9/d14/d17/d1c/dd8/fe4 x:0 0 0 2024-02-22T15:50:43.008 INFO:tasks.workunit.client.0.smithi019.stdout:1/762: dwrite d8/fd [0,4194304] 0 2024-02-22T15:50:43.008 INFO:tasks.workunit.client.0.smithi019.stdout:8/676: readlink d9/d14/d17/d3d/d83/ld2 0 2024-02-22T15:50:43.009 INFO:tasks.workunit.client.0.smithi019.stdout:2/738: truncate d1/d17/f30 2622005 0 2024-02-22T15:50:43.013 INFO:tasks.workunit.client.0.smithi019.stdout:2/739: mknod d1/d17/d67/c10b 0 2024-02-22T15:50:43.016 INFO:tasks.workunit.client.0.smithi019.stdout:8/677: mkdir d9/d14/d17/d3d/de5 0 2024-02-22T15:50:43.018 INFO:tasks.workunit.client.0.smithi019.stdout:5/733: dread d8/dd/d19/d77/d7a/f97 [0,4194304] 0 2024-02-22T15:50:43.019 INFO:tasks.workunit.client.0.smithi019.stdout:2/740: fdatasync d1/d17/d21/d51/f72 0 2024-02-22T15:50:43.019 INFO:tasks.workunit.client.0.smithi019.stdout:9/722: dwrite d2/d1f/db5/fde [0,4194304] 0 2024-02-22T15:50:43.022 INFO:tasks.workunit.client.0.smithi019.stdout:9/723: dread - d2/d7/d52/ff7 zero size 2024-02-22T15:50:43.023 INFO:tasks.workunit.client.1.smithi062.stdout:6/624: mkdir d5/df/dbc/dce 0 2024-02-22T15:50:43.024 INFO:tasks.workunit.client.1.smithi062.stdout:5/451: creat d0/d3/da/d32/f8d x:0 0 0 2024-02-22T15:50:43.026 INFO:tasks.workunit.client.0.smithi019.stdout:8/678: fdatasync d9/d14/d1f/fda 0 2024-02-22T15:50:43.026 INFO:tasks.workunit.client.0.smithi019.stdout:5/734: chown d8/dd/d19/d77/c84 157915 1 2024-02-22T15:50:43.026 INFO:tasks.workunit.client.0.smithi019.stdout:9/724: chown d2/d1f/d2e/d48/d45/d5a/l99 1618487 1 2024-02-22T15:50:43.031 INFO:tasks.workunit.client.0.smithi019.stdout:1/763: link d8/d15/l22 d8/d23/l102 0 2024-02-22T15:50:43.036 INFO:tasks.workunit.client.0.smithi019.stdout:8/679: dwrite d9/d14/d1f/d5c/fc8 [0,4194304] 0 2024-02-22T15:50:43.036 INFO:tasks.workunit.client.1.smithi062.stdout:6/625: creat d5/d8a/fcf x:0 0 0 2024-02-22T15:50:43.037 INFO:tasks.workunit.client.1.smithi062.stdout:6/626: mkdir d5/df/dbc/dd0 0 2024-02-22T15:50:43.037 INFO:tasks.workunit.client.1.smithi062.stdout:5/452: symlink d0/d3/da/d13/d23/d4e/d5a/l8e 0 2024-02-22T15:50:43.037 INFO:tasks.workunit.client.0.smithi019.stdout:1/764: read - d8/d2a/d97/f9a zero size 2024-02-22T15:50:43.037 INFO:tasks.workunit.client.1.smithi062.stdout:4/539: dwrite d4/f3f [0,4194304] 0 2024-02-22T15:50:43.039 INFO:tasks.workunit.client.1.smithi062.stdout:6/627: write d5/df/d40/d44/d78/f70 [867890,73005] 0 2024-02-22T15:50:43.039 INFO:tasks.workunit.client.0.smithi019.stdout:5/735: creat d8/dd/d19/d77/d7a/ffd x:0 0 0 2024-02-22T15:50:43.040 INFO:tasks.workunit.client.1.smithi062.stdout:5/453: rmdir d0/d3/da/d13/d23/d78/d81 39 2024-02-22T15:50:43.041 INFO:tasks.workunit.client.0.smithi019.stdout:1/765: chown d8/d2a/c46 253532 1 2024-02-22T15:50:43.042 INFO:tasks.workunit.client.0.smithi019.stdout:9/725: dwrite d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:43.045 INFO:tasks.workunit.client.1.smithi062.stdout:6/628: mkdir d5/df/d40/d44/d4a/d41/d85/dd1 0 2024-02-22T15:50:43.046 INFO:tasks.workunit.client.0.smithi019.stdout:8/680: mkdir d9/d14/d17/d5d/de6 0 2024-02-22T15:50:43.047 INFO:tasks.workunit.client.1.smithi062.stdout:8/603: rmdir d8 39 2024-02-22T15:50:43.050 INFO:tasks.workunit.client.0.smithi019.stdout:2/741: dwrite d1/f76 [0,4194304] 0 2024-02-22T15:50:43.052 INFO:tasks.workunit.client.0.smithi019.stdout:1/766: rename d8/d15/d5b/le2 to d8/d15/da4/l103 0 2024-02-22T15:50:43.052 INFO:tasks.workunit.client.0.smithi019.stdout:8/681: readlink d9/d14/l46 0 2024-02-22T15:50:43.053 INFO:tasks.workunit.client.1.smithi062.stdout:6/629: mkdir d5/df/d40/dd2 0 2024-02-22T15:50:43.054 INFO:tasks.workunit.client.1.smithi062.stdout:8/604: truncate d8/f39 2045975 0 2024-02-22T15:50:43.056 INFO:tasks.workunit.client.1.smithi062.stdout:6/630: mkdir d5/df/d32/dd3 0 2024-02-22T15:50:43.058 INFO:tasks.workunit.client.0.smithi019.stdout:1/767: creat d8/d2a/db3/f104 x:0 0 0 2024-02-22T15:50:43.061 INFO:tasks.workunit.client.1.smithi062.stdout:8/605: fsync d8/d9/d44/d4d/d51/d55/f92 0 2024-02-22T15:50:43.062 INFO:tasks.workunit.client.0.smithi019.stdout:2/742: unlink d1/d17/d4b/c52 0 2024-02-22T15:50:43.062 INFO:tasks.workunit.client.1.smithi062.stdout:8/606: rename d8/d9/d44/d4d/d51/d55/c84 to d8/d42/d87/cc0 0 2024-02-22T15:50:43.062 INFO:tasks.workunit.client.1.smithi062.stdout:6/631: getdents d5/df/d40/d44/d4a 0 2024-02-22T15:50:43.063 INFO:tasks.workunit.client.0.smithi019.stdout:9/726: dwrite d2/fb [4194304,4194304] 0 2024-02-22T15:50:43.063 INFO:tasks.workunit.client.0.smithi019.stdout:2/743: stat d1/db4/fd2 0 2024-02-22T15:50:43.064 INFO:tasks.workunit.client.1.smithi062.stdout:8/607: fsync d8/d9/d44/f56 0 2024-02-22T15:50:43.065 INFO:tasks.workunit.client.0.smithi019.stdout:1/768: symlink d8/d18/d64/d7f/de3/l105 0 2024-02-22T15:50:43.067 INFO:tasks.workunit.client.1.smithi062.stdout:8/608: dread - d8/d9/d2c/d49/fb8 zero size 2024-02-22T15:50:43.068 INFO:tasks.workunit.client.0.smithi019.stdout:1/769: creat d8/d15/d5b/f106 x:0 0 0 2024-02-22T15:50:43.069 INFO:tasks.workunit.client.1.smithi062.stdout:5/454: truncate d0/d3/da/d13/d23/f24 1008133 0 2024-02-22T15:50:43.069 INFO:tasks.workunit.client.0.smithi019.stdout:9/727: creat d2/d7/d52/f104 x:0 0 0 2024-02-22T15:50:43.070 INFO:tasks.workunit.client.0.smithi019.stdout:2/744: creat d1/d17/d21/d71/f10c x:0 0 0 2024-02-22T15:50:43.070 INFO:tasks.workunit.client.0.smithi019.stdout:9/728: chown d2/d7/d52/dc0/de4 381198 1 2024-02-22T15:50:43.071 INFO:tasks.workunit.client.0.smithi019.stdout:1/770: creat d8/d18/d72/f107 x:0 0 0 2024-02-22T15:50:43.072 INFO:tasks.workunit.client.1.smithi062.stdout:6/632: dwrite d5/df/d40/f53 [0,4194304] 0 2024-02-22T15:50:43.073 INFO:tasks.workunit.client.0.smithi019.stdout:2/745: dread - d1/d17/d4b/f96 zero size 2024-02-22T15:50:43.073 INFO:tasks.workunit.client.0.smithi019.stdout:1/771: dread - d8/d23/d33/fd4 zero size 2024-02-22T15:50:43.073 INFO:tasks.workunit.client.1.smithi062.stdout:6/633: stat d5/df/d32 0 2024-02-22T15:50:43.075 INFO:tasks.workunit.client.1.smithi062.stdout:8/609: rename d8/d42/d8f to d8/d9/d2c/d49/d94/dc1 0 2024-02-22T15:50:43.075 INFO:tasks.workunit.client.1.smithi062.stdout:8/610: stat d8/d9/d44/d4d/l7b 0 2024-02-22T15:50:43.076 INFO:tasks.workunit.client.0.smithi019.stdout:1/772: fdatasync d8/d2a/d75/d81/f96 0 2024-02-22T15:50:43.077 INFO:tasks.workunit.client.0.smithi019.stdout:2/746: write d1/d17/f47 [3008113,13796] 0 2024-02-22T15:50:43.078 INFO:tasks.workunit.client.1.smithi062.stdout:5/455: symlink d0/d3/da/d32/d4f/d72/l8f 0 2024-02-22T15:50:43.079 INFO:tasks.workunit.client.0.smithi019.stdout:5/736: dread d8/f28 [0,4194304] 0 2024-02-22T15:50:43.079 INFO:tasks.workunit.client.1.smithi062.stdout:7/427: sync 2024-02-22T15:50:43.079 INFO:tasks.workunit.client.1.smithi062.stdout:0/626: sync 2024-02-22T15:50:43.081 INFO:tasks.workunit.client.1.smithi062.stdout:6/634: write d5/df/d40/d44/d4a/fad [694242,83966] 0 2024-02-22T15:50:43.082 INFO:tasks.workunit.client.1.smithi062.stdout:8/611: chown d8/d9/c25 235 1 2024-02-22T15:50:43.082 INFO:tasks.workunit.client.0.smithi019.stdout:1/773: symlink d8/d2a/d75/d81/ddb/l108 0 2024-02-22T15:50:43.083 INFO:tasks.workunit.client.1.smithi062.stdout:4/540: dread d4/f11 [0,4194304] 0 2024-02-22T15:50:43.083 INFO:tasks.workunit.client.0.smithi019.stdout:9/729: dwrite d2/dec/d79/fb2 [0,4194304] 0 2024-02-22T15:50:43.083 INFO:tasks.workunit.client.0.smithi019.stdout:2/747: mkdir d1/d46/ddb/d10d 0 2024-02-22T15:50:43.084 INFO:tasks.workunit.client.0.smithi019.stdout:5/737: dread - d8/dd/d19/d38/d54/f95 zero size 2024-02-22T15:50:43.088 INFO:tasks.workunit.client.0.smithi019.stdout:9/730: dread - d2/d1f/d2e/d48/d45/d5a/fd0 zero size 2024-02-22T15:50:43.089 INFO:tasks.workunit.client.1.smithi062.stdout:7/428: readlink d0/d14/l18 0 2024-02-22T15:50:43.090 INFO:tasks.workunit.client.0.smithi019.stdout:2/748: rmdir d1/d17/d1c/d66/db9/dfe 0 2024-02-22T15:50:43.090 INFO:tasks.workunit.client.1.smithi062.stdout:9/525: sync 2024-02-22T15:50:43.092 INFO:tasks.workunit.client.0.smithi019.stdout:5/738: symlink d8/dd/d19/d77/dd2/dd9/lfe 0 2024-02-22T15:50:43.094 INFO:tasks.workunit.client.0.smithi019.stdout:9/731: fsync d2/dc8/fb0 0 2024-02-22T15:50:43.096 INFO:tasks.workunit.client.0.smithi019.stdout:5/739: creat d8/dd/d19/d77/dd2/dd9/fff x:0 0 0 2024-02-22T15:50:43.096 INFO:tasks.workunit.client.1.smithi062.stdout:6/635: rmdir d5/df/d40/d44/d78 39 2024-02-22T15:50:43.098 INFO:tasks.workunit.client.1.smithi062.stdout:4/541: creat d4/d2a/fb8 x:0 0 0 2024-02-22T15:50:43.099 INFO:tasks.workunit.client.0.smithi019.stdout:5/740: stat d8/dd/d19/d3a/d46/c8d 0 2024-02-22T15:50:43.099 INFO:tasks.workunit.client.0.smithi019.stdout:2/749: getdents d1/db4 0 2024-02-22T15:50:43.101 INFO:tasks.workunit.client.0.smithi019.stdout:5/741: write d8/db1/fb7 [92781,87994] 0 2024-02-22T15:50:43.103 INFO:tasks.workunit.client.1.smithi062.stdout:7/429: dread - d0/d6/d2e/d4a/d5a/d5e/f6c zero size 2024-02-22T15:50:43.103 INFO:tasks.workunit.client.0.smithi019.stdout:8/682: write d9/d14/d17/fbe [424464,97006] 0 2024-02-22T15:50:43.104 INFO:tasks.workunit.client.1.smithi062.stdout:0/627: dwrite d1/d3/d4/d1a/d1d/d3b/fd1 [0,4194304] 0 2024-02-22T15:50:43.105 INFO:tasks.workunit.client.1.smithi062.stdout:6/636: unlink d5/df/f17 0 2024-02-22T15:50:43.105 INFO:tasks.workunit.client.1.smithi062.stdout:5/456: dwrite d0/d3/da/d13/d23/d4e/d6d/f80 [0,4194304] 0 2024-02-22T15:50:43.105 INFO:tasks.workunit.client.0.smithi019.stdout:8/683: chown d9/d14/d1f/d6d/fc5 11950 1 2024-02-22T15:50:43.107 INFO:tasks.workunit.client.1.smithi062.stdout:8/612: write d8/d9/fe [2650926,57537] 0 2024-02-22T15:50:43.107 INFO:tasks.workunit.client.1.smithi062.stdout:4/542: dread - d4/d2a/d2d/d46/fad zero size 2024-02-22T15:50:43.109 INFO:tasks.workunit.client.0.smithi019.stdout:5/742: creat d8/d11/d33/f100 x:0 0 0 2024-02-22T15:50:43.110 INFO:tasks.workunit.client.0.smithi019.stdout:2/750: dwrite d1/da4/fe6 [0,4194304] 0 2024-02-22T15:50:43.112 INFO:tasks.workunit.client.0.smithi019.stdout:1/774: dread d8/d18/d85/f98 [0,4194304] 0 2024-02-22T15:50:43.115 INFO:tasks.workunit.client.0.smithi019.stdout:5/743: rename d8/f4e to d8/dd/d19/d77/dd2/f101 0 2024-02-22T15:50:43.115 INFO:tasks.workunit.client.1.smithi062.stdout:6/637: write d5/fbb [478218,41296] 0 2024-02-22T15:50:43.116 INFO:tasks.workunit.client.1.smithi062.stdout:8/613: fsync d8/d42/d87/fa7 0 2024-02-22T15:50:43.118 INFO:tasks.workunit.client.0.smithi019.stdout:8/684: symlink d9/le7 0 2024-02-22T15:50:43.119 INFO:tasks.workunit.client.0.smithi019.stdout:5/744: read d8/f23 [2090486,4519] 0 2024-02-22T15:50:43.120 INFO:tasks.workunit.client.1.smithi062.stdout:6/638: readlink d5/df/d40/d44/d4a/l64 0 2024-02-22T15:50:43.120 INFO:tasks.workunit.client.1.smithi062.stdout:5/457: unlink d0/d3/da/d13/c76 0 2024-02-22T15:50:43.120 INFO:tasks.workunit.client.1.smithi062.stdout:7/430: creat d0/f89 x:0 0 0 2024-02-22T15:50:43.121 INFO:tasks.workunit.client.0.smithi019.stdout:9/732: write d2/d7/dc/f29 [99851,64383] 0 2024-02-22T15:50:43.123 INFO:tasks.workunit.client.0.smithi019.stdout:8/685: unlink d9/d14/d17/d3d/d3f/l4f 0 2024-02-22T15:50:43.124 INFO:tasks.workunit.client.1.smithi062.stdout:0/628: rename d1/dd/l7f to d1/d3/d4/d1a/d76/d8c/d5d/dc1/ld3 0 2024-02-22T15:50:43.125 INFO:tasks.workunit.client.0.smithi019.stdout:8/686: fsync d9/d14/d17/f45 0 2024-02-22T15:50:43.126 INFO:tasks.workunit.client.0.smithi019.stdout:2/751: getdents d1/d17/d21/d51 0 2024-02-22T15:50:43.126 INFO:tasks.workunit.client.0.smithi019.stdout:9/733: creat d2/d7/dc/d27/dbc/f105 x:0 0 0 2024-02-22T15:50:43.127 INFO:tasks.workunit.client.0.smithi019.stdout:5/745: getdents d8/dd/d19/d3a/dc7 0 2024-02-22T15:50:43.127 INFO:tasks.workunit.client.1.smithi062.stdout:4/543: dwrite d4/d1c/d2c/f56 [0,4194304] 0 2024-02-22T15:50:43.128 INFO:tasks.workunit.client.1.smithi062.stdout:9/526: dread d4/d15/d5f/d75/f9a [0,4194304] 0 2024-02-22T15:50:43.129 INFO:tasks.workunit.client.0.smithi019.stdout:5/746: chown d8/dd/d19/d38/d47/ddb 444087225 1 2024-02-22T15:50:43.134 INFO:tasks.workunit.client.1.smithi062.stdout:4/544: mkdir d4/d1c/d2c/db9 0 2024-02-22T15:50:43.135 INFO:tasks.workunit.client.0.smithi019.stdout:8/687: write d9/d14/d1f/d20/f61 [5212007,123503] 0 2024-02-22T15:50:43.135 INFO:tasks.workunit.client.1.smithi062.stdout:8/614: dread d8/d9/d2c/d74/f89 [0,4194304] 0 2024-02-22T15:50:43.136 INFO:tasks.workunit.client.1.smithi062.stdout:9/527: write d4/d35/d3f/f39 [916123,28176] 0 2024-02-22T15:50:43.137 INFO:tasks.workunit.client.0.smithi019.stdout:2/752: dwrite d1/d46/f98 [0,4194304] 0 2024-02-22T15:50:43.140 INFO:tasks.workunit.client.0.smithi019.stdout:5/747: symlink d8/dd/d19/d3a/d46/d7e/dbe/l102 0 2024-02-22T15:50:43.140 INFO:tasks.workunit.client.1.smithi062.stdout:4/545: chown d4/f51 7942978 1 2024-02-22T15:50:43.140 INFO:tasks.workunit.client.1.smithi062.stdout:8/615: read - d8/d9/d44/d4d/f82 zero size 2024-02-22T15:50:43.141 INFO:tasks.workunit.client.1.smithi062.stdout:6/639: dwrite d5/df/d32/f3d [0,4194304] 0 2024-02-22T15:50:43.143 INFO:tasks.workunit.client.1.smithi062.stdout:7/431: dwrite d0/d6/d2e/d58/f5c [0,4194304] 0 2024-02-22T15:50:43.144 INFO:tasks.workunit.client.0.smithi019.stdout:9/734: link d2/d1f/d2e/d48/d45/f61 d2/dec/f106 0 2024-02-22T15:50:43.144 INFO:tasks.workunit.client.1.smithi062.stdout:8/616: readlink d8/d9/d2c/lad 0 2024-02-22T15:50:43.146 INFO:tasks.workunit.client.0.smithi019.stdout:8/688: creat d9/d14/d17/d5d/de6/fe8 x:0 0 0 2024-02-22T15:50:43.146 INFO:tasks.workunit.client.0.smithi019.stdout:1/775: write d8/d18/d64/f89 [698543,131042] 0 2024-02-22T15:50:43.147 INFO:tasks.workunit.client.1.smithi062.stdout:9/528: fsync d4/d15/f23 0 2024-02-22T15:50:43.147 INFO:tasks.workunit.client.1.smithi062.stdout:8/617: dread - d8/d9/d44/f9b zero size 2024-02-22T15:50:43.150 INFO:tasks.workunit.client.1.smithi062.stdout:7/432: dread d0/f10 [4194304,4194304] 0 2024-02-22T15:50:43.151 INFO:tasks.workunit.client.0.smithi019.stdout:2/753: creat d1/de/d15/f10e x:0 0 0 2024-02-22T15:50:43.151 INFO:tasks.workunit.client.1.smithi062.stdout:4/546: creat d4/d1c/d2c/db9/fba x:0 0 0 2024-02-22T15:50:43.154 INFO:tasks.workunit.client.0.smithi019.stdout:1/776: fsync d8/d15/fba 0 2024-02-22T15:50:43.154 INFO:tasks.workunit.client.1.smithi062.stdout:9/529: rename d4/d35/d3f to d4/d15/d5e/d63/da0 0 2024-02-22T15:50:43.156 INFO:tasks.workunit.client.1.smithi062.stdout:4/547: fdatasync d4/f39 0 2024-02-22T15:50:43.158 INFO:tasks.workunit.client.1.smithi062.stdout:7/433: mkdir d0/d6/d2e/d55/d80/d8a 0 2024-02-22T15:50:43.159 INFO:tasks.workunit.client.0.smithi019.stdout:6/893: sync 2024-02-22T15:50:43.160 INFO:tasks.workunit.client.1.smithi062.stdout:8/618: dwrite d8/d9/d2c/f78 [0,4194304] 0 2024-02-22T15:50:43.160 INFO:tasks.workunit.client.1.smithi062.stdout:6/640: read d5/dd/d33/fa9 [15128,117126] 0 2024-02-22T15:50:43.160 INFO:tasks.workunit.client.0.smithi019.stdout:2/754: chown d1/de/f1e 31 1 2024-02-22T15:50:43.160 INFO:tasks.workunit.client.1.smithi062.stdout:4/548: write d4/d1c/d30/f32 [2893830,80471] 0 2024-02-22T15:50:43.161 INFO:tasks.workunit.client.0.smithi019.stdout:9/735: creat d2/d7/f107 x:0 0 0 2024-02-22T15:50:43.164 INFO:tasks.workunit.client.0.smithi019.stdout:1/777: link d8/d15/d34/fa3 d8/d15/d5b/dbc/f109 0 2024-02-22T15:50:43.164 INFO:tasks.workunit.client.0.smithi019.stdout:8/689: dread d9/d14/fac [0,4194304] 0 2024-02-22T15:50:43.165 INFO:tasks.workunit.client.0.smithi019.stdout:2/755: dread d1/d17/d21/d51/f6e [0,4194304] 0 2024-02-22T15:50:43.166 INFO:tasks.workunit.client.1.smithi062.stdout:8/619: mknod d8/d28/cc2 0 2024-02-22T15:50:43.166 INFO:tasks.workunit.client.0.smithi019.stdout:2/756: readlink d1/de/d9a/lc1 0 2024-02-22T15:50:43.168 INFO:tasks.workunit.client.0.smithi019.stdout:6/894: mkdir d6/d4b/d71/d88/ded/d121 0 2024-02-22T15:50:43.168 INFO:tasks.workunit.client.1.smithi062.stdout:7/434: creat d0/d14/f8b x:0 0 0 2024-02-22T15:50:43.171 INFO:tasks.workunit.client.1.smithi062.stdout:7/435: fsync d0/d6/f23 0 2024-02-22T15:50:43.171 INFO:tasks.workunit.client.0.smithi019.stdout:2/757: symlink d1/db4/l10f 0 2024-02-22T15:50:43.173 INFO:tasks.workunit.client.1.smithi062.stdout:7/436: chown d0/d14/d76 244826446 1 2024-02-22T15:50:43.174 INFO:tasks.workunit.client.1.smithi062.stdout:4/549: link d4/d2a/d2d/d9d/fa2 d4/df/fbb 0 2024-02-22T15:50:43.174 INFO:tasks.workunit.client.1.smithi062.stdout:9/530: dwrite d4/d15/d5f/d75/f9a [0,4194304] 0 2024-02-22T15:50:43.175 INFO:tasks.workunit.client.1.smithi062.stdout:8/620: unlink d8/d9/d44/d7e/cbb 0 2024-02-22T15:50:43.176 INFO:tasks.workunit.client.0.smithi019.stdout:6/895: rename d6/d4b/d6d/dda/d105 to d6/d31/d48/d122 0 2024-02-22T15:50:43.177 INFO:tasks.workunit.client.0.smithi019.stdout:2/758: creat d1/d17/d38/f110 x:0 0 0 2024-02-22T15:50:43.178 INFO:tasks.workunit.client.1.smithi062.stdout:7/437: mknod d0/d6/d2e/d35/c8c 0 2024-02-22T15:50:43.178 INFO:tasks.workunit.client.1.smithi062.stdout:8/621: read - d8/d28/d4e/f8e zero size 2024-02-22T15:50:43.180 INFO:tasks.workunit.client.0.smithi019.stdout:8/690: dwrite d9/d14/d17/d3d/fcb [0,4194304] 0 2024-02-22T15:50:43.180 INFO:tasks.workunit.client.1.smithi062.stdout:4/550: truncate d4/d1c/d30/d3b/f67 1478472 0 2024-02-22T15:50:43.185 INFO:tasks.workunit.client.1.smithi062.stdout:7/438: truncate d0/d14/d19/d30/d37/f63 4209363 0 2024-02-22T15:50:43.185 INFO:tasks.workunit.client.1.smithi062.stdout:9/531: rename d4/d35/f36 to d4/fa1 0 2024-02-22T15:50:43.186 INFO:tasks.workunit.client.0.smithi019.stdout:8/691: chown d9/d14/d17/d1c/dd8 18 1 2024-02-22T15:50:43.186 INFO:tasks.workunit.client.1.smithi062.stdout:5/458: dread d0/d3/d30/f46 [0,4194304] 0 2024-02-22T15:50:43.188 INFO:tasks.workunit.client.1.smithi062.stdout:8/622: mknod d8/d9/d2c/d49/d5e/cc3 0 2024-02-22T15:50:43.188 INFO:tasks.workunit.client.1.smithi062.stdout:9/532: write d4/fd [1140476,76662] 0 2024-02-22T15:50:43.189 INFO:tasks.workunit.client.0.smithi019.stdout:5/748: dread d8/dd/d8e/d2c/f31 [0,4194304] 0 2024-02-22T15:50:43.189 INFO:tasks.workunit.client.1.smithi062.stdout:4/551: fsync d4/d1c/d30/f32 0 2024-02-22T15:50:43.192 INFO:tasks.workunit.client.0.smithi019.stdout:9/736: write d2/d1f/f4b [3609397,28400] 0 2024-02-22T15:50:43.194 INFO:tasks.workunit.client.0.smithi019.stdout:2/759: dwrite d1/d17/d21/d71/ff1 [0,4194304] 0 2024-02-22T15:50:43.198 INFO:tasks.workunit.client.0.smithi019.stdout:6/896: dwrite d6/d4b/d6d/fbb [0,4194304] 0 2024-02-22T15:50:43.200 INFO:tasks.workunit.client.0.smithi019.stdout:5/749: creat d8/d11/f103 x:0 0 0 2024-02-22T15:50:43.204 INFO:tasks.workunit.client.1.smithi062.stdout:9/533: rename d4/d35/f37 to d4/d15/d5f/fa2 0 2024-02-22T15:50:43.205 INFO:tasks.workunit.client.0.smithi019.stdout:6/897: chown d6/d3e/l80 4270547 1 2024-02-22T15:50:43.205 INFO:tasks.workunit.client.0.smithi019.stdout:2/760: rmdir d1/d17/d21 39 2024-02-22T15:50:43.205 INFO:tasks.workunit.client.0.smithi019.stdout:1/778: dwrite d8/d18/d85/f8d [0,4194304] 0 2024-02-22T15:50:43.205 INFO:tasks.workunit.client.0.smithi019.stdout:8/692: dread d9/d14/d1f/d6d/f8f [0,4194304] 0 2024-02-22T15:50:43.207 INFO:tasks.workunit.client.1.smithi062.stdout:5/459: dwrite d0/d3/f14 [0,4194304] 0 2024-02-22T15:50:43.207 INFO:tasks.workunit.client.1.smithi062.stdout:6/641: dread d5/dd/d33/f48 [0,4194304] 0 2024-02-22T15:50:43.210 INFO:tasks.workunit.client.1.smithi062.stdout:8/623: link d8/d86/fa9 d8/d9/fc4 0 2024-02-22T15:50:43.210 INFO:tasks.workunit.client.0.smithi019.stdout:9/737: creat d2/d1f/d2e/d7d/f108 x:0 0 0 2024-02-22T15:50:43.211 INFO:tasks.workunit.client.0.smithi019.stdout:5/750: getdents d8/dd/d19/d3a/dc7 0 2024-02-22T15:50:43.212 INFO:tasks.workunit.client.0.smithi019.stdout:8/693: write d9/d14/d17/d3d/d83/fa6 [6685070,37552] 0 2024-02-22T15:50:43.212 INFO:tasks.workunit.client.1.smithi062.stdout:8/624: dread - d8/d86/fa6 zero size 2024-02-22T15:50:43.213 INFO:tasks.workunit.client.1.smithi062.stdout:5/460: write d0/d3/f14 [5029307,70077] 0 2024-02-22T15:50:43.214 INFO:tasks.workunit.client.0.smithi019.stdout:9/738: mknod d2/d7/c109 0 2024-02-22T15:50:43.217 INFO:tasks.workunit.client.0.smithi019.stdout:2/761: creat d1/d17/d21/f111 x:0 0 0 2024-02-22T15:50:43.220 INFO:tasks.workunit.client.0.smithi019.stdout:8/694: unlink d9/d14/d6f/d7e/d94/d9c/fb2 0 2024-02-22T15:50:43.225 INFO:tasks.workunit.client.1.smithi062.stdout:8/625: creat d8/d28/d70/fc5 x:0 0 0 2024-02-22T15:50:43.225 INFO:tasks.workunit.client.1.smithi062.stdout:4/552: dwrite d4/d1c/d2c/d6b/d70/f93 [0,4194304] 0 2024-02-22T15:50:43.226 INFO:tasks.workunit.client.0.smithi019.stdout:5/751: link d8/dd/d19/d3a/l50 d8/dd/d19/l104 0 2024-02-22T15:50:43.226 INFO:tasks.workunit.client.0.smithi019.stdout:9/739: write d2/d1f/d2e/d48/d36/f83 [2616946,35217] 0 2024-02-22T15:50:43.228 INFO:tasks.workunit.client.1.smithi062.stdout:5/461: getdents d0/d3/da/d13/d23/d78/d8c 0 2024-02-22T15:50:43.229 INFO:tasks.workunit.client.0.smithi019.stdout:2/762: mkdir d1/d17/d21/d71/d60/d112 0 2024-02-22T15:50:43.229 INFO:tasks.workunit.client.0.smithi019.stdout:5/752: readlink d8/dd/d19/d38/d54/l6c 0 2024-02-22T15:50:43.230 INFO:tasks.workunit.client.1.smithi062.stdout:8/626: stat d8/d9/d44/d4d/c72 0 2024-02-22T15:50:43.232 INFO:tasks.workunit.client.0.smithi019.stdout:2/763: creat d1/de/d9a/f113 x:0 0 0 2024-02-22T15:50:43.233 INFO:tasks.workunit.client.1.smithi062.stdout:4/553: read d4/d2a/d2d/f85 [782038,103875] 0 2024-02-22T15:50:43.235 INFO:tasks.workunit.client.0.smithi019.stdout:6/898: dread d6/d31/d69/fa6 [0,4194304] 0 2024-02-22T15:50:43.236 INFO:tasks.workunit.client.1.smithi062.stdout:5/462: symlink d0/d3/da/d13/d57/l90 0 2024-02-22T15:50:43.237 INFO:tasks.workunit.client.0.smithi019.stdout:6/899: write d6/f17 [4969679,126162] 0 2024-02-22T15:50:43.238 INFO:tasks.workunit.client.0.smithi019.stdout:5/753: rename d8/d11/d33/c7f to d8/dd/d19/d77/d7a/c105 0 2024-02-22T15:50:43.239 INFO:tasks.workunit.client.1.smithi062.stdout:7/439: dwrite d0/d4f/f59 [0,4194304] 0 2024-02-22T15:50:43.244 INFO:tasks.workunit.client.0.smithi019.stdout:5/754: chown d8/dd/d19/d38/d47/df3/cf4 0 1 2024-02-22T15:50:43.244 INFO:tasks.workunit.client.1.smithi062.stdout:8/627: creat d8/d28/fc6 x:0 0 0 2024-02-22T15:50:43.244 INFO:tasks.workunit.client.1.smithi062.stdout:7/440: fsync d0/d14/d13/f48 0 2024-02-22T15:50:43.244 INFO:tasks.workunit.client.1.smithi062.stdout:5/463: dread d0/d3/da/d13/d23/f3e [0,4194304] 0 2024-02-22T15:50:43.247 INFO:tasks.workunit.client.1.smithi062.stdout:5/464: dread - d0/d3/da/d13/d23/d78/f7d zero size 2024-02-22T15:50:43.250 INFO:tasks.workunit.client.1.smithi062.stdout:8/628: mknod d8/d9/d2c/d74/cc7 0 2024-02-22T15:50:43.250 INFO:tasks.workunit.client.1.smithi062.stdout:7/441: creat d0/d14/d19/d30/f8d x:0 0 0 2024-02-22T15:50:43.250 INFO:tasks.workunit.client.0.smithi019.stdout:5/755: rename d8/dd/d19/d38/l6f to d8/dd/d19/d3a/dce/l106 0 2024-02-22T15:50:43.252 INFO:tasks.workunit.client.1.smithi062.stdout:9/534: truncate d4/fd 452601 0 2024-02-22T15:50:43.252 INFO:tasks.workunit.client.0.smithi019.stdout:0/714: sync 2024-02-22T15:50:43.254 INFO:tasks.workunit.client.0.smithi019.stdout:6/900: dwrite d6/f41 [8388608,4194304] 0 2024-02-22T15:50:43.256 INFO:tasks.workunit.client.0.smithi019.stdout:0/715: fsync d4/db/d1b/d63/db0/da1/dbb/f39 0 2024-02-22T15:50:43.256 INFO:tasks.workunit.client.1.smithi062.stdout:5/465: readlink d0/d3/d2f/d59/d67/d69/d3f/d66/l6e 0 2024-02-22T15:50:43.257 INFO:tasks.workunit.client.0.smithi019.stdout:5/756: rmdir d8/dd/d19/d38/d47/ddb/dde/deb 39 2024-02-22T15:50:43.257 INFO:tasks.workunit.client.1.smithi062.stdout:6/642: dwrite d5/dd/d33/d66/f79 [0,4194304] 0 2024-02-22T15:50:43.258 INFO:tasks.workunit.client.1.smithi062.stdout:7/442: dread - d0/d14/d13/f3e zero size 2024-02-22T15:50:43.258 INFO:tasks.workunit.client.1.smithi062.stdout:8/629: mkdir d8/d28/dc8 0 2024-02-22T15:50:43.259 INFO:tasks.workunit.client.1.smithi062.stdout:9/535: write d4/d15/d5e/d63/da0/f8c [485314,102036] 0 2024-02-22T15:50:43.263 INFO:tasks.workunit.client.1.smithi062.stdout:5/466: readlink d0/d3/da/d32/d4f/d72/l8f 0 2024-02-22T15:50:43.264 INFO:tasks.workunit.client.0.smithi019.stdout:6/901: mknod d6/c123 0 2024-02-22T15:50:43.264 INFO:tasks.workunit.client.0.smithi019.stdout:1/779: dread d8/d18/d64/d7f/de3/fed [0,4194304] 0 2024-02-22T15:50:43.265 INFO:tasks.workunit.client.1.smithi062.stdout:5/467: chown d0/d3/c5 1142223 1 2024-02-22T15:50:43.266 INFO:tasks.workunit.client.1.smithi062.stdout:4/554: dwrite d4/d2a/d2d/d9d/fa2 [0,4194304] 0 2024-02-22T15:50:43.266 INFO:tasks.workunit.client.1.smithi062.stdout:9/536: chown d4/d15/d5e/d63 107 1 2024-02-22T15:50:43.267 INFO:tasks.workunit.client.0.smithi019.stdout:8/695: dread d9/d14/d17/d5d/f68 [0,4194304] 0 2024-02-22T15:50:43.271 INFO:tasks.workunit.client.0.smithi019.stdout:6/902: symlink d6/d58/d6a/l124 0 2024-02-22T15:50:43.271 INFO:tasks.workunit.client.1.smithi062.stdout:9/537: stat d4/d1c/d83/d87/c90 0 2024-02-22T15:50:43.272 INFO:tasks.workunit.client.1.smithi062.stdout:8/630: fdatasync d8/d9/f18 0 2024-02-22T15:50:43.273 INFO:tasks.workunit.client.0.smithi019.stdout:9/740: dwrite d2/d1f/d2e/d57/d60/f75 [4194304,4194304] 0 2024-02-22T15:50:43.274 INFO:tasks.workunit.client.0.smithi019.stdout:1/780: mkdir d8/d2a/d75/da0/d10a 0 2024-02-22T15:50:43.274 INFO:tasks.workunit.client.1.smithi062.stdout:1/532: sync 2024-02-22T15:50:43.275 INFO:tasks.workunit.client.1.smithi062.stdout:3/494: sync 2024-02-22T15:50:43.275 INFO:tasks.workunit.client.1.smithi062.stdout:6/643: creat d5/df/d40/d44/d4a/d41/d43/dc6/fd4 x:0 0 0 2024-02-22T15:50:43.275 INFO:tasks.workunit.client.0.smithi019.stdout:0/716: dwrite d4/f56 [0,4194304] 0 2024-02-22T15:50:43.276 INFO:tasks.workunit.client.0.smithi019.stdout:8/696: symlink d9/d14/le9 0 2024-02-22T15:50:43.276 INFO:tasks.workunit.client.1.smithi062.stdout:8/631: symlink d8/d28/d4e/lc9 0 2024-02-22T15:50:43.277 INFO:tasks.workunit.client.1.smithi062.stdout:6/644: readlink d5/lc 0 2024-02-22T15:50:43.278 INFO:tasks.workunit.client.1.smithi062.stdout:4/555: mkdir d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc 0 2024-02-22T15:50:43.278 INFO:tasks.workunit.client.1.smithi062.stdout:6/645: stat d5/f1e 0 2024-02-22T15:50:43.278 INFO:tasks.workunit.client.1.smithi062.stdout:9/538: chown d4/d15/d5f/d75/d84/f5b 25 1 2024-02-22T15:50:43.279 INFO:tasks.workunit.client.1.smithi062.stdout:1/533: chown d1/d6/d14/c3c 1810850 1 2024-02-22T15:50:43.283 INFO:tasks.workunit.client.0.smithi019.stdout:1/781: creat d8/d18/d72/f10b x:0 0 0 2024-02-22T15:50:43.284 INFO:tasks.workunit.client.0.smithi019.stdout:9/741: mknod d2/d1f/d2e/d48/d45/d5a/c10a 0 2024-02-22T15:50:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:0/717: mkdir d4/d23/df5 0 2024-02-22T15:50:43.286 INFO:tasks.workunit.client.0.smithi019.stdout:8/697: mkdir d9/d14/d6f/d7e/d94/daa/dea 0 2024-02-22T15:50:43.287 INFO:tasks.workunit.client.1.smithi062.stdout:3/495: mkdir d5/d26/d2d/da2 0 2024-02-22T15:50:43.289 INFO:tasks.workunit.client.1.smithi062.stdout:5/468: dwrite d0/d3/d2f/f3a [0,4194304] 0 2024-02-22T15:50:43.290 INFO:tasks.workunit.client.1.smithi062.stdout:1/534: truncate d1/d6/d36/d76/fa1 865039 0 2024-02-22T15:50:43.290 INFO:tasks.workunit.client.0.smithi019.stdout:9/742: chown d2/d1f/d2e/d48/d45/f61 29592 1 2024-02-22T15:50:43.291 INFO:tasks.workunit.client.1.smithi062.stdout:7/443: link d0/d14/d13/c27 d0/d6/c8e 0 2024-02-22T15:50:43.291 INFO:tasks.workunit.client.1.smithi062.stdout:3/496: chown d5/d26/f38 55342 1 2024-02-22T15:50:43.292 INFO:tasks.workunit.client.0.smithi019.stdout:0/718: fsync d4/d22/d2c/d68/f6a 0 2024-02-22T15:50:43.292 INFO:tasks.workunit.client.1.smithi062.stdout:8/632: mknod d8/d9/d2c/d49/d5e/cca 0 2024-02-22T15:50:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:6/903: dwrite d6/d31/fc9 [0,4194304] 0 2024-02-22T15:50:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:1/782: chown d8/d2a/d75/cc2 197428138 1 2024-02-22T15:50:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:2/764: dread d1/d17/d4b/f4a [0,4194304] 0 2024-02-22T15:50:43.297 INFO:tasks.workunit.client.1.smithi062.stdout:6/646: mkdir d5/df/dbc/dd0/dd5 0 2024-02-22T15:50:43.297 INFO:tasks.workunit.client.1.smithi062.stdout:5/469: dread - d0/d3/da/d32/d4f/f50 zero size 2024-02-22T15:50:43.299 INFO:tasks.workunit.client.1.smithi062.stdout:3/497: mknod d5/d16/d21/d36/ca3 0 2024-02-22T15:50:43.301 INFO:tasks.workunit.client.0.smithi019.stdout:0/719: write d4/db/d76/d77/faa [2176588,59105] 0 2024-02-22T15:50:43.303 INFO:tasks.workunit.client.0.smithi019.stdout:1/783: rmdir d8/d18/d85 39 2024-02-22T15:50:43.303 INFO:tasks.workunit.client.1.smithi062.stdout:8/633: mkdir d8/d42/dcb 0 2024-02-22T15:50:43.304 INFO:tasks.workunit.client.1.smithi062.stdout:4/556: link d4/d66/c90 d4/d2a/d2d/d46/cbd 0 2024-02-22T15:50:43.305 INFO:tasks.workunit.client.0.smithi019.stdout:9/743: write d2/d7/dc/fb7 [2888251,103568] 0 2024-02-22T15:50:43.307 INFO:tasks.workunit.client.0.smithi019.stdout:6/904: chown d6/d31/c112 105526307 1 2024-02-22T15:50:43.307 INFO:tasks.workunit.client.1.smithi062.stdout:9/539: getdents d4/d15 0 2024-02-22T15:50:43.310 INFO:tasks.workunit.client.1.smithi062.stdout:7/444: dread d0/d14/d19/d30/d37/f53 [0,4194304] 0 2024-02-22T15:50:43.312 INFO:tasks.workunit.client.0.smithi019.stdout:0/720: mknod d4/d22/d3d/de8/cf6 0 2024-02-22T15:50:43.314 INFO:tasks.workunit.client.0.smithi019.stdout:2/765: rename d1/d17/d21/f41 to d1/d17/d21/d71/dd1/f114 0 2024-02-22T15:50:43.315 INFO:tasks.workunit.client.1.smithi062.stdout:9/540: mkdir d4/d28/d50/da3 0 2024-02-22T15:50:43.315 INFO:tasks.workunit.client.1.smithi062.stdout:7/445: fsync d0/d6/d2e/d4a/f4c 0 2024-02-22T15:50:43.315 INFO:tasks.workunit.client.1.smithi062.stdout:4/557: dread d4/d66/f6f [0,4194304] 0 2024-02-22T15:50:43.316 INFO:tasks.workunit.client.0.smithi019.stdout:4/702: sync 2024-02-22T15:50:43.316 INFO:tasks.workunit.client.0.smithi019.stdout:3/599: sync 2024-02-22T15:50:43.317 INFO:tasks.workunit.client.0.smithi019.stdout:5/757: dwrite d8/d11/fb6 [0,4194304] 0 2024-02-22T15:50:43.317 INFO:tasks.workunit.client.0.smithi019.stdout:1/784: mknod d8/d2a/d75/def/c10c 0 2024-02-22T15:50:43.317 INFO:tasks.workunit.client.0.smithi019.stdout:8/698: dwrite d9/d14/d17/f4a [0,4194304] 0 2024-02-22T15:50:43.318 INFO:tasks.workunit.client.1.smithi062.stdout:6/647: dwrite d5/df/d40/d44/d4a/d41/d43/f58 [0,4194304] 0 2024-02-22T15:50:43.324 INFO:tasks.workunit.client.1.smithi062.stdout:9/541: mknod d4/d15/d5f/d6a/ca4 0 2024-02-22T15:50:43.327 INFO:tasks.workunit.client.1.smithi062.stdout:8/634: dwrite d8/d9/d2c/d49/d94/fa2 [0,4194304] 0 2024-02-22T15:50:43.327 INFO:tasks.workunit.client.0.smithi019.stdout:6/905: creat d6/d4b/da9/d67/d8a/f125 x:0 0 0 2024-02-22T15:50:43.328 INFO:tasks.workunit.client.1.smithi062.stdout:7/446: creat d0/d6/d2e/d55/d80/d8a/f8f x:0 0 0 2024-02-22T15:50:43.330 INFO:tasks.workunit.client.0.smithi019.stdout:2/766: creat d1/d17/d67/f115 x:0 0 0 2024-02-22T15:50:43.332 INFO:tasks.workunit.client.0.smithi019.stdout:4/703: dread - d2/d3f/d48/d76/fcb zero size 2024-02-22T15:50:43.333 INFO:tasks.workunit.client.0.smithi019.stdout:0/721: mkdir d4/db/d1b/d40/d44/d75/df7 0 2024-02-22T15:50:43.333 INFO:tasks.workunit.client.1.smithi062.stdout:9/542: creat d4/d15/d31/fa5 x:0 0 0 2024-02-22T15:50:43.333 INFO:tasks.workunit.client.1.smithi062.stdout:5/470: dread d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:43.334 INFO:tasks.workunit.client.0.smithi019.stdout:9/744: dwrite d2/d7/dc/fb7 [0,4194304] 0 2024-02-22T15:50:43.334 INFO:tasks.workunit.client.0.smithi019.stdout:1/785: fdatasync d8/d15/f26 0 2024-02-22T15:50:43.334 INFO:tasks.workunit.client.1.smithi062.stdout:4/558: dwrite d4/d2a/f8b [0,4194304] 0 2024-02-22T15:50:43.335 INFO:tasks.workunit.client.0.smithi019.stdout:3/600: mkdir d9/d26/d2d/dc7 0 2024-02-22T15:50:43.337 INFO:tasks.workunit.client.0.smithi019.stdout:0/722: readlink d4/d67/l7c 0 2024-02-22T15:50:43.337 INFO:tasks.workunit.client.0.smithi019.stdout:3/601: stat d9/d26/l37 0 2024-02-22T15:50:43.337 INFO:tasks.workunit.client.0.smithi019.stdout:8/699: mkdir d9/d14/d6f/d7e/d94/d9a/deb 0 2024-02-22T15:50:43.339 INFO:tasks.workunit.client.0.smithi019.stdout:2/767: creat d1/d17/d1c/d66/db9/dd0/dee/f116 x:0 0 0 2024-02-22T15:50:43.342 INFO:tasks.workunit.client.0.smithi019.stdout:1/786: fdatasync d8/d18/d72/fd3 0 2024-02-22T15:50:43.342 INFO:tasks.workunit.client.1.smithi062.stdout:8/635: creat d8/d9/d2c/d74/fcc x:0 0 0 2024-02-22T15:50:43.343 INFO:tasks.workunit.client.1.smithi062.stdout:5/471: mkdir d0/d3/d2f/d59/d91 0 2024-02-22T15:50:43.343 INFO:tasks.workunit.client.1.smithi062.stdout:4/559: symlink d4/d1c/d30/da7/lbe 0 2024-02-22T15:50:43.343 INFO:tasks.workunit.client.1.smithi062.stdout:8/636: chown d8/d28/d4e 17768386 1 2024-02-22T15:50:43.344 INFO:tasks.workunit.client.1.smithi062.stdout:1/535: write d1/d6/de/d9/fd [2800349,22349] 0 2024-02-22T15:50:43.346 INFO:tasks.workunit.client.1.smithi062.stdout:8/637: stat d8/d9/d2c/d49/d94/da1 0 2024-02-22T15:50:43.348 INFO:tasks.workunit.client.0.smithi019.stdout:3/602: fsync d9/f2f 0 2024-02-22T15:50:43.348 INFO:tasks.workunit.client.0.smithi019.stdout:5/758: truncate d8/d11/d33/fe1 199145 0 2024-02-22T15:50:43.349 INFO:tasks.workunit.client.1.smithi062.stdout:6/648: dwrite d5/df/d32/d8f/f91 [0,4194304] 0 2024-02-22T15:50:43.350 INFO:tasks.workunit.client.0.smithi019.stdout:2/768: symlink d1/d17/d1c/l117 0 2024-02-22T15:50:43.350 INFO:tasks.workunit.client.1.smithi062.stdout:5/472: symlink d0/d3/da/l92 0 2024-02-22T15:50:43.350 INFO:tasks.workunit.client.0.smithi019.stdout:1/787: write d8/d18/f21 [1208100,11720] 0 2024-02-22T15:50:43.355 INFO:tasks.workunit.client.0.smithi019.stdout:8/700: symlink d9/d14/d6f/d7e/d94/daa/dea/lec 0 2024-02-22T15:50:43.355 INFO:tasks.workunit.client.0.smithi019.stdout:5/759: symlink d8/dd/d8e/d2c/l107 0 2024-02-22T15:50:43.356 INFO:tasks.workunit.client.1.smithi062.stdout:5/473: write d0/d3/da/d13/f2a [2237116,64540] 0 2024-02-22T15:50:43.356 INFO:tasks.workunit.client.1.smithi062.stdout:4/560: creat d4/d2a/d2d/d46/fbf x:0 0 0 2024-02-22T15:50:43.356 INFO:tasks.workunit.client.0.smithi019.stdout:2/769: mkdir d1/db4/d118 0 2024-02-22T15:50:43.357 INFO:tasks.workunit.client.0.smithi019.stdout:0/723: dwrite d4/db/d1b/d40/f5d [0,4194304] 0 2024-02-22T15:50:43.357 INFO:tasks.workunit.client.0.smithi019.stdout:3/603: rmdir d9/d22/d58 39 2024-02-22T15:50:43.359 INFO:tasks.workunit.client.1.smithi062.stdout:8/638: unlink d8/d9/f16 0 2024-02-22T15:50:43.359 INFO:tasks.workunit.client.1.smithi062.stdout:5/474: dread d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:43.359 INFO:tasks.workunit.client.0.smithi019.stdout:4/704: link d2/db/d14/d23/d21/f3a d2/d3f/fe4 0 2024-02-22T15:50:43.359 INFO:tasks.workunit.client.1.smithi062.stdout:6/649: symlink d5/dd/ld6 0 2024-02-22T15:50:43.362 INFO:tasks.workunit.client.0.smithi019.stdout:5/760: write d8/dd/d19/d77/d7a/f97 [1798124,34062] 0 2024-02-22T15:50:43.365 INFO:tasks.workunit.client.0.smithi019.stdout:0/724: creat d4/d22/d3d/de8/ff8 x:0 0 0 2024-02-22T15:50:43.366 INFO:tasks.workunit.client.0.smithi019.stdout:5/761: read - d8/dd/d19/d38/d47/fd8 zero size 2024-02-22T15:50:43.367 INFO:tasks.workunit.client.0.smithi019.stdout:3/604: mkdir d9/dc8 0 2024-02-22T15:50:43.368 INFO:tasks.workunit.client.0.smithi019.stdout:2/770: creat d1/d17/f119 x:0 0 0 2024-02-22T15:50:43.370 INFO:tasks.workunit.client.0.smithi019.stdout:0/725: creat d4/db/d24/d80/ff9 x:0 0 0 2024-02-22T15:50:43.371 INFO:tasks.workunit.client.1.smithi062.stdout:2/636: sync 2024-02-22T15:50:43.371 INFO:tasks.workunit.client.0.smithi019.stdout:8/701: dwrite d9/d14/d6f/d7e/d94/d9a/fb7 [0,4194304] 0 2024-02-22T15:50:43.372 INFO:tasks.workunit.client.1.smithi062.stdout:4/561: creat d4/d1c/d2c/d6b/d9b/fc0 x:0 0 0 2024-02-22T15:50:43.373 INFO:tasks.workunit.client.0.smithi019.stdout:3/605: creat d9/d65/da7/fc9 x:0 0 0 2024-02-22T15:50:43.373 INFO:tasks.workunit.client.0.smithi019.stdout:8/702: write d9/d14/fce [796508,81897] 0 2024-02-22T15:50:43.377 INFO:tasks.workunit.client.0.smithi019.stdout:8/703: truncate d9/d14/d6f/d7e/d94/db8/fe2 855778 0 2024-02-22T15:50:43.378 INFO:tasks.workunit.client.1.smithi062.stdout:4/562: write d4/d13/fa6 [444064,19604] 0 2024-02-22T15:50:43.378 INFO:tasks.workunit.client.1.smithi062.stdout:1/536: dread d1/f82 [0,4194304] 0 2024-02-22T15:50:43.379 INFO:tasks.workunit.client.1.smithi062.stdout:2/637: fsync d1/d28/fc2 0 2024-02-22T15:50:43.379 INFO:tasks.workunit.client.0.smithi019.stdout:0/726: rename d4/db/d49/cb3 to d4/d23/d7d/cfa 0 2024-02-22T15:50:43.380 INFO:tasks.workunit.client.0.smithi019.stdout:2/771: mknod d1/d46/ddb/d10d/c11a 0 2024-02-22T15:50:43.382 INFO:tasks.workunit.client.0.smithi019.stdout:0/727: write d4/db/d24/d80/ff9 [639360,114583] 0 2024-02-22T15:50:43.382 INFO:tasks.workunit.client.1.smithi062.stdout:6/650: rename d5/df/d32/d8f/l63 to d5/d8a/ld7 0 2024-02-22T15:50:43.384 INFO:tasks.workunit.client.1.smithi062.stdout:3/498: dread d5/f8a [0,4194304] 0 2024-02-22T15:50:43.385 INFO:tasks.workunit.client.1.smithi062.stdout:1/537: rmdir d1/d6/d7/d8a 39 2024-02-22T15:50:43.386 INFO:tasks.workunit.client.0.smithi019.stdout:4/705: dread d2/db/d14/f5e [0,4194304] 0 2024-02-22T15:50:43.386 INFO:tasks.workunit.client.1.smithi062.stdout:8/639: symlink d8/lcd 0 2024-02-22T15:50:43.386 INFO:tasks.workunit.client.1.smithi062.stdout:2/638: fdatasync d1/d8/d62/d73/dbd/f9f 0 2024-02-22T15:50:43.387 INFO:tasks.workunit.client.0.smithi019.stdout:3/606: write d9/d22/f4f [1926886,90001] 0 2024-02-22T15:50:43.389 INFO:tasks.workunit.client.0.smithi019.stdout:2/772: rename d1/d17/d4b to d1/d17/d21/d71/d60/d11b 0 2024-02-22T15:50:43.390 INFO:tasks.workunit.client.1.smithi062.stdout:3/499: mknod d5/d1a/d3c/d62/ca4 0 2024-02-22T15:50:43.390 INFO:tasks.workunit.client.0.smithi019.stdout:8/704: link d9/d14/d6f/d7e/d94/db8/faf d9/d14/d1f/d4c/fed 0 2024-02-22T15:50:43.391 INFO:tasks.workunit.client.1.smithi062.stdout:1/538: symlink d1/lb3 0 2024-02-22T15:50:43.392 INFO:tasks.workunit.client.1.smithi062.stdout:9/543: dread d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:43.392 INFO:tasks.workunit.client.1.smithi062.stdout:2/639: symlink d1/d47/d32/lce 0 2024-02-22T15:50:43.393 INFO:tasks.workunit.client.0.smithi019.stdout:5/762: getdents d8/dd/d19/d38/d47/ddb 0 2024-02-22T15:50:43.393 INFO:tasks.workunit.client.1.smithi062.stdout:1/539: chown d1/d6/d7/d2b/d56/c59 8854626 1 2024-02-22T15:50:43.394 INFO:tasks.workunit.client.1.smithi062.stdout:8/640: chown d8/d9/d44/d4d/d51/f95 34055 1 2024-02-22T15:50:43.394 INFO:tasks.workunit.client.1.smithi062.stdout:9/544: truncate d4/d1c/f56 623929 0 2024-02-22T15:50:43.395 INFO:tasks.workunit.client.0.smithi019.stdout:2/773: dread - d1/d17/fc8 zero size 2024-02-22T15:50:43.395 INFO:tasks.workunit.client.0.smithi019.stdout:6/906: truncate d6/d98/fa5 2685064 0 2024-02-22T15:50:43.397 INFO:tasks.workunit.client.1.smithi062.stdout:9/545: read d4/f26 [524891,10451] 0 2024-02-22T15:50:43.397 INFO:tasks.workunit.client.1.smithi062.stdout:3/500: creat d5/d16/d21/fa5 x:0 0 0 2024-02-22T15:50:43.398 INFO:tasks.workunit.client.0.smithi019.stdout:0/728: symlink d4/db/d1b/d40/lfb 0 2024-02-22T15:50:43.398 INFO:tasks.workunit.client.0.smithi019.stdout:3/607: dwrite d9/d26/f35 [4194304,4194304] 0 2024-02-22T15:50:43.398 INFO:tasks.workunit.client.0.smithi019.stdout:5/763: read - d8/dd/d19/d77/d7a/ffd zero size 2024-02-22T15:50:43.399 INFO:tasks.workunit.client.1.smithi062.stdout:2/640: creat d1/d47/d39/d5d/fcf x:0 0 0 2024-02-22T15:50:43.400 INFO:tasks.workunit.client.1.smithi062.stdout:1/540: symlink d1/d6/d36/lb4 0 2024-02-22T15:50:43.400 INFO:tasks.workunit.client.0.smithi019.stdout:6/907: write d6/d4b/d8d/f95 [2413589,83246] 0 2024-02-22T15:50:43.401 INFO:tasks.workunit.client.1.smithi062.stdout:2/641: dread - d1/d8/df/f8b zero size 2024-02-22T15:50:43.402 INFO:tasks.workunit.client.1.smithi062.stdout:4/563: dread d4/d1c/d30/d3b/f67 [0,4194304] 0 2024-02-22T15:50:43.403 INFO:tasks.workunit.client.1.smithi062.stdout:8/641: write d8/f53 [3301273,32674] 0 2024-02-22T15:50:43.403 INFO:tasks.workunit.client.0.smithi019.stdout:8/705: mknod d9/d14/d1f/cee 0 2024-02-22T15:50:43.404 INFO:tasks.workunit.client.1.smithi062.stdout:7/447: truncate d0/d6/d5b/d60/f66 1819572 0 2024-02-22T15:50:43.405 INFO:tasks.workunit.client.1.smithi062.stdout:0/629: sync 2024-02-22T15:50:43.406 INFO:tasks.workunit.client.1.smithi062.stdout:9/546: symlink d4/d1c/d83/d87/la6 0 2024-02-22T15:50:43.407 INFO:tasks.workunit.client.0.smithi019.stdout:0/729: symlink d4/db/d24/d80/daf/lfc 0 2024-02-22T15:50:43.410 INFO:tasks.workunit.client.0.smithi019.stdout:5/764: rename d8/d11/fb8 to d8/dd/d19/d38/d54/f108 0 2024-02-22T15:50:43.411 INFO:tasks.workunit.client.0.smithi019.stdout:1/788: dwrite d8/d2a/f2b [0,4194304] 0 2024-02-22T15:50:43.411 INFO:tasks.workunit.client.1.smithi062.stdout:7/448: dread d0/d4f/f59 [0,4194304] 0 2024-02-22T15:50:43.413 INFO:tasks.workunit.client.1.smithi062.stdout:0/630: dread - d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/d97/fbc zero size 2024-02-22T15:50:43.415 INFO:tasks.workunit.client.0.smithi019.stdout:8/706: fdatasync d9/d14/d6f/fde 0 2024-02-22T15:50:43.416 INFO:tasks.workunit.client.1.smithi062.stdout:3/501: mkdir d5/da6 0 2024-02-22T15:50:43.417 INFO:tasks.workunit.client.1.smithi062.stdout:8/642: symlink d8/d9/d2c/d49/d94/lce 0 2024-02-22T15:50:43.418 INFO:tasks.workunit.client.0.smithi019.stdout:3/608: creat d9/d22/d27/d96/fca x:0 0 0 2024-02-22T15:50:43.420 INFO:tasks.workunit.client.0.smithi019.stdout:0/730: creat d4/d9/ffd x:0 0 0 2024-02-22T15:50:43.421 INFO:tasks.workunit.client.0.smithi019.stdout:6/908: symlink d6/d31/d43/d120/l126 0 2024-02-22T15:50:43.422 INFO:tasks.workunit.client.1.smithi062.stdout:0/631: dread - d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/fd2 zero size 2024-02-22T15:50:43.426 INFO:tasks.workunit.client.1.smithi062.stdout:9/547: mkdir d4/d15/d5f/d75/d97/da7 0 2024-02-22T15:50:43.426 INFO:tasks.workunit.client.1.smithi062.stdout:1/541: mknod d1/d6/d7/d8a/cb5 0 2024-02-22T15:50:43.427 INFO:tasks.workunit.client.1.smithi062.stdout:5/475: truncate d0/d3/ff 812911 0 2024-02-22T15:50:43.428 INFO:tasks.workunit.client.0.smithi019.stdout:0/731: creat d4/d32/ffe x:0 0 0 2024-02-22T15:50:43.429 INFO:tasks.workunit.client.1.smithi062.stdout:8/643: dwrite d8/d9/d44/f56 [0,4194304] 0 2024-02-22T15:50:43.429 INFO:tasks.workunit.client.1.smithi062.stdout:6/651: write d5/dd/d33/fa9 [845314,25622] 0 2024-02-22T15:50:43.430 INFO:tasks.workunit.client.1.smithi062.stdout:7/449: creat d0/d6/d2e/d55/d80/d8a/f90 x:0 0 0 2024-02-22T15:50:43.430 INFO:tasks.workunit.client.0.smithi019.stdout:5/765: dwrite d8/d11/fb6 [0,4194304] 0 2024-02-22T15:50:43.431 INFO:tasks.workunit.client.0.smithi019.stdout:6/909: mknod d6/d4b/d6d/d7b/d7f/c127 0 2024-02-22T15:50:43.437 INFO:tasks.workunit.client.1.smithi062.stdout:2/642: creat d1/d47/d32/fd0 x:0 0 0 2024-02-22T15:50:43.437 INFO:tasks.workunit.client.1.smithi062.stdout:9/548: rename d4/d28/f67 to d4/d15/d5f/d6a/fa8 0 2024-02-22T15:50:43.439 INFO:tasks.workunit.client.1.smithi062.stdout:6/652: unlink d5/df/d32/d47/c92 0 2024-02-22T15:50:43.440 INFO:tasks.workunit.client.1.smithi062.stdout:3/502: write d5/d16/d21/f4e [521066,113633] 0 2024-02-22T15:50:43.443 INFO:tasks.workunit.client.1.smithi062.stdout:9/549: write d4/d1c/f53 [423351,110102] 0 2024-02-22T15:50:43.443 INFO:tasks.workunit.client.1.smithi062.stdout:0/632: symlink d1/d3/d4/d1a/d76/d8c/ld4 0 2024-02-22T15:50:43.445 INFO:tasks.workunit.client.0.smithi019.stdout:4/706: truncate d2/d3f/f51 257574 0 2024-02-22T15:50:43.446 INFO:tasks.workunit.client.0.smithi019.stdout:1/789: dwrite d8/d18/f47 [0,4194304] 0 2024-02-22T15:50:43.449 INFO:tasks.workunit.client.1.smithi062.stdout:8/644: dwrite d8/d9/d44/d4d/d51/d55/f92 [0,4194304] 0 2024-02-22T15:50:43.449 INFO:tasks.workunit.client.1.smithi062.stdout:4/564: getdents d4/d2a/d2d/d2e 0 2024-02-22T15:50:43.455 INFO:tasks.workunit.client.1.smithi062.stdout:2/643: creat d1/d8/d62/d73/dbd/d7d/d84/fd1 x:0 0 0 2024-02-22T15:50:43.456 INFO:tasks.workunit.client.1.smithi062.stdout:7/450: rename d0/d4f to d0/d91 0 2024-02-22T15:50:43.457 INFO:tasks.workunit.client.1.smithi062.stdout:3/503: creat d5/d26/d2d/d3f/fa7 x:0 0 0 2024-02-22T15:50:43.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:43 smithi062 ceph-mon[48130]: pgmap v37: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 256 MiB/s wr, 478 op/s 2024-02-22T15:50:43.459 INFO:tasks.workunit.client.1.smithi062.stdout:0/633: rmdir d1/d3/d4/d1a/d76/d8c/d2c/d87 39 2024-02-22T15:50:43.462 INFO:tasks.workunit.client.1.smithi062.stdout:9/550: dwrite d4/f5 [0,4194304] 0 2024-02-22T15:50:43.463 INFO:tasks.workunit.client.1.smithi062.stdout:4/565: mknod d4/d1c/d30/da7/cc1 0 2024-02-22T15:50:43.463 INFO:tasks.workunit.client.0.smithi019.stdout:3/609: dwrite d9/d26/d3f/d59/d66/fb3 [0,4194304] 0 2024-02-22T15:50:43.464 INFO:tasks.workunit.client.0.smithi019.stdout:8/707: dread d9/d88/f8b [0,4194304] 0 2024-02-22T15:50:43.466 INFO:tasks.workunit.client.1.smithi062.stdout:7/451: stat d0/d6/d2e/d4a/d5a/d5e/c7a 0 2024-02-22T15:50:43.468 INFO:tasks.workunit.client.0.smithi019.stdout:2/774: dwrite d1/d17/d21/d51/f72 [0,4194304] 0 2024-02-22T15:50:43.469 INFO:tasks.workunit.client.1.smithi062.stdout:3/504: dread - d5/d16/d21/d32/f8b zero size 2024-02-22T15:50:43.471 INFO:tasks.workunit.client.1.smithi062.stdout:5/476: dread d0/d3/da/d13/d23/d31/f43 [8388608,4194304] 0 2024-02-22T15:50:43.473 INFO:tasks.workunit.client.0.smithi019.stdout:5/766: rename d8/dd/d19/d38/d54/db0/cc2 to d8/dd/d19/d3a/d46/d7e/c109 0 2024-02-22T15:50:43.473 INFO:tasks.workunit.client.1.smithi062.stdout:7/452: chown d0/d14/d28/l3c 990 1 2024-02-22T15:50:43.475 INFO:tasks.workunit.client.1.smithi062.stdout:4/566: dread d4/f11 [0,4194304] 0 2024-02-22T15:50:43.475 INFO:tasks.workunit.client.1.smithi062.stdout:8/645: getdents d8/d26/db5 0 2024-02-22T15:50:43.476 INFO:tasks.workunit.client.0.smithi019.stdout:5/767: truncate d8/dd/d19/d38/d47/fd8 260358 0 2024-02-22T15:50:43.477 INFO:tasks.workunit.client.1.smithi062.stdout:0/634: chown d1/d3/d4/d1a/d76/d8c/d5d/dc3 11088 1 2024-02-22T15:50:43.477 INFO:tasks.workunit.client.1.smithi062.stdout:4/567: readlink d4/d34/l96 0 2024-02-22T15:50:43.478 INFO:tasks.workunit.client.0.smithi019.stdout:6/910: mkdir d6/d3e/de5/d128 0 2024-02-22T15:50:43.480 INFO:tasks.workunit.client.1.smithi062.stdout:6/653: write d5/f7 [9214826,78683] 0 2024-02-22T15:50:43.480 INFO:tasks.workunit.client.1.smithi062.stdout:0/635: readlink d1/d3/l7c 0 2024-02-22T15:50:43.482 INFO:tasks.workunit.client.0.smithi019.stdout:8/708: symlink d9/d14/d17/d1c/lef 0 2024-02-22T15:50:43.483 INFO:tasks.workunit.client.1.smithi062.stdout:2/644: mknod d1/d1e/cd2 0 2024-02-22T15:50:43.485 INFO:tasks.workunit.client.1.smithi062.stdout:7/453: symlink d0/d14/d13/l92 0 2024-02-22T15:50:43.485 INFO:tasks.workunit.client.0.smithi019.stdout:3/610: creat d9/d26/d2d/d53/da0/fcb x:0 0 0 2024-02-22T15:50:43.486 INFO:tasks.workunit.client.0.smithi019.stdout:2/775: stat d1/d17/d1c/d66/db9/dd0/c104 0 2024-02-22T15:50:43.487 INFO:tasks.workunit.client.1.smithi062.stdout:4/568: creat d4/d13/d20/fc2 x:0 0 0 2024-02-22T15:50:43.487 INFO:tasks.workunit.client.1.smithi062.stdout:3/505: mknod d5/da6/ca8 0 2024-02-22T15:50:43.491 INFO:tasks.workunit.client.0.smithi019.stdout:8/709: truncate d9/d14/d6f/d7e/d94/db8/fa1 138796 0 2024-02-22T15:50:43.492 INFO:tasks.workunit.client.1.smithi062.stdout:7/454: chown d0/d14/f29 34577817 1 2024-02-22T15:50:43.494 INFO:tasks.workunit.client.1.smithi062.stdout:6/654: symlink d5/df/d40/dac/db6/ld8 0 2024-02-22T15:50:43.495 INFO:tasks.workunit.client.0.smithi019.stdout:5/768: dwrite d8/dd/f75 [4194304,4194304] 0 2024-02-22T15:50:43.495 INFO:tasks.workunit.client.0.smithi019.stdout:3/611: write d9/d26/d32/f33 [1179866,62900] 0 2024-02-22T15:50:43.495 INFO:tasks.workunit.client.0.smithi019.stdout:8/710: stat d9/d88/f8b 0 2024-02-22T15:50:43.499 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:43 smithi019 ceph-mon[67418]: pgmap v37: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 256 MiB/s wr, 478 op/s 2024-02-22T15:50:43.501 INFO:tasks.workunit.client.0.smithi019.stdout:0/732: dread d4/db/d49/d65/fc7 [0,4194304] 0 2024-02-22T15:50:43.503 INFO:tasks.workunit.client.0.smithi019.stdout:3/612: read f3 [10372628,9287] 0 2024-02-22T15:50:43.503 INFO:tasks.workunit.client.1.smithi062.stdout:3/506: dread d5/d26/d2d/f90 [0,4194304] 0 2024-02-22T15:50:43.504 INFO:tasks.workunit.client.1.smithi062.stdout:4/569: creat d4/d2a/d2d/d2e/d6a/d9e/fc3 x:0 0 0 2024-02-22T15:50:43.504 INFO:tasks.workunit.client.1.smithi062.stdout:9/551: dwrite d4/d15/d5f/d6a/f81 [0,4194304] 0 2024-02-22T15:50:43.505 INFO:tasks.workunit.client.1.smithi062.stdout:5/477: dwrite d0/d3/da/d32/d42/f65 [0,4194304] 0 2024-02-22T15:50:43.505 INFO:tasks.workunit.client.1.smithi062.stdout:2/645: write d1/d8/df/f25 [3660863,20133] 0 2024-02-22T15:50:43.508 INFO:tasks.workunit.client.0.smithi019.stdout:5/769: mkdir d8/dd/d19/d77/dd2/dec/d10a 0 2024-02-22T15:50:43.511 INFO:tasks.workunit.client.0.smithi019.stdout:2/776: link d1/d17/d21/d71/d60/d11b/f79 d1/d17/d1c/d66/db9/dd0/f11c 0 2024-02-22T15:50:43.512 INFO:tasks.workunit.client.1.smithi062.stdout:8/646: dwrite d8/d9/d2c/d49/d94/fb3 [0,4194304] 0 2024-02-22T15:50:43.513 INFO:tasks.workunit.client.1.smithi062.stdout:7/455: truncate d0/d14/d19/f7b 849820 0 2024-02-22T15:50:43.514 INFO:tasks.workunit.client.0.smithi019.stdout:3/613: symlink d9/d26/lcc 0 2024-02-22T15:50:43.516 INFO:tasks.workunit.client.1.smithi062.stdout:8/647: stat d8/d26/l33 0 2024-02-22T15:50:43.517 INFO:tasks.workunit.client.0.smithi019.stdout:8/711: creat d9/d14/d6f/d7e/d94/daa/dd0/ff0 x:0 0 0 2024-02-22T15:50:43.518 INFO:tasks.workunit.client.0.smithi019.stdout:1/790: dwrite d8/d2a/fbf [4194304,4194304] 0 2024-02-22T15:50:43.518 INFO:tasks.workunit.client.1.smithi062.stdout:7/456: write d0/d14/d13/f41 [8858597,44323] 0 2024-02-22T15:50:43.518 INFO:tasks.workunit.client.1.smithi062.stdout:0/636: dwrite d1/d13/f2f [0,4194304] 0 2024-02-22T15:50:43.519 INFO:tasks.workunit.client.0.smithi019.stdout:0/733: truncate d4/d22/d2c/d68/f6a 4325992 0 2024-02-22T15:50:43.520 INFO:tasks.workunit.client.1.smithi062.stdout:5/478: creat d0/d3/da/d13/d23/d4e/d6d/f93 x:0 0 0 2024-02-22T15:50:43.521 INFO:tasks.workunit.client.1.smithi062.stdout:4/570: unlink d4/d2a/d2d/d64/c82 0 2024-02-22T15:50:43.522 INFO:tasks.workunit.client.0.smithi019.stdout:2/777: truncate d1/d17/d21/d71/f10c 722066 0 2024-02-22T15:50:43.523 INFO:tasks.workunit.client.1.smithi062.stdout:6/655: truncate d5/dd/f20 2814976 0 2024-02-22T15:50:43.523 INFO:tasks.workunit.client.0.smithi019.stdout:6/911: dread d6/f28 [0,4194304] 0 2024-02-22T15:50:43.525 INFO:tasks.workunit.client.1.smithi062.stdout:3/507: creat d5/d1a/fa9 x:0 0 0 2024-02-22T15:50:43.528 INFO:tasks.workunit.client.1.smithi062.stdout:4/571: symlink d4/d1c/d30/lc4 0 2024-02-22T15:50:43.529 INFO:tasks.workunit.client.0.smithi019.stdout:3/614: fdatasync d9/d22/d27/f7a 0 2024-02-22T15:50:43.530 INFO:tasks.workunit.client.1.smithi062.stdout:8/648: getdents d8/d42/dcb 0 2024-02-22T15:50:43.530 INFO:tasks.workunit.client.1.smithi062.stdout:6/656: read - d5/dd/d33/f97 zero size 2024-02-22T15:50:43.531 INFO:tasks.workunit.client.1.smithi062.stdout:0/637: mkdir d1/d3/dd5 0 2024-02-22T15:50:43.531 INFO:tasks.workunit.client.0.smithi019.stdout:6/912: fsync d6/d4b/d71/f119 0 2024-02-22T15:50:43.531 INFO:tasks.workunit.client.0.smithi019.stdout:5/770: mkdir d8/dd/d19/d10b 0 2024-02-22T15:50:43.531 INFO:tasks.workunit.client.0.smithi019.stdout:0/734: write d4/db/d1b/d63/db0/da1/dbb/d52/dee/ff4 [788767,130657] 0 2024-02-22T15:50:43.533 INFO:tasks.workunit.client.1.smithi062.stdout:5/479: dwrite d0/f16 [4194304,4194304] 0 2024-02-22T15:50:43.536 INFO:tasks.workunit.client.0.smithi019.stdout:4/707: dwrite d2/db/d14/d23/d33/f74 [0,4194304] 0 2024-02-22T15:50:43.538 INFO:tasks.workunit.client.1.smithi062.stdout:3/508: truncate d5/d26/d2d/fa1 592878 0 2024-02-22T15:50:43.538 INFO:tasks.workunit.client.1.smithi062.stdout:8/649: creat d8/d28/d4e/fcf x:0 0 0 2024-02-22T15:50:43.538 INFO:tasks.workunit.client.1.smithi062.stdout:5/480: dread - d0/d3/f83 zero size 2024-02-22T15:50:43.539 INFO:tasks.workunit.client.0.smithi019.stdout:6/913: write d6/d58/fe3 [778202,70903] 0 2024-02-22T15:50:43.540 INFO:tasks.workunit.client.1.smithi062.stdout:5/481: fdatasync d0/d3/f14 0 2024-02-22T15:50:43.541 INFO:tasks.workunit.client.1.smithi062.stdout:4/572: creat d4/d2a/d2d/d2e/d6a/d9e/da4/db6/fc5 x:0 0 0 2024-02-22T15:50:43.543 INFO:tasks.workunit.client.0.smithi019.stdout:0/735: fdatasync d4/db/d24/d80/fa0 0 2024-02-22T15:50:43.543 INFO:tasks.workunit.client.0.smithi019.stdout:3/615: write d9/d26/d3f/d59/d66/f69 [4147057,51516] 0 2024-02-22T15:50:43.543 INFO:tasks.workunit.client.0.smithi019.stdout:8/712: unlink d9/d14/d17/l2e 0 2024-02-22T15:50:43.544 INFO:tasks.workunit.client.1.smithi062.stdout:0/638: creat d1/d3/d4/d1a/d76/fd6 x:0 0 0 2024-02-22T15:50:43.544 INFO:tasks.workunit.client.1.smithi062.stdout:8/650: mknod d8/d28/d4e/cd0 0 2024-02-22T15:50:43.545 INFO:tasks.workunit.client.0.smithi019.stdout:4/708: write d2/fd [3561310,27641] 0 2024-02-22T15:50:43.545 INFO:tasks.workunit.client.0.smithi019.stdout:1/791: creat d8/d2a/d75/d81/f10d x:0 0 0 2024-02-22T15:50:43.547 INFO:tasks.workunit.client.1.smithi062.stdout:5/482: fsync d0/d3/da/d13/d23/d31/f43 0 2024-02-22T15:50:43.548 INFO:tasks.workunit.client.0.smithi019.stdout:0/736: write d4/db/d1b/d40/ff3 [653201,104366] 0 2024-02-22T15:50:43.549 INFO:tasks.workunit.client.0.smithi019.stdout:0/737: chown d4/d22/d2c/c47 1 1 2024-02-22T15:50:43.549 INFO:tasks.workunit.client.1.smithi062.stdout:8/651: read - d8/d9/d2c/d49/d94/da1/fbc zero size 2024-02-22T15:50:43.550 INFO:tasks.workunit.client.1.smithi062.stdout:3/509: rename d5/d26/c86 to d5/d16/d21/caa 0 2024-02-22T15:50:43.551 INFO:tasks.workunit.client.1.smithi062.stdout:0/639: fsync d1/fc4 0 2024-02-22T15:50:43.551 INFO:tasks.workunit.client.0.smithi019.stdout:1/792: mknod d8/d15/df5/c10e 0 2024-02-22T15:50:43.553 INFO:tasks.workunit.client.1.smithi062.stdout:5/483: creat d0/d3/f94 x:0 0 0 2024-02-22T15:50:43.554 INFO:tasks.workunit.client.1.smithi062.stdout:4/573: dwrite d4/f39 [4194304,4194304] 0 2024-02-22T15:50:43.554 INFO:tasks.workunit.client.1.smithi062.stdout:3/510: write d5/d16/f48 [4379222,72444] 0 2024-02-22T15:50:43.554 INFO:tasks.workunit.client.1.smithi062.stdout:8/652: mkdir d8/d42/dcb/dd1 0 2024-02-22T15:50:43.558 INFO:tasks.workunit.client.1.smithi062.stdout:3/511: fdatasync d5/d16/d21/d36/d5a/d85/f93 0 2024-02-22T15:50:43.558 INFO:tasks.workunit.client.1.smithi062.stdout:5/484: dread - d0/d3/da/d13/d23/f49 zero size 2024-02-22T15:50:43.559 INFO:tasks.workunit.client.1.smithi062.stdout:4/574: write d4/f45 [3906958,93755] 0 2024-02-22T15:50:43.559 INFO:tasks.workunit.client.0.smithi019.stdout:6/914: dwrite d6/d4b/da9/d67/d8a/fb6 [0,4194304] 0 2024-02-22T15:50:43.559 INFO:tasks.workunit.client.0.smithi019.stdout:1/793: read d8/d15/f68 [5113625,30975] 0 2024-02-22T15:50:43.561 INFO:tasks.workunit.client.0.smithi019.stdout:6/915: chown d6/d31/d43/l107 4009551 1 2024-02-22T15:50:43.562 INFO:tasks.workunit.client.0.smithi019.stdout:6/916: stat d6/f10 0 2024-02-22T15:50:43.563 INFO:tasks.workunit.client.1.smithi062.stdout:8/653: symlink d8/d9/d2c/d49/d94/da1/ld2 0 2024-02-22T15:50:43.564 INFO:tasks.workunit.client.1.smithi062.stdout:0/640: mkdir d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8d/dd7 0 2024-02-22T15:50:43.566 INFO:tasks.workunit.client.0.smithi019.stdout:6/917: symlink d6/d4b/da9/l129 0 2024-02-22T15:50:43.567 INFO:tasks.workunit.client.1.smithi062.stdout:8/654: truncate d8/d28/d70/f97 52276 0 2024-02-22T15:50:43.568 INFO:tasks.workunit.client.0.smithi019.stdout:6/918: chown d6/d4b/d71/f119 8 1 2024-02-22T15:50:43.568 INFO:tasks.workunit.client.1.smithi062.stdout:4/575: mkdir d4/d1c/d2c/d6b/d9b/db1/dc6 0 2024-02-22T15:50:43.569 INFO:tasks.workunit.client.1.smithi062.stdout:8/655: fsync d8/f3a 0 2024-02-22T15:50:43.569 INFO:tasks.workunit.client.1.smithi062.stdout:3/512: dwrite d5/f6f [0,4194304] 0 2024-02-22T15:50:43.570 INFO:tasks.workunit.client.0.smithi019.stdout:1/794: dwrite d8/d44/ffd [0,4194304] 0 2024-02-22T15:50:43.573 INFO:tasks.workunit.client.1.smithi062.stdout:4/576: creat d4/d2a/d2d/d9d/fc7 x:0 0 0 2024-02-22T15:50:43.575 INFO:tasks.workunit.client.1.smithi062.stdout:8/656: fdatasync d8/d9/d48/fa4 0 2024-02-22T15:50:43.579 INFO:tasks.workunit.client.0.smithi019.stdout:6/919: dwrite d6/d4b/d6d/da3/ff0 [0,4194304] 0 2024-02-22T15:50:43.579 INFO:tasks.workunit.client.0.smithi019.stdout:8/713: dread d9/f10 [0,4194304] 0 2024-02-22T15:50:43.579 INFO:tasks.workunit.client.1.smithi062.stdout:4/577: stat d4/d2a/d2d/d2e/d6a/d8a/l8d 0 2024-02-22T15:50:43.580 INFO:tasks.workunit.client.0.smithi019.stdout:6/920: chown d6/f28 1 1 2024-02-22T15:50:43.580 INFO:tasks.workunit.client.1.smithi062.stdout:8/657: mknod d8/d9/d44/d7e/cd3 0 2024-02-22T15:50:43.581 INFO:tasks.workunit.client.0.smithi019.stdout:6/921: stat d6/f28 0 2024-02-22T15:50:43.581 INFO:tasks.workunit.client.0.smithi019.stdout:0/738: dread d4/d22/d2c/f9a [0,4194304] 0 2024-02-22T15:50:43.584 INFO:tasks.workunit.client.1.smithi062.stdout:3/513: dwrite d5/d16/d21/d32/d6a/f78 [0,4194304] 0 2024-02-22T15:50:43.584 INFO:tasks.workunit.client.0.smithi019.stdout:6/922: read - d6/d3e/fce zero size 2024-02-22T15:50:43.586 INFO:tasks.workunit.client.0.smithi019.stdout:8/714: rename d9/d14/d1f/d20/fdb to d9/d14/d6f/d7e/d94/d9a/ff1 0 2024-02-22T15:50:43.587 INFO:tasks.workunit.client.0.smithi019.stdout:6/923: chown d6/d4b/d6d/d11f/c101 190 1 2024-02-22T15:50:43.588 INFO:tasks.workunit.client.1.smithi062.stdout:0/641: dread d1/d3/d4/d1a/d76/d8c/f4c [0,4194304] 0 2024-02-22T15:50:43.591 INFO:tasks.workunit.client.1.smithi062.stdout:6/657: dwrite d5/dd/f20 [0,4194304] 0 2024-02-22T15:50:43.592 INFO:tasks.workunit.client.1.smithi062.stdout:3/514: mkdir d5/d26/d2d/da2/dab 0 2024-02-22T15:50:43.592 INFO:tasks.workunit.client.0.smithi019.stdout:0/739: rename d4/d22/d3d/f4f to d4/d23/d7d/fff 0 2024-02-22T15:50:43.593 INFO:tasks.workunit.client.0.smithi019.stdout:6/924: mkdir d6/d4b/d71/dc2/d12a 0 2024-02-22T15:50:43.595 INFO:tasks.workunit.client.1.smithi062.stdout:4/578: dwrite d4/d2a/d2d/d2e/d6a/d8a/f9a [0,4194304] 0 2024-02-22T15:50:43.595 INFO:tasks.workunit.client.0.smithi019.stdout:1/795: dread d8/d15/d5b/dbc/f109 [0,4194304] 0 2024-02-22T15:50:43.595 INFO:tasks.workunit.client.1.smithi062.stdout:0/642: write d1/d3/d4/d1a/d1d/d3b/fd1 [4394444,130443] 0 2024-02-22T15:50:43.596 INFO:tasks.workunit.client.0.smithi019.stdout:6/925: symlink d6/l12b 0 2024-02-22T15:50:43.598 INFO:tasks.workunit.client.1.smithi062.stdout:8/658: dread d8/d9/d2c/d49/d94/fb3 [0,4194304] 0 2024-02-22T15:50:43.601 INFO:tasks.workunit.client.0.smithi019.stdout:6/926: mkdir d6/d4b/d6d/d12c 0 2024-02-22T15:50:43.602 INFO:tasks.workunit.client.1.smithi062.stdout:4/579: truncate d4/d2a/d2d/d2e/d6a/d9e/fc3 851139 0 2024-02-22T15:50:43.603 INFO:tasks.workunit.client.0.smithi019.stdout:6/927: readlink d6/l9f 0 2024-02-22T15:50:43.604 INFO:tasks.workunit.client.0.smithi019.stdout:1/796: creat d8/d18/d64/d7f/d94/df9/f10f x:0 0 0 2024-02-22T15:50:43.606 INFO:tasks.workunit.client.0.smithi019.stdout:0/740: dwrite d4/d22/d2c/f9a [0,4194304] 0 2024-02-22T15:50:43.608 INFO:tasks.workunit.client.0.smithi019.stdout:6/928: symlink d6/d3e/l12d 0 2024-02-22T15:50:43.611 INFO:tasks.workunit.client.0.smithi019.stdout:1/797: creat d8/d2a/d75/d81/ddb/f110 x:0 0 0 2024-02-22T15:50:43.611 INFO:tasks.workunit.client.0.smithi019.stdout:6/929: dread - d6/d4b/d6d/dda/f10d zero size 2024-02-22T15:50:43.612 INFO:tasks.workunit.client.0.smithi019.stdout:0/741: creat d4/db/d24/f100 x:0 0 0 2024-02-22T15:50:43.612 INFO:tasks.workunit.client.1.smithi062.stdout:6/658: dwrite d5/df/d32/d8f/f7f [0,4194304] 0 2024-02-22T15:50:43.612 INFO:tasks.workunit.client.1.smithi062.stdout:4/580: read d4/d2a/d2d/d9d/fa2 [1472166,82296] 0 2024-02-22T15:50:43.615 INFO:tasks.workunit.client.0.smithi019.stdout:6/930: mkdir d6/d4b/da9/d67/d11b/d12e 0 2024-02-22T15:50:43.616 INFO:tasks.workunit.client.0.smithi019.stdout:0/742: symlink d4/db/d1b/d63/db0/da1/l101 0 2024-02-22T15:50:43.619 INFO:tasks.workunit.client.0.smithi019.stdout:1/798: creat d8/d2a/f111 x:0 0 0 2024-02-22T15:50:43.621 INFO:tasks.workunit.client.1.smithi062.stdout:2/646: dread d1/d47/f88 [0,4194304] 0 2024-02-22T15:50:43.621 INFO:tasks.workunit.client.0.smithi019.stdout:0/743: write d4/db/d49/d65/fc7 [4606735,70060] 0 2024-02-22T15:50:43.623 INFO:tasks.workunit.client.0.smithi019.stdout:6/931: fdatasync d6/d98/fa5 0 2024-02-22T15:50:43.623 INFO:tasks.workunit.client.1.smithi062.stdout:6/659: read d5/df/d32/f9f [56459,69421] 0 2024-02-22T15:50:43.623 INFO:tasks.workunit.client.1.smithi062.stdout:2/647: chown d1/d8/df/f78 23210477 1 2024-02-22T15:50:43.627 INFO:tasks.workunit.client.0.smithi019.stdout:6/932: truncate d6/d4b/d71/f119 583079 0 2024-02-22T15:50:43.627 INFO:tasks.workunit.client.0.smithi019.stdout:1/799: creat d8/d2a/d75/f112 x:0 0 0 2024-02-22T15:50:43.628 INFO:tasks.workunit.client.1.smithi062.stdout:8/659: mknod d8/cd4 0 2024-02-22T15:50:43.629 INFO:tasks.workunit.client.0.smithi019.stdout:1/800: mknod d8/d18/d64/d7f/de3/c113 0 2024-02-22T15:50:43.631 INFO:tasks.workunit.client.0.smithi019.stdout:6/933: creat d6/d4b/d6d/d7b/f12f x:0 0 0 2024-02-22T15:50:43.632 INFO:tasks.workunit.client.0.smithi019.stdout:9/745: sync 2024-02-22T15:50:43.633 INFO:tasks.workunit.client.0.smithi019.stdout:9/746: read - d2/d7/dc/dd6/fe3 zero size 2024-02-22T15:50:43.633 INFO:tasks.workunit.client.0.smithi019.stdout:6/934: mkdir d6/d4b/d71/dc2/d130 0 2024-02-22T15:50:43.635 INFO:tasks.workunit.client.0.smithi019.stdout:6/935: truncate d6/d31/d43/f11e 917388 0 2024-02-22T15:50:43.635 INFO:tasks.workunit.client.0.smithi019.stdout:9/747: write d2/f4 [7071942,70861] 0 2024-02-22T15:50:43.636 INFO:tasks.workunit.client.0.smithi019.stdout:1/801: rename d8/d15/da4/fe4 to d8/d2a/d75/d81/d8e/f114 0 2024-02-22T15:50:43.638 INFO:tasks.workunit.client.1.smithi062.stdout:4/581: dwrite d4/d2a/d2d/d9d/fc7 [0,4194304] 0 2024-02-22T15:50:43.638 INFO:tasks.workunit.client.0.smithi019.stdout:1/802: read d8/d18/fb8 [259761,59518] 0 2024-02-22T15:50:43.639 INFO:tasks.workunit.client.0.smithi019.stdout:6/936: mknod d6/d4b/d71/d88/ded/c131 0 2024-02-22T15:50:43.640 INFO:tasks.workunit.client.0.smithi019.stdout:1/803: write d8/d18/d64/d7f/ffc [826807,74540] 0 2024-02-22T15:50:43.640 INFO:tasks.workunit.client.0.smithi019.stdout:9/748: read d2/d7/dc/d27/dbc/f3f [3228504,10837] 0 2024-02-22T15:50:43.641 INFO:tasks.workunit.client.0.smithi019.stdout:6/937: chown d6/d31/d43/fb0 257711 1 2024-02-22T15:50:43.643 INFO:tasks.workunit.client.0.smithi019.stdout:9/749: fsync d2/d1f/d2e/d57/d60/f75 0 2024-02-22T15:50:43.643 INFO:tasks.workunit.client.0.smithi019.stdout:1/804: symlink d8/d2a/db3/l115 0 2024-02-22T15:50:43.644 INFO:tasks.workunit.client.1.smithi062.stdout:2/648: truncate d1/d8/df/f85 1508450 0 2024-02-22T15:50:43.644 INFO:tasks.workunit.client.0.smithi019.stdout:9/750: fdatasync d2/dc8/fea 0 2024-02-22T15:50:43.645 INFO:tasks.workunit.client.0.smithi019.stdout:6/938: mkdir d6/d4b/d71/d88/ded/d121/d132 0 2024-02-22T15:50:43.645 INFO:tasks.workunit.client.0.smithi019.stdout:1/805: write d8/fe1 [4114956,82043] 0 2024-02-22T15:50:43.648 INFO:tasks.workunit.client.0.smithi019.stdout:1/806: write d8/fb4 [4183434,24433] 0 2024-02-22T15:50:43.649 INFO:tasks.workunit.client.1.smithi062.stdout:8/660: truncate d8/d86/f96 938501 0 2024-02-22T15:50:43.651 INFO:tasks.workunit.client.0.smithi019.stdout:6/939: mkdir d6/d3e/d133 0 2024-02-22T15:50:43.651 INFO:tasks.workunit.client.0.smithi019.stdout:6/940: dread - d6/f86 zero size 2024-02-22T15:50:43.652 INFO:tasks.workunit.client.0.smithi019.stdout:9/751: creat d2/d1f/d2e/d7d/f10b x:0 0 0 2024-02-22T15:50:43.654 INFO:tasks.workunit.client.0.smithi019.stdout:9/752: chown d2/d1f/d2e/d48/d45/d5a/daf/db6/lb9 89 1 2024-02-22T15:50:43.654 INFO:tasks.workunit.client.1.smithi062.stdout:3/515: dwrite d5/d16/d21/f22 [0,4194304] 0 2024-02-22T15:50:43.655 INFO:tasks.workunit.client.0.smithi019.stdout:1/807: mkdir d8/d2a/d116 0 2024-02-22T15:50:43.656 INFO:tasks.workunit.client.1.smithi062.stdout:6/660: dwrite d5/dd/d33/d66/f79 [0,4194304] 0 2024-02-22T15:50:43.657 INFO:tasks.workunit.client.0.smithi019.stdout:6/941: creat d6/d4b/da9/d67/f134 x:0 0 0 2024-02-22T15:50:43.658 INFO:tasks.workunit.client.1.smithi062.stdout:4/582: mknod d4/df/daf/cc8 0 2024-02-22T15:50:43.658 INFO:tasks.workunit.client.1.smithi062.stdout:3/516: dread - d5/d16/d88/f91 zero size 2024-02-22T15:50:43.661 INFO:tasks.workunit.client.0.smithi019.stdout:9/753: read d2/f4f [3966313,68796] 0 2024-02-22T15:50:43.661 INFO:tasks.workunit.client.0.smithi019.stdout:1/808: unlink d8/d15/d34/d9e/lab 0 2024-02-22T15:50:43.663 INFO:tasks.workunit.client.1.smithi062.stdout:2/649: rename d1/d47/d32/d52/d80/fa8 to d1/d47/d32/fd3 0 2024-02-22T15:50:43.665 INFO:tasks.workunit.client.0.smithi019.stdout:1/809: readlink d8/d2a/d75/ld0 0 2024-02-22T15:50:43.667 INFO:tasks.workunit.client.0.smithi019.stdout:6/942: dwrite d6/d4b/d8d/d111/f11a [0,4194304] 0 2024-02-22T15:50:43.668 INFO:tasks.workunit.client.1.smithi062.stdout:6/661: truncate d5/df/d40/d44/d78/f5c 993018 0 2024-02-22T15:50:43.668 INFO:tasks.workunit.client.1.smithi062.stdout:3/517: creat d5/d16/d21/d32/d6a/fac x:0 0 0 2024-02-22T15:50:43.672 INFO:tasks.workunit.client.0.smithi019.stdout:1/810: creat d8/d2a/d75/def/df8/f117 x:0 0 0 2024-02-22T15:50:43.672 INFO:tasks.workunit.client.0.smithi019.stdout:9/754: symlink d2/d7/d52/l10c 0 2024-02-22T15:50:43.676 INFO:tasks.workunit.client.1.smithi062.stdout:3/518: unlink d5/d16/c1f 0 2024-02-22T15:50:43.676 INFO:tasks.workunit.client.0.smithi019.stdout:1/811: unlink d8/d2a/d75/d81/d8e/lbb 0 2024-02-22T15:50:43.677 INFO:tasks.workunit.client.1.smithi062.stdout:2/650: mkdir d1/d8/d62/d73/dbd/d65/dd4 0 2024-02-22T15:50:43.680 INFO:tasks.workunit.client.1.smithi062.stdout:6/662: dwrite d5/df/d40/f53 [4194304,4194304] 0 2024-02-22T15:50:43.682 INFO:tasks.workunit.client.1.smithi062.stdout:2/651: creat d1/d8/d62/d73/dbd/d7d/d84/fd5 x:0 0 0 2024-02-22T15:50:43.682 INFO:tasks.workunit.client.1.smithi062.stdout:2/652: stat d1/d8/d62/d73/da6/la9 0 2024-02-22T15:50:43.683 INFO:tasks.workunit.client.1.smithi062.stdout:6/663: chown d5/df/d32/f9f 32325 1 2024-02-22T15:50:43.689 INFO:tasks.workunit.client.1.smithi062.stdout:2/653: rename d1/d1e/d38/d76/c8a to d1/d28/cd6 0 2024-02-22T15:50:43.693 INFO:tasks.workunit.client.1.smithi062.stdout:6/664: dwrite d5/df/d40/d44/f82 [0,4194304] 0 2024-02-22T15:50:43.694 INFO:tasks.workunit.client.1.smithi062.stdout:2/654: creat d1/d8/d62/d73/dbd/d7d/d84/fd7 x:0 0 0 2024-02-22T15:50:43.695 INFO:tasks.workunit.client.1.smithi062.stdout:6/665: fdatasync d5/df/d40/d44/d4a/faa 0 2024-02-22T15:50:43.699 INFO:tasks.workunit.client.1.smithi062.stdout:0/643: dread d1/d3/d4/d1a/d76/d8c/d5d/dc3/f49 [0,4194304] 0 2024-02-22T15:50:43.700 INFO:tasks.workunit.client.1.smithi062.stdout:6/666: readlink d5/dd/d33/d66/la7 0 2024-02-22T15:50:43.700 INFO:tasks.workunit.client.1.smithi062.stdout:6/667: chown d5/d6e/dc1 40300461 1 2024-02-22T15:50:43.701 INFO:tasks.workunit.client.1.smithi062.stdout:6/668: fdatasync d5/dd/d33/f46 0 2024-02-22T15:50:43.705 INFO:tasks.workunit.client.1.smithi062.stdout:6/669: mknod d5/df/d40/db3/cd9 0 2024-02-22T15:50:43.706 INFO:tasks.workunit.client.1.smithi062.stdout:6/670: stat d5/df/d40/d44/d76/d56 0 2024-02-22T15:50:43.707 INFO:tasks.workunit.client.1.smithi062.stdout:6/671: symlink d5/df/dbc/dd0/lda 0 2024-02-22T15:50:43.708 INFO:tasks.workunit.client.0.smithi019.stdout:3/616: write d9/d26/f55 [114175,52432] 0 2024-02-22T15:50:43.710 INFO:tasks.workunit.client.0.smithi019.stdout:3/617: dread - d9/d14/f9b zero size 2024-02-22T15:50:43.711 INFO:tasks.workunit.client.0.smithi019.stdout:2/778: dwrite d1/d17/d21/d71/d60/d11b/f96 [0,4194304] 0 2024-02-22T15:50:43.711 INFO:tasks.workunit.client.0.smithi019.stdout:3/618: readlink d9/d26/d3f/d59/d66/l6f 0 2024-02-22T15:50:43.712 INFO:tasks.workunit.client.0.smithi019.stdout:3/619: dread - d9/d26/fb5 zero size 2024-02-22T15:50:43.715 INFO:tasks.workunit.client.0.smithi019.stdout:3/620: fdatasync d9/d26/d3f/f8c 0 2024-02-22T15:50:43.717 INFO:tasks.workunit.client.0.smithi019.stdout:0/744: dread d4/db/d1b/d40/ff3 [0,4194304] 0 2024-02-22T15:50:43.717 INFO:tasks.workunit.client.0.smithi019.stdout:2/779: fsync d1/d17/d21/d71/d60/d11b/f4a 0 2024-02-22T15:50:43.719 INFO:tasks.workunit.client.0.smithi019.stdout:0/745: chown d4/db/d1b/d40/f5d 24 1 2024-02-22T15:50:43.720 INFO:tasks.workunit.client.1.smithi062.stdout:8/661: truncate d8/d42/f7c 260343 0 2024-02-22T15:50:43.722 INFO:tasks.workunit.client.0.smithi019.stdout:3/621: rmdir d9/d22/d27/d6d 39 2024-02-22T15:50:43.722 INFO:tasks.workunit.client.0.smithi019.stdout:2/780: truncate d1/d17/d21/f105 653396 0 2024-02-22T15:50:43.725 INFO:tasks.workunit.client.0.smithi019.stdout:0/746: symlink d4/l102 0 2024-02-22T15:50:43.725 INFO:tasks.workunit.client.0.smithi019.stdout:4/709: dwrite d2/db/d14/d23/f65 [0,4194304] 0 2024-02-22T15:50:43.731 INFO:tasks.workunit.client.1.smithi062.stdout:4/583: dwrite d4/d1c/d2c/d6b/d70/f76 [0,4194304] 0 2024-02-22T15:50:43.734 INFO:tasks.workunit.client.1.smithi062.stdout:2/655: truncate d1/d1e/d38/f4f 2463940 0 2024-02-22T15:50:43.734 INFO:tasks.workunit.client.1.smithi062.stdout:4/584: readlink d4/df/l60 0 2024-02-22T15:50:43.738 INFO:tasks.workunit.client.1.smithi062.stdout:2/656: creat d1/d1e/d38/fd8 x:0 0 0 2024-02-22T15:50:43.739 INFO:tasks.workunit.client.0.smithi019.stdout:0/747: rename d4/db/d1b/d40/d44/d75/db7/c4c to d4/db/d1b/d63/db0/da1/dbb/c103 0 2024-02-22T15:50:43.739 INFO:tasks.workunit.client.1.smithi062.stdout:4/585: chown d4/d1c/d30 410 1 2024-02-22T15:50:43.740 INFO:tasks.workunit.client.0.smithi019.stdout:2/781: link d1/d17/d21/d71/d60/d11b/f92 d1/d17/d21/d71/d60/d11b/f11d 0 2024-02-22T15:50:43.742 INFO:tasks.workunit.client.0.smithi019.stdout:2/782: chown d1/d17/d21/d71/dd1/d106/f10a 385607 1 2024-02-22T15:50:43.742 INFO:tasks.workunit.client.0.smithi019.stdout:0/748: write d4/db/d1b/d40/d44/d75/db7/f90 [4516179,119018] 0 2024-02-22T15:50:43.742 INFO:tasks.workunit.client.1.smithi062.stdout:4/586: getdents d4/d2a/d7d 0 2024-02-22T15:50:43.743 INFO:tasks.workunit.client.1.smithi062.stdout:4/587: dread - d4/d2a/d2d/d46/fbf zero size 2024-02-22T15:50:43.744 INFO:tasks.workunit.client.0.smithi019.stdout:2/783: write d1/d17/d67/f100 [211566,115001] 0 2024-02-22T15:50:43.744 INFO:tasks.workunit.client.1.smithi062.stdout:4/588: stat l0 0 2024-02-22T15:50:43.745 INFO:tasks.workunit.client.1.smithi062.stdout:4/589: write d4/d1c/d30/f65 [2830295,85925] 0 2024-02-22T15:50:43.746 INFO:tasks.workunit.client.1.smithi062.stdout:2/657: link d1/d1e/c22 d1/d8/d62/d73/dbd/cd9 0 2024-02-22T15:50:43.750 INFO:tasks.workunit.client.0.smithi019.stdout:0/749: dwrite d4/db/d1b/d40/f5d [0,4194304] 0 2024-02-22T15:50:43.751 INFO:tasks.workunit.client.0.smithi019.stdout:6/943: truncate d6/d3e/f4e 293852 0 2024-02-22T15:50:43.753 INFO:tasks.workunit.client.0.smithi019.stdout:9/755: truncate d2/d1f/d2e/d48/d36/f83 480936 0 2024-02-22T15:50:43.754 INFO:tasks.workunit.client.1.smithi062.stdout:2/658: mkdir d1/d1e/dda 0 2024-02-22T15:50:43.758 INFO:tasks.workunit.client.1.smithi062.stdout:4/590: read d4/d1c/d30/f65 [3902692,125095] 0 2024-02-22T15:50:43.761 INFO:tasks.workunit.client.1.smithi062.stdout:8/662: dwrite d8/d26/f41 [0,4194304] 0 2024-02-22T15:50:43.762 INFO:tasks.workunit.client.0.smithi019.stdout:9/756: truncate d2/d1f/d2e/d48/fc4 1331873 0 2024-02-22T15:50:43.763 INFO:tasks.workunit.client.1.smithi062.stdout:4/591: rename d4/d1c/d30/c89 to d4/d1c/d30/da5/cc9 0 2024-02-22T15:50:43.763 INFO:tasks.workunit.client.1.smithi062.stdout:8/663: fsync d8/d9/d44/d4d/fb0 0 2024-02-22T15:50:43.768 INFO:tasks.workunit.client.0.smithi019.stdout:1/812: dwrite d8/d15/fba [0,4194304] 0 2024-02-22T15:50:43.768 INFO:tasks.workunit.client.0.smithi019.stdout:0/750: dwrite d4/db/d1b/d63/db0/da1/dbb/d7/fcb [4194304,4194304] 0 2024-02-22T15:50:43.770 INFO:tasks.workunit.client.1.smithi062.stdout:4/592: creat d4/d1c/d2c/d6b/fca x:0 0 0 2024-02-22T15:50:43.770 INFO:tasks.workunit.client.1.smithi062.stdout:8/664: mkdir d8/d9/d2c/dd5 0 2024-02-22T15:50:43.770 INFO:tasks.workunit.client.0.smithi019.stdout:4/710: dwrite d2/db/d14/d23/d21/d35/f38 [0,4194304] 0 2024-02-22T15:50:43.772 INFO:tasks.workunit.client.0.smithi019.stdout:9/757: truncate d2/dc8/fa2 357764 0 2024-02-22T15:50:43.773 INFO:tasks.workunit.client.1.smithi062.stdout:4/593: dread d4/d2a/f8b [0,4194304] 0 2024-02-22T15:50:43.773 INFO:tasks.workunit.client.0.smithi019.stdout:1/813: rmdir d8/d15/df5 39 2024-02-22T15:50:43.774 INFO:tasks.workunit.client.0.smithi019.stdout:6/944: dwrite d6/d4b/d6d/d7b/f62 [0,4194304] 0 2024-02-22T15:50:43.774 INFO:tasks.workunit.client.1.smithi062.stdout:4/594: write d4/d2a/d2d/d2e/f97 [360788,31048] 0 2024-02-22T15:50:43.775 INFO:tasks.workunit.client.1.smithi062.stdout:4/595: readlink d4/d1c/d30/lc4 0 2024-02-22T15:50:43.775 INFO:tasks.workunit.client.0.smithi019.stdout:0/751: mknod d4/db/d49/dc2/dc1/c104 0 2024-02-22T15:50:43.778 INFO:tasks.workunit.client.0.smithi019.stdout:6/945: creat d6/d31/d43/d120/f135 x:0 0 0 2024-02-22T15:50:43.779 INFO:tasks.workunit.client.0.smithi019.stdout:0/752: mkdir d4/db/d24/d105 0 2024-02-22T15:50:43.780 INFO:tasks.workunit.client.1.smithi062.stdout:4/596: chown d4/d2a/d2d/d64/ca9 20617 1 2024-02-22T15:50:43.781 INFO:tasks.workunit.client.0.smithi019.stdout:6/946: write d6/d4b/d6d/d7b/f62 [531141,107460] 0 2024-02-22T15:50:43.783 INFO:tasks.workunit.client.0.smithi019.stdout:9/758: rename d2/d7/dc/d27/l2d to d2/d1f/d2e/d57/d60/d8c/l10d 0 2024-02-22T15:50:43.784 INFO:tasks.workunit.client.0.smithi019.stdout:6/947: mknod d6/d4b/d8d/c136 0 2024-02-22T15:50:43.786 INFO:tasks.workunit.client.0.smithi019.stdout:9/759: symlink d2/d7/dc/d27/d68/l10e 0 2024-02-22T15:50:43.788 INFO:tasks.workunit.client.0.smithi019.stdout:0/753: link d4/d22/cb4 d4/db/d76/dc6/c106 0 2024-02-22T15:50:43.791 INFO:tasks.workunit.client.0.smithi019.stdout:6/948: rename d6/d58/d6a/cb3 to d6/d4b/da9/d67/d8a/c137 0 2024-02-22T15:50:43.791 INFO:tasks.workunit.client.0.smithi019.stdout:6/949: readlink d6/lc5 0 2024-02-22T15:50:43.792 INFO:tasks.workunit.client.0.smithi019.stdout:0/754: creat d4/d22/d2c/d68/d6d/f107 x:0 0 0 2024-02-22T15:50:43.792 INFO:tasks.workunit.client.0.smithi019.stdout:6/950: chown d6/d58/d6a 34 1 2024-02-22T15:50:43.794 INFO:tasks.workunit.client.0.smithi019.stdout:0/755: write d4/d22/d3d/fd9 [785000,73429] 0 2024-02-22T15:50:43.797 INFO:tasks.workunit.client.0.smithi019.stdout:0/756: truncate d4/d22/d2c/fdd 427153 0 2024-02-22T15:50:43.797 INFO:tasks.workunit.client.0.smithi019.stdout:6/951: creat d6/d3e/de5/d128/f138 x:0 0 0 2024-02-22T15:50:43.800 INFO:tasks.workunit.client.0.smithi019.stdout:0/757: creat d4/d32/f108 x:0 0 0 2024-02-22T15:50:43.801 INFO:tasks.workunit.client.0.smithi019.stdout:6/952: symlink d6/d4b/da9/d67/d8a/d90/l139 0 2024-02-22T15:50:43.803 INFO:tasks.workunit.client.0.smithi019.stdout:0/758: mkdir d4/d22/d2c/d109 0 2024-02-22T15:50:43.805 INFO:tasks.workunit.client.1.smithi062.stdout:2/659: dwrite d1/d1e/f94 [0,4194304] 0 2024-02-22T15:50:43.805 INFO:tasks.workunit.client.0.smithi019.stdout:6/953: unlink d6/d4b/da9/d67/d8a/fcd 0 2024-02-22T15:50:43.806 INFO:tasks.workunit.client.0.smithi019.stdout:6/954: write d6/d4b/f103 [3047092,47485] 0 2024-02-22T15:50:43.810 INFO:tasks.workunit.client.0.smithi019.stdout:2/784: write d1/d17/d1c/f34 [4543404,22803] 0 2024-02-22T15:50:43.811 INFO:tasks.workunit.client.1.smithi062.stdout:4/597: dread d4/df/f14 [0,4194304] 0 2024-02-22T15:50:43.812 INFO:tasks.workunit.client.1.smithi062.stdout:4/598: fsync d4/d1c/f27 0 2024-02-22T15:50:43.812 INFO:tasks.workunit.client.0.smithi019.stdout:9/760: dread d2/d7/f1b [4194304,4194304] 0 2024-02-22T15:50:43.814 INFO:tasks.workunit.client.1.smithi062.stdout:4/599: mkdir d4/d1c/d30/da7/dcb 0 2024-02-22T15:50:43.814 INFO:tasks.workunit.client.0.smithi019.stdout:6/955: creat d6/d4b/d71/d88/ded/d121/d132/f13a x:0 0 0 2024-02-22T15:50:43.815 INFO:tasks.workunit.client.1.smithi062.stdout:4/600: fsync d4/d2a/d2d/d46/fad 0 2024-02-22T15:50:43.815 INFO:tasks.workunit.client.0.smithi019.stdout:9/761: rmdir d2/d7/dc/d27 39 2024-02-22T15:50:43.816 INFO:tasks.workunit.client.1.smithi062.stdout:4/601: stat d4/d2a/d2d/d2e/d6a/d9e/fc3 0 2024-02-22T15:50:43.817 INFO:tasks.workunit.client.1.smithi062.stdout:4/602: write d4/d1c/d2c/d6b/d70/f93 [608960,39946] 0 2024-02-22T15:50:43.818 INFO:tasks.workunit.client.1.smithi062.stdout:4/603: fdatasync d4/d1c/d2c/d6b/fca 0 2024-02-22T15:50:43.820 INFO:tasks.workunit.client.1.smithi062.stdout:4/604: fsync d4/d2a/fb8 0 2024-02-22T15:50:43.822 INFO:tasks.workunit.client.0.smithi019.stdout:9/762: readlink d2/d1f/d2e/d48/d89/l38 0 2024-02-22T15:50:43.822 INFO:tasks.workunit.client.0.smithi019.stdout:6/956: dread d6/d58/fe3 [0,4194304] 0 2024-02-22T15:50:43.825 INFO:tasks.workunit.client.0.smithi019.stdout:2/785: dwrite d1/d17/d21/d51/fed [0,4194304] 0 2024-02-22T15:50:43.827 INFO:tasks.workunit.client.1.smithi062.stdout:4/605: fdatasync d4/d1c/d30/f65 0 2024-02-22T15:50:43.830 INFO:tasks.workunit.client.0.smithi019.stdout:9/763: write d2/d7/dc/d27/dbc/fd8 [479186,92620] 0 2024-02-22T15:50:43.830 INFO:tasks.workunit.client.1.smithi062.stdout:4/606: mknod d4/d1c/d2c/d6b/d9b/db1/ccc 0 2024-02-22T15:50:43.830 INFO:tasks.workunit.client.0.smithi019.stdout:4/711: dread d2/db/d14/f18 [0,4194304] 0 2024-02-22T15:50:43.831 INFO:tasks.workunit.client.1.smithi062.stdout:4/607: readlink d4/d13/d20/l24 0 2024-02-22T15:50:43.832 INFO:tasks.workunit.client.0.smithi019.stdout:2/786: unlink d1/d46/f98 0 2024-02-22T15:50:43.833 INFO:tasks.workunit.client.1.smithi062.stdout:2/660: dread d1/d1e/faf [0,4194304] 0 2024-02-22T15:50:43.834 INFO:tasks.workunit.client.1.smithi062.stdout:8/665: dread d8/d42/f7c [0,4194304] 0 2024-02-22T15:50:43.834 INFO:tasks.workunit.client.0.smithi019.stdout:4/712: write d2/db/d14/d23/d21/f2d [9219439,18525] 0 2024-02-22T15:50:43.834 INFO:tasks.workunit.client.0.smithi019.stdout:2/787: fdatasync d1/d17/d38/fd9 0 2024-02-22T15:50:43.835 INFO:tasks.workunit.client.1.smithi062.stdout:4/608: chown d4/d1c/l1d 45 1 2024-02-22T15:50:43.837 INFO:tasks.workunit.client.1.smithi062.stdout:8/666: read - d8/d9/d44/f9b zero size 2024-02-22T15:50:43.837 INFO:tasks.workunit.client.0.smithi019.stdout:6/957: dwrite d6/d31/fc9 [0,4194304] 0 2024-02-22T15:50:43.837 INFO:tasks.workunit.client.0.smithi019.stdout:4/713: readlink d2/d3f/d48/d76/lb4 0 2024-02-22T15:50:43.838 INFO:tasks.workunit.client.1.smithi062.stdout:4/609: symlink d4/d1c/dac/lcd 0 2024-02-22T15:50:43.839 INFO:tasks.workunit.client.0.smithi019.stdout:2/788: rmdir d1/d17 39 2024-02-22T15:50:43.839 INFO:tasks.workunit.client.0.smithi019.stdout:4/714: chown d2/db/d14/d23/d21/d42/dc4 5 1 2024-02-22T15:50:43.840 INFO:tasks.workunit.client.0.smithi019.stdout:9/764: dwrite d2/d1f/db5/fde [4194304,4194304] 0 2024-02-22T15:50:43.841 INFO:tasks.workunit.client.1.smithi062.stdout:2/661: link d1/d47/d32/l60 d1/d8/d62/d73/dbd/d7d/ldb 0 2024-02-22T15:50:43.843 INFO:tasks.workunit.client.0.smithi019.stdout:2/789: read - d1/d17/d1c/d66/db9/dd0/dee/f116 zero size 2024-02-22T15:50:43.844 INFO:tasks.workunit.client.0.smithi019.stdout:4/715: write d2/db/d14/d23/d21/d35/f38 [260994,122292] 0 2024-02-22T15:50:43.844 INFO:tasks.workunit.client.0.smithi019.stdout:2/790: chown d1/db4/fd2 3 1 2024-02-22T15:50:43.845 INFO:tasks.workunit.client.1.smithi062.stdout:2/662: write d1/d8/d62/d73/da6/fb1 [592476,18486] 0 2024-02-22T15:50:43.845 INFO:tasks.workunit.client.0.smithi019.stdout:9/765: mknod d2/d1f/d2e/d57/d60/d8c/d66/c10f 0 2024-02-22T15:50:43.847 INFO:tasks.workunit.client.0.smithi019.stdout:9/766: read - d2/d1f/d2e/d48/d36/ff1 zero size 2024-02-22T15:50:43.847 INFO:tasks.workunit.client.0.smithi019.stdout:4/716: creat d2/db/d14/d23/d67/fe5 x:0 0 0 2024-02-22T15:50:43.848 INFO:tasks.workunit.client.1.smithi062.stdout:4/610: dwrite d4/d1c/d2c/d6b/d70/f76 [4194304,4194304] 0 2024-02-22T15:50:43.852 INFO:tasks.workunit.client.1.smithi062.stdout:2/663: dread - d1/d8/d62/d73/dbd/f9f zero size 2024-02-22T15:50:43.853 INFO:tasks.workunit.client.0.smithi019.stdout:2/791: mkdir d1/d17/d67/dfa/d11e 0 2024-02-22T15:50:43.853 INFO:tasks.workunit.client.0.smithi019.stdout:9/767: mkdir d2/d7/dc/d27/dbc/d110 0 2024-02-22T15:50:43.853 INFO:tasks.workunit.client.0.smithi019.stdout:2/792: truncate d1/d46/f95 633198 0 2024-02-22T15:50:43.853 INFO:tasks.workunit.client.0.smithi019.stdout:4/717: creat d2/db/d14/d23/d21/d42/dc4/dd5/fe6 x:0 0 0 2024-02-22T15:50:43.854 INFO:tasks.workunit.client.1.smithi062.stdout:8/667: dwrite d8/d9/d44/d4d/d51/d55/f77 [0,4194304] 0 2024-02-22T15:50:43.854 INFO:tasks.workunit.client.1.smithi062.stdout:2/664: chown d1/d8/d62/d73/dbd/dbb/lc3 10576069 1 2024-02-22T15:50:43.854 INFO:tasks.workunit.client.1.smithi062.stdout:8/668: fdatasync d8/f24 0 2024-02-22T15:50:43.855 INFO:tasks.workunit.client.1.smithi062.stdout:4/611: write d4/d1c/d30/d3b/f67 [500089,5701] 0 2024-02-22T15:50:43.856 INFO:tasks.workunit.client.0.smithi019.stdout:4/718: symlink d2/db/d14/d23/d9e/dac/le7 0 2024-02-22T15:50:43.858 INFO:tasks.workunit.client.1.smithi062.stdout:4/612: creat d4/d1c/dac/fce x:0 0 0 2024-02-22T15:50:43.859 INFO:tasks.workunit.client.0.smithi019.stdout:4/719: creat d2/db/d14/d23/d21/d35/d3d/fe8 x:0 0 0 2024-02-22T15:50:43.859 INFO:tasks.workunit.client.0.smithi019.stdout:9/768: link d2/dec/l3d d2/d7/dc/d97/l111 0 2024-02-22T15:50:43.859 INFO:tasks.workunit.client.1.smithi062.stdout:8/669: mknod d8/d9/d44/d4d/d51/cd6 0 2024-02-22T15:50:43.861 INFO:tasks.workunit.client.1.smithi062.stdout:8/670: creat d8/d26/fd7 x:0 0 0 2024-02-22T15:50:43.863 INFO:tasks.workunit.client.1.smithi062.stdout:8/671: write d8/d28/d4e/f8e [388045,89273] 0 2024-02-22T15:50:43.864 INFO:tasks.workunit.client.1.smithi062.stdout:8/672: write d8/d28/fc6 [111080,103103] 0 2024-02-22T15:50:43.866 INFO:tasks.workunit.client.1.smithi062.stdout:4/613: rename d4/d2a/d2d/d46/la8 to d4/d2a/d2d/d2e/d6a/lcf 0 2024-02-22T15:50:43.866 INFO:tasks.workunit.client.1.smithi062.stdout:8/673: fdatasync d8/d9/d2c/d49/d94/da1/fbc 0 2024-02-22T15:50:43.870 INFO:tasks.workunit.client.1.smithi062.stdout:4/614: truncate d4/d2a/d2d/d2e/d6a/d9e/da4/db6/fc5 160455 0 2024-02-22T15:50:43.870 INFO:tasks.workunit.client.1.smithi062.stdout:4/615: chown d4/df 251638 1 2024-02-22T15:50:43.871 INFO:tasks.workunit.client.1.smithi062.stdout:8/674: rename d8/d9/d44/d4d/c72 to d8/d28/d70/cd8 0 2024-02-22T15:50:43.872 INFO:tasks.workunit.client.1.smithi062.stdout:8/675: write d8/d26/f6c [164503,5415] 0 2024-02-22T15:50:43.874 INFO:tasks.workunit.client.1.smithi062.stdout:2/665: dread d1/d47/d32/d52/dba/fc8 [0,4194304] 0 2024-02-22T15:50:43.876 INFO:tasks.workunit.client.1.smithi062.stdout:2/666: readlink d1/d47/d39/d5d/la7 0 2024-02-22T15:50:43.878 INFO:tasks.workunit.client.0.smithi019.stdout:5/771: sync 2024-02-22T15:50:43.878 INFO:tasks.workunit.client.1.smithi062.stdout:8/676: rename d8/d9/d44/f5c to d8/d28/fd9 0 2024-02-22T15:50:43.879 INFO:tasks.workunit.client.0.smithi019.stdout:9/769: dread d2/d7/dc/d27/d68/f6b [0,4194304] 0 2024-02-22T15:50:43.879 INFO:tasks.workunit.client.0.smithi019.stdout:5/772: fdatasync d8/d11/fa8 0 2024-02-22T15:50:43.880 INFO:tasks.workunit.client.1.smithi062.stdout:2/667: creat d1/d47/d32/d52/d80/fdc x:0 0 0 2024-02-22T15:50:43.880 INFO:tasks.workunit.client.0.smithi019.stdout:9/770: chown d2/d1f/d2e/d48/d36/f71 111478973 1 2024-02-22T15:50:43.881 INFO:tasks.workunit.client.0.smithi019.stdout:9/771: readlink d2/d1f/d2e/d48/d36/lad 0 2024-02-22T15:50:43.881 INFO:tasks.workunit.client.0.smithi019.stdout:5/773: mknod d8/dd/d19/c10c 0 2024-02-22T15:50:43.882 INFO:tasks.workunit.client.0.smithi019.stdout:5/774: fdatasync d8/d11/fbb 0 2024-02-22T15:50:43.884 INFO:tasks.workunit.client.1.smithi062.stdout:2/668: rename d1/f2c to d1/d8/fdd 0 2024-02-22T15:50:43.886 INFO:tasks.workunit.client.1.smithi062.stdout:8/677: dread d8/d9/f27 [0,4194304] 0 2024-02-22T15:50:43.886 INFO:tasks.workunit.client.0.smithi019.stdout:9/772: link d2/c25 d2/d1f/d2e/d48/d45/d5a/daf/db6/c112 0 2024-02-22T15:50:43.887 INFO:tasks.workunit.client.0.smithi019.stdout:9/773: dread - d2/d7/dc/d27/dbc/f105 zero size 2024-02-22T15:50:43.887 INFO:tasks.workunit.client.1.smithi062.stdout:8/678: symlink d8/d9/d2c/lda 0 2024-02-22T15:50:43.888 INFO:tasks.workunit.client.0.smithi019.stdout:9/774: stat d2/d1f/d2e/d48/d45/d5a/c9b 0 2024-02-22T15:50:43.888 INFO:tasks.workunit.client.1.smithi062.stdout:8/679: stat d8/d26/fb4 0 2024-02-22T15:50:43.891 INFO:tasks.workunit.client.0.smithi019.stdout:9/775: link d2/d1f/d2e/d48/d36/f71 d2/d7/dc/d27/dbc/dcc/f113 0 2024-02-22T15:50:43.893 INFO:tasks.workunit.client.1.smithi062.stdout:8/680: dwrite d8/d9/d44/d4d/d51/d55/f77 [0,4194304] 0 2024-02-22T15:50:43.899 INFO:tasks.workunit.client.1.smithi062.stdout:8/681: dwrite d8/d9/d2c/d74/fcc [0,4194304] 0 2024-02-22T15:50:43.911 INFO:tasks.workunit.client.1.smithi062.stdout:8/682: dread d8/d9/d2c/d49/f6b [0,4194304] 0 2024-02-22T15:50:43.915 INFO:tasks.workunit.client.1.smithi062.stdout:8/683: dread d8/d9/d44/f56 [0,4194304] 0 2024-02-22T15:50:43.917 INFO:tasks.workunit.client.1.smithi062.stdout:8/684: creat d8/d9/d44/d58/fdb x:0 0 0 2024-02-22T15:50:43.919 INFO:tasks.workunit.client.1.smithi062.stdout:8/685: creat d8/d9/d44/d4d/d8a/fdc x:0 0 0 2024-02-22T15:50:43.919 INFO:tasks.workunit.client.1.smithi062.stdout:8/686: chown d8/d9/d2c/dd5 0 1 2024-02-22T15:50:43.920 INFO:tasks.workunit.client.1.smithi062.stdout:8/687: stat d8/d42/d87/fa7 0 2024-02-22T15:50:43.921 INFO:tasks.workunit.client.1.smithi062.stdout:8/688: write d8/d9/d48/fbf [854408,3813] 0 2024-02-22T15:50:43.923 INFO:tasks.workunit.client.1.smithi062.stdout:8/689: unlink d8/d42/d87/fa7 0 2024-02-22T15:50:43.925 INFO:tasks.workunit.client.1.smithi062.stdout:8/690: symlink d8/d9/d2c/d49/d94/da1/ldd 0 2024-02-22T15:50:43.925 INFO:tasks.workunit.client.1.smithi062.stdout:8/691: truncate d8/d28/d4e/faa 1103233 0 2024-02-22T15:50:43.927 INFO:tasks.workunit.client.1.smithi062.stdout:8/692: fsync d8/d9/f71 0 2024-02-22T15:50:43.927 INFO:tasks.workunit.client.1.smithi062.stdout:8/693: chown d8/f12 1930 1 2024-02-22T15:50:43.928 INFO:tasks.workunit.client.1.smithi062.stdout:8/694: chown d8/d9/d44 919912405 1 2024-02-22T15:50:43.930 INFO:tasks.workunit.client.1.smithi062.stdout:8/695: creat d8/d9/d44/fde x:0 0 0 2024-02-22T15:50:43.934 INFO:tasks.workunit.client.1.smithi062.stdout:8/696: link d8/d9/d44/d4d/d51/f68 d8/d86/db6/fdf 0 2024-02-22T15:50:43.939 INFO:tasks.workunit.client.1.smithi062.stdout:8/697: dread d8/d9/f37 [0,4194304] 0 2024-02-22T15:50:43.946 INFO:tasks.workunit.client.1.smithi062.stdout:8/698: getdents d8/d42/dcb 0 2024-02-22T15:50:43.948 INFO:tasks.workunit.client.0.smithi019.stdout:1/814: write d8/d44/f45 [3136561,6976] 0 2024-02-22T15:50:43.950 INFO:tasks.workunit.client.0.smithi019.stdout:1/815: dread - d8/d23/faf zero size 2024-02-22T15:50:43.974 INFO:tasks.workunit.client.0.smithi019.stdout:0/759: dwrite d4/d9/f13 [0,4194304] 0 2024-02-22T15:50:43.977 INFO:tasks.workunit.client.0.smithi019.stdout:0/760: write d4/db/d1b/d40/d44/d75/f82 [783675,44823] 0 2024-02-22T15:50:43.983 INFO:tasks.workunit.client.0.smithi019.stdout:6/958: dwrite d6/d4b/f52 [0,4194304] 0 2024-02-22T15:50:43.983 INFO:tasks.workunit.client.0.smithi019.stdout:1/816: dread d8/d44/d5d/fc4 [0,4194304] 0 2024-02-22T15:50:43.986 INFO:tasks.workunit.client.0.smithi019.stdout:6/959: rmdir d6/d4b/d8d/d111 39 2024-02-22T15:50:43.987 INFO:tasks.workunit.client.0.smithi019.stdout:1/817: creat d8/d2a/d75/d81/d8e/f118 x:0 0 0 2024-02-22T15:50:43.988 INFO:tasks.workunit.client.0.smithi019.stdout:6/960: unlink d6/d98/fa5 0 2024-02-22T15:50:43.992 INFO:tasks.workunit.client.0.smithi019.stdout:1/818: read d8/d15/f17 [2631636,115549] 0 2024-02-22T15:50:43.992 INFO:tasks.workunit.client.1.smithi062.stdout:1/542: sync 2024-02-22T15:50:43.993 INFO:tasks.workunit.client.0.smithi019.stdout:1/819: write d8/d18/d64/d7f/ffc [1037582,123530] 0 2024-02-22T15:50:43.994 INFO:tasks.workunit.client.0.smithi019.stdout:1/820: readlink d8/d23/d33/laa 0 2024-02-22T15:50:43.996 INFO:tasks.workunit.client.1.smithi062.stdout:1/543: mkdir d1/d6/d7/d2b/d47/d63/db6 0 2024-02-22T15:50:43.998 INFO:tasks.workunit.client.0.smithi019.stdout:1/821: rename d8/d23/d33/laa to d8/d15/d34/d9e/l119 0 2024-02-22T15:50:43.999 INFO:tasks.workunit.client.0.smithi019.stdout:1/822: creat d8/d18/d72/f11a x:0 0 0 2024-02-22T15:50:44.000 INFO:tasks.workunit.client.0.smithi019.stdout:1/823: chown d8/fb4 122547820 1 2024-02-22T15:50:44.000 INFO:tasks.workunit.client.1.smithi062.stdout:1/544: link d1/d6/d7/d8a/l9c d1/d6/d36/d37/lb7 0 2024-02-22T15:50:44.002 INFO:tasks.workunit.client.0.smithi019.stdout:1/824: creat d8/d15/da4/f11b x:0 0 0 2024-02-22T15:50:44.004 INFO:tasks.workunit.client.0.smithi019.stdout:1/825: rmdir d8/d2a/db3 39 2024-02-22T15:50:44.004 INFO:tasks.workunit.client.1.smithi062.stdout:1/545: dread - d1/d6/d7/d2b/d49/d74/f85 zero size 2024-02-22T15:50:44.006 INFO:tasks.workunit.client.0.smithi019.stdout:1/826: mkdir d8/d18/d64/d7f/d94/df9/d11c 0 2024-02-22T15:50:44.007 INFO:tasks.workunit.client.1.smithi062.stdout:2/669: read - d1/d47/d32/fd0 zero size 2024-02-22T15:50:44.008 INFO:tasks.workunit.client.1.smithi062.stdout:4/616: dwrite d4/df/f14 [4194304,4194304] 0 2024-02-22T15:50:44.011 INFO:tasks.workunit.client.0.smithi019.stdout:1/827: rename d8/d2a/d97/ldd to d8/d15/df5/l11d 0 2024-02-22T15:50:44.012 INFO:tasks.workunit.client.0.smithi019.stdout:9/776: dread d2/f6 [8388608,4194304] 0 2024-02-22T15:50:44.014 INFO:tasks.workunit.client.0.smithi019.stdout:9/777: write d2/dec/d79/fb2 [5371210,45870] 0 2024-02-22T15:50:44.014 INFO:tasks.workunit.client.0.smithi019.stdout:4/720: truncate d2/db/d14/d23/d21/f9b 2939936 0 2024-02-22T15:50:44.015 INFO:tasks.workunit.client.1.smithi062.stdout:1/546: creat d1/d6/d7/d2b/d49/da9/fb8 x:0 0 0 2024-02-22T15:50:44.018 INFO:tasks.workunit.client.1.smithi062.stdout:2/670: mknod d1/d28/cde 0 2024-02-22T15:50:44.018 INFO:tasks.workunit.client.1.smithi062.stdout:1/547: rmdir d1/d6/d7/d4a/d65 39 2024-02-22T15:50:44.019 INFO:tasks.workunit.client.0.smithi019.stdout:4/721: mkdir d2/d3f/d48/de9 0 2024-02-22T15:50:44.021 INFO:tasks.workunit.client.1.smithi062.stdout:2/671: mknod d1/d47/d39/d5d/cdf 0 2024-02-22T15:50:44.022 INFO:tasks.workunit.client.1.smithi062.stdout:2/672: readlink d1/l6b 0 2024-02-22T15:50:44.022 INFO:tasks.workunit.client.1.smithi062.stdout:8/699: dwrite d8/d9/d2c/d49/f6b [0,4194304] 0 2024-02-22T15:50:44.023 INFO:tasks.workunit.client.0.smithi019.stdout:8/715: sync 2024-02-22T15:50:44.023 INFO:tasks.workunit.client.0.smithi019.stdout:9/778: rmdir d2/d1f/d2e/d7d 39 2024-02-22T15:50:44.024 INFO:tasks.workunit.client.1.smithi062.stdout:1/548: mkdir d1/d6/d7/d2b/d94/db9 0 2024-02-22T15:50:44.026 INFO:tasks.workunit.client.0.smithi019.stdout:1/828: getdents d8/d18/d64 0 2024-02-22T15:50:44.027 INFO:tasks.workunit.client.1.smithi062.stdout:8/700: dread d8/d9/d44/d4d/d51/d55/f6d [0,4194304] 0 2024-02-22T15:50:44.030 INFO:tasks.workunit.client.0.smithi019.stdout:9/779: symlink d2/d7/d52/dc0/dcd/l114 0 2024-02-22T15:50:44.032 INFO:tasks.workunit.client.1.smithi062.stdout:1/549: rmdir d1/d6/d7/d8a 39 2024-02-22T15:50:44.032 INFO:tasks.workunit.client.1.smithi062.stdout:8/701: dread d8/d28/d70/f97 [0,4194304] 0 2024-02-22T15:50:44.032 INFO:tasks.workunit.client.1.smithi062.stdout:2/673: rename d1/c2e to d1/d8/d62/d73/dbd/dbb/ce0 0 2024-02-22T15:50:44.033 INFO:tasks.workunit.client.1.smithi062.stdout:1/550: rmdir d1/d6/d7/d2b/d96 39 2024-02-22T15:50:44.035 INFO:tasks.workunit.client.0.smithi019.stdout:4/722: fdatasync d2/db/f25 0 2024-02-22T15:50:44.038 INFO:tasks.workunit.client.0.smithi019.stdout:1/829: rmdir d8/d15/d34/d9e 39 2024-02-22T15:50:44.039 INFO:tasks.workunit.client.0.smithi019.stdout:0/761: dread d4/db/d1b/d63/db0/da1/dbb/fe [0,4194304] 0 2024-02-22T15:50:44.040 INFO:tasks.workunit.client.0.smithi019.stdout:9/780: truncate d2/d7/dc/d27/d68/f6b 632920 0 2024-02-22T15:50:44.040 INFO:tasks.workunit.client.1.smithi062.stdout:4/617: truncate d4/d2a/d2d/d64/fb5 599253 0 2024-02-22T15:50:44.041 INFO:tasks.workunit.client.0.smithi019.stdout:8/716: dwrite d9/d14/d17/d3d/fbb [0,4194304] 0 2024-02-22T15:50:44.043 INFO:tasks.workunit.client.1.smithi062.stdout:4/618: getdents d4/d2a/d2d/d2e/d6a/d7b 0 2024-02-22T15:50:44.046 INFO:tasks.workunit.client.0.smithi019.stdout:8/717: fsync d9/d14/d6f/fe3 0 2024-02-22T15:50:44.047 INFO:tasks.workunit.client.1.smithi062.stdout:2/674: chown d1/d8 22388880 1 2024-02-22T15:50:44.048 INFO:tasks.workunit.client.1.smithi062.stdout:4/619: mkdir d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd0 0 2024-02-22T15:50:44.051 INFO:tasks.workunit.client.0.smithi019.stdout:4/723: mkdir d2/db/dea 0 2024-02-22T15:50:44.051 INFO:tasks.workunit.client.0.smithi019.stdout:1/830: dwrite d8/d18/d72/f10b [0,4194304] 0 2024-02-22T15:50:44.051 INFO:tasks.workunit.client.0.smithi019.stdout:0/762: mknod d4/d22/d3d/c10a 0 2024-02-22T15:50:44.052 INFO:tasks.workunit.client.1.smithi062.stdout:2/675: dread - d1/d8/d62/d73/dbd/d7d/d84/fd7 zero size 2024-02-22T15:50:44.053 INFO:tasks.workunit.client.0.smithi019.stdout:1/831: chown d8/d2a/f5e 1 1 2024-02-22T15:50:44.056 INFO:tasks.workunit.client.1.smithi062.stdout:8/702: dwrite d8/d9/fc4 [0,4194304] 0 2024-02-22T15:50:44.056 INFO:tasks.workunit.client.0.smithi019.stdout:4/724: chown d2/db/d14/d23/f36 443340710 1 2024-02-22T15:50:44.058 INFO:tasks.workunit.client.1.smithi062.stdout:4/620: getdents d4/d2a/d2d/d9d 0 2024-02-22T15:50:44.058 INFO:tasks.workunit.client.0.smithi019.stdout:0/763: mknod d4/d22/d2c/d68/d6d/c10b 0 2024-02-22T15:50:44.060 INFO:tasks.workunit.client.0.smithi019.stdout:0/764: write d4/d23/d7d/f88 [374305,115393] 0 2024-02-22T15:50:44.061 INFO:tasks.workunit.client.0.smithi019.stdout:9/781: creat d2/d1f/d2e/d48/d36/f115 x:0 0 0 2024-02-22T15:50:44.062 INFO:tasks.workunit.client.1.smithi062.stdout:2/676: unlink d1/d1e/d38/d76/f92 0 2024-02-22T15:50:44.063 INFO:tasks.workunit.client.1.smithi062.stdout:4/621: creat d4/d2a/fd1 x:0 0 0 2024-02-22T15:50:44.064 INFO:tasks.workunit.client.0.smithi019.stdout:8/718: creat d9/d14/d17/ff2 x:0 0 0 2024-02-22T15:50:44.066 INFO:tasks.workunit.client.1.smithi062.stdout:4/622: read d4/d1c/f23 [430704,4294] 0 2024-02-22T15:50:44.066 INFO:tasks.workunit.client.0.smithi019.stdout:0/765: write d4/db/d1b/d63/db0/da1/dbb/d52/f62 [3634587,18912] 0 2024-02-22T15:50:44.067 INFO:tasks.workunit.client.0.smithi019.stdout:1/832: getdents d8/d2a/d97 0 2024-02-22T15:50:44.069 INFO:tasks.workunit.client.1.smithi062.stdout:2/677: dread - d1/d47/d32/d52/d80/fdc zero size 2024-02-22T15:50:44.069 INFO:tasks.workunit.client.1.smithi062.stdout:8/703: dwrite d8/d9/d2c/d49/f6b [0,4194304] 0 2024-02-22T15:50:44.071 INFO:tasks.workunit.client.1.smithi062.stdout:4/623: chown d4/d1c/d30/da7/cab 54196134 1 2024-02-22T15:50:44.075 INFO:tasks.workunit.client.0.smithi019.stdout:0/766: dwrite d4/db/d1b/d63/db0/da1/dbb/f39 [0,4194304] 0 2024-02-22T15:50:44.075 INFO:tasks.workunit.client.1.smithi062.stdout:4/624: chown d4/d2a/d2d/d2e/d6a/d8a 211074931 1 2024-02-22T15:50:44.075 INFO:tasks.workunit.client.1.smithi062.stdout:2/678: mkdir d1/de1 0 2024-02-22T15:50:44.075 INFO:tasks.workunit.client.0.smithi019.stdout:0/767: chown d4/d32/f9c 32323 1 2024-02-22T15:50:44.076 INFO:tasks.workunit.client.1.smithi062.stdout:4/625: chown d4/df/l16 4 1 2024-02-22T15:50:44.076 INFO:tasks.workunit.client.1.smithi062.stdout:8/704: unlink d8/d9/c1a 0 2024-02-22T15:50:44.078 INFO:tasks.workunit.client.0.smithi019.stdout:0/768: write d4/db/d24/f100 [1199,117215] 0 2024-02-22T15:50:44.080 INFO:tasks.workunit.client.1.smithi062.stdout:2/679: mknod d1/d47/d39/d5d/ce2 0 2024-02-22T15:50:44.081 INFO:tasks.workunit.client.1.smithi062.stdout:8/705: mkdir d8/d9/d44/d58/de0 0 2024-02-22T15:50:44.082 INFO:tasks.workunit.client.1.smithi062.stdout:4/626: dread d4/f11 [0,4194304] 0 2024-02-22T15:50:44.082 INFO:tasks.workunit.client.0.smithi019.stdout:0/769: symlink d4/db/d1b/d40/l10c 0 2024-02-22T15:50:44.085 INFO:tasks.workunit.client.1.smithi062.stdout:8/706: write d8/d9/d44/f9b [881260,35374] 0 2024-02-22T15:50:44.086 INFO:tasks.workunit.client.1.smithi062.stdout:8/707: readlink d8/d9/d2c/lda 0 2024-02-22T15:50:44.087 INFO:tasks.workunit.client.0.smithi019.stdout:4/725: truncate d2/db/d54/d8c/f96 1544168 0 2024-02-22T15:50:44.088 INFO:tasks.workunit.client.0.smithi019.stdout:1/833: write d8/d15/d34/f71 [226182,22847] 0 2024-02-22T15:50:44.089 INFO:tasks.workunit.client.1.smithi062.stdout:8/708: mknod d8/d26/ce1 0 2024-02-22T15:50:44.090 INFO:tasks.workunit.client.0.smithi019.stdout:1/834: dread - d8/d2a/d75/d81/f10d zero size 2024-02-22T15:50:44.091 INFO:tasks.workunit.client.1.smithi062.stdout:8/709: chown d8/f53 399393146 1 2024-02-22T15:50:44.092 INFO:tasks.workunit.client.1.smithi062.stdout:2/680: dwrite d1/d8/df/f25 [0,4194304] 0 2024-02-22T15:50:44.092 INFO:tasks.workunit.client.1.smithi062.stdout:8/710: chown d8/f3a 407 1 2024-02-22T15:50:44.093 INFO:tasks.workunit.client.0.smithi019.stdout:1/835: fsync d8/d18/d64/f83 0 2024-02-22T15:50:44.093 INFO:tasks.workunit.client.1.smithi062.stdout:4/627: rename d4/d2a/c47 to d4/d1c/cd2 0 2024-02-22T15:50:44.095 INFO:tasks.workunit.client.0.smithi019.stdout:4/726: dwrite d2/db/d92/f91 [0,4194304] 0 2024-02-22T15:50:44.096 INFO:tasks.workunit.client.0.smithi019.stdout:1/836: write d8/d18/d64/fad [510538,94988] 0 2024-02-22T15:50:44.098 INFO:tasks.workunit.client.0.smithi019.stdout:8/719: dread d9/f12 [0,4194304] 0 2024-02-22T15:50:44.099 INFO:tasks.workunit.client.1.smithi062.stdout:4/628: unlink d4/d1c/d30/d37/l42 0 2024-02-22T15:50:44.100 INFO:tasks.workunit.client.1.smithi062.stdout:8/711: creat d8/d9/d2c/d49/d5e/fe2 x:0 0 0 2024-02-22T15:50:44.100 INFO:tasks.workunit.client.0.smithi019.stdout:4/727: creat d2/db/d59/feb x:0 0 0 2024-02-22T15:50:44.102 INFO:tasks.workunit.client.1.smithi062.stdout:2/681: mknod d1/d28/ce3 0 2024-02-22T15:50:44.103 INFO:tasks.workunit.client.0.smithi019.stdout:0/770: dwrite d4/d22/dcc/fd1 [0,4194304] 0 2024-02-22T15:50:44.104 INFO:tasks.workunit.client.1.smithi062.stdout:8/712: write d8/d9/f5f [52425,53056] 0 2024-02-22T15:50:44.106 INFO:tasks.workunit.client.0.smithi019.stdout:8/720: symlink d9/d14/d17/d3d/d3f/lf3 0 2024-02-22T15:50:44.107 INFO:tasks.workunit.client.1.smithi062.stdout:2/682: mknod d1/d8/df/ce4 0 2024-02-22T15:50:44.107 INFO:tasks.workunit.client.1.smithi062.stdout:4/629: rename d4/df/l60 to d4/d1c/d2c/d6b/d9b/ld3 0 2024-02-22T15:50:44.110 INFO:tasks.workunit.client.1.smithi062.stdout:2/683: chown d1/d8/d62/d73/dbd/d7d/d84/fd1 0 1 2024-02-22T15:50:44.111 INFO:tasks.workunit.client.0.smithi019.stdout:1/837: getdents d8/d2a/d75/da0 0 2024-02-22T15:50:44.112 INFO:tasks.workunit.client.1.smithi062.stdout:4/630: fdatasync d4/df/fbb 0 2024-02-22T15:50:44.113 INFO:tasks.workunit.client.0.smithi019.stdout:8/721: mknod d9/d14/d6f/d7e/d94/d9a/cf4 0 2024-02-22T15:50:44.114 INFO:tasks.workunit.client.0.smithi019.stdout:4/728: write d2/db/f25 [1367711,106896] 0 2024-02-22T15:50:44.115 INFO:tasks.workunit.client.1.smithi062.stdout:8/713: rename d8/d9/d44/d4d/d51/d55/f92 to d8/d86/db6/fe3 0 2024-02-22T15:50:44.116 INFO:tasks.workunit.client.1.smithi062.stdout:2/684: symlink d1/d47/d39/d5d/le5 0 2024-02-22T15:50:44.116 INFO:tasks.workunit.client.0.smithi019.stdout:1/838: chown d8/d18/d72/fd3 6482418 1 2024-02-22T15:50:44.117 INFO:tasks.workunit.client.0.smithi019.stdout:8/722: chown d9/d14/d17/f4a 18854 1 2024-02-22T15:50:44.117 INFO:tasks.workunit.client.0.smithi019.stdout:4/729: symlink d2/d3f/lec 0 2024-02-22T15:50:44.117 INFO:tasks.workunit.client.1.smithi062.stdout:2/685: chown d1/d8/f15 10 1 2024-02-22T15:50:44.119 INFO:tasks.workunit.client.1.smithi062.stdout:4/631: creat d4/d1c/d30/da7/dcb/fd4 x:0 0 0 2024-02-22T15:50:44.119 INFO:tasks.workunit.client.0.smithi019.stdout:8/723: dread - d9/d14/d17/d3d/d3f/f9e zero size 2024-02-22T15:50:44.120 INFO:tasks.workunit.client.1.smithi062.stdout:8/714: mknod d8/d26/db5/ce4 0 2024-02-22T15:50:44.121 INFO:tasks.workunit.client.1.smithi062.stdout:8/715: dread - d8/d28/d70/fc5 zero size 2024-02-22T15:50:44.122 INFO:tasks.workunit.client.0.smithi019.stdout:1/839: rmdir d8/d2a/d75/da0/d10a 0 2024-02-22T15:50:44.122 INFO:tasks.workunit.client.1.smithi062.stdout:8/716: stat d8/d28/d70/fc5 0 2024-02-22T15:50:44.122 INFO:tasks.workunit.client.1.smithi062.stdout:4/632: truncate d4/d1c/d30/f65 370798 0 2024-02-22T15:50:44.123 INFO:tasks.workunit.client.0.smithi019.stdout:4/730: mkdir d2/db/dea/ded 0 2024-02-22T15:50:44.123 INFO:tasks.workunit.client.1.smithi062.stdout:8/717: write d8/d9/f71 [1039432,23449] 0 2024-02-22T15:50:44.125 INFO:tasks.workunit.client.0.smithi019.stdout:8/724: symlink d9/d14/d17/d1c/lf5 0 2024-02-22T15:50:44.125 INFO:tasks.workunit.client.0.smithi019.stdout:4/731: fdatasync d2/db/d14/f5e 0 2024-02-22T15:50:44.126 INFO:tasks.workunit.client.1.smithi062.stdout:2/686: fsync d1/d8/df/f25 0 2024-02-22T15:50:44.126 INFO:tasks.workunit.client.1.smithi062.stdout:8/718: chown d8/d9/l66 5 1 2024-02-22T15:50:44.127 INFO:tasks.workunit.client.1.smithi062.stdout:4/633: mkdir d4/d1c/d2c/d6b/d9b/db1/dc6/dd5 0 2024-02-22T15:50:44.129 INFO:tasks.workunit.client.0.smithi019.stdout:8/725: dread d9/d14/d6f/d7e/d94/daa/fb6 [0,4194304] 0 2024-02-22T15:50:44.129 INFO:tasks.workunit.client.1.smithi062.stdout:8/719: write d8/d9/d44/d4d/d8a/fdc [884202,61754] 0 2024-02-22T15:50:44.132 INFO:tasks.workunit.client.0.smithi019.stdout:8/726: mkdir d9/d14/d17/d3d/d83/df6 0 2024-02-22T15:50:44.132 INFO:tasks.workunit.client.0.smithi019.stdout:4/732: dread d2/db/fa3 [0,4194304] 0 2024-02-22T15:50:44.134 INFO:tasks.workunit.client.1.smithi062.stdout:4/634: rename d4/d2a/d2d/d9d to d4/d1c/d2c/d6b/d70/dd6 0 2024-02-22T15:50:44.136 INFO:tasks.workunit.client.0.smithi019.stdout:0/771: dwrite d4/d23/d7d/fff [0,4194304] 0 2024-02-22T15:50:44.136 INFO:tasks.workunit.client.1.smithi062.stdout:4/635: chown d4/d1c/d30/da7/dcb/fd4 2174507 1 2024-02-22T15:50:44.137 INFO:tasks.workunit.client.0.smithi019.stdout:8/727: unlink d9/d14/d17/d5d/lc4 0 2024-02-22T15:50:44.138 INFO:tasks.workunit.client.1.smithi062.stdout:4/636: read - d4/d1c/d30/d3b/d49/f95 zero size 2024-02-22T15:50:44.138 INFO:tasks.workunit.client.1.smithi062.stdout:2/687: dread d1/d47/f2b [0,4194304] 0 2024-02-22T15:50:44.140 INFO:tasks.workunit.client.0.smithi019.stdout:4/733: symlink d2/db/dea/ded/lee 0 2024-02-22T15:50:44.141 INFO:tasks.workunit.client.1.smithi062.stdout:2/688: chown d1/d47/d39/d5d/cdf 32209417 1 2024-02-22T15:50:44.142 INFO:tasks.workunit.client.1.smithi062.stdout:4/637: truncate d4/d2a/fd1 511705 0 2024-02-22T15:50:44.143 INFO:tasks.workunit.client.1.smithi062.stdout:8/720: dwrite d8/d9/f2a [0,4194304] 0 2024-02-22T15:50:44.144 INFO:tasks.workunit.client.0.smithi019.stdout:1/840: dwrite d8/d23/d33/f93 [0,4194304] 0 2024-02-22T15:50:44.145 INFO:tasks.workunit.client.1.smithi062.stdout:2/689: symlink d1/d8/df/le6 0 2024-02-22T15:50:44.146 INFO:tasks.workunit.client.0.smithi019.stdout:1/841: chown d8/d18/c38 154813353 1 2024-02-22T15:50:44.147 INFO:tasks.workunit.client.1.smithi062.stdout:2/690: stat d1/d1e/f66 0 2024-02-22T15:50:44.148 INFO:tasks.workunit.client.1.smithi062.stdout:8/721: symlink d8/d9/d44/d4d/le5 0 2024-02-22T15:50:44.151 INFO:tasks.workunit.client.1.smithi062.stdout:4/638: rename d4/d2a/d2d/d2e/d6a/d8a/dae to d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd7 0 2024-02-22T15:50:44.152 INFO:tasks.workunit.client.0.smithi019.stdout:1/842: symlink d8/d2a/d75/def/df8/l11e 0 2024-02-22T15:50:44.152 INFO:tasks.workunit.client.0.smithi019.stdout:0/772: dwrite d4/db/f43 [0,4194304] 0 2024-02-22T15:50:44.154 INFO:tasks.workunit.client.1.smithi062.stdout:4/639: dread d4/d1c/d30/d3b/f67 [0,4194304] 0 2024-02-22T15:50:44.155 INFO:tasks.workunit.client.1.smithi062.stdout:8/722: creat d8/d9/d2c/fe6 x:0 0 0 2024-02-22T15:50:44.156 INFO:tasks.workunit.client.1.smithi062.stdout:2/691: rename d1/d1e/d38/fd8 to d1/d8/d62/d73/dbd/dbb/dc4/fe7 0 2024-02-22T15:50:44.157 INFO:tasks.workunit.client.1.smithi062.stdout:8/723: readlink d8/d9/d44/d4d/d51/d55/l93 0 2024-02-22T15:50:44.159 INFO:tasks.workunit.client.0.smithi019.stdout:1/843: fsync d8/d15/d34/fac 0 2024-02-22T15:50:44.160 INFO:tasks.workunit.client.0.smithi019.stdout:0/773: symlink d4/db/d1b/d63/db0/da1/dbb/d52/l10d 0 2024-02-22T15:50:44.161 INFO:tasks.workunit.client.1.smithi062.stdout:4/640: rename d4/d1c/d2c/d6b/f7e to d4/d1c/d2c/d6b/d9b/db1/dc6/fd8 0 2024-02-22T15:50:44.162 INFO:tasks.workunit.client.0.smithi019.stdout:1/844: write d8/d2a/d75/d81/d8e/f118 [546212,122315] 0 2024-02-22T15:50:44.162 INFO:tasks.workunit.client.0.smithi019.stdout:0/774: write d4/d9/f26 [8914015,9486] 0 2024-02-22T15:50:44.165 INFO:tasks.workunit.client.1.smithi062.stdout:9/552: sync 2024-02-22T15:50:44.165 INFO:tasks.workunit.client.1.smithi062.stdout:2/692: mkdir d1/d1e/dda/de8 0 2024-02-22T15:50:44.165 INFO:tasks.workunit.client.0.smithi019.stdout:0/775: readlink d4/db/d1b/d40/lfb 0 2024-02-22T15:50:44.166 INFO:tasks.workunit.client.1.smithi062.stdout:8/724: creat d8/d9/d2c/dd5/fe7 x:0 0 0 2024-02-22T15:50:44.166 INFO:tasks.workunit.client.1.smithi062.stdout:4/641: symlink d4/d1c/dac/ld9 0 2024-02-22T15:50:44.167 INFO:tasks.workunit.client.0.smithi019.stdout:0/776: chown d4/d22/f5f 10781721 1 2024-02-22T15:50:44.168 INFO:tasks.workunit.client.1.smithi062.stdout:4/642: write d4/d2a/fb8 [102653,50915] 0 2024-02-22T15:50:44.169 INFO:tasks.workunit.client.1.smithi062.stdout:2/693: mknod d1/d1e/d38/ce9 0 2024-02-22T15:50:44.169 INFO:tasks.workunit.client.0.smithi019.stdout:4/734: dwrite d2/db/d14/d23/fc3 [0,4194304] 0 2024-02-22T15:50:44.170 INFO:tasks.workunit.client.1.smithi062.stdout:9/553: mknod d4/d15/d5f/d75/d84/ca9 0 2024-02-22T15:50:44.171 INFO:tasks.workunit.client.1.smithi062.stdout:8/725: rename d8/d28/d4e/cd0 to d8/d86/db6/ce8 0 2024-02-22T15:50:44.172 INFO:tasks.workunit.client.1.smithi062.stdout:2/694: creat d1/d8/d62/fea x:0 0 0 2024-02-22T15:50:44.172 INFO:tasks.workunit.client.0.smithi019.stdout:1/845: rename d8/d15/df5/l11d to d8/d2a/d75/d81/d8e/l11f 0 2024-02-22T15:50:44.172 INFO:tasks.workunit.client.1.smithi062.stdout:9/554: fdatasync d4/f1b 0 2024-02-22T15:50:44.173 INFO:tasks.workunit.client.0.smithi019.stdout:0/777: write d4/db/d1b/d63/db0/da1/dbb/d52/f58 [3078114,120252] 0 2024-02-22T15:50:44.173 INFO:tasks.workunit.client.0.smithi019.stdout:1/846: dread - d8/d2a/d75/f112 zero size 2024-02-22T15:50:44.174 INFO:tasks.workunit.client.0.smithi019.stdout:0/778: chown d4/db/d1b/d63/db0 1449097 1 2024-02-22T15:50:44.175 INFO:tasks.workunit.client.1.smithi062.stdout:4/643: rename d4/d1c/d30/d3b/f67 to d4/df/fda 0 2024-02-22T15:50:44.176 INFO:tasks.workunit.client.0.smithi019.stdout:0/779: dread - d4/d32/fe6 zero size 2024-02-22T15:50:44.176 INFO:tasks.workunit.client.1.smithi062.stdout:2/695: symlink d1/d1e/d38/leb 0 2024-02-22T15:50:44.178 INFO:tasks.workunit.client.0.smithi019.stdout:1/847: write d8/d18/d85/f8d [2218940,19230] 0 2024-02-22T15:50:44.180 INFO:tasks.workunit.client.1.smithi062.stdout:4/644: fdatasync d4/d1c/d2c/d6b/d70/dd6/fa2 0 2024-02-22T15:50:44.181 INFO:tasks.workunit.client.0.smithi019.stdout:4/735: dwrite d2/db/d14/d23/d33/db1/f95 [0,4194304] 0 2024-02-22T15:50:44.181 INFO:tasks.workunit.client.0.smithi019.stdout:8/728: dread f3 [0,4194304] 0 2024-02-22T15:50:44.182 INFO:tasks.workunit.client.1.smithi062.stdout:8/726: rename d8/d42/dcb to d8/d9/d2c/d49/d94/dc1/de9 0 2024-02-22T15:50:44.183 INFO:tasks.workunit.client.0.smithi019.stdout:4/736: read - d2/db/d54/d8c/dbb/fe2 zero size 2024-02-22T15:50:44.184 INFO:tasks.workunit.client.1.smithi062.stdout:8/727: write d8/d9/d44/d58/fdb [403978,76354] 0 2024-02-22T15:50:44.184 INFO:tasks.workunit.client.1.smithi062.stdout:9/555: link d4/f14 d4/d15/faa 0 2024-02-22T15:50:44.186 INFO:tasks.workunit.client.0.smithi019.stdout:1/848: rename d8/d15/d34/fa3 to d8/d15/d5b/f120 0 2024-02-22T15:50:44.187 INFO:tasks.workunit.client.1.smithi062.stdout:4/645: dread d4/d2a/d2d/d2e/d6a/d9e/fc3 [0,4194304] 0 2024-02-22T15:50:44.187 INFO:tasks.workunit.client.1.smithi062.stdout:9/556: read d4/f10 [141741,69183] 0 2024-02-22T15:50:44.190 INFO:tasks.workunit.client.0.smithi019.stdout:1/849: creat d8/d18/d64/d7f/d94/f121 x:0 0 0 2024-02-22T15:50:44.191 INFO:tasks.workunit.client.0.smithi019.stdout:8/729: rmdir d9/d14/d17/d3d/d83/df6 0 2024-02-22T15:50:44.192 INFO:tasks.workunit.client.1.smithi062.stdout:0/644: sync 2024-02-22T15:50:44.192 INFO:tasks.workunit.client.1.smithi062.stdout:6/672: sync 2024-02-22T15:50:44.192 INFO:tasks.workunit.client.1.smithi062.stdout:1/551: sync 2024-02-22T15:50:44.192 INFO:tasks.workunit.client.1.smithi062.stdout:7/457: sync 2024-02-22T15:50:44.193 INFO:tasks.workunit.client.1.smithi062.stdout:5/485: sync 2024-02-22T15:50:44.193 INFO:tasks.workunit.client.1.smithi062.stdout:3/519: sync 2024-02-22T15:50:44.193 INFO:tasks.workunit.client.1.smithi062.stdout:2/696: mkdir d1/d1e/d38/d76/dc5/dec 0 2024-02-22T15:50:44.195 INFO:tasks.workunit.client.1.smithi062.stdout:2/697: chown d1/d47/d32/d52 21 1 2024-02-22T15:50:44.195 INFO:tasks.workunit.client.0.smithi019.stdout:1/850: rename d8/d18/d72/c87 to d8/d2a/db3/c122 0 2024-02-22T15:50:44.196 INFO:tasks.workunit.client.1.smithi062.stdout:5/486: write d0/d3/f83 [413967,82763] 0 2024-02-22T15:50:44.196 INFO:tasks.workunit.client.0.smithi019.stdout:4/737: getdents d2/db/d14/d23/d33 0 2024-02-22T15:50:44.197 INFO:tasks.workunit.client.1.smithi062.stdout:4/646: mkdir d4/d2a/d2d/d2e/d6a/d8a/ddb 0 2024-02-22T15:50:44.198 INFO:tasks.workunit.client.0.smithi019.stdout:8/730: mknod d9/d14/d17/d1c/da0/cf7 0 2024-02-22T15:50:44.199 INFO:tasks.workunit.client.1.smithi062.stdout:8/728: link d8/d28/d4e/f8e d8/d9/d2c/fea 0 2024-02-22T15:50:44.200 INFO:tasks.workunit.client.0.smithi019.stdout:4/738: dread d2/db/d14/d23/d33/f7d [0,4194304] 0 2024-02-22T15:50:44.200 INFO:tasks.workunit.client.0.smithi019.stdout:1/851: dread d8/d2a/db3/fb1 [0,4194304] 0 2024-02-22T15:50:44.201 INFO:tasks.workunit.client.0.smithi019.stdout:1/852: stat d8/d15/d5b/fb2 0 2024-02-22T15:50:44.202 INFO:tasks.workunit.client.1.smithi062.stdout:0/645: read d1/d3/d4/d1e/fb2 [1053947,128793] 0 2024-02-22T15:50:44.202 INFO:tasks.workunit.client.1.smithi062.stdout:7/458: read d0/d6/d2e/d2f/f3a [1255797,91517] 0 2024-02-22T15:50:44.203 INFO:tasks.workunit.client.0.smithi019.stdout:1/853: readlink d8/d15/l2e 0 2024-02-22T15:50:44.204 INFO:tasks.workunit.client.0.smithi019.stdout:4/739: fsync d2/d3f/fe4 0 2024-02-22T15:50:44.204 INFO:tasks.workunit.client.1.smithi062.stdout:5/487: dread d0/d3/d2f/d59/d67/d69/d3f/d5f/f89 [0,4194304] 0 2024-02-22T15:50:44.208 INFO:tasks.workunit.client.0.smithi019.stdout:3/622: sync 2024-02-22T15:50:44.208 INFO:tasks.workunit.client.0.smithi019.stdout:5/775: sync 2024-02-22T15:50:44.208 INFO:tasks.workunit.client.0.smithi019.stdout:2/793: sync 2024-02-22T15:50:44.208 INFO:tasks.workunit.client.0.smithi019.stdout:1/854: link d8/d15/f68 d8/d2a/db3/f123 0 2024-02-22T15:50:44.210 INFO:tasks.workunit.client.0.smithi019.stdout:4/740: mkdir d2/db/d14/d23/d21/d35/d85/def 0 2024-02-22T15:50:44.215 INFO:tasks.workunit.client.0.smithi019.stdout:5/776: chown d8/dd/d19/d3a/d46/f7b 687267575 1 2024-02-22T15:50:44.216 INFO:tasks.workunit.client.1.smithi062.stdout:4/647: dwrite d4/d1c/faa [0,4194304] 0 2024-02-22T15:50:44.217 INFO:tasks.workunit.client.0.smithi019.stdout:2/794: mkdir d1/d86/d11f 0 2024-02-22T15:50:44.217 INFO:tasks.workunit.client.0.smithi019.stdout:8/731: dread d9/d14/d17/f47 [4194304,4194304] 0 2024-02-22T15:50:44.219 INFO:tasks.workunit.client.0.smithi019.stdout:2/795: write d1/d17/fc8 [25741,127627] 0 2024-02-22T15:50:44.220 INFO:tasks.workunit.client.0.smithi019.stdout:1/855: rmdir d8/d18/d64/d7f/d94 39 2024-02-22T15:50:44.224 INFO:tasks.workunit.client.0.smithi019.stdout:4/741: creat d2/db/d14/d23/d21/d42/dc4/ff0 x:0 0 0 2024-02-22T15:50:44.224 INFO:tasks.workunit.client.0.smithi019.stdout:8/732: symlink d9/d14/d17/d5d/de6/lf8 0 2024-02-22T15:50:44.228 INFO:tasks.workunit.client.0.smithi019.stdout:2/796: fdatasync d1/d46/f95 0 2024-02-22T15:50:44.228 INFO:tasks.workunit.client.1.smithi062.stdout:6/673: dread d5/df/f26 [0,4194304] 0 2024-02-22T15:50:44.229 INFO:tasks.workunit.client.0.smithi019.stdout:1/856: creat d8/d2a/d75/d81/ddb/f124 x:0 0 0 2024-02-22T15:50:44.230 INFO:tasks.workunit.client.0.smithi019.stdout:8/733: chown d9/d14/f52 429134 1 2024-02-22T15:50:44.232 INFO:tasks.workunit.client.0.smithi019.stdout:3/623: getdents d9/d14 0 2024-02-22T15:50:44.232 INFO:tasks.workunit.client.1.smithi062.stdout:8/729: mknod d8/d9/d44/d7e/ceb 0 2024-02-22T15:50:44.232 INFO:tasks.workunit.client.0.smithi019.stdout:5/777: write d8/dd/fc8 [660871,17365] 0 2024-02-22T15:50:44.233 INFO:tasks.workunit.client.1.smithi062.stdout:8/730: fsync d8/f12 0 2024-02-22T15:50:44.233 INFO:tasks.workunit.client.0.smithi019.stdout:4/742: creat d2/d3f/d48/d76/ff1 x:0 0 0 2024-02-22T15:50:44.235 INFO:tasks.workunit.client.0.smithi019.stdout:2/797: mkdir d1/d17/d1c/d66/db9/dd0/dee/d120 0 2024-02-22T15:50:44.238 INFO:tasks.workunit.client.1.smithi062.stdout:8/731: read d8/d9/d2c/f78 [70885,90296] 0 2024-02-22T15:50:44.239 INFO:tasks.workunit.client.0.smithi019.stdout:5/778: mkdir d8/dd/d19/d38/d54/db0/d10d 0 2024-02-22T15:50:44.240 INFO:tasks.workunit.client.0.smithi019.stdout:2/798: mknod d1/d17/d67/c121 0 2024-02-22T15:50:44.243 INFO:tasks.workunit.client.0.smithi019.stdout:5/779: truncate d8/dd/d19/d77/d7a/fdd 546993 0 2024-02-22T15:50:44.243 INFO:tasks.workunit.client.0.smithi019.stdout:3/624: creat d9/d26/d2d/d39/fcd x:0 0 0 2024-02-22T15:50:44.243 INFO:tasks.workunit.client.1.smithi062.stdout:0/646: dread d1/d3/de/f90 [0,4194304] 0 2024-02-22T15:50:44.244 INFO:tasks.workunit.client.1.smithi062.stdout:0/647: fdatasync d1/d3/d4/d1e/d74/f70 0 2024-02-22T15:50:44.245 INFO:tasks.workunit.client.0.smithi019.stdout:8/734: write d9/d14/d17/d3d/d3f/f56 [330760,18027] 0 2024-02-22T15:50:44.247 INFO:tasks.workunit.client.0.smithi019.stdout:1/857: getdents d8/d15/d34/d9e 0 2024-02-22T15:50:44.247 INFO:tasks.workunit.client.1.smithi062.stdout:0/648: chown d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/d97/fab 2749 1 2024-02-22T15:50:44.247 INFO:tasks.workunit.client.0.smithi019.stdout:2/799: dread d1/f2e [0,4194304] 0 2024-02-22T15:50:44.248 INFO:tasks.workunit.client.0.smithi019.stdout:8/735: chown d9/f12 0 1 2024-02-22T15:50:44.252 INFO:tasks.workunit.client.0.smithi019.stdout:2/800: mknod d1/def/c122 0 2024-02-22T15:50:44.252 INFO:tasks.workunit.client.0.smithi019.stdout:1/858: write d8/d23/d33/f93 [4273314,107400] 0 2024-02-22T15:50:44.256 INFO:tasks.workunit.client.0.smithi019.stdout:3/625: dread d9/d26/d2d/d53/f56 [0,4194304] 0 2024-02-22T15:50:44.260 INFO:tasks.workunit.client.0.smithi019.stdout:1/859: chown d8/d15/d34/f65 0 1 2024-02-22T15:50:44.260 INFO:tasks.workunit.client.0.smithi019.stdout:5/780: dwrite d8/d11/f65 [0,4194304] 0 2024-02-22T15:50:44.261 INFO:tasks.workunit.client.0.smithi019.stdout:2/801: rename d1/d17/d21/l45 to d1/de/d9a/l123 0 2024-02-22T15:50:44.267 INFO:tasks.workunit.client.0.smithi019.stdout:8/736: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:44.268 INFO:tasks.workunit.client.0.smithi019.stdout:5/781: mkdir d8/dd/d8e/d2c/d10e 0 2024-02-22T15:50:44.273 INFO:tasks.workunit.client.1.smithi062.stdout:4/648: mknod d4/d1c/d30/da7/cdc 0 2024-02-22T15:50:44.273 INFO:tasks.workunit.client.0.smithi019.stdout:4/743: dread d2/db/d14/d23/f4d [0,4194304] 0 2024-02-22T15:50:44.275 INFO:tasks.workunit.client.0.smithi019.stdout:3/626: dread d9/d11/f1f [0,4194304] 0 2024-02-22T15:50:44.276 INFO:tasks.workunit.client.0.smithi019.stdout:4/744: chown d2/db/d14/d23/d9e/dac/dc2/dc9 122 1 2024-02-22T15:50:44.278 INFO:tasks.workunit.client.0.smithi019.stdout:3/627: dread - d9/d22/d27/d96/fca zero size 2024-02-22T15:50:44.281 INFO:tasks.workunit.client.1.smithi062.stdout:5/488: dwrite d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:44.285 INFO:tasks.workunit.client.1.smithi062.stdout:1/552: write d1/d6/d10/f4b [3122933,103212] 0 2024-02-22T15:50:44.286 INFO:tasks.workunit.client.1.smithi062.stdout:0/649: chown d1/d3/d4/d1a/d1d/c9f 35312747 1 2024-02-22T15:50:44.287 INFO:tasks.workunit.client.0.smithi019.stdout:5/782: write d8/dd/d19/d77/dd2/f101 [815260,441] 0 2024-02-22T15:50:44.287 INFO:tasks.workunit.client.0.smithi019.stdout:8/737: write f3 [3059029,49076] 0 2024-02-22T15:50:44.291 INFO:tasks.workunit.client.0.smithi019.stdout:4/745: dwrite d2/db/d14/d23/d33/db1/f95 [0,4194304] 0 2024-02-22T15:50:44.291 INFO:tasks.workunit.client.0.smithi019.stdout:1/860: dread d8/d15/f26 [0,4194304] 0 2024-02-22T15:50:44.297 INFO:tasks.workunit.client.0.smithi019.stdout:3/628: dwrite d9/d22/d27/fb0 [0,4194304] 0 2024-02-22T15:50:44.298 INFO:tasks.workunit.client.1.smithi062.stdout:6/674: creat d5/df/d40/d44/d76/fdb x:0 0 0 2024-02-22T15:50:44.298 INFO:tasks.workunit.client.0.smithi019.stdout:5/783: creat d8/dd/d19/d38/d47/df3/f10f x:0 0 0 2024-02-22T15:50:44.298 INFO:tasks.workunit.client.1.smithi062.stdout:4/649: mknod d4/d2a/d2d/d2e/d6a/d94/cdd 0 2024-02-22T15:50:44.299 INFO:tasks.workunit.client.1.smithi062.stdout:3/520: dwrite d5/d26/f61 [0,4194304] 0 2024-02-22T15:50:44.300 INFO:tasks.workunit.client.0.smithi019.stdout:4/746: write d2/db/d14/d23/d9e/dac/dc2/fd2 [100269,38288] 0 2024-02-22T15:50:44.300 INFO:tasks.workunit.client.0.smithi019.stdout:8/738: dread d9/d14/d1f/fa5 [0,4194304] 0 2024-02-22T15:50:44.301 INFO:tasks.workunit.client.0.smithi019.stdout:3/629: dread - d9/d14/d19/fbb zero size 2024-02-22T15:50:44.306 INFO:tasks.workunit.client.1.smithi062.stdout:9/557: creat d4/d15/d5f/d75/fab x:0 0 0 2024-02-22T15:50:44.307 INFO:tasks.workunit.client.0.smithi019.stdout:8/739: rmdir d9/d14/d6f 39 2024-02-22T15:50:44.310 INFO:tasks.workunit.client.1.smithi062.stdout:3/521: readlink d5/la 0 2024-02-22T15:50:44.310 INFO:tasks.workunit.client.1.smithi062.stdout:1/553: dwrite d1/d6/d7/d2b/d47/d7e/fa7 [0,4194304] 0 2024-02-22T15:50:44.313 INFO:tasks.workunit.client.0.smithi019.stdout:2/802: dread d1/d17/d38/fd9 [0,4194304] 0 2024-02-22T15:50:44.315 INFO:tasks.workunit.client.1.smithi062.stdout:4/650: truncate d4/d66/f6f 825655 0 2024-02-22T15:50:44.316 INFO:tasks.workunit.client.0.smithi019.stdout:2/803: write d1/de/d15/f8c [1172464,90846] 0 2024-02-22T15:50:44.317 INFO:tasks.workunit.client.1.smithi062.stdout:6/675: symlink d5/d6e/dc1/ldc 0 2024-02-22T15:50:44.318 INFO:tasks.workunit.client.0.smithi019.stdout:4/747: creat d2/db/d14/d23/d33/db1/ff2 x:0 0 0 2024-02-22T15:50:44.319 INFO:tasks.workunit.client.1.smithi062.stdout:6/676: fsync d5/df/d40/d44/d4a/faa 0 2024-02-22T15:50:44.322 INFO:tasks.workunit.client.0.smithi019.stdout:4/748: write d2/db/d14/fc7 [2708357,129175] 0 2024-02-22T15:50:44.325 INFO:tasks.workunit.client.0.smithi019.stdout:2/804: fsync d1/d17/d1c/d66/f83 0 2024-02-22T15:50:44.325 INFO:tasks.workunit.client.1.smithi062.stdout:9/558: mknod d4/d15/d5f/d75/cac 0 2024-02-22T15:50:44.325 INFO:tasks.workunit.client.1.smithi062.stdout:4/651: symlink d4/d2a/d2d/d2e/d6a/d7b/lde 0 2024-02-22T15:50:44.325 INFO:tasks.workunit.client.1.smithi062.stdout:3/522: fsync d5/d26/f31 0 2024-02-22T15:50:44.326 INFO:tasks.workunit.client.0.smithi019.stdout:5/784: creat d8/dd/d19/d3a/f110 x:0 0 0 2024-02-22T15:50:44.329 INFO:tasks.workunit.client.0.smithi019.stdout:3/630: dread d9/d26/d3f/d59/d66/f7f [0,4194304] 0 2024-02-22T15:50:44.329 INFO:tasks.workunit.client.0.smithi019.stdout:1/861: dread d8/d2a/f32 [0,4194304] 0 2024-02-22T15:50:44.330 INFO:tasks.workunit.client.1.smithi062.stdout:3/523: read - d5/d16/d21/d32/d6a/f8e zero size 2024-02-22T15:50:44.331 INFO:tasks.workunit.client.0.smithi019.stdout:4/749: creat d2/db/ff3 x:0 0 0 2024-02-22T15:50:44.332 INFO:tasks.workunit.client.1.smithi062.stdout:5/489: dread d0/d3/f14 [0,4194304] 0 2024-02-22T15:50:44.332 INFO:tasks.workunit.client.0.smithi019.stdout:8/740: truncate d9/d14/d6f/d7e/d94/daa/fb6 78355 0 2024-02-22T15:50:44.333 INFO:tasks.workunit.client.1.smithi062.stdout:5/490: stat d0/d3/d2f/f3a 0 2024-02-22T15:50:44.333 INFO:tasks.workunit.client.1.smithi062.stdout:9/559: dread d4/d15/d31/f60 [0,4194304] 0 2024-02-22T15:50:44.333 INFO:tasks.workunit.client.1.smithi062.stdout:3/524: mknod d5/d16/d56/cad 0 2024-02-22T15:50:44.334 INFO:tasks.workunit.client.0.smithi019.stdout:2/805: rename d1/d17/d21/d51/fc9 to d1/d17/d1c/d66/f124 0 2024-02-22T15:50:44.335 INFO:tasks.workunit.client.1.smithi062.stdout:3/525: chown d5/d16/d21/d36/d4a 666 1 2024-02-22T15:50:44.336 INFO:tasks.workunit.client.0.smithi019.stdout:4/750: write d2/f32 [3441427,121667] 0 2024-02-22T15:50:44.337 INFO:tasks.workunit.client.0.smithi019.stdout:8/741: chown d9/d14/d17/d3d/d3f/dd4 144962300 1 2024-02-22T15:50:44.338 INFO:tasks.workunit.client.0.smithi019.stdout:4/751: chown d2/db/fa3 832171485 1 2024-02-22T15:50:44.340 INFO:tasks.workunit.client.0.smithi019.stdout:2/806: symlink d1/d17/d67/l125 0 2024-02-22T15:50:44.341 INFO:tasks.workunit.client.0.smithi019.stdout:1/862: dread d8/d15/d34/d9e/fe7 [0,4194304] 0 2024-02-22T15:50:44.345 INFO:tasks.workunit.client.0.smithi019.stdout:3/631: dread d9/d22/d27/fb0 [0,4194304] 0 2024-02-22T15:50:44.346 INFO:tasks.workunit.client.0.smithi019.stdout:2/807: unlink d1/d17/d21/d71/d60/d11b/f79 0 2024-02-22T15:50:44.347 INFO:tasks.workunit.client.1.smithi062.stdout:2/698: rename d1/d47/d32/c90 to d1/de1/ced 0 2024-02-22T15:50:44.349 INFO:tasks.workunit.client.0.smithi019.stdout:3/632: write d9/d22/d27/d96/fca [913776,95957] 0 2024-02-22T15:50:44.350 INFO:tasks.workunit.client.1.smithi062.stdout:5/491: creat d0/d3/d2f/d59/d67/f95 x:0 0 0 2024-02-22T15:50:44.350 INFO:tasks.workunit.client.0.smithi019.stdout:4/752: mknod d2/db/d14/d23/d21/d35/d85/def/cf4 0 2024-02-22T15:50:44.353 INFO:tasks.workunit.client.0.smithi019.stdout:8/742: dwrite d9/fa [0,4194304] 0 2024-02-22T15:50:44.354 INFO:tasks.workunit.client.1.smithi062.stdout:6/677: rename d5/df/dbc/dd0 to d5/d8a/ddd 0 2024-02-22T15:50:44.354 INFO:tasks.workunit.client.1.smithi062.stdout:2/699: dread - d1/d8/df/f61 zero size 2024-02-22T15:50:44.354 INFO:tasks.workunit.client.1.smithi062.stdout:5/492: read d0/d3/da/d13/d23/d31/f34 [2090225,47339] 0 2024-02-22T15:50:44.356 INFO:tasks.workunit.client.1.smithi062.stdout:4/652: link d4/la d4/d2a/d2d/d2e/d6a/ldf 0 2024-02-22T15:50:44.356 INFO:tasks.workunit.client.1.smithi062.stdout:2/700: read d1/d47/d39/f4b [1228413,16815] 0 2024-02-22T15:50:44.357 INFO:tasks.workunit.client.1.smithi062.stdout:3/526: link d5/c12 d5/d16/d21/d32/d75/cae 0 2024-02-22T15:50:44.357 INFO:tasks.workunit.client.0.smithi019.stdout:5/785: dread d8/dd/d19/d3a/d46/f53 [0,4194304] 0 2024-02-22T15:50:44.360 INFO:tasks.workunit.client.1.smithi062.stdout:9/560: dread d4/d15/d5f/f7b [0,4194304] 0 2024-02-22T15:50:44.360 INFO:tasks.workunit.client.0.smithi019.stdout:1/863: dwrite d8/d44/ffd [0,4194304] 0 2024-02-22T15:50:44.360 INFO:tasks.workunit.client.0.smithi019.stdout:8/743: dread - d9/d14/d6f/d7e/d94/d9a/fba zero size 2024-02-22T15:50:44.366 INFO:tasks.workunit.client.0.smithi019.stdout:1/864: write d8/d18/d85/f98 [4804779,125591] 0 2024-02-22T15:50:44.367 INFO:tasks.workunit.client.0.smithi019.stdout:5/786: mknod d8/dd/d19/d38/d47/ddb/dde/df6/c111 0 2024-02-22T15:50:44.369 INFO:tasks.workunit.client.1.smithi062.stdout:2/701: creat d1/d8/d62/d73/dbd/dbb/fee x:0 0 0 2024-02-22T15:50:44.369 INFO:tasks.workunit.client.0.smithi019.stdout:3/633: dwrite d9/d22/f9d [4194304,4194304] 0 2024-02-22T15:50:44.370 INFO:tasks.workunit.client.0.smithi019.stdout:3/634: readlink d9/d26/d2d/d39/l52 0 2024-02-22T15:50:44.370 INFO:tasks.workunit.client.0.smithi019.stdout:8/744: mknod d9/d14/d6f/d7e/d94/d9a/dd3/cf9 0 2024-02-22T15:50:44.370 INFO:tasks.workunit.client.0.smithi019.stdout:5/787: dread - d8/d11/d33/fe2 zero size 2024-02-22T15:50:44.372 INFO:tasks.workunit.client.0.smithi019.stdout:4/753: truncate d2/db/f17 2795511 0 2024-02-22T15:50:44.375 INFO:tasks.workunit.client.1.smithi062.stdout:4/653: creat d4/d2a/d2d/d2e/d6a/d8a/ddb/fe0 x:0 0 0 2024-02-22T15:50:44.375 INFO:tasks.workunit.client.1.smithi062.stdout:4/654: readlink d4/d13/d20/l24 0 2024-02-22T15:50:44.376 INFO:tasks.workunit.client.1.smithi062.stdout:3/527: creat d5/d26/d2d/da2/dab/faf x:0 0 0 2024-02-22T15:50:44.376 INFO:tasks.workunit.client.0.smithi019.stdout:3/635: unlink d9/d14/d19/fbb 0 2024-02-22T15:50:44.379 INFO:tasks.workunit.client.1.smithi062.stdout:9/561: creat d4/d15/d5f/d75/d97/da7/fad x:0 0 0 2024-02-22T15:50:44.381 INFO:tasks.workunit.client.0.smithi019.stdout:3/636: chown d9/d26/d32/l7e 1371746 1 2024-02-22T15:50:44.381 INFO:tasks.workunit.client.0.smithi019.stdout:8/745: link d9/d14/d6f/d7e/d94/d9a/cf4 d9/d14/d1f/d6d/cfa 0 2024-02-22T15:50:44.382 INFO:tasks.workunit.client.0.smithi019.stdout:5/788: dwrite d8/dd/d19/d77/dd2/dec/ff2 [0,4194304] 0 2024-02-22T15:50:44.386 INFO:tasks.workunit.client.0.smithi019.stdout:5/789: write d8/d11/d33/fa9 [124746,80838] 0 2024-02-22T15:50:44.386 INFO:tasks.workunit.client.0.smithi019.stdout:3/637: truncate d9/d26/d2d/d53/da0/fcb 275603 0 2024-02-22T15:50:44.387 INFO:tasks.workunit.client.0.smithi019.stdout:8/746: mkdir d9/d14/d1f/dca/dfb 0 2024-02-22T15:50:44.387 INFO:tasks.workunit.client.1.smithi062.stdout:9/562: rmdir d4/d15/d5e 39 2024-02-22T15:50:44.391 INFO:tasks.workunit.client.1.smithi062.stdout:4/655: dwrite d4/d2a/d2d/d2e/d6a/d8a/f9a [0,4194304] 0 2024-02-22T15:50:44.391 INFO:tasks.workunit.client.1.smithi062.stdout:2/702: dwrite d1/d1e/d38/f74 [4194304,4194304] 0 2024-02-22T15:50:44.391 INFO:tasks.workunit.client.1.smithi062.stdout:7/459: sync 2024-02-22T15:50:44.392 INFO:tasks.workunit.client.0.smithi019.stdout:4/754: dread d2/db/f26 [0,4194304] 0 2024-02-22T15:50:44.392 INFO:tasks.workunit.client.0.smithi019.stdout:8/747: symlink d9/d14/d6f/d7e/d94/daa/dea/lfc 0 2024-02-22T15:50:44.395 INFO:tasks.workunit.client.1.smithi062.stdout:4/656: readlink d4/d2a/d2d/l9f 0 2024-02-22T15:50:44.395 INFO:tasks.workunit.client.0.smithi019.stdout:4/755: chown d2/db/d14/d23/d33/db1/lca 10509014 1 2024-02-22T15:50:44.397 INFO:tasks.workunit.client.0.smithi019.stdout:8/748: fdatasync d9/d14/d17/d3d/d3f/f9e 0 2024-02-22T15:50:44.397 INFO:tasks.workunit.client.0.smithi019.stdout:4/756: dread d2/db/fa3 [0,4194304] 0 2024-02-22T15:50:44.398 INFO:tasks.workunit.client.1.smithi062.stdout:0/650: write d1/fc4 [1456769,114194] 0 2024-02-22T15:50:44.399 INFO:tasks.workunit.client.0.smithi019.stdout:5/790: dread d8/d11/d33/f3c [4194304,4194304] 0 2024-02-22T15:50:44.400 INFO:tasks.workunit.client.0.smithi019.stdout:4/757: mknod d2/db/d14/d23/d9e/dac/cf5 0 2024-02-22T15:50:44.406 INFO:tasks.workunit.client.1.smithi062.stdout:4/657: truncate d4/d1c/d30/f32 3249508 0 2024-02-22T15:50:44.408 INFO:tasks.workunit.client.0.smithi019.stdout:8/749: creat d9/d14/d1f/ffd x:0 0 0 2024-02-22T15:50:44.409 INFO:tasks.workunit.client.1.smithi062.stdout:9/563: rename d4/d15/d5f/d6a/f81 to d4/d15/d5e/d63/da0/fae 0 2024-02-22T15:50:44.410 INFO:tasks.workunit.client.1.smithi062.stdout:4/658: chown d4/d1c/d30/da7/cc1 2482 1 2024-02-22T15:50:44.410 INFO:tasks.workunit.client.1.smithi062.stdout:9/564: chown d4/d1c/d83/d87 1 1 2024-02-22T15:50:44.412 INFO:tasks.workunit.client.0.smithi019.stdout:5/791: write d8/f25 [8350782,24427] 0 2024-02-22T15:50:44.413 INFO:tasks.workunit.client.1.smithi062.stdout:4/659: dread d4/df/fda [0,4194304] 0 2024-02-22T15:50:44.416 INFO:tasks.workunit.client.0.smithi019.stdout:4/758: dwrite d2/fa [0,4194304] 0 2024-02-22T15:50:44.416 INFO:tasks.workunit.client.1.smithi062.stdout:9/565: creat d4/d15/d5f/d75/d97/faf x:0 0 0 2024-02-22T15:50:44.418 INFO:tasks.workunit.client.1.smithi062.stdout:9/566: chown d4/d35 550 1 2024-02-22T15:50:44.419 INFO:tasks.workunit.client.0.smithi019.stdout:5/792: rename d8/d11/l1e to d8/dd/d19/d3a/d46/l112 0 2024-02-22T15:50:44.420 INFO:tasks.workunit.client.1.smithi062.stdout:4/660: mknod d4/ce1 0 2024-02-22T15:50:44.423 INFO:tasks.workunit.client.1.smithi062.stdout:1/554: dwrite d1/d6/de/f72 [0,4194304] 0 2024-02-22T15:50:44.427 INFO:tasks.workunit.client.1.smithi062.stdout:9/567: read d4/d15/d31/f60 [668790,17526] 0 2024-02-22T15:50:44.428 INFO:tasks.workunit.client.1.smithi062.stdout:4/661: mkdir d4/d1c/d30/da7/dcb/de2 0 2024-02-22T15:50:44.431 INFO:tasks.workunit.client.1.smithi062.stdout:4/662: fsync d4/d2a/d2d/d2e/f84 0 2024-02-22T15:50:44.432 INFO:tasks.workunit.client.0.smithi019.stdout:4/759: dwrite d2/db/fd1 [0,4194304] 0 2024-02-22T15:50:44.434 INFO:tasks.workunit.client.1.smithi062.stdout:1/555: dread d1/d6/f61 [0,4194304] 0 2024-02-22T15:50:44.435 INFO:tasks.workunit.client.1.smithi062.stdout:9/568: rename d4/d28/d50/l76 to d4/d1c/d83/lb0 0 2024-02-22T15:50:44.436 INFO:tasks.workunit.client.1.smithi062.stdout:8/732: rmdir d8/d9/d44/d7e 39 2024-02-22T15:50:44.439 INFO:tasks.workunit.client.1.smithi062.stdout:9/569: creat d4/d15/d5f/d6a/fb1 x:0 0 0 2024-02-22T15:50:44.440 INFO:tasks.workunit.client.1.smithi062.stdout:1/556: mknod d1/d6/d7/d2b/d56/cba 0 2024-02-22T15:50:44.441 INFO:tasks.workunit.client.1.smithi062.stdout:2/703: dread d1/d8/fa [0,4194304] 0 2024-02-22T15:50:44.441 INFO:tasks.workunit.client.0.smithi019.stdout:4/760: write d2/db/d59/fe3 [747376,35600] 0 2024-02-22T15:50:44.445 INFO:tasks.workunit.client.1.smithi062.stdout:4/663: creat d4/d1c/d2c/d6b/fe3 x:0 0 0 2024-02-22T15:50:44.445 INFO:tasks.workunit.client.1.smithi062.stdout:7/460: dread d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:44.446 INFO:tasks.workunit.client.0.smithi019.stdout:6/961: sync 2024-02-22T15:50:44.454 INFO:tasks.workunit.client.1.smithi062.stdout:5/493: write d0/d3/ff [285065,88113] 0 2024-02-22T15:50:44.454 INFO:tasks.workunit.client.0.smithi019.stdout:3/638: dread d9/d14/f1b [0,4194304] 0 2024-02-22T15:50:44.455 INFO:tasks.workunit.client.1.smithi062.stdout:6/678: dwrite d5/df/d40/d44/d76/d56/f5b [0,4194304] 0 2024-02-22T15:50:44.456 INFO:tasks.workunit.client.0.smithi019.stdout:6/962: chown d6/d4b/d8d/c136 451647733 1 2024-02-22T15:50:44.458 INFO:tasks.workunit.client.1.smithi062.stdout:3/528: truncate d5/d16/d56/f7e 3948018 0 2024-02-22T15:50:44.459 INFO:tasks.workunit.client.0.smithi019.stdout:5/793: dwrite d8/f23 [0,4194304] 0 2024-02-22T15:50:44.461 INFO:tasks.workunit.client.1.smithi062.stdout:1/557: dwrite d1/d66/f75 [0,4194304] 0 2024-02-22T15:50:44.461 INFO:tasks.workunit.client.1.smithi062.stdout:6/679: chown d5/df/d40/d44/d4a/fad 94898241 1 2024-02-22T15:50:44.461 INFO:tasks.workunit.client.0.smithi019.stdout:9/782: sync 2024-02-22T15:50:44.463 INFO:tasks.workunit.client.1.smithi062.stdout:4/664: creat d4/d1c/fe4 x:0 0 0 2024-02-22T15:50:44.467 INFO:tasks.workunit.client.1.smithi062.stdout:3/529: truncate d5/d16/d21/d32/f51 98265 0 2024-02-22T15:50:44.467 INFO:tasks.workunit.client.1.smithi062.stdout:0/651: truncate d1/d3/d4/d1a/d76/d8c/f88 573343 0 2024-02-22T15:50:44.468 INFO:tasks.workunit.client.1.smithi062.stdout:6/680: rmdir d5/d8a/ddd 39 2024-02-22T15:50:44.468 INFO:tasks.workunit.client.1.smithi062.stdout:2/704: dread d1/d1e/f66 [0,4194304] 0 2024-02-22T15:50:44.469 INFO:tasks.workunit.client.1.smithi062.stdout:5/494: creat d0/d3/da/f96 x:0 0 0 2024-02-22T15:50:44.471 INFO:tasks.workunit.client.1.smithi062.stdout:4/665: mkdir d4/d66/de5 0 2024-02-22T15:50:44.473 INFO:tasks.workunit.client.1.smithi062.stdout:8/733: dread d8/d9/f71 [0,4194304] 0 2024-02-22T15:50:44.475 INFO:tasks.workunit.client.1.smithi062.stdout:0/652: read - d1/d3/d4/d1e/d74/d68/fc7 zero size 2024-02-22T15:50:44.477 INFO:tasks.workunit.client.1.smithi062.stdout:0/653: readlink d1/d3/d4/d1a/l91 0 2024-02-22T15:50:44.479 INFO:tasks.workunit.client.0.smithi019.stdout:5/794: stat d8/dd/d19/d38/d47/ddb/dde/deb/fe7 0 2024-02-22T15:50:44.480 INFO:tasks.workunit.client.1.smithi062.stdout:4/666: chown d4/d1c/d2c/d6b/d70/dd6/fa2 18602631 1 2024-02-22T15:50:44.480 INFO:tasks.workunit.client.1.smithi062.stdout:8/734: write d8/d9/f2a [2951624,117957] 0 2024-02-22T15:50:44.481 INFO:tasks.workunit.client.0.smithi019.stdout:5/795: fsync d8/dd/d19/d77/dd2/dd9/fff 0 2024-02-22T15:50:44.482 INFO:tasks.workunit.client.0.smithi019.stdout:3/639: link d9/d26/fb5 d9/d14/fce 0 2024-02-22T15:50:44.482 INFO:tasks.workunit.client.1.smithi062.stdout:6/681: read - d5/df/d40/d44/d76/d56/f65 zero size 2024-02-22T15:50:44.484 INFO:tasks.workunit.client.0.smithi019.stdout:5/796: dread - d8/dd/f8f zero size 2024-02-22T15:50:44.484 INFO:tasks.workunit.client.1.smithi062.stdout:0/654: write d1/d3/d4/d1e/f79 [1675988,92280] 0 2024-02-22T15:50:44.487 INFO:tasks.workunit.client.1.smithi062.stdout:2/705: symlink d1/d8/d62/d73/dbd/dbb/lef 0 2024-02-22T15:50:44.488 INFO:tasks.workunit.client.0.smithi019.stdout:5/797: read d8/dd/d19/d38/d47/ddb/dde/deb/fd4 [71656,73325] 0 2024-02-22T15:50:44.490 INFO:tasks.workunit.client.0.smithi019.stdout:5/798: fsync d8/d11/d33/fc1 0 2024-02-22T15:50:44.490 INFO:tasks.workunit.client.1.smithi062.stdout:3/530: dwrite d5/d16/d21/d32/f70 [0,4194304] 0 2024-02-22T15:50:44.490 INFO:tasks.workunit.client.1.smithi062.stdout:8/735: creat d8/d26/fec x:0 0 0 2024-02-22T15:50:44.491 INFO:tasks.workunit.client.1.smithi062.stdout:1/558: link d1/d6/d10/f1c d1/d6/fbb 0 2024-02-22T15:50:44.492 INFO:tasks.workunit.client.1.smithi062.stdout:4/667: symlink d4/d2a/d2d/d2e/d6a/d7b/le6 0 2024-02-22T15:50:44.502 INFO:tasks.workunit.client.1.smithi062.stdout:5/495: getdents d0/d3/da/d32/d42 0 2024-02-22T15:50:44.503 INFO:tasks.workunit.client.0.smithi019.stdout:3/640: truncate d9/d26/d3f/d59/f6b 251953 0 2024-02-22T15:50:44.504 INFO:tasks.workunit.client.1.smithi062.stdout:1/559: rmdir d1/d6/d7/d2b/d49/d74 39 2024-02-22T15:50:44.505 INFO:tasks.workunit.client.0.smithi019.stdout:3/641: mkdir d9/d16/dcf 0 2024-02-22T15:50:44.505 INFO:tasks.workunit.client.1.smithi062.stdout:9/570: dwrite d4/d15/d31/f60 [0,4194304] 0 2024-02-22T15:50:44.506 INFO:tasks.workunit.client.0.smithi019.stdout:3/642: dread - d9/d14/f9b zero size 2024-02-22T15:50:44.506 INFO:tasks.workunit.client.0.smithi019.stdout:3/643: read - d9/d65/da7/fc9 zero size 2024-02-22T15:50:44.507 INFO:tasks.workunit.client.1.smithi062.stdout:3/531: creat d5/d16/d56/fb0 x:0 0 0 2024-02-22T15:50:44.507 INFO:tasks.workunit.client.0.smithi019.stdout:3/644: readlink d9/d26/d76/l94 0 2024-02-22T15:50:44.510 INFO:tasks.workunit.client.0.smithi019.stdout:3/645: mkdir d9/d22/d27/dd0 0 2024-02-22T15:50:44.510 INFO:tasks.workunit.client.1.smithi062.stdout:7/461: dread d0/d14/f67 [0,4194304] 0 2024-02-22T15:50:44.511 INFO:tasks.workunit.client.1.smithi062.stdout:9/571: truncate d4/d15/faa 5134166 0 2024-02-22T15:50:44.511 INFO:tasks.workunit.client.1.smithi062.stdout:8/736: fsync d8/f21 0 2024-02-22T15:50:44.511 INFO:tasks.workunit.client.1.smithi062.stdout:2/706: symlink d1/d47/d32/d52/dbc/lf0 0 2024-02-22T15:50:44.512 INFO:tasks.workunit.client.0.smithi019.stdout:3/646: symlink d9/d26/d3f/ld1 0 2024-02-22T15:50:44.513 INFO:tasks.workunit.client.1.smithi062.stdout:3/532: write d5/d26/f6c [4476320,17841] 0 2024-02-22T15:50:44.515 INFO:tasks.workunit.client.0.smithi019.stdout:3/647: creat d9/d26/d3f/d59/d66/fd2 x:0 0 0 2024-02-22T15:50:44.518 INFO:tasks.workunit.client.1.smithi062.stdout:8/737: chown d8/d9/d2c/d49/d94 114427 1 2024-02-22T15:50:44.519 INFO:tasks.workunit.client.0.smithi019.stdout:3/648: read d9/d26/d2d/d53/f56 [174233,97856] 0 2024-02-22T15:50:44.521 INFO:tasks.workunit.client.0.smithi019.stdout:6/963: dwrite d6/fe9 [0,4194304] 0 2024-02-22T15:50:44.523 INFO:tasks.workunit.client.1.smithi062.stdout:6/682: dread d5/df/d40/d44/d78/f31 [0,4194304] 0 2024-02-22T15:50:44.523 INFO:tasks.workunit.client.1.smithi062.stdout:0/655: write d1/d3/d4/d1a/d76/d8c/d5d/dc3/f24 [3089534,28534] 0 2024-02-22T15:50:44.524 INFO:tasks.workunit.client.1.smithi062.stdout:1/560: dwrite d1/d6/d36/d37/f8e [0,4194304] 0 2024-02-22T15:50:44.524 INFO:tasks.workunit.client.1.smithi062.stdout:9/572: creat d4/d15/d5e/d63/fb2 x:0 0 0 2024-02-22T15:50:44.524 INFO:tasks.workunit.client.0.smithi019.stdout:9/783: dwrite d2/d1f/d2e/d48/d45/d5a/f63 [0,4194304] 0 2024-02-22T15:50:44.527 INFO:tasks.workunit.client.1.smithi062.stdout:7/462: truncate d0/d14/f20 3073486 0 2024-02-22T15:50:44.528 INFO:tasks.workunit.client.1.smithi062.stdout:2/707: write d1/d8/df/f8b [683654,89951] 0 2024-02-22T15:50:44.528 INFO:tasks.workunit.client.0.smithi019.stdout:6/964: rename d6/d4b/da9/cd8 to d6/d31/d48/d122/c13b 0 2024-02-22T15:50:44.531 INFO:tasks.workunit.client.0.smithi019.stdout:6/965: mkdir d6/d4b/d6d/d11f/d13c 0 2024-02-22T15:50:44.532 INFO:tasks.workunit.client.1.smithi062.stdout:2/708: write d1/d47/d32/d52/d80/f9b [1855921,124159] 0 2024-02-22T15:50:44.533 INFO:tasks.workunit.client.1.smithi062.stdout:5/496: creat d0/d3/d2f/d59/d67/d69/d3f/d66/f97 x:0 0 0 2024-02-22T15:50:44.533 INFO:tasks.workunit.client.1.smithi062.stdout:6/683: rmdir d5/dd/d33/d66 39 2024-02-22T15:50:44.535 INFO:tasks.workunit.client.1.smithi062.stdout:1/561: mknod d1/d6/d7/d2b/d49/da9/cbc 0 2024-02-22T15:50:44.535 INFO:tasks.workunit.client.1.smithi062.stdout:0/656: creat d1/d3/d4/d1e/fd8 x:0 0 0 2024-02-22T15:50:44.535 INFO:tasks.workunit.client.1.smithi062.stdout:3/533: creat d5/d16/d97/fb1 x:0 0 0 2024-02-22T15:50:44.539 INFO:tasks.workunit.client.1.smithi062.stdout:3/534: chown d5/d26/d2d 0 1 2024-02-22T15:50:44.539 INFO:tasks.workunit.client.0.smithi019.stdout:6/966: link d6/l12b d6/d4b/da9/d67/d8a/l13d 0 2024-02-22T15:50:44.540 INFO:tasks.workunit.client.0.smithi019.stdout:6/967: fdatasync d6/d58/fe3 0 2024-02-22T15:50:44.541 INFO:tasks.workunit.client.1.smithi062.stdout:3/535: fdatasync d5/f6f 0 2024-02-22T15:50:44.541 INFO:tasks.workunit.client.1.smithi062.stdout:8/738: unlink d8/d9/d44/d7e/ceb 0 2024-02-22T15:50:44.541 INFO:tasks.workunit.client.0.smithi019.stdout:6/968: read - d6/d4b/d6d/da3/d117/f11d zero size 2024-02-22T15:50:44.541 INFO:tasks.workunit.client.1.smithi062.stdout:4/668: link d4/d2a/d2d/d46/l6e d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/le7 0 2024-02-22T15:50:44.543 INFO:tasks.workunit.client.0.smithi019.stdout:6/969: mknod d6/d58/c13e 0 2024-02-22T15:50:44.545 INFO:tasks.workunit.client.1.smithi062.stdout:2/709: write d1/f16 [2880692,19050] 0 2024-02-22T15:50:44.545 INFO:tasks.workunit.client.0.smithi019.stdout:6/970: fsync d6/f29 0 2024-02-22T15:50:44.546 INFO:tasks.workunit.client.1.smithi062.stdout:1/562: rmdir d1/d6/d7/d8a 39 2024-02-22T15:50:44.550 INFO:tasks.workunit.client.1.smithi062.stdout:4/669: mkdir d4/d1c/d2c/d6b/d70/de8 0 2024-02-22T15:50:44.550 INFO:tasks.workunit.client.0.smithi019.stdout:6/971: rename d6/f20 to d6/d3e/de5/f13f 0 2024-02-22T15:50:44.551 INFO:tasks.workunit.client.1.smithi062.stdout:1/563: write d1/d6/d36/d76/fa1 [10753,16638] 0 2024-02-22T15:50:44.551 INFO:tasks.workunit.client.1.smithi062.stdout:5/497: getdents d0/d3/d2f/d59 0 2024-02-22T15:50:44.554 INFO:tasks.workunit.client.1.smithi062.stdout:2/710: unlink d1/d47/d32/l60 0 2024-02-22T15:50:44.556 INFO:tasks.workunit.client.1.smithi062.stdout:4/670: readlink d4/d1c/d2c/l68 0 2024-02-22T15:50:44.558 INFO:tasks.workunit.client.1.smithi062.stdout:0/657: link d1/d3/d4/d1e/fd8 d1/d3/d4/d1a/d76/d8c/d5d/dc1/fd9 0 2024-02-22T15:50:44.558 INFO:tasks.workunit.client.0.smithi019.stdout:6/972: truncate d6/d31/d43/f11e 370499 0 2024-02-22T15:50:44.561 INFO:tasks.workunit.client.1.smithi062.stdout:2/711: rmdir d1/de1 39 2024-02-22T15:50:44.562 INFO:tasks.workunit.client.0.smithi019.stdout:6/973: rename d6/d3e/l80 to d6/d4b/da9/d67/l140 0 2024-02-22T15:50:44.566 INFO:tasks.workunit.client.1.smithi062.stdout:4/671: truncate d4/d1c/f3d 5198709 0 2024-02-22T15:50:44.566 INFO:tasks.workunit.client.1.smithi062.stdout:8/739: link d8/d9/d44/c80 d8/d9/d2c/d49/d94/dc1/de9/dd1/ced 0 2024-02-22T15:50:44.569 INFO:tasks.workunit.client.1.smithi062.stdout:9/573: dread d4/d15/d5f/d75/d84/f3e [0,4194304] 0 2024-02-22T15:50:44.569 INFO:tasks.workunit.client.0.smithi019.stdout:6/974: write d6/f57 [1054005,113478] 0 2024-02-22T15:50:44.570 INFO:tasks.workunit.client.1.smithi062.stdout:3/536: getdents d5/d16/d21/d55 0 2024-02-22T15:50:44.571 INFO:tasks.workunit.client.1.smithi062.stdout:0/658: write d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/fd2 [651074,115978] 0 2024-02-22T15:50:44.575 INFO:tasks.workunit.client.1.smithi062.stdout:2/712: write d1/d8/d62/d73/dbd/d7d/d84/fd5 [509263,10114] 0 2024-02-22T15:50:44.575 INFO:tasks.workunit.client.0.smithi019.stdout:6/975: write d6/d4b/d8d/f96 [2588635,5975] 0 2024-02-22T15:50:44.576 INFO:tasks.workunit.client.1.smithi062.stdout:4/672: dread - d4/d2a/d2d/d46/fbf zero size 2024-02-22T15:50:44.576 INFO:tasks.workunit.client.0.smithi019.stdout:6/976: fdatasync d6/fe9 0 2024-02-22T15:50:44.579 INFO:tasks.workunit.client.1.smithi062.stdout:1/564: dwrite d1/d6/de/f9a [0,4194304] 0 2024-02-22T15:50:44.580 INFO:tasks.workunit.client.0.smithi019.stdout:6/977: getdents d6/d4b/d6d/d11f 0 2024-02-22T15:50:44.580 INFO:tasks.workunit.client.1.smithi062.stdout:6/684: dread d5/df/d40/d44/d76/d56/f61 [0,4194304] 0 2024-02-22T15:50:44.580 INFO:tasks.workunit.client.0.smithi019.stdout:6/978: chown d6/d4b/d6d/c83 11975334 1 2024-02-22T15:50:44.581 INFO:tasks.workunit.client.0.smithi019.stdout:6/979: chown d6/d98 0 1 2024-02-22T15:50:44.585 INFO:tasks.workunit.client.1.smithi062.stdout:2/713: write d1/d8/f15 [316948,88859] 0 2024-02-22T15:50:44.588 INFO:tasks.workunit.client.1.smithi062.stdout:0/659: read d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/fa3 [826683,87218] 0 2024-02-22T15:50:44.588 INFO:tasks.workunit.client.1.smithi062.stdout:8/740: creat d8/d9/d44/d4d/fee x:0 0 0 2024-02-22T15:50:44.588 INFO:tasks.workunit.client.1.smithi062.stdout:9/574: rename d4/d15/d5f/d6a/c78 to d4/d15/d31/cb3 0 2024-02-22T15:50:44.593 INFO:tasks.workunit.client.1.smithi062.stdout:5/498: dread d0/d3/da/d32/d42/f65 [0,4194304] 0 2024-02-22T15:50:44.593 INFO:tasks.workunit.client.0.smithi019.stdout:6/980: truncate d6/f41 2354189 0 2024-02-22T15:50:44.594 INFO:tasks.workunit.client.1.smithi062.stdout:6/685: rmdir d5/df/d40/dac/db6/dc0 39 2024-02-22T15:50:44.594 INFO:tasks.workunit.client.1.smithi062.stdout:7/463: dwrite d0/d6/d2e/f86 [0,4194304] 0 2024-02-22T15:50:44.595 INFO:tasks.workunit.client.0.smithi019.stdout:6/981: rmdir d6/d4b/d71/d88/ded/d121 39 2024-02-22T15:50:44.596 INFO:tasks.workunit.client.1.smithi062.stdout:7/464: fdatasync d0/d6/d2e/d35/f70 0 2024-02-22T15:50:44.596 INFO:tasks.workunit.client.0.smithi019.stdout:6/982: symlink d6/d4b/d6d/d7b/d7f/l141 0 2024-02-22T15:50:44.598 INFO:tasks.workunit.client.1.smithi062.stdout:4/673: rename d4/d2a/d2d/d46/fbf to d4/d13/d20/fe9 0 2024-02-22T15:50:44.599 INFO:tasks.workunit.client.0.smithi019.stdout:6/983: getdents d6/d4b/d6d/da3/d117 0 2024-02-22T15:50:44.600 INFO:tasks.workunit.client.1.smithi062.stdout:8/741: rmdir d8/d42/d87 39 2024-02-22T15:50:44.600 INFO:tasks.workunit.client.1.smithi062.stdout:8/742: fdatasync d8/f12 0 2024-02-22T15:50:44.601 INFO:tasks.workunit.client.1.smithi062.stdout:9/575: mknod d4/d1c/d83/cb4 0 2024-02-22T15:50:44.603 INFO:tasks.workunit.client.0.smithi019.stdout:6/984: mkdir d6/d4b/da9/d67/d11b/d142 0 2024-02-22T15:50:44.605 INFO:tasks.workunit.client.1.smithi062.stdout:1/565: fsync d1/d6/d7/d4a/d65/d83/faa 0 2024-02-22T15:50:44.609 INFO:tasks.workunit.client.1.smithi062.stdout:1/566: dread - d1/d6/d7/d2b/d47/d63/fa6 zero size 2024-02-22T15:50:44.611 INFO:tasks.workunit.client.1.smithi062.stdout:6/686: chown d5/df/f21 14 1 2024-02-22T15:50:44.612 INFO:tasks.workunit.client.1.smithi062.stdout:5/499: unlink d0/d3/da/d13/d23/d31/f34 0 2024-02-22T15:50:44.612 INFO:tasks.workunit.client.1.smithi062.stdout:2/714: dwrite d1/d47/d32/d52/dba/fc8 [0,4194304] 0 2024-02-22T15:50:44.613 INFO:tasks.workunit.client.1.smithi062.stdout:3/537: truncate d5/f8 735367 0 2024-02-22T15:50:44.615 INFO:tasks.workunit.client.1.smithi062.stdout:7/465: chown d0/d14/d19/d30/f7f 738767 1 2024-02-22T15:50:44.616 INFO:tasks.workunit.client.0.smithi019.stdout:6/985: write d6/d4b/d71/f7d [32631,92296] 0 2024-02-22T15:50:44.618 INFO:tasks.workunit.client.1.smithi062.stdout:4/674: creat d4/d13/d20/fea x:0 0 0 2024-02-22T15:50:44.618 INFO:tasks.workunit.client.1.smithi062.stdout:8/743: unlink d8/d9/d44/d4d/d51/cd6 0 2024-02-22T15:50:44.620 INFO:tasks.workunit.client.1.smithi062.stdout:9/576: symlink d4/d15/d5f/d75/d97/lb5 0 2024-02-22T15:50:44.621 INFO:tasks.workunit.client.1.smithi062.stdout:0/660: dread d1/dd/f36 [0,4194304] 0 2024-02-22T15:50:44.622 INFO:tasks.workunit.client.0.smithi019.stdout:6/986: rename d6/d58/d6a/fc7 to d6/d4b/da9/f143 0 2024-02-22T15:50:44.622 INFO:tasks.workunit.client.1.smithi062.stdout:0/661: chown d1/dd/f36 3701371 1 2024-02-22T15:50:44.624 INFO:tasks.workunit.client.1.smithi062.stdout:6/687: creat d5/df/d40/d44/d76/dae/fde x:0 0 0 2024-02-22T15:50:44.627 INFO:tasks.workunit.client.1.smithi062.stdout:1/567: rename d1/d6/d7/d2b/d47/d7e/fa7 to d1/d6/d7/d4a/d65/d83/fbd 0 2024-02-22T15:50:44.631 INFO:tasks.workunit.client.1.smithi062.stdout:4/675: creat d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/feb x:0 0 0 2024-02-22T15:50:44.632 INFO:tasks.workunit.client.1.smithi062.stdout:3/538: chown d5/d1a/c58 1 1 2024-02-22T15:50:44.635 INFO:tasks.workunit.client.1.smithi062.stdout:5/500: mknod d0/d3/da/d13/d23/d4e/c98 0 2024-02-22T15:50:44.644 INFO:tasks.workunit.client.1.smithi062.stdout:6/688: dwrite d5/df/d40/d44/d4a/fad [0,4194304] 0 2024-02-22T15:50:44.645 INFO:tasks.workunit.client.1.smithi062.stdout:4/676: unlink d4/f11 0 2024-02-22T15:50:44.647 INFO:tasks.workunit.client.1.smithi062.stdout:7/466: dwrite d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:44.648 INFO:tasks.workunit.client.1.smithi062.stdout:8/744: write d8/d9/d2c/d49/d94/dc1/f9a [1442959,111717] 0 2024-02-22T15:50:44.651 INFO:tasks.workunit.client.1.smithi062.stdout:0/662: write d1/d3/d4/d1e/fd8 [675075,56309] 0 2024-02-22T15:50:44.653 INFO:tasks.workunit.client.1.smithi062.stdout:7/467: fsync d0/d6/d2e/d35/f70 0 2024-02-22T15:50:44.654 INFO:tasks.workunit.client.1.smithi062.stdout:8/745: dread - d8/d9/d2c/d49/d5e/fe2 zero size 2024-02-22T15:50:44.657 INFO:tasks.workunit.client.1.smithi062.stdout:2/715: dread d1/d47/d32/f44 [0,4194304] 0 2024-02-22T15:50:44.660 INFO:tasks.workunit.client.1.smithi062.stdout:1/568: creat d1/d6/d7/d2b/d47/d63/db2/fbe x:0 0 0 2024-02-22T15:50:44.661 INFO:tasks.workunit.client.1.smithi062.stdout:3/539: dwrite d5/d1a/d3c/f54 [0,4194304] 0 2024-02-22T15:50:44.662 INFO:tasks.workunit.client.1.smithi062.stdout:5/501: creat d0/d3/da/d32/d4f/d72/f99 x:0 0 0 2024-02-22T15:50:44.665 INFO:tasks.workunit.client.1.smithi062.stdout:8/746: creat d8/d9/d44/d58/fef x:0 0 0 2024-02-22T15:50:44.666 INFO:tasks.workunit.client.1.smithi062.stdout:9/577: getdents d4/d15/d5f/d75 0 2024-02-22T15:50:44.668 INFO:tasks.workunit.client.1.smithi062.stdout:4/677: mknod d4/d2a/d2d/cec 0 2024-02-22T15:50:44.671 INFO:tasks.workunit.client.1.smithi062.stdout:0/663: mknod d1/d3/d4/d1a/d1d/cda 0 2024-02-22T15:50:44.675 INFO:tasks.workunit.client.1.smithi062.stdout:0/664: write d1/d3/d4/d1e/d74/f70 [1688705,7809] 0 2024-02-22T15:50:44.678 INFO:tasks.workunit.client.1.smithi062.stdout:9/578: dwrite d4/d15/d31/f60 [4194304,4194304] 0 2024-02-22T15:50:44.680 INFO:tasks.workunit.client.1.smithi062.stdout:6/689: rename d5/fbb to d5/df/d40/d44/fdf 0 2024-02-22T15:50:44.682 INFO:tasks.workunit.client.1.smithi062.stdout:8/747: mknod d8/d9/d44/d58/de0/cf0 0 2024-02-22T15:50:44.686 INFO:tasks.workunit.client.1.smithi062.stdout:7/468: getdents d0/d6/d2e 0 2024-02-22T15:50:44.688 INFO:tasks.workunit.client.1.smithi062.stdout:2/716: getdents d1/d8/d62 0 2024-02-22T15:50:44.689 INFO:tasks.workunit.client.1.smithi062.stdout:4/678: dwrite d4/d1c/d2c/d6b/d9b/fc0 [0,4194304] 0 2024-02-22T15:50:44.691 INFO:tasks.workunit.client.1.smithi062.stdout:7/469: read d0/d6/f23 [4558199,15767] 0 2024-02-22T15:50:44.693 INFO:tasks.workunit.client.1.smithi062.stdout:9/579: mkdir d4/d15/d5e/d63/da0/d92/db6 0 2024-02-22T15:50:44.695 INFO:tasks.workunit.client.1.smithi062.stdout:8/748: mkdir d8/d9/d44/df1 0 2024-02-22T15:50:44.697 INFO:tasks.workunit.client.1.smithi062.stdout:0/665: dwrite d1/d3/d4/d1e/d74/f70 [0,4194304] 0 2024-02-22T15:50:44.697 INFO:tasks.workunit.client.1.smithi062.stdout:4/679: stat d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd7 0 2024-02-22T15:50:44.699 INFO:tasks.workunit.client.1.smithi062.stdout:3/540: write d5/f1d [4749886,130765] 0 2024-02-22T15:50:44.699 INFO:tasks.workunit.client.1.smithi062.stdout:2/717: dread - d1/d8/d62/d73/dbd/dbb/dc4/fe7 zero size 2024-02-22T15:50:44.703 INFO:tasks.workunit.client.1.smithi062.stdout:1/569: dread d1/d6/d10/f5c [0,4194304] 0 2024-02-22T15:50:44.704 INFO:tasks.workunit.client.1.smithi062.stdout:6/690: creat d5/dd/d33/d66/db9/fe0 x:0 0 0 2024-02-22T15:50:44.707 INFO:tasks.workunit.client.1.smithi062.stdout:4/680: rename d4/d66/c72 to d4/d1c/d2c/d6b/d70/ced 0 2024-02-22T15:50:44.708 INFO:tasks.workunit.client.1.smithi062.stdout:8/749: dread d8/d9/d2c/d74/fa0 [0,4194304] 0 2024-02-22T15:50:44.709 INFO:tasks.workunit.client.1.smithi062.stdout:3/541: creat d5/d16/d21/d36/d5a/d85/fb2 x:0 0 0 2024-02-22T15:50:44.710 INFO:tasks.workunit.client.1.smithi062.stdout:1/570: symlink d1/d6/d7/d2b/d47/d7e/lbf 0 2024-02-22T15:50:44.711 INFO:tasks.workunit.client.1.smithi062.stdout:0/666: mkdir d1/d3/d4/d1a/d1d/ddb 0 2024-02-22T15:50:44.712 INFO:tasks.workunit.client.1.smithi062.stdout:8/750: fdatasync d8/d9/d44/f45 0 2024-02-22T15:50:44.712 INFO:tasks.workunit.client.1.smithi062.stdout:4/681: write d4/d1c/f23 [673979,40244] 0 2024-02-22T15:50:44.716 INFO:tasks.workunit.client.1.smithi062.stdout:3/542: write d5/d1a/d3c/f54 [5916881,102446] 0 2024-02-22T15:50:44.717 INFO:tasks.workunit.client.1.smithi062.stdout:4/682: fsync d4/d13/d20/fea 0 2024-02-22T15:50:44.720 INFO:tasks.workunit.client.1.smithi062.stdout:8/751: read d8/d9/d48/fbf [578450,121554] 0 2024-02-22T15:50:44.722 INFO:tasks.workunit.client.1.smithi062.stdout:9/580: dwrite d4/fa1 [4194304,4194304] 0 2024-02-22T15:50:44.726 INFO:tasks.workunit.client.1.smithi062.stdout:2/718: dwrite d1/d1e/faf [4194304,4194304] 0 2024-02-22T15:50:44.727 INFO:tasks.workunit.client.1.smithi062.stdout:9/581: readlink d4/d15/l16 0 2024-02-22T15:50:44.727 INFO:tasks.workunit.client.1.smithi062.stdout:6/691: dwrite d5/dd/d33/d66/db9/fa6 [0,4194304] 0 2024-02-22T15:50:44.729 INFO:tasks.workunit.client.1.smithi062.stdout:0/667: mkdir d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/d97/ddc 0 2024-02-22T15:50:44.735 INFO:tasks.workunit.client.1.smithi062.stdout:0/668: chown d1/d3/d4/d1a/d76/d8c 445754865 1 2024-02-22T15:50:44.738 INFO:tasks.workunit.client.1.smithi062.stdout:2/719: mknod d1/d47/cf1 0 2024-02-22T15:50:44.741 INFO:tasks.workunit.client.1.smithi062.stdout:3/543: dwrite d5/d16/d21/d36/d5a/d85/fb2 [0,4194304] 0 2024-02-22T15:50:44.743 INFO:tasks.workunit.client.1.smithi062.stdout:2/720: write d1/fb8 [2895967,120109] 0 2024-02-22T15:50:44.750 INFO:tasks.workunit.client.1.smithi062.stdout:0/669: creat d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8e/fdd x:0 0 0 2024-02-22T15:50:44.752 INFO:tasks.workunit.client.1.smithi062.stdout:2/721: chown d1/d8/d62/d73/dbd/d65/lc0 281 1 2024-02-22T15:50:44.755 INFO:tasks.workunit.client.1.smithi062.stdout:0/670: read d1/d3/f39 [715996,107456] 0 2024-02-22T15:50:44.756 INFO:tasks.workunit.client.1.smithi062.stdout:1/571: dwrite d1/f30 [0,4194304] 0 2024-02-22T15:50:44.756 INFO:tasks.workunit.client.1.smithi062.stdout:0/671: chown d1/d38 0 1 2024-02-22T15:50:44.759 INFO:tasks.workunit.client.1.smithi062.stdout:0/672: chown d1/d3/d4/d1a/d76/d8c/d5d/dc3/d85/d8d/fba 3537 1 2024-02-22T15:50:44.761 INFO:tasks.workunit.client.1.smithi062.stdout:4/683: dread d4/d1c/d2c/f87 [0,4194304] 0 2024-02-22T15:50:44.762 INFO:tasks.workunit.client.1.smithi062.stdout:4/684: chown d4/d1c/d2c/d6b/d9b/ca0 8 1 2024-02-22T15:50:44.765 INFO:tasks.workunit.client.1.smithi062.stdout:4/685: symlink d4/d2a/d2d/d64/lee 0 2024-02-22T15:50:44.769 INFO:tasks.workunit.client.1.smithi062.stdout:1/572: dread d1/d6/f1a [0,4194304] 0 2024-02-22T15:50:44.770 INFO:tasks.workunit.client.1.smithi062.stdout:4/686: chown d4/d2a/c62 10923145 1 2024-02-22T15:50:44.770 INFO:tasks.workunit.client.1.smithi062.stdout:9/582: dread d4/f1b [0,4194304] 0 2024-02-22T15:50:44.771 INFO:tasks.workunit.client.1.smithi062.stdout:0/673: rename d1/d3/d4/d1a to d1/d13/dde 0 2024-02-22T15:50:44.772 INFO:tasks.workunit.client.1.smithi062.stdout:8/752: dwrite d8/d9/d2c/f78 [0,4194304] 0 2024-02-22T15:50:44.774 INFO:tasks.workunit.client.1.smithi062.stdout:9/583: chown d4/d15/d5f/d75/d97/lb5 3490951 1 2024-02-22T15:50:44.776 INFO:tasks.workunit.client.1.smithi062.stdout:0/674: stat d1/d13/dde/l27 0 2024-02-22T15:50:44.779 INFO:tasks.workunit.client.1.smithi062.stdout:1/573: getdents d1/d6 0 2024-02-22T15:50:44.788 INFO:tasks.workunit.client.1.smithi062.stdout:0/675: symlink d1/d13/dde/d76/d8c/d5d/dc3/d85/ldf 0 2024-02-22T15:50:44.789 INFO:tasks.workunit.client.1.smithi062.stdout:9/584: getdents d4/d28/d50 0 2024-02-22T15:50:44.793 INFO:tasks.workunit.client.1.smithi062.stdout:2/722: dwrite d1/d1e/f9a [4194304,4194304] 0 2024-02-22T15:50:44.793 INFO:tasks.workunit.client.1.smithi062.stdout:4/687: dwrite d4/d1c/dac/fce [0,4194304] 0 2024-02-22T15:50:44.794 INFO:tasks.workunit.client.1.smithi062.stdout:1/574: truncate d1/d6/d7/d2b/d56/d95/f9b 564296 0 2024-02-22T15:50:44.802 INFO:tasks.workunit.client.1.smithi062.stdout:1/575: mknod d1/d6/d7/d4a/d65/d83/cc0 0 2024-02-22T15:50:44.804 INFO:tasks.workunit.client.1.smithi062.stdout:9/585: fsync d4/d28/f30 0 2024-02-22T15:50:44.804 INFO:tasks.workunit.client.1.smithi062.stdout:8/753: dwrite d8/d9/d44/fb2 [0,4194304] 0 2024-02-22T15:50:44.805 INFO:tasks.workunit.client.1.smithi062.stdout:8/754: stat d8/d9/d2c/d49/d94/da1/la8 0 2024-02-22T15:50:44.815 INFO:tasks.workunit.client.1.smithi062.stdout:9/586: creat d4/d28/d50/da3/fb7 x:0 0 0 2024-02-22T15:50:44.816 INFO:tasks.workunit.client.1.smithi062.stdout:8/755: fsync d8/d28/f4c 0 2024-02-22T15:50:44.817 INFO:tasks.workunit.client.1.smithi062.stdout:1/576: creat d1/d6/d7/d4a/d65/d83/fc1 x:0 0 0 2024-02-22T15:50:44.818 INFO:tasks.workunit.client.1.smithi062.stdout:4/688: dwrite d4/d1c/d2c/d6b/d9b/fc0 [4194304,4194304] 0 2024-02-22T15:50:44.819 INFO:tasks.workunit.client.1.smithi062.stdout:2/723: dread d1/d8/d62/d73/dbd/d7d/d84/fd5 [0,4194304] 0 2024-02-22T15:50:44.829 INFO:tasks.workunit.client.1.smithi062.stdout:2/724: write d1/d1e/d38/f74 [2254864,8471] 0 2024-02-22T15:50:44.831 INFO:tasks.workunit.client.1.smithi062.stdout:9/587: dwrite d4/f7d [0,4194304] 0 2024-02-22T15:50:44.833 INFO:tasks.workunit.client.1.smithi062.stdout:9/588: readlink d4/l41 0 2024-02-22T15:50:44.833 INFO:tasks.workunit.client.0.smithi019.stdout:1/865: sync 2024-02-22T15:50:44.833 INFO:tasks.workunit.client.0.smithi019.stdout:2/808: sync 2024-02-22T15:50:44.833 INFO:tasks.workunit.client.0.smithi019.stdout:0/780: sync 2024-02-22T15:50:44.835 INFO:tasks.workunit.client.0.smithi019.stdout:1/866: fdatasync d8/d18/d64/d7f/de3/ffa 0 2024-02-22T15:50:44.837 INFO:tasks.workunit.client.0.smithi019.stdout:0/781: truncate f1 335814 0 2024-02-22T15:50:44.837 INFO:tasks.workunit.client.0.smithi019.stdout:2/809: mkdir d1/d17/d67/dfa/d11e/d126 0 2024-02-22T15:50:44.839 INFO:tasks.workunit.client.0.smithi019.stdout:1/867: read d8/d2a/f3f [4036024,85520] 0 2024-02-22T15:50:44.840 INFO:tasks.workunit.client.1.smithi062.stdout:9/589: dread d4/d15/d5f/d75/d84/f59 [0,4194304] 0 2024-02-22T15:50:44.840 INFO:tasks.workunit.client.0.smithi019.stdout:2/810: write d1/d17/d1c/f34 [3247370,28572] 0 2024-02-22T15:50:44.843 INFO:tasks.workunit.client.1.smithi062.stdout:4/689: rename d4/d1c/d30/f32 to d4/d1c/d2c/d6b/d9b/db1/dc6/fef 0 2024-02-22T15:50:44.844 INFO:tasks.workunit.client.0.smithi019.stdout:1/868: link d8/d15/df5/c10e d8/d15/da4/c125 0 2024-02-22T15:50:44.849 INFO:tasks.workunit.client.1.smithi062.stdout:9/590: creat d4/d15/d5f/d75/fb8 x:0 0 0 2024-02-22T15:50:44.849 INFO:tasks.workunit.client.1.smithi062.stdout:8/756: dread d8/d9/fe [0,4194304] 0 2024-02-22T15:50:44.849 INFO:tasks.workunit.client.1.smithi062.stdout:0/676: dread d1/d13/dde/d76/d8c/d5d/dc3/f47 [0,4194304] 0 2024-02-22T15:50:44.850 INFO:tasks.workunit.client.0.smithi019.stdout:0/782: truncate d4/db/d76/d77/faa 914785 0 2024-02-22T15:50:44.851 INFO:tasks.workunit.client.1.smithi062.stdout:9/591: stat d4/d15/f23 0 2024-02-22T15:50:44.852 INFO:tasks.workunit.client.0.smithi019.stdout:0/783: dread - d4/d22/d3d/de8/ff8 zero size 2024-02-22T15:50:44.856 INFO:tasks.workunit.client.1.smithi062.stdout:2/725: dwrite d1/d47/f2b [0,4194304] 0 2024-02-22T15:50:44.856 INFO:tasks.workunit.client.1.smithi062.stdout:0/677: chown d1/d13/dde/d76/d8c/d5d/dc3/l59 20252264 1 2024-02-22T15:50:44.858 INFO:tasks.workunit.client.0.smithi019.stdout:1/869: dwrite d8/d23/faf [0,4194304] 0 2024-02-22T15:50:44.858 INFO:tasks.workunit.client.0.smithi019.stdout:0/784: mknod d4/db/d1b/d63/db0/da1/dbb/c10e 0 2024-02-22T15:50:44.860 INFO:tasks.workunit.client.1.smithi062.stdout:0/678: write d1/d38/f98 [91237,88682] 0 2024-02-22T15:50:44.860 INFO:tasks.workunit.client.0.smithi019.stdout:1/870: write d8/d18/d64/d7f/d94/f121 [106547,51864] 0 2024-02-22T15:50:44.861 INFO:tasks.workunit.client.1.smithi062.stdout:2/726: creat d1/d8/d62/d73/dbd/ff2 x:0 0 0 2024-02-22T15:50:44.861 INFO:tasks.workunit.client.0.smithi019.stdout:0/785: symlink d4/d23/df5/l10f 0 2024-02-22T15:50:44.862 INFO:tasks.workunit.client.0.smithi019.stdout:1/871: write d8/d18/f27 [1615541,108656] 0 2024-02-22T15:50:44.864 INFO:tasks.workunit.client.1.smithi062.stdout:2/727: write d1/f16 [2151841,82412] 0 2024-02-22T15:50:44.867 INFO:tasks.workunit.client.0.smithi019.stdout:1/872: rename d8/d18/d72 to d8/d15/d34/d9e/d126 0 2024-02-22T15:50:44.872 INFO:tasks.workunit.client.0.smithi019.stdout:1/873: rmdir d8/d18/d85 39 2024-02-22T15:50:44.874 INFO:tasks.workunit.client.1.smithi062.stdout:2/728: dread d1/d8/d62/d73/da6/fb1 [0,4194304] 0 2024-02-22T15:50:44.875 INFO:tasks.workunit.client.0.smithi019.stdout:2/811: dread d1/d17/d1c/d66/db9/dd0/fe5 [0,4194304] 0 2024-02-22T15:50:44.876 INFO:tasks.workunit.client.0.smithi019.stdout:0/786: dwrite d4/db/d1b/d63/db0/da1/fbc [0,4194304] 0 2024-02-22T15:50:44.879 INFO:tasks.workunit.client.1.smithi062.stdout:8/757: truncate d8/f29 1722917 0 2024-02-22T15:50:44.879 INFO:tasks.workunit.client.0.smithi019.stdout:8/750: sync 2024-02-22T15:50:44.880 INFO:tasks.workunit.client.0.smithi019.stdout:9/784: sync 2024-02-22T15:50:44.880 INFO:tasks.workunit.client.0.smithi019.stdout:6/987: sync 2024-02-22T15:50:44.880 INFO:tasks.workunit.client.0.smithi019.stdout:4/761: sync 2024-02-22T15:50:44.880 INFO:tasks.workunit.client.0.smithi019.stdout:5/799: sync 2024-02-22T15:50:44.880 INFO:tasks.workunit.client.0.smithi019.stdout:3/649: sync 2024-02-22T15:50:44.881 INFO:tasks.workunit.client.1.smithi062.stdout:8/758: symlink d8/d28/d70/lf2 0 2024-02-22T15:50:44.884 INFO:tasks.workunit.client.1.smithi062.stdout:9/592: link d4/d15/d5f/d75/d97/lb5 d4/d15/d31/lb9 0 2024-02-22T15:50:44.884 INFO:tasks.workunit.client.0.smithi019.stdout:1/874: link d8/d18/l28 d8/d2a/d75/l127 0 2024-02-22T15:50:44.884 INFO:tasks.workunit.client.1.smithi062.stdout:8/759: chown d8/cd 20037 1 2024-02-22T15:50:44.885 INFO:tasks.workunit.client.0.smithi019.stdout:3/650: truncate d9/d14/f9b 335423 0 2024-02-22T15:50:44.886 INFO:tasks.workunit.client.0.smithi019.stdout:1/875: chown d8/d2a/c48 3819 1 2024-02-22T15:50:44.886 INFO:tasks.workunit.client.0.smithi019.stdout:3/651: fsync d9/d22/f9d 0 2024-02-22T15:50:44.886 INFO:tasks.workunit.client.1.smithi062.stdout:0/679: dwrite d1/dd/f63 [0,4194304] 0 2024-02-22T15:50:44.887 INFO:tasks.workunit.client.0.smithi019.stdout:6/988: dread d6/f3c [0,4194304] 0 2024-02-22T15:50:44.888 INFO:tasks.workunit.client.0.smithi019.stdout:0/787: symlink d4/db/d1b/d63/db0/da1/dbb/de7/l110 0 2024-02-22T15:50:44.890 INFO:tasks.workunit.client.0.smithi019.stdout:9/785: fsync d2/d1f/d2e/d57/fc7 0 2024-02-22T15:50:44.891 INFO:tasks.workunit.client.1.smithi062.stdout:8/760: creat d8/d9/d2c/d49/d94/dc1/de9/ff3 x:0 0 0 2024-02-22T15:50:44.891 INFO:tasks.workunit.client.1.smithi062.stdout:9/593: symlink d4/d15/d5f/d75/d97/da7/lba 0 2024-02-22T15:50:44.893 INFO:tasks.workunit.client.1.smithi062.stdout:8/761: chown d8/d26/fd7 141210050 1 2024-02-22T15:50:44.894 INFO:tasks.workunit.client.1.smithi062.stdout:0/680: creat d1/d13/dde/d1d/ddb/fe0 x:0 0 0 2024-02-22T15:50:44.895 INFO:tasks.workunit.client.0.smithi019.stdout:4/762: truncate d2/db/d54/d8c/f96 1034254 0 2024-02-22T15:50:44.895 INFO:tasks.workunit.client.0.smithi019.stdout:2/812: rename d1/d17/d21/d71/d60/d112 to d1/da4/d127 0 2024-02-22T15:50:44.897 INFO:tasks.workunit.client.0.smithi019.stdout:4/763: chown d2/db/d54/c5d 108935687 1 2024-02-22T15:50:44.898 INFO:tasks.workunit.client.1.smithi062.stdout:9/594: symlink d4/d15/lbb 0 2024-02-22T15:50:44.900 INFO:tasks.workunit.client.0.smithi019.stdout:5/800: write d8/fe [2403843,34321] 0 2024-02-22T15:50:44.900 INFO:tasks.workunit.client.0.smithi019.stdout:8/751: symlink d9/d14/d6f/d7e/d94/d9a/deb/lfe 0 2024-02-22T15:50:44.903 INFO:tasks.workunit.client.0.smithi019.stdout:1/876: rename d8/d2a/d75/d81/d8e/lf2 to d8/d44/d5d/l128 0 2024-02-22T15:50:44.903 INFO:tasks.workunit.client.1.smithi062.stdout:0/681: chown d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fa4 24163 1 2024-02-22T15:50:44.904 INFO:tasks.workunit.client.0.smithi019.stdout:0/788: mknod d4/c111 0 2024-02-22T15:50:44.906 INFO:tasks.workunit.client.1.smithi062.stdout:9/595: dread d4/d1c/f58 [0,4194304] 0 2024-02-22T15:50:44.908 INFO:tasks.workunit.client.0.smithi019.stdout:0/789: fsync d4/db/d49/fb1 0 2024-02-22T15:50:44.908 INFO:tasks.workunit.client.1.smithi062.stdout:2/729: dwrite d1/d1e/d38/f81 [0,4194304] 0 2024-02-22T15:50:44.909 INFO:tasks.workunit.client.0.smithi019.stdout:4/764: dwrite d2/db/d92/f91 [4194304,4194304] 0 2024-02-22T15:50:44.911 INFO:tasks.workunit.client.0.smithi019.stdout:1/877: creat d8/d2a/d75/d81/d8e/f129 x:0 0 0 2024-02-22T15:50:44.911 INFO:tasks.workunit.client.1.smithi062.stdout:8/762: dwrite d8/d28/d4e/faa [0,4194304] 0 2024-02-22T15:50:44.913 INFO:tasks.workunit.client.0.smithi019.stdout:5/801: dread d8/d11/f65 [0,4194304] 0 2024-02-22T15:50:44.915 INFO:tasks.workunit.client.0.smithi019.stdout:2/813: rename d1/d17/d1c/d66 to d1/d17/d1c/d128 0 2024-02-22T15:50:44.922 INFO:tasks.workunit.client.1.smithi062.stdout:2/730: mkdir d1/d1e/d38/d76/dc5/df3 0 2024-02-22T15:50:44.923 INFO:tasks.workunit.client.0.smithi019.stdout:1/878: dread d8/d15/f26 [0,4194304] 0 2024-02-22T15:50:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:2/814: creat d1/d86/d11f/f129 x:0 0 0 2024-02-22T15:50:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:6/989: dwrite d6/ff [0,4194304] 0 2024-02-22T15:50:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:1/879: dread - d8/d18/ffb zero size 2024-02-22T15:50:44.925 INFO:tasks.workunit.client.0.smithi019.stdout:4/765: getdents d2/db/d14/d23/d21/d35/d85/def 0 2024-02-22T15:50:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:6/990: readlink d6/l24 0 2024-02-22T15:50:44.927 INFO:tasks.workunit.client.0.smithi019.stdout:5/802: rename d8/f25 to d8/dd/d19/d38/d54/db0/f113 0 2024-02-22T15:50:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:2/815: chown d1/d17/d21/d51/f6e 73 1 2024-02-22T15:50:44.928 INFO:tasks.workunit.client.0.smithi019.stdout:6/991: readlink d6/d4b/d8d/leb 0 2024-02-22T15:50:44.929 INFO:tasks.workunit.client.1.smithi062.stdout:2/731: link d1/d8/d62/d73/dbd/dbb/lef d1/d8/d62/d73/dbd/dbb/dc4/lf4 0 2024-02-22T15:50:44.931 INFO:tasks.workunit.client.1.smithi062.stdout:2/732: mkdir d1/d1e/d38/d76/df5 0 2024-02-22T15:50:44.932 INFO:tasks.workunit.client.1.smithi062.stdout:2/733: dread - d1/d8/d62/d73/dbd/ff2 zero size 2024-02-22T15:50:44.936 INFO:tasks.workunit.client.0.smithi019.stdout:6/992: fdatasync d6/f29 0 2024-02-22T15:50:44.936 INFO:tasks.workunit.client.0.smithi019.stdout:5/803: readlink d8/dd/lf5 0 2024-02-22T15:50:44.937 INFO:tasks.workunit.client.0.smithi019.stdout:0/790: dwrite d4/db/d1b/d40/ff3 [0,4194304] 0 2024-02-22T15:50:44.940 INFO:tasks.workunit.client.0.smithi019.stdout:3/652: dread d9/d14/f95 [4194304,4194304] 0 2024-02-22T15:50:44.940 INFO:tasks.workunit.client.0.smithi019.stdout:4/766: creat d2/db/d14/d23/d21/dbf/dd7/ff6 x:0 0 0 2024-02-22T15:50:44.943 INFO:tasks.workunit.client.0.smithi019.stdout:0/791: write d4/d23/f4a [4467377,28950] 0 2024-02-22T15:50:44.946 INFO:tasks.workunit.client.0.smithi019.stdout:9/786: dread d2/d1f/d2e/d48/d89/f54 [0,4194304] 0 2024-02-22T15:50:44.948 INFO:tasks.workunit.client.0.smithi019.stdout:3/653: truncate d9/f2f 1688732 0 2024-02-22T15:50:44.949 INFO:tasks.workunit.client.1.smithi062.stdout:0/682: dread d1/f12 [0,4194304] 0 2024-02-22T15:50:44.950 INFO:tasks.workunit.client.0.smithi019.stdout:4/767: symlink d2/db/d14/d23/d9e/dac/dc2/dc9/lf7 0 2024-02-22T15:50:44.950 INFO:tasks.workunit.client.0.smithi019.stdout:0/792: getdents d4/d22/d2c/d109 0 2024-02-22T15:50:44.951 INFO:tasks.workunit.client.0.smithi019.stdout:9/787: symlink d2/d7/dee/l116 0 2024-02-22T15:50:44.954 INFO:tasks.workunit.client.0.smithi019.stdout:6/993: dwrite d6/d31/d69/fa6 [0,4194304] 0 2024-02-22T15:50:44.959 INFO:tasks.workunit.client.0.smithi019.stdout:3/654: symlink d9/d22/d27/ld3 0 2024-02-22T15:50:44.960 INFO:tasks.workunit.client.0.smithi019.stdout:9/788: mknod d2/d7/d52/c117 0 2024-02-22T15:50:44.962 INFO:tasks.workunit.client.0.smithi019.stdout:3/655: write d9/d26/d3f/d59/d66/fd2 [934955,31230] 0 2024-02-22T15:50:44.962 INFO:tasks.workunit.client.0.smithi019.stdout:9/789: chown d2/d1f/d2e/d48/d45/d5a/daf/db6 1 1 2024-02-22T15:50:44.963 INFO:tasks.workunit.client.0.smithi019.stdout:3/656: dread - d9/d14/fce zero size 2024-02-22T15:50:44.965 INFO:tasks.workunit.client.0.smithi019.stdout:2/816: dread d1/de/f5a [0,4194304] 0 2024-02-22T15:50:44.969 INFO:tasks.workunit.client.0.smithi019.stdout:1/880: write d8/d18/d85/f8a [813123,57204] 0 2024-02-22T15:50:44.971 INFO:tasks.workunit.client.0.smithi019.stdout:4/768: link d2/db/d44/c72 d2/db/d54/cf8 0 2024-02-22T15:50:44.971 INFO:tasks.workunit.client.0.smithi019.stdout:9/790: write d2/d1f/d2e/d7d/f108 [844409,6483] 0 2024-02-22T15:50:44.973 INFO:tasks.workunit.client.0.smithi019.stdout:0/793: getdents d4/db/d24 0 2024-02-22T15:50:44.973 INFO:tasks.workunit.client.0.smithi019.stdout:5/804: write d8/d11/f99 [535395,61070] 0 2024-02-22T15:50:44.974 INFO:tasks.workunit.client.0.smithi019.stdout:2/817: creat d1/d17/d67/dfa/f12a x:0 0 0 2024-02-22T15:50:44.976 INFO:tasks.workunit.client.0.smithi019.stdout:3/657: mknod d9/d26/d3f/cd4 0 2024-02-22T15:50:44.977 INFO:tasks.workunit.client.0.smithi019.stdout:1/881: creat d8/d15/d5b/dbc/f12a x:0 0 0 2024-02-22T15:50:44.982 INFO:tasks.workunit.client.0.smithi019.stdout:0/794: mknod d4/db/d1b/c112 0 2024-02-22T15:50:44.982 INFO:tasks.workunit.client.0.smithi019.stdout:6/994: truncate d6/ff 3887989 0 2024-02-22T15:50:44.985 INFO:tasks.workunit.client.0.smithi019.stdout:9/791: rename d2/d1f/d2e/d48/d45/d5a/daf/db6 to d2/d1f/d2e/d48/d45/d5a/d118 0 2024-02-22T15:50:44.986 INFO:tasks.workunit.client.0.smithi019.stdout:1/882: mknod d8/d2a/d75/def/c12b 0 2024-02-22T15:50:44.987 INFO:tasks.workunit.client.0.smithi019.stdout:9/792: stat d2/d7/d52/dc0/dcd/cfc 0 2024-02-22T15:50:44.988 INFO:tasks.workunit.client.0.smithi019.stdout:3/658: symlink d9/d11/ld5 0 2024-02-22T15:50:44.990 INFO:tasks.workunit.client.0.smithi019.stdout:0/795: dwrite d4/d22/f5f [0,4194304] 0 2024-02-22T15:50:44.991 INFO:tasks.workunit.client.0.smithi019.stdout:6/995: symlink d6/d4b/da9/d67/d8a/d90/l144 0 2024-02-22T15:50:44.994 INFO:tasks.workunit.client.0.smithi019.stdout:1/883: symlink d8/d15/df5/l12c 0 2024-02-22T15:50:44.995 INFO:tasks.workunit.client.0.smithi019.stdout:0/796: write d4/d67/de5/fef [598708,79376] 0 2024-02-22T15:50:44.996 INFO:tasks.workunit.client.0.smithi019.stdout:9/793: mkdir d2/dec/d79/d119 0 2024-02-22T15:50:44.996 INFO:tasks.workunit.client.0.smithi019.stdout:6/996: write d6/f17 [6116546,129322] 0 2024-02-22T15:50:44.997 INFO:tasks.workunit.client.0.smithi019.stdout:2/818: dread d1/d86/f9e [0,4194304] 0 2024-02-22T15:50:44.999 INFO:tasks.workunit.client.0.smithi019.stdout:3/659: fsync d9/f6a 0 2024-02-22T15:50:44.999 INFO:tasks.workunit.client.0.smithi019.stdout:4/769: link d2/db/d14/d23/d9e/fab d2/db/d59/ff9 0 2024-02-22T15:50:45.001 INFO:tasks.workunit.client.1.smithi062.stdout:9/596: write d4/f26 [1247205,60213] 0 2024-02-22T15:50:45.004 INFO:tasks.workunit.client.0.smithi019.stdout:6/997: read - d6/d4b/da9/d67/d8a/f125 zero size 2024-02-22T15:50:45.005 INFO:tasks.workunit.client.0.smithi019.stdout:4/770: chown d2/db/f26 29 1 2024-02-22T15:50:45.006 INFO:tasks.workunit.client.0.smithi019.stdout:5/805: dwrite d8/dd/d8e/f8a [0,4194304] 0 2024-02-22T15:50:45.007 INFO:tasks.workunit.client.0.smithi019.stdout:6/998: write d6/f44 [4588433,42780] 0 2024-02-22T15:50:45.008 INFO:tasks.workunit.client.0.smithi019.stdout:1/884: dread d8/d2a/d75/d81/d8e/f118 [0,4194304] 0 2024-02-22T15:50:45.008 INFO:tasks.workunit.client.1.smithi062.stdout:9/597: dread d4/d15/d5f/d75/d84/f59 [0,4194304] 0 2024-02-22T15:50:45.010 INFO:tasks.workunit.client.0.smithi019.stdout:2/819: fdatasync d1/d17/d21/d51/f6a 0 2024-02-22T15:50:45.010 INFO:tasks.workunit.client.1.smithi062.stdout:8/763: dwrite d8/d42/f7c [0,4194304] 0 2024-02-22T15:50:45.013 INFO:tasks.workunit.client.1.smithi062.stdout:0/683: dwrite d1/d13/dde/d76/d8c/d5d/dc3/f24 [4194304,4194304] 0 2024-02-22T15:50:45.013 INFO:tasks.workunit.client.0.smithi019.stdout:0/797: symlink d4/db/d1b/d63/db0/da1/dbb/l113 0 2024-02-22T15:50:45.018 INFO:tasks.workunit.client.0.smithi019.stdout:2/820: chown d1/d17/d1c/d128/db9/dd0/dee/ffc 21 1 2024-02-22T15:50:45.018 INFO:tasks.workunit.client.0.smithi019.stdout:5/806: mknod d8/dd/d19/d3a/d46/d7e/c114 0 2024-02-22T15:50:45.020 INFO:tasks.workunit.client.1.smithi062.stdout:9/598: symlink d4/d28/d50/d5a/lbc 0 2024-02-22T15:50:45.021 INFO:tasks.workunit.client.0.smithi019.stdout:6/999: rename d6/d31/f3b to d6/d31/d48/f145 0 2024-02-22T15:50:45.021 INFO:tasks.workunit.client.1.smithi062.stdout:8/764: creat d8/d9/d2c/d74/ff4 x:0 0 0 2024-02-22T15:50:45.021 INFO:tasks.workunit.client.1.smithi062.stdout:0/684: dread - d1/d13/f41 zero size 2024-02-22T15:50:45.022 INFO:tasks.workunit.client.1.smithi062.stdout:9/599: readlink d4/d1c/d83/d87/la6 0 2024-02-22T15:50:45.022 INFO:tasks.workunit.client.1.smithi062.stdout:0/685: readlink d1/dd/l89 0 2024-02-22T15:50:45.024 INFO:tasks.workunit.client.1.smithi062.stdout:9/600: write d4/f26 [1087054,54229] 0 2024-02-22T15:50:45.024 INFO:tasks.workunit.client.1.smithi062.stdout:8/765: read d8/d9/d2c/d49/d94/dc1/f9a [501504,121670] 0 2024-02-22T15:50:45.025 INFO:tasks.workunit.client.0.smithi019.stdout:5/807: write d8/dd/d19/d77/d7a/ffd [430870,56332] 0 2024-02-22T15:50:45.025 INFO:tasks.workunit.client.0.smithi019.stdout:1/885: mknod d8/d2a/d75/da0/c12d 0 2024-02-22T15:50:45.026 INFO:tasks.workunit.client.1.smithi062.stdout:8/766: chown d8/d26/fb4 39 1 2024-02-22T15:50:45.027 INFO:tasks.workunit.client.0.smithi019.stdout:3/660: dwrite d9/d26/d2d/f4b [0,4194304] 0 2024-02-22T15:50:45.027 INFO:tasks.workunit.client.0.smithi019.stdout:0/798: creat d4/db/d24/d80/daf/f114 x:0 0 0 2024-02-22T15:50:45.028 INFO:tasks.workunit.client.0.smithi019.stdout:2/821: mkdir d1/d17/d1c/d128/d12b 0 2024-02-22T15:50:45.028 INFO:tasks.workunit.client.0.smithi019.stdout:1/886: chown d8/d15/d34/fc8 2496 1 2024-02-22T15:50:45.029 INFO:tasks.workunit.client.0.smithi019.stdout:4/771: write d2/d3f/d48/d76/fa8 [1023308,108316] 0 2024-02-22T15:50:45.030 INFO:tasks.workunit.client.1.smithi062.stdout:8/767: mkdir d8/d9/d2c/d74/df5 0 2024-02-22T15:50:45.032 INFO:tasks.workunit.client.1.smithi062.stdout:0/686: rename d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/ccf to d1/d13/dde/d76/d8c/d2c/d84/ce1 0 2024-02-22T15:50:45.033 INFO:tasks.workunit.client.0.smithi019.stdout:0/799: unlink d4/l12 0 2024-02-22T15:50:45.033 INFO:tasks.workunit.client.0.smithi019.stdout:9/794: dwrite d2/d7/f1b [0,4194304] 0 2024-02-22T15:50:45.034 INFO:tasks.workunit.client.0.smithi019.stdout:2/822: mkdir d1/d46/ddb/d10d/d12c 0 2024-02-22T15:50:45.036 INFO:tasks.workunit.client.0.smithi019.stdout:0/800: readlink d4/db/d1b/d40/d44/l48 0 2024-02-22T15:50:45.037 INFO:tasks.workunit.client.0.smithi019.stdout:2/823: chown d1/d17/d21/d51/f6e 120947 1 2024-02-22T15:50:45.039 INFO:tasks.workunit.client.0.smithi019.stdout:9/795: fsync d2/dc8/fb0 0 2024-02-22T15:50:45.041 INFO:tasks.workunit.client.0.smithi019.stdout:0/801: creat d4/d67/f115 x:0 0 0 2024-02-22T15:50:45.042 INFO:tasks.workunit.client.0.smithi019.stdout:5/808: dread d8/f9b [0,4194304] 0 2024-02-22T15:50:45.042 INFO:tasks.workunit.client.0.smithi019.stdout:0/802: fdatasync d4/d67/de5/fef 0 2024-02-22T15:50:45.045 INFO:tasks.workunit.client.0.smithi019.stdout:1/887: link d8/d15/d34/d9e/d126/fd3 d8/d2a/d75/d81/f12e 0 2024-02-22T15:50:45.046 INFO:tasks.workunit.client.0.smithi019.stdout:5/809: truncate d8/d11/d33/fa9 4601350 0 2024-02-22T15:50:45.047 INFO:tasks.workunit.client.0.smithi019.stdout:2/824: creat d1/d17/d1c/d128/db9/dd0/df4/f12d x:0 0 0 2024-02-22T15:50:45.048 INFO:tasks.workunit.client.0.smithi019.stdout:8/752: sync 2024-02-22T15:50:45.049 INFO:tasks.workunit.client.0.smithi019.stdout:0/803: symlink d4/d23/df5/l116 0 2024-02-22T15:50:45.050 INFO:tasks.workunit.client.0.smithi019.stdout:9/796: dwrite d2/d1f/d2e/d57/d60/f75 [0,4194304] 0 2024-02-22T15:50:45.051 INFO:tasks.workunit.client.1.smithi062.stdout:0/687: dread d1/d13/dde/d76/d8c/d5d/f37 [0,4194304] 0 2024-02-22T15:50:45.051 INFO:tasks.workunit.client.0.smithi019.stdout:8/753: readlink d9/d14/d17/d3d/d3f/lf3 0 2024-02-22T15:50:45.052 INFO:tasks.workunit.client.0.smithi019.stdout:5/810: mknod d8/dd/d19/d3a/d46/d7e/dbe/de8/c115 0 2024-02-22T15:50:45.053 INFO:tasks.workunit.client.0.smithi019.stdout:0/804: stat d4/db/d1b/d63/db0/da1/dbb/d7/c69 0 2024-02-22T15:50:45.053 INFO:tasks.workunit.client.1.smithi062.stdout:0/688: creat d1/d13/dde/d76/fe2 x:0 0 0 2024-02-22T15:50:45.053 INFO:tasks.workunit.client.1.smithi062.stdout:0/689: dread - d1/d13/dde/d1d/ddb/fe0 zero size 2024-02-22T15:50:45.054 INFO:tasks.workunit.client.0.smithi019.stdout:2/825: mknod d1/d17/d21/d71/dd1/c12e 0 2024-02-22T15:50:45.055 INFO:tasks.workunit.client.1.smithi062.stdout:0/690: fsync d1/d13/dde/f34 0 2024-02-22T15:50:45.056 INFO:tasks.workunit.client.0.smithi019.stdout:8/754: mkdir d9/d14/d6f/d7e/dff 0 2024-02-22T15:50:45.056 INFO:tasks.workunit.client.1.smithi062.stdout:0/691: rename d1/d13/dde to d1/d13/dde/d76/d8c/de3 22 2024-02-22T15:50:45.057 INFO:tasks.workunit.client.0.smithi019.stdout:1/888: link d8/f5a d8/d18/f12f 0 2024-02-22T15:50:45.058 INFO:tasks.workunit.client.0.smithi019.stdout:5/811: symlink d8/dd/d19/d38/d54/l116 0 2024-02-22T15:50:45.058 INFO:tasks.workunit.client.0.smithi019.stdout:1/889: fdatasync d8/d18/d85/f8d 0 2024-02-22T15:50:45.059 INFO:tasks.workunit.client.1.smithi062.stdout:0/692: truncate d1/d13/dde/d76/d8c/d5d/dc3/d85/fa3 1062718 0 2024-02-22T15:50:45.061 INFO:tasks.workunit.client.1.smithi062.stdout:0/693: truncate d1/d13/dde/d1d/d3b/fd1 4928763 0 2024-02-22T15:50:45.062 INFO:tasks.workunit.client.0.smithi019.stdout:0/805: mknod d4/d22/d2c/c117 0 2024-02-22T15:50:45.063 INFO:tasks.workunit.client.0.smithi019.stdout:9/797: rename d2/d1f/d2e/d48/l33 to d2/d7/dc/d27/dbc/d110/l11a 0 2024-02-22T15:50:45.063 INFO:tasks.workunit.client.1.smithi062.stdout:0/694: truncate d1/d13/dde/d76/d8c/d2c/d87/fd0 958027 0 2024-02-22T15:50:45.063 INFO:tasks.workunit.client.0.smithi019.stdout:1/890: write d8/d15/d34/f39 [7817504,76563] 0 2024-02-22T15:50:45.065 INFO:tasks.workunit.client.0.smithi019.stdout:8/755: creat d9/d14/d6f/d7e/d94/daa/dea/f100 x:0 0 0 2024-02-22T15:50:45.065 INFO:tasks.workunit.client.1.smithi062.stdout:0/695: creat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/fe4 x:0 0 0 2024-02-22T15:50:45.066 INFO:tasks.workunit.client.1.smithi062.stdout:0/696: chown d1/c61 468 1 2024-02-22T15:50:45.067 INFO:tasks.workunit.client.1.smithi062.stdout:0/697: readlink d1/d13/dde/d76/d8c/d5d/dc3/l86 0 2024-02-22T15:50:45.067 INFO:tasks.workunit.client.0.smithi019.stdout:2/826: dwrite d1/de/d9a/f113 [0,4194304] 0 2024-02-22T15:50:45.068 INFO:tasks.workunit.client.0.smithi019.stdout:1/891: creat d8/d2a/dde/f130 x:0 0 0 2024-02-22T15:50:45.070 INFO:tasks.workunit.client.0.smithi019.stdout:8/756: rmdir d9/d14/d17/d3d/d3f/dd4 39 2024-02-22T15:50:45.070 INFO:tasks.workunit.client.0.smithi019.stdout:2/827: chown d1/d17/d1c/d128/l102 1346 1 2024-02-22T15:50:45.070 INFO:tasks.workunit.client.0.smithi019.stdout:9/798: fdatasync d2/d7/dc/d27/d87/fb1 0 2024-02-22T15:50:45.071 INFO:tasks.workunit.client.1.smithi062.stdout:0/698: dread d1/fc4 [0,4194304] 0 2024-02-22T15:50:45.071 INFO:tasks.workunit.client.0.smithi019.stdout:1/892: chown d8/d18/lb7 811936 1 2024-02-22T15:50:45.072 INFO:tasks.workunit.client.1.smithi062.stdout:0/699: chown d1/d13/dde/d76/d8c/d2c/db5 23542 1 2024-02-22T15:50:45.075 INFO:tasks.workunit.client.1.smithi062.stdout:0/700: rename d1/d3/de/f90 to d1/d3/d4/fe5 0 2024-02-22T15:50:45.076 INFO:tasks.workunit.client.1.smithi062.stdout:0/701: write d1/dd/f63 [3328432,123907] 0 2024-02-22T15:50:45.078 INFO:tasks.workunit.client.0.smithi019.stdout:2/828: truncate d1/d17/d67/dfa/f12a 300635 0 2024-02-22T15:50:45.078 INFO:tasks.workunit.client.0.smithi019.stdout:9/799: truncate d2/d7/d52/f104 892564 0 2024-02-22T15:50:45.079 INFO:tasks.workunit.client.0.smithi019.stdout:8/757: rename d9/d14/d6f/d7e/d94/daa/fd5 to d9/d14/d17/d3d/d3f/dd4/f101 0 2024-02-22T15:50:45.080 INFO:tasks.workunit.client.1.smithi062.stdout:0/702: rename d1/dd/f36 to d1/fe6 0 2024-02-22T15:50:45.084 INFO:tasks.workunit.client.0.smithi019.stdout:9/800: fsync d2/d1f/d2e/d57/d60/d8c/d66/fef 0 2024-02-22T15:50:45.084 INFO:tasks.workunit.client.0.smithi019.stdout:8/758: mknod d9/d14/d17/d3d/d3f/dd4/c102 0 2024-02-22T15:50:45.084 INFO:tasks.workunit.client.0.smithi019.stdout:9/801: chown d2/d7/dc/d27/dbc/lbe 187063 1 2024-02-22T15:50:45.084 INFO:tasks.workunit.client.0.smithi019.stdout:8/759: readlink d9/d14/d1f/d4c/l5b 0 2024-02-22T15:50:45.084 INFO:tasks.workunit.client.0.smithi019.stdout:9/802: dread - d2/d1f/d2e/d57/d60/d8c/d66/fef zero size 2024-02-22T15:50:45.085 INFO:tasks.workunit.client.1.smithi062.stdout:0/703: link d1/d13/dde/d76/d8c/d92/cb6 d1/d13/ce7 0 2024-02-22T15:50:45.085 INFO:tasks.workunit.client.0.smithi019.stdout:2/829: read d1/d17/fbd [2859780,44670] 0 2024-02-22T15:50:45.085 INFO:tasks.workunit.client.0.smithi019.stdout:8/760: mkdir d9/d14/d1f/d6d/d103 0 2024-02-22T15:50:45.086 INFO:tasks.workunit.client.0.smithi019.stdout:0/806: dread d4/db/d1b/d63/db0/da1/dbb/d7/fcb [0,4194304] 0 2024-02-22T15:50:45.087 INFO:tasks.workunit.client.1.smithi062.stdout:0/704: read d1/dd/f6c [657875,79175] 0 2024-02-22T15:50:45.088 INFO:tasks.workunit.client.0.smithi019.stdout:9/803: write d2/d1f/d2e/d48/d45/d5a/fd0 [115157,5588] 0 2024-02-22T15:50:45.088 INFO:tasks.workunit.client.1.smithi062.stdout:0/705: chown d1/d13/dde/d76/d8c/d5d/dc3/ccd 34373 1 2024-02-22T15:50:45.090 INFO:tasks.workunit.client.0.smithi019.stdout:2/830: readlink d1/d17/d1c/d128/db9/dd0/dee/l108 0 2024-02-22T15:50:45.090 INFO:tasks.workunit.client.0.smithi019.stdout:8/761: readlink d9/d14/d17/d3d/d83/lad 0 2024-02-22T15:50:45.091 INFO:tasks.workunit.client.1.smithi062.stdout:0/706: creat d1/d3/d4/fe8 x:0 0 0 2024-02-22T15:50:45.091 INFO:tasks.workunit.client.1.smithi062.stdout:0/707: chown d1/dd/l89 72620 1 2024-02-22T15:50:45.093 INFO:tasks.workunit.client.0.smithi019.stdout:2/831: creat d1/d46/f12f x:0 0 0 2024-02-22T15:50:45.095 INFO:tasks.workunit.client.0.smithi019.stdout:9/804: link d2/d7/f1b d2/d1f/d2e/d57/d60/f11b 0 2024-02-22T15:50:45.096 INFO:tasks.workunit.client.0.smithi019.stdout:8/762: mkdir d9/d14/d6f/d7e/d94/d9a/dd3/d104 0 2024-02-22T15:50:45.097 INFO:tasks.workunit.client.0.smithi019.stdout:0/807: dwrite d4/d32/f9c [4194304,4194304] 0 2024-02-22T15:50:45.098 INFO:tasks.workunit.client.0.smithi019.stdout:8/763: chown d9/d14/d17/d3d/d83/la7 129 1 2024-02-22T15:50:45.100 INFO:tasks.workunit.client.0.smithi019.stdout:0/808: write d4/db/d1b/d40/d8c/fd0 [954264,65713] 0 2024-02-22T15:50:45.105 INFO:tasks.workunit.client.0.smithi019.stdout:9/805: dwrite d2/dc8/fea [0,4194304] 0 2024-02-22T15:50:45.106 INFO:tasks.workunit.client.1.smithi062.stdout:0/708: dread d1/fc2 [0,4194304] 0 2024-02-22T15:50:45.108 INFO:tasks.workunit.client.0.smithi019.stdout:0/809: getdents d4/db/d1b/d63/db0/da1/dbb/d52 0 2024-02-22T15:50:45.109 INFO:tasks.workunit.client.1.smithi062.stdout:0/709: dread d1/fc2 [0,4194304] 0 2024-02-22T15:50:45.111 INFO:tasks.workunit.client.1.smithi062.stdout:0/710: creat d1/d5e/fe9 x:0 0 0 2024-02-22T15:50:45.129 INFO:tasks.workunit.client.0.smithi019.stdout:5/812: fdatasync d8/dd/d19/d77/d7a/ffd 0 2024-02-22T15:50:45.130 INFO:tasks.workunit.client.0.smithi019.stdout:5/813: write d8/d11/fb6 [2683067,98381] 0 2024-02-22T15:50:45.132 INFO:tasks.workunit.client.0.smithi019.stdout:5/814: fsync d8/dd/d8e/f8a 0 2024-02-22T15:50:45.135 INFO:tasks.workunit.client.0.smithi019.stdout:5/815: readlink d8/dd/d19/d3a/la7 0 2024-02-22T15:50:45.138 INFO:tasks.workunit.client.0.smithi019.stdout:5/816: truncate d8/d11/d33/f74 564264 0 2024-02-22T15:50:45.139 INFO:tasks.workunit.client.1.smithi062.stdout:9/601: write d4/d15/d5f/d75/d84/f59 [667478,92850] 0 2024-02-22T15:50:45.141 INFO:tasks.workunit.client.0.smithi019.stdout:4/772: truncate d2/db/d92/f91 1418038 0 2024-02-22T15:50:45.143 INFO:tasks.workunit.client.1.smithi062.stdout:9/602: link d4/d15/d5e/d63/fb2 d4/d28/d50/fbd 0 2024-02-22T15:50:45.144 INFO:tasks.workunit.client.1.smithi062.stdout:9/603: read d4/d1c/f58 [195111,108749] 0 2024-02-22T15:50:45.145 INFO:tasks.workunit.client.0.smithi019.stdout:4/773: creat d2/d3f/ffa x:0 0 0 2024-02-22T15:50:45.145 INFO:tasks.workunit.client.1.smithi062.stdout:9/604: chown d4/d15/d31/f94 300033 1 2024-02-22T15:50:45.146 INFO:tasks.workunit.client.0.smithi019.stdout:5/817: getdents d8/dd/d19/d38/d47/df3 0 2024-02-22T15:50:45.147 INFO:tasks.workunit.client.0.smithi019.stdout:5/818: chown d8/dd/d19/d3a/c9c 22554609 1 2024-02-22T15:50:45.147 INFO:tasks.workunit.client.1.smithi062.stdout:9/605: symlink d4/d15/d5f/lbe 0 2024-02-22T15:50:45.148 INFO:tasks.workunit.client.1.smithi062.stdout:9/606: write d4/d15/d5e/d63/da0/f39 [4836990,45645] 0 2024-02-22T15:50:45.154 INFO:tasks.workunit.client.1.smithi062.stdout:0/711: write d1/d13/dde/f56 [757056,37803] 0 2024-02-22T15:50:45.157 INFO:tasks.workunit.client.0.smithi019.stdout:5/819: creat d8/dd/d19/d3a/dc7/f117 x:0 0 0 2024-02-22T15:50:45.159 INFO:tasks.workunit.client.0.smithi019.stdout:5/820: write d8/dd/d19/d38/d47/fd8 [257661,109171] 0 2024-02-22T15:50:45.160 INFO:tasks.workunit.client.1.smithi062.stdout:8/768: dread d8/f22 [0,4194304] 0 2024-02-22T15:50:45.166 INFO:tasks.workunit.client.1.smithi062.stdout:0/712: dwrite d1/f45 [0,4194304] 0 2024-02-22T15:50:45.169 INFO:tasks.workunit.client.1.smithi062.stdout:9/607: dread d4/d35/f8a [0,4194304] 0 2024-02-22T15:50:45.170 INFO:tasks.workunit.client.0.smithi019.stdout:5/821: dwrite d8/dd/d19/d77/dd2/dec/ff2 [0,4194304] 0 2024-02-22T15:50:45.172 INFO:tasks.workunit.client.1.smithi062.stdout:0/713: rmdir d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce 39 2024-02-22T15:50:45.174 INFO:tasks.workunit.client.0.smithi019.stdout:5/822: write d8/dd/d8e/f8a [5214109,20342] 0 2024-02-22T15:50:45.174 INFO:tasks.workunit.client.0.smithi019.stdout:2/832: truncate d1/db4/fca 832176 0 2024-02-22T15:50:45.176 INFO:tasks.workunit.client.0.smithi019.stdout:8/764: write d9/d14/d17/d3d/f91 [100747,3239] 0 2024-02-22T15:50:45.176 INFO:tasks.workunit.client.1.smithi062.stdout:9/608: symlink d4/d1c/d83/d87/lbf 0 2024-02-22T15:50:45.177 INFO:tasks.workunit.client.0.smithi019.stdout:1/893: dwrite d8/d2a/d75/d81/d8e/f114 [0,4194304] 0 2024-02-22T15:50:45.178 INFO:tasks.workunit.client.1.smithi062.stdout:0/714: write d1/d13/dde/d76/d8c/f64 [2284851,77243] 0 2024-02-22T15:50:45.178 INFO:tasks.workunit.client.1.smithi062.stdout:9/609: dread - d4/d28/d50/fbd zero size 2024-02-22T15:50:45.179 INFO:tasks.workunit.client.1.smithi062.stdout:8/769: dread d8/d9/f1f [0,4194304] 0 2024-02-22T15:50:45.181 INFO:tasks.workunit.client.1.smithi062.stdout:9/610: truncate d4/d15/d5f/f7b 4699805 0 2024-02-22T15:50:45.182 INFO:tasks.workunit.client.0.smithi019.stdout:4/774: rmdir d2/db 39 2024-02-22T15:50:45.183 INFO:tasks.workunit.client.1.smithi062.stdout:8/770: truncate d8/d9/d2c/fe6 484335 0 2024-02-22T15:50:45.185 INFO:tasks.workunit.client.1.smithi062.stdout:0/715: rename d1/c61 to d1/d13/dde/d1d/cea 0 2024-02-22T15:50:45.185 INFO:tasks.workunit.client.1.smithi062.stdout:8/771: truncate d8/d9/d44/d4d/f69 611885 0 2024-02-22T15:50:45.185 INFO:tasks.workunit.client.1.smithi062.stdout:9/611: symlink d4/d15/d5f/d75/d84/lc0 0 2024-02-22T15:50:45.186 INFO:tasks.workunit.client.1.smithi062.stdout:8/772: fdatasync d8/d9/d44/fb2 0 2024-02-22T15:50:45.187 INFO:tasks.workunit.client.1.smithi062.stdout:9/612: chown d4/d1c/f2c 95616406 1 2024-02-22T15:50:45.187 INFO:tasks.workunit.client.1.smithi062.stdout:0/716: creat d1/d5e/feb x:0 0 0 2024-02-22T15:50:45.191 INFO:tasks.workunit.client.1.smithi062.stdout:8/773: chown d8/d9/d2c/d49/d94/dc1/f91 108 1 2024-02-22T15:50:45.192 INFO:tasks.workunit.client.0.smithi019.stdout:5/823: symlink d8/dd/d19/d77/dd2/l118 0 2024-02-22T15:50:45.192 INFO:tasks.workunit.client.0.smithi019.stdout:9/806: dwrite d2/d1f/d2e/d48/d36/f83 [0,4194304] 0 2024-02-22T15:50:45.192 INFO:tasks.workunit.client.1.smithi062.stdout:9/613: rename d4/d15/d5f/d6a/fb1 to d4/d1c/fc1 0 2024-02-22T15:50:45.194 INFO:tasks.workunit.client.1.smithi062.stdout:0/717: symlink d1/d3/de/d50/lec 0 2024-02-22T15:50:45.194 INFO:tasks.workunit.client.0.smithi019.stdout:4/775: symlink d2/db/d92/lfb 0 2024-02-22T15:50:45.197 INFO:tasks.workunit.client.1.smithi062.stdout:0/718: dread - d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fdd zero size 2024-02-22T15:50:45.198 INFO:tasks.workunit.client.0.smithi019.stdout:5/824: mkdir d8/dd/d19/d3a/d46/d7e/dbe/dc3/d119 0 2024-02-22T15:50:45.203 INFO:tasks.workunit.client.0.smithi019.stdout:0/810: dwrite d4/d22/d2c/d68/f6a [0,4194304] 0 2024-02-22T15:50:45.203 INFO:tasks.workunit.client.1.smithi062.stdout:0/719: link d1/d3/de/ff d1/d3/d5f/fed 0 2024-02-22T15:50:45.204 INFO:tasks.workunit.client.0.smithi019.stdout:9/807: rename d2/d1f/d2e/d57/d60/d8c/fc3 to d2/d7/dc/d27/dbc/dcc/f11c 0 2024-02-22T15:50:45.204 INFO:tasks.workunit.client.0.smithi019.stdout:1/894: creat d8/d15/f131 x:0 0 0 2024-02-22T15:50:45.205 INFO:tasks.workunit.client.0.smithi019.stdout:0/811: stat d4/d22/d2c/d68/d6d/fe3 0 2024-02-22T15:50:45.206 INFO:tasks.workunit.client.0.smithi019.stdout:5/825: truncate d8/dd/d19/d38/d47/fa3 597770 0 2024-02-22T15:50:45.206 INFO:tasks.workunit.client.1.smithi062.stdout:0/720: truncate d1/d13/dde/d76/d8c/d2c/d87/fa2 788513 0 2024-02-22T15:50:45.206 INFO:tasks.workunit.client.1.smithi062.stdout:0/721: readlink d1/dd/l89 0 2024-02-22T15:50:45.208 INFO:tasks.workunit.client.0.smithi019.stdout:4/776: rename d2/db/d14/d23/f4d to d2/db/d92/ffc 0 2024-02-22T15:50:45.209 INFO:tasks.workunit.client.1.smithi062.stdout:0/722: rmdir d1/d3/de/d50 39 2024-02-22T15:50:45.210 INFO:tasks.workunit.client.0.smithi019.stdout:9/808: symlink d2/d1f/l11d 0 2024-02-22T15:50:45.210 INFO:tasks.workunit.client.0.smithi019.stdout:2/833: write d1/d17/d21/d71/d60/d11b/fea [461793,63762] 0 2024-02-22T15:50:45.212 INFO:tasks.workunit.client.1.smithi062.stdout:0/723: symlink d1/d13/dde/lee 0 2024-02-22T15:50:45.213 INFO:tasks.workunit.client.1.smithi062.stdout:8/774: write d8/d9/f1f [4984668,105986] 0 2024-02-22T15:50:45.214 INFO:tasks.workunit.client.1.smithi062.stdout:9/614: write d4/d15/faa [5439015,110803] 0 2024-02-22T15:50:45.214 INFO:tasks.workunit.client.0.smithi019.stdout:4/777: symlink d2/db/dea/ded/lfd 0 2024-02-22T15:50:45.216 INFO:tasks.workunit.client.0.smithi019.stdout:1/895: link d8/d44/f9f d8/d18/d64/d7f/de3/f132 0 2024-02-22T15:50:45.217 INFO:tasks.workunit.client.0.smithi019.stdout:5/826: dwrite d8/d11/f103 [0,4194304] 0 2024-02-22T15:50:45.217 INFO:tasks.workunit.client.1.smithi062.stdout:9/615: read - d4/d15/d5e/d63/f86 zero size 2024-02-22T15:50:45.218 INFO:tasks.workunit.client.0.smithi019.stdout:1/896: mkdir d8/d44/d5d/d133 0 2024-02-22T15:50:45.220 INFO:tasks.workunit.client.1.smithi062.stdout:0/724: link d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fd2 d1/d3/dd5/fef 0 2024-02-22T15:50:45.221 INFO:tasks.workunit.client.1.smithi062.stdout:9/616: unlink d4/d15/d31/c6c 0 2024-02-22T15:50:45.222 INFO:tasks.workunit.client.0.smithi019.stdout:5/827: truncate d8/d11/fed 841380 0 2024-02-22T15:50:45.222 INFO:tasks.workunit.client.1.smithi062.stdout:0/725: chown d1/d13/c15 510319 1 2024-02-22T15:50:45.223 INFO:tasks.workunit.client.1.smithi062.stdout:8/775: dwrite d8/d9/d44/d4d/f69 [0,4194304] 0 2024-02-22T15:50:45.224 INFO:tasks.workunit.client.0.smithi019.stdout:2/834: dwrite d1/fbe [0,4194304] 0 2024-02-22T15:50:45.227 INFO:tasks.workunit.client.0.smithi019.stdout:1/897: creat d8/d15/da4/f134 x:0 0 0 2024-02-22T15:50:45.227 INFO:tasks.workunit.client.0.smithi019.stdout:5/828: creat d8/dd/d19/d38/d54/f11a x:0 0 0 2024-02-22T15:50:45.229 INFO:tasks.workunit.client.0.smithi019.stdout:1/898: readlink d8/d15/leb 0 2024-02-22T15:50:45.230 INFO:tasks.workunit.client.1.smithi062.stdout:8/776: fsync d8/d86/db6/fe3 0 2024-02-22T15:50:45.232 INFO:tasks.workunit.client.1.smithi062.stdout:9/617: getdents d4/d28/d50/da3 0 2024-02-22T15:50:45.233 INFO:tasks.workunit.client.0.smithi019.stdout:4/778: rename d2/db/d14/d23/d21/f9b to d2/d3f/d48/d76/ffe 0 2024-02-22T15:50:45.233 INFO:tasks.workunit.client.1.smithi062.stdout:0/726: mkdir d1/d3/de/df0 0 2024-02-22T15:50:45.235 INFO:tasks.workunit.client.0.smithi019.stdout:5/829: creat d8/dd/d19/d38/f11b x:0 0 0 2024-02-22T15:50:45.237 INFO:tasks.workunit.client.0.smithi019.stdout:2/835: mkdir d1/d17/d21/d71/d60/d11b/d130 0 2024-02-22T15:50:45.238 INFO:tasks.workunit.client.1.smithi062.stdout:8/777: fdatasync d8/d28/d4e/f8e 0 2024-02-22T15:50:45.239 INFO:tasks.workunit.client.0.smithi019.stdout:1/899: rename d8/d2a/d75/def/df8/fd5 to d8/d23/d33/f135 0 2024-02-22T15:50:45.239 INFO:tasks.workunit.client.0.smithi019.stdout:3/661: sync 2024-02-22T15:50:45.240 INFO:tasks.workunit.client.0.smithi019.stdout:4/779: mknod d2/d3f/d6e/cff 0 2024-02-22T15:50:45.241 INFO:tasks.workunit.client.1.smithi062.stdout:9/618: mknod d4/d28/d50/cc2 0 2024-02-22T15:50:45.241 INFO:tasks.workunit.client.1.smithi062.stdout:0/727: dwrite d1/d13/dde/d76/d8c/d5d/dc3/f24 [0,4194304] 0 2024-02-22T15:50:45.241 INFO:tasks.workunit.client.0.smithi019.stdout:1/900: write d8/d44/ffd [107686,16930] 0 2024-02-22T15:50:45.243 INFO:tasks.workunit.client.0.smithi019.stdout:5/830: readlink d8/dd/d19/d77/l82 0 2024-02-22T15:50:45.243 INFO:tasks.workunit.client.1.smithi062.stdout:0/728: write d1/d3/d4/d1e/fd8 [1235066,130839] 0 2024-02-22T15:50:45.246 INFO:tasks.workunit.client.0.smithi019.stdout:3/662: truncate d9/f6a 967422 0 2024-02-22T15:50:45.246 INFO:tasks.workunit.client.0.smithi019.stdout:1/901: dread - d8/d18/fcb zero size 2024-02-22T15:50:45.246 INFO:tasks.workunit.client.1.smithi062.stdout:8/778: rmdir d8/d9/d2c/d49/d94/dc1 39 2024-02-22T15:50:45.247 INFO:tasks.workunit.client.1.smithi062.stdout:9/619: rmdir d4/d15/d5f 39 2024-02-22T15:50:45.248 INFO:tasks.workunit.client.0.smithi019.stdout:1/902: readlink d8/d2a/db3/l115 0 2024-02-22T15:50:45.249 INFO:tasks.workunit.client.1.smithi062.stdout:0/729: symlink d1/d13/dde/d1d/ddb/lf1 0 2024-02-22T15:50:45.250 INFO:tasks.workunit.client.0.smithi019.stdout:3/663: rmdir d9 39 2024-02-22T15:50:45.251 INFO:tasks.workunit.client.0.smithi019.stdout:2/836: rename d1/db4/fd2 to d1/d17/d21/d71/d60/d11b/f131 0 2024-02-22T15:50:45.255 INFO:tasks.workunit.client.0.smithi019.stdout:1/903: mkdir d8/d2a/d75/d136 0 2024-02-22T15:50:45.256 INFO:tasks.workunit.client.0.smithi019.stdout:5/831: truncate d8/dd/d19/d38/d47/ddb/dde/deb/fa4 434533 0 2024-02-22T15:50:45.256 INFO:tasks.workunit.client.0.smithi019.stdout:4/780: dwrite d2/db/ff3 [0,4194304] 0 2024-02-22T15:50:45.257 INFO:tasks.workunit.client.1.smithi062.stdout:0/730: dwrite d1/d13/dde/d76/d8c/d5d/dc3/f49 [0,4194304] 0 2024-02-22T15:50:45.263 INFO:tasks.workunit.client.0.smithi019.stdout:3/664: truncate d9/d26/d32/f9c 3138671 0 2024-02-22T15:50:45.264 INFO:tasks.workunit.client.0.smithi019.stdout:3/665: fsync d9/d65/da7/fc9 0 2024-02-22T15:50:45.265 INFO:tasks.workunit.client.1.smithi062.stdout:5/502: sync 2024-02-22T15:50:45.267 INFO:tasks.workunit.client.0.smithi019.stdout:2/837: truncate d1/d46/d9d/fe2 2827093 0 2024-02-22T15:50:45.268 INFO:tasks.workunit.client.1.smithi062.stdout:9/620: write d4/d15/f5c [755634,90667] 0 2024-02-22T15:50:45.269 INFO:tasks.workunit.client.1.smithi062.stdout:0/731: rename d1/d13/dde/d76/d8c/d5d/dc3/d85/ldf to d1/d3/d4/lf2 0 2024-02-22T15:50:45.270 INFO:tasks.workunit.client.0.smithi019.stdout:2/838: dread - d1/db4/fe9 zero size 2024-02-22T15:50:45.273 INFO:tasks.workunit.client.1.smithi062.stdout:9/621: write d4/d15/d5f/d75/fab [605823,65541] 0 2024-02-22T15:50:45.273 INFO:tasks.workunit.client.1.smithi062.stdout:0/732: chown d1/d13/dde/d1d/d3b 147081 1 2024-02-22T15:50:45.273 INFO:tasks.workunit.client.0.smithi019.stdout:3/666: stat d9/d22/d27/d6d/c97 0 2024-02-22T15:50:45.274 INFO:tasks.workunit.client.0.smithi019.stdout:3/667: fdatasync d9/d26/f55 0 2024-02-22T15:50:45.275 INFO:tasks.workunit.client.0.smithi019.stdout:2/839: mkdir d1/da4/d132 0 2024-02-22T15:50:45.276 INFO:tasks.workunit.client.0.smithi019.stdout:2/840: write d1/d17/fc8 [93469,49558] 0 2024-02-22T15:50:45.276 INFO:tasks.workunit.client.1.smithi062.stdout:0/733: rmdir d1/d13/dde/d76/d8c/d5d/dc3/d85 39 2024-02-22T15:50:45.279 INFO:tasks.workunit.client.1.smithi062.stdout:0/734: stat d1/d13/dde/d76/d8c/d5d/dc3/l86 0 2024-02-22T15:50:45.281 INFO:tasks.workunit.client.0.smithi019.stdout:3/668: unlink d9/d26/d2d/d53/da0/la6 0 2024-02-22T15:50:45.283 INFO:tasks.workunit.client.0.smithi019.stdout:3/669: read - d9/d26/d2d/f91 zero size 2024-02-22T15:50:45.284 INFO:tasks.workunit.client.1.smithi062.stdout:9/622: dwrite d4/d1c/f2c [0,4194304] 0 2024-02-22T15:50:45.285 INFO:tasks.workunit.client.0.smithi019.stdout:1/904: write d8/d15/d34/d9e/d126/fd3 [1106643,91188] 0 2024-02-22T15:50:45.286 INFO:tasks.workunit.client.1.smithi062.stdout:5/503: truncate d0/d3/da/d13/d23/d4e/f5d 3371087 0 2024-02-22T15:50:45.290 INFO:tasks.workunit.client.1.smithi062.stdout:0/735: creat d1/ff3 x:0 0 0 2024-02-22T15:50:45.291 INFO:tasks.workunit.client.1.smithi062.stdout:9/623: creat d4/d15/d5e/d63/da0/d92/db6/fc3 x:0 0 0 2024-02-22T15:50:45.295 INFO:tasks.workunit.client.0.smithi019.stdout:5/832: dwrite d8/dd/d19/d38/d54/db0/fbd [0,4194304] 0 2024-02-22T15:50:45.298 INFO:tasks.workunit.client.0.smithi019.stdout:4/781: dread d2/db/d14/d23/d67/f68 [0,4194304] 0 2024-02-22T15:50:45.299 INFO:tasks.workunit.client.0.smithi019.stdout:1/905: getdents d8/d18/d64/d7f/de3 0 2024-02-22T15:50:45.300 INFO:tasks.workunit.client.1.smithi062.stdout:0/736: truncate d1/d3/f39 718528 0 2024-02-22T15:50:45.300 INFO:tasks.workunit.client.0.smithi019.stdout:2/841: dread d1/d17/d21/d71/d60/d11b/f4a [0,4194304] 0 2024-02-22T15:50:45.300 INFO:tasks.workunit.client.0.smithi019.stdout:4/782: chown d2/fda 536017095 1 2024-02-22T15:50:45.302 INFO:tasks.workunit.client.0.smithi019.stdout:5/833: read d8/d11/f3b [746398,110992] 0 2024-02-22T15:50:45.303 INFO:tasks.workunit.client.1.smithi062.stdout:9/624: link d4/d1c/f1d d4/d15/d5e/d63/da0/d92/db6/fc4 0 2024-02-22T15:50:45.303 INFO:tasks.workunit.client.0.smithi019.stdout:3/670: dwrite f6 [0,4194304] 0 2024-02-22T15:50:45.305 INFO:tasks.workunit.client.0.smithi019.stdout:4/783: dread - d2/db/d14/d23/d21/dbf/dd7/ff6 zero size 2024-02-22T15:50:45.305 INFO:tasks.workunit.client.0.smithi019.stdout:1/906: creat d8/d2a/f137 x:0 0 0 2024-02-22T15:50:45.308 INFO:tasks.workunit.client.0.smithi019.stdout:3/671: chown d9/d22/d27/d96/fca 2050 1 2024-02-22T15:50:45.308 INFO:tasks.workunit.client.0.smithi019.stdout:1/907: truncate d8/d2a/f111 951646 0 2024-02-22T15:50:45.308 INFO:tasks.workunit.client.1.smithi062.stdout:5/504: dwrite d0/d3/d2f/d59/d67/d69/d3f/f5c [0,4194304] 0 2024-02-22T15:50:45.309 INFO:tasks.workunit.client.1.smithi062.stdout:9/625: creat d4/d1c/d83/fc5 x:0 0 0 2024-02-22T15:50:45.310 INFO:tasks.workunit.client.0.smithi019.stdout:2/842: getdents d1/d17/d1c/d128/db9/ddd 0 2024-02-22T15:50:45.312 INFO:tasks.workunit.client.0.smithi019.stdout:3/672: fdatasync d9/d26/d3f/d59/d66/f87 0 2024-02-22T15:50:45.312 INFO:tasks.workunit.client.1.smithi062.stdout:9/626: write d4/d15/d5e/d63/da0/d92/db6/fc3 [443944,81456] 0 2024-02-22T15:50:45.314 INFO:tasks.workunit.client.0.smithi019.stdout:5/834: dwrite d8/dd/d19/d3a/d46/fd5 [4194304,4194304] 0 2024-02-22T15:50:45.314 INFO:tasks.workunit.client.0.smithi019.stdout:1/908: unlink d8/d2a/d75/l127 0 2024-02-22T15:50:45.316 INFO:tasks.workunit.client.0.smithi019.stdout:2/843: read d1/d86/f9e [499211,74629] 0 2024-02-22T15:50:45.317 INFO:tasks.workunit.client.1.smithi062.stdout:5/505: link d0/d3/da/d13/f3d d0/d3/da/d32/d4f/d72/f9a 0 2024-02-22T15:50:45.318 INFO:tasks.workunit.client.0.smithi019.stdout:4/784: getdents d2/db/d14/d23/d21/dbf 0 2024-02-22T15:50:45.319 INFO:tasks.workunit.client.1.smithi062.stdout:9/627: creat d4/d15/d5f/d75/d97/da7/fc6 x:0 0 0 2024-02-22T15:50:45.320 INFO:tasks.workunit.client.0.smithi019.stdout:2/844: symlink d1/da4/l133 0 2024-02-22T15:50:45.321 INFO:tasks.workunit.client.0.smithi019.stdout:4/785: fdatasync d2/db/d14/d23/d21/d35/f64 0 2024-02-22T15:50:45.321 INFO:tasks.workunit.client.1.smithi062.stdout:0/737: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/df4 0 2024-02-22T15:50:45.325 INFO:tasks.workunit.client.0.smithi019.stdout:5/835: rename d8/dd/d19/d38/d54/db0/f113 to d8/dd/d19/d3a/d46/d7e/dbe/dc3/f11c 0 2024-02-22T15:50:45.326 INFO:tasks.workunit.client.1.smithi062.stdout:5/506: write d0/d3/d30/f46 [991608,105881] 0 2024-02-22T15:50:45.326 INFO:tasks.workunit.client.0.smithi019.stdout:2/845: creat d1/d17/d67/dfa/d11e/f134 x:0 0 0 2024-02-22T15:50:45.326 INFO:tasks.workunit.client.0.smithi019.stdout:5/836: dread - d8/d11/fe4 zero size 2024-02-22T15:50:45.328 INFO:tasks.workunit.client.0.smithi019.stdout:5/837: chown d8/f23 17 1 2024-02-22T15:50:45.329 INFO:tasks.workunit.client.0.smithi019.stdout:2/846: symlink d1/d86/d11f/l135 0 2024-02-22T15:50:45.329 INFO:tasks.workunit.client.0.smithi019.stdout:5/838: dread - d8/d11/fdc zero size 2024-02-22T15:50:45.331 INFO:tasks.workunit.client.0.smithi019.stdout:5/839: write d8/dd/f5b [3438835,7341] 0 2024-02-22T15:50:45.334 INFO:tasks.workunit.client.0.smithi019.stdout:2/847: rename d1/d17/d21/d71/cad to d1/d17/d21/d71/dd1/c136 0 2024-02-22T15:50:45.335 INFO:tasks.workunit.client.0.smithi019.stdout:1/909: dread d8/d18/d85/f8a [0,4194304] 0 2024-02-22T15:50:45.335 INFO:tasks.workunit.client.0.smithi019.stdout:5/840: mknod d8/dd/d19/c11d 0 2024-02-22T15:50:45.336 INFO:tasks.workunit.client.1.smithi062.stdout:9/628: rename d4/d15/d5f/d75/d97/da7/lba to d4/d28/lc7 0 2024-02-22T15:50:45.339 INFO:tasks.workunit.client.0.smithi019.stdout:2/848: rename d1/l82 to d1/l137 0 2024-02-22T15:50:45.340 INFO:tasks.workunit.client.1.smithi062.stdout:9/629: dread d4/d35/f8a [0,4194304] 0 2024-02-22T15:50:45.340 INFO:tasks.workunit.client.1.smithi062.stdout:9/630: chown d4/d1c/d83 398 1 2024-02-22T15:50:45.342 INFO:tasks.workunit.client.1.smithi062.stdout:9/631: write d4/d15/faa [2936723,15048] 0 2024-02-22T15:50:45.342 INFO:tasks.workunit.client.0.smithi019.stdout:5/841: link d8/dd/d19/d38/d54/f6b d8/dd/d19/d77/dd2/dec/f11e 0 2024-02-22T15:50:45.342 INFO:tasks.workunit.client.0.smithi019.stdout:1/910: dread d8/d2a/d75/d81/f12e [0,4194304] 0 2024-02-22T15:50:45.346 INFO:tasks.workunit.client.0.smithi019.stdout:2/849: dread d1/d17/fb1 [0,4194304] 0 2024-02-22T15:50:45.347 INFO:tasks.workunit.client.0.smithi019.stdout:3/673: truncate d9/d14/f9b 31470 0 2024-02-22T15:50:45.349 INFO:tasks.workunit.client.1.smithi062.stdout:5/507: write d0/f26 [917272,96741] 0 2024-02-22T15:50:45.350 INFO:tasks.workunit.client.1.smithi062.stdout:5/508: readlink d0/d3/da/d13/l29 0 2024-02-22T15:50:45.351 INFO:tasks.workunit.client.0.smithi019.stdout:1/911: rename d8/d18/d64/d7f/d94/df9/f10f to d8/d2a/d75/d81/d8e/f138 0 2024-02-22T15:50:45.351 INFO:tasks.workunit.client.1.smithi062.stdout:9/632: rename d4/d1c/d83/d87/c90 to d4/d15/d5e/d63/da0/d92/cc8 0 2024-02-22T15:50:45.351 INFO:tasks.workunit.client.1.smithi062.stdout:5/509: chown d0/f45 0 1 2024-02-22T15:50:45.351 INFO:tasks.workunit.client.0.smithi019.stdout:1/912: readlink d8/d15/d34/l70 0 2024-02-22T15:50:45.354 INFO:tasks.workunit.client.1.smithi062.stdout:5/510: fdatasync d0/d3/d2f/d59/d67/d69/d3f/f51 0 2024-02-22T15:50:45.355 INFO:tasks.workunit.client.0.smithi019.stdout:3/674: fdatasync d9/f6a 0 2024-02-22T15:50:45.355 INFO:tasks.workunit.client.1.smithi062.stdout:5/511: fdatasync d0/f45 0 2024-02-22T15:50:45.358 INFO:tasks.workunit.client.1.smithi062.stdout:5/512: mknod d0/d3/da/d32/c9b 0 2024-02-22T15:50:45.358 INFO:tasks.workunit.client.0.smithi019.stdout:2/850: link d1/d17/f47 d1/d17/d67/dfa/d11e/d126/f138 0 2024-02-22T15:50:45.359 INFO:tasks.workunit.client.0.smithi019.stdout:4/786: dwrite d2/d3f/f51 [0,4194304] 0 2024-02-22T15:50:45.360 INFO:tasks.workunit.client.1.smithi062.stdout:9/633: dread d4/d15/f23 [0,4194304] 0 2024-02-22T15:50:45.361 INFO:tasks.workunit.client.0.smithi019.stdout:2/851: dread - d1/def/fc3 zero size 2024-02-22T15:50:45.365 INFO:tasks.workunit.client.0.smithi019.stdout:1/913: dwrite d8/d2a/d75/d81/d8e/f118 [0,4194304] 0 2024-02-22T15:50:45.365 INFO:tasks.workunit.client.1.smithi062.stdout:9/634: mknod d4/d15/d5f/d75/d97/da7/cc9 0 2024-02-22T15:50:45.367 INFO:tasks.workunit.client.1.smithi062.stdout:5/513: dwrite d0/d3/ff [0,4194304] 0 2024-02-22T15:50:45.368 INFO:tasks.workunit.client.0.smithi019.stdout:2/852: getdents d1/d17/d1c/d128/db9/dd0/dee/d120 0 2024-02-22T15:50:45.368 INFO:tasks.workunit.client.1.smithi062.stdout:9/635: chown d4/d15/d5f/lbe 14591 1 2024-02-22T15:50:45.370 INFO:tasks.workunit.client.0.smithi019.stdout:2/853: readlink d1/da4/l133 0 2024-02-22T15:50:45.370 INFO:tasks.workunit.client.0.smithi019.stdout:4/787: dwrite d2/db/d92/ffc [4194304,4194304] 0 2024-02-22T15:50:45.372 INFO:tasks.workunit.client.0.smithi019.stdout:2/854: chown d1/de/f1e 13853 1 2024-02-22T15:50:45.374 INFO:tasks.workunit.client.1.smithi062.stdout:5/514: mknod d0/d3/da/d32/d4f/c9c 0 2024-02-22T15:50:45.374 INFO:tasks.workunit.client.0.smithi019.stdout:4/788: creat d2/d3f/d6e/f100 x:0 0 0 2024-02-22T15:50:45.376 INFO:tasks.workunit.client.0.smithi019.stdout:2/855: mknod d1/da4/c139 0 2024-02-22T15:50:45.376 INFO:tasks.workunit.client.0.smithi019.stdout:2/856: readlink d1/d86/la7 0 2024-02-22T15:50:45.379 INFO:tasks.workunit.client.0.smithi019.stdout:2/857: write d1/d17/d38/f110 [727746,23960] 0 2024-02-22T15:50:45.381 INFO:tasks.workunit.client.1.smithi062.stdout:5/515: creat d0/d3/da/d13/d23/d78/d81/f9d x:0 0 0 2024-02-22T15:50:45.381 INFO:tasks.workunit.client.0.smithi019.stdout:2/858: write d1/d17/fae [803256,1448] 0 2024-02-22T15:50:45.385 INFO:tasks.workunit.client.0.smithi019.stdout:2/859: write d1/de/f5a [1238091,108165] 0 2024-02-22T15:50:45.387 INFO:tasks.workunit.client.1.smithi062.stdout:5/516: mknod d0/d3/da/d32/c9e 0 2024-02-22T15:50:45.387 INFO:tasks.workunit.client.1.smithi062.stdout:9/636: dread d4/d1c/f53 [0,4194304] 0 2024-02-22T15:50:45.389 INFO:tasks.workunit.client.1.smithi062.stdout:9/637: creat d4/d1c/d83/fca x:0 0 0 2024-02-22T15:50:45.390 INFO:tasks.workunit.client.0.smithi019.stdout:2/860: creat d1/d17/d67/dfa/d11e/d126/f13a x:0 0 0 2024-02-22T15:50:45.393 INFO:tasks.workunit.client.0.smithi019.stdout:2/861: getdents d1/d86 0 2024-02-22T15:50:45.394 INFO:tasks.workunit.client.1.smithi062.stdout:9/638: dread d4/d15/d5f/d75/d84/f5b [0,4194304] 0 2024-02-22T15:50:45.395 INFO:tasks.workunit.client.0.smithi019.stdout:2/862: mkdir d1/d17/d1c/d128/d13b 0 2024-02-22T15:50:45.396 INFO:tasks.workunit.client.1.smithi062.stdout:5/517: dwrite d0/d3/da/d32/d33/f88 [0,4194304] 0 2024-02-22T15:50:45.397 INFO:tasks.workunit.client.0.smithi019.stdout:2/863: mkdir d1/d46/ddb/d10d/d12c/d13c 0 2024-02-22T15:50:45.399 INFO:tasks.workunit.client.0.smithi019.stdout:2/864: truncate d1/d17/d21/d71/d60/d11b/f131 229026 0 2024-02-22T15:50:45.399 INFO:tasks.workunit.client.1.smithi062.stdout:5/518: write d0/d3/d2f/d59/d67/d69/d3f/f4b [716136,38620] 0 2024-02-22T15:50:45.403 INFO:tasks.workunit.client.1.smithi062.stdout:5/519: creat d0/d3/d2f/d59/f9f x:0 0 0 2024-02-22T15:50:45.406 INFO:tasks.workunit.client.1.smithi062.stdout:5/520: mknod d0/d3/d30/d71/d7c/ca0 0 2024-02-22T15:50:45.408 INFO:tasks.workunit.client.1.smithi062.stdout:5/521: mknod d0/d3/d30/d71/ca1 0 2024-02-22T15:50:45.409 INFO:tasks.workunit.client.0.smithi019.stdout:2/865: getdents d1/d17/d38 0 2024-02-22T15:50:45.409 INFO:tasks.workunit.client.0.smithi019.stdout:2/866: write d1/de/d9a/fcb [80507,22012] 0 2024-02-22T15:50:45.409 INFO:tasks.workunit.client.0.smithi019.stdout:1/914: dread d8/d15/f17 [0,4194304] 0 2024-02-22T15:50:45.409 INFO:tasks.workunit.client.0.smithi019.stdout:1/915: chown d8/d18/f27 4858 1 2024-02-22T15:50:45.410 INFO:tasks.workunit.client.0.smithi019.stdout:2/867: creat d1/d17/d67/dfa/d11e/f13d x:0 0 0 2024-02-22T15:50:45.411 INFO:tasks.workunit.client.0.smithi019.stdout:1/916: chown d8/d15/df5/c10e 40 1 2024-02-22T15:50:45.411 INFO:tasks.workunit.client.0.smithi019.stdout:2/868: write d1/de/f103 [415548,89202] 0 2024-02-22T15:50:45.413 INFO:tasks.workunit.client.1.smithi062.stdout:9/639: truncate d4/d15/d5e/d63/f64 2768266 0 2024-02-22T15:50:45.414 INFO:tasks.workunit.client.0.smithi019.stdout:1/917: symlink d8/d15/d5b/dbc/l139 0 2024-02-22T15:50:45.422 INFO:tasks.workunit.client.0.smithi019.stdout:1/918: rmdir d8/d15/d5b 39 2024-02-22T15:50:45.424 INFO:tasks.workunit.client.0.smithi019.stdout:1/919: stat d8/d15/d34/d9e/d126/f107 0 2024-02-22T15:50:45.424 INFO:tasks.workunit.client.1.smithi062.stdout:5/522: dwrite d0/d3/f14 [0,4194304] 0 2024-02-22T15:50:45.427 INFO:tasks.workunit.client.1.smithi062.stdout:5/523: creat d0/d3/da/d32/d4f/fa2 x:0 0 0 2024-02-22T15:50:45.428 INFO:tasks.workunit.client.1.smithi062.stdout:5/524: chown d0/d3/d30 55 1 2024-02-22T15:50:45.428 INFO:tasks.workunit.client.0.smithi019.stdout:2/869: dwrite d1/d17/d21/d71/d60/d11b/f9b [0,4194304] 0 2024-02-22T15:50:45.430 INFO:tasks.workunit.client.1.smithi062.stdout:5/525: fsync d0/d3/d2f/f44 0 2024-02-22T15:50:45.431 INFO:tasks.workunit.client.0.smithi019.stdout:2/870: creat d1/d17/d21/d71/dd1/d106/f13e x:0 0 0 2024-02-22T15:50:45.432 INFO:tasks.workunit.client.1.smithi062.stdout:5/526: write d0/d3/f83 [1052647,82019] 0 2024-02-22T15:50:45.437 INFO:tasks.workunit.client.0.smithi019.stdout:2/871: creat d1/f13f x:0 0 0 2024-02-22T15:50:45.438 INFO:tasks.workunit.client.0.smithi019.stdout:2/872: fdatasync d1/d46/fab 0 2024-02-22T15:50:45.439 INFO:tasks.workunit.client.0.smithi019.stdout:2/873: dread - d1/fc4 zero size 2024-02-22T15:50:45.441 INFO:tasks.workunit.client.0.smithi019.stdout:2/874: creat d1/def/f140 x:0 0 0 2024-02-22T15:50:45.444 INFO:tasks.workunit.client.0.smithi019.stdout:1/920: dread d8/d18/d64/f83 [0,4194304] 0 2024-02-22T15:50:45.445 INFO:tasks.workunit.client.0.smithi019.stdout:2/875: dread d1/d17/d38/f110 [0,4194304] 0 2024-02-22T15:50:45.446 INFO:tasks.workunit.client.0.smithi019.stdout:1/921: mkdir d8/d2a/d97/d13a 0 2024-02-22T15:50:45.449 INFO:tasks.workunit.client.0.smithi019.stdout:1/922: symlink d8/d2a/d75/d81/l13b 0 2024-02-22T15:50:45.451 INFO:tasks.workunit.client.0.smithi019.stdout:1/923: write d8/d18/dd2/ff4 [2426884,83178] 0 2024-02-22T15:50:45.451 INFO:tasks.workunit.client.0.smithi019.stdout:2/876: write d1/d17/f47 [153241,14569] 0 2024-02-22T15:50:45.451 INFO:tasks.workunit.client.1.smithi062.stdout:5/527: fsync d0/d3/da/d13/f3d 0 2024-02-22T15:50:45.453 INFO:tasks.workunit.client.0.smithi019.stdout:2/877: creat d1/d17/d21/d71/d60/f141 x:0 0 0 2024-02-22T15:50:45.456 INFO:tasks.workunit.client.1.smithi062.stdout:7/470: sync 2024-02-22T15:50:45.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:45 smithi062 ceph-mon[48130]: pgmap v38: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 256 MiB/s wr, 478 op/s 2024-02-22T15:50:45.458 INFO:tasks.workunit.client.1.smithi062.stdout:5/528: dwrite d0/d3/da/fb [0,4194304] 0 2024-02-22T15:50:45.459 INFO:tasks.workunit.client.1.smithi062.stdout:5/529: chown d0/d3/d2f/d59/d67/d69/d3f/d66 48905 1 2024-02-22T15:50:45.461 INFO:tasks.workunit.client.1.smithi062.stdout:5/530: chown d0/d3/da/d13/d23/d4e/c98 13812549 1 2024-02-22T15:50:45.469 INFO:tasks.workunit.client.0.smithi019.stdout:1/924: dwrite d8/f5a [4194304,4194304] 0 2024-02-22T15:50:45.470 INFO:tasks.workunit.client.0.smithi019.stdout:1/925: dread - d8/d15/da4/f134 zero size 2024-02-22T15:50:45.475 INFO:tasks.workunit.client.1.smithi062.stdout:7/471: truncate d0/d14/d19/d30/f42 2373908 0 2024-02-22T15:50:45.476 INFO:tasks.workunit.client.1.smithi062.stdout:5/531: truncate d0/d3/da/f18 121323 0 2024-02-22T15:50:45.480 INFO:tasks.workunit.client.0.smithi019.stdout:2/878: dread - d1/d17/d21/fa5 zero size 2024-02-22T15:50:45.480 INFO:tasks.workunit.client.1.smithi062.stdout:5/532: creat d0/d3/da/d13/d23/d4e/fa3 x:0 0 0 2024-02-22T15:50:45.486 INFO:tasks.workunit.client.0.smithi019.stdout:2/879: dwrite d1/de/d9a/f113 [0,4194304] 0 2024-02-22T15:50:45.487 INFO:tasks.workunit.client.1.smithi062.stdout:5/533: dwrite d0/d3/da/d13/d23/d4e/fa3 [0,4194304] 0 2024-02-22T15:50:45.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:45 smithi019 ceph-mon[67418]: pgmap v38: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 256 MiB/s wr, 478 op/s 2024-02-22T15:50:45.499 INFO:tasks.workunit.client.0.smithi019.stdout:1/926: dread d8/fb4 [0,4194304] 0 2024-02-22T15:50:45.500 INFO:tasks.workunit.client.0.smithi019.stdout:1/927: truncate d8/fe1 4199685 0 2024-02-22T15:50:45.505 INFO:tasks.workunit.client.0.smithi019.stdout:1/928: dwrite d8/d15/d34/d9e/d126/f10b [0,4194304] 0 2024-02-22T15:50:45.510 INFO:tasks.workunit.client.0.smithi019.stdout:1/929: link d8/d23/d33/f135 d8/d18/d64/d7f/d94/df9/f13c 0 2024-02-22T15:50:45.511 INFO:tasks.workunit.client.0.smithi019.stdout:8/765: sync 2024-02-22T15:50:45.514 INFO:tasks.workunit.client.0.smithi019.stdout:1/930: symlink d8/l13d 0 2024-02-22T15:50:45.516 INFO:tasks.workunit.client.0.smithi019.stdout:1/931: symlink d8/d15/d34/d9e/d126/l13e 0 2024-02-22T15:50:45.518 INFO:tasks.workunit.client.0.smithi019.stdout:1/932: creat d8/d2a/d116/f13f x:0 0 0 2024-02-22T15:50:45.522 INFO:tasks.workunit.client.1.smithi062.stdout:7/472: dread d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:45.523 INFO:tasks.workunit.client.0.smithi019.stdout:1/933: write d8/d44/d5d/fd9 [532673,19034] 0 2024-02-22T15:50:45.527 INFO:tasks.workunit.client.0.smithi019.stdout:1/934: dread d8/d15/d5b/fb2 [0,4194304] 0 2024-02-22T15:50:45.536 INFO:tasks.workunit.client.0.smithi019.stdout:8/766: dread d9/d14/d17/f50 [0,4194304] 0 2024-02-22T15:50:45.542 INFO:tasks.workunit.client.1.smithi062.stdout:7/473: dread d0/d6/d2e/d58/f5c [0,4194304] 0 2024-02-22T15:50:45.544 INFO:tasks.workunit.client.0.smithi019.stdout:1/935: dread d8/d18/d85/f8d [0,4194304] 0 2024-02-22T15:50:45.545 INFO:tasks.workunit.client.0.smithi019.stdout:1/936: write d8/d2a/f2b [95525,14214] 0 2024-02-22T15:50:45.545 INFO:tasks.workunit.client.1.smithi062.stdout:7/474: creat d0/d6/f93 x:0 0 0 2024-02-22T15:50:45.551 INFO:tasks.workunit.client.1.smithi062.stdout:7/475: dwrite d0/d14/d13/f48 [0,4194304] 0 2024-02-22T15:50:45.554 INFO:tasks.workunit.client.0.smithi019.stdout:8/767: dread d9/d14/f19 [0,4194304] 0 2024-02-22T15:50:45.554 INFO:tasks.workunit.client.1.smithi062.stdout:7/476: write d0/d14/f17 [1729336,105173] 0 2024-02-22T15:50:45.555 INFO:tasks.workunit.client.0.smithi019.stdout:8/768: chown d9/d14/d1f/d6d/f95 23 1 2024-02-22T15:50:45.556 INFO:tasks.workunit.client.0.smithi019.stdout:8/769: chown d9/d14/d17/d1c/da0/fcc 0 1 2024-02-22T15:50:45.562 INFO:tasks.workunit.client.1.smithi062.stdout:7/477: write d0/d6/d2e/d55/f69 [1412974,55148] 0 2024-02-22T15:50:45.563 INFO:tasks.workunit.client.0.smithi019.stdout:8/770: dread d9/f66 [0,4194304] 0 2024-02-22T15:50:45.563 INFO:tasks.workunit.client.1.smithi062.stdout:7/478: write d0/f89 [877764,125669] 0 2024-02-22T15:50:45.564 INFO:tasks.workunit.client.1.smithi062.stdout:7/479: fdatasync d0/d6/d2e/d4a/f4c 0 2024-02-22T15:50:45.565 INFO:tasks.workunit.client.0.smithi019.stdout:8/771: truncate d9/d14/d1f/f41 5063784 0 2024-02-22T15:50:45.566 INFO:tasks.workunit.client.0.smithi019.stdout:8/772: rmdir d9/d14 39 2024-02-22T15:50:45.568 INFO:tasks.workunit.client.1.smithi062.stdout:7/480: rename d0/d6/d5b/d60/f66 to d0/d6/d2e/d35/f94 0 2024-02-22T15:50:45.571 INFO:tasks.workunit.client.0.smithi019.stdout:8/773: dread d9/d14/d17/fbe [0,4194304] 0 2024-02-22T15:50:45.574 INFO:tasks.workunit.client.1.smithi062.stdout:7/481: dread d0/d6/f52 [0,4194304] 0 2024-02-22T15:50:45.588 INFO:tasks.workunit.client.1.smithi062.stdout:7/482: dread d0/f10 [0,4194304] 0 2024-02-22T15:50:45.590 INFO:tasks.workunit.client.1.smithi062.stdout:7/483: creat d0/d14/d19/d30/d37/f95 x:0 0 0 2024-02-22T15:50:45.591 INFO:tasks.workunit.client.1.smithi062.stdout:7/484: symlink d0/l96 0 2024-02-22T15:50:45.593 INFO:tasks.workunit.client.1.smithi062.stdout:7/485: mknod d0/d6/d2e/d55/d80/c97 0 2024-02-22T15:50:45.596 INFO:tasks.workunit.client.1.smithi062.stdout:7/486: link d0/d14/d13/fc d0/d14/d19/f98 0 2024-02-22T15:50:45.606 INFO:tasks.workunit.client.1.smithi062.stdout:7/487: dwrite d0/d14/d13/f3e [0,4194304] 0 2024-02-22T15:50:45.607 INFO:tasks.workunit.client.1.smithi062.stdout:7/488: mknod d0/d14/d28/c99 0 2024-02-22T15:50:45.608 INFO:tasks.workunit.client.1.smithi062.stdout:6/692: sync 2024-02-22T15:50:45.608 INFO:tasks.workunit.client.1.smithi062.stdout:1/577: sync 2024-02-22T15:50:45.609 INFO:tasks.workunit.client.1.smithi062.stdout:3/544: sync 2024-02-22T15:50:45.609 INFO:tasks.workunit.client.1.smithi062.stdout:7/489: chown d0/d14/f67 1542242834 1 2024-02-22T15:50:45.610 INFO:tasks.workunit.client.1.smithi062.stdout:3/545: fsync d5/d26/f38 0 2024-02-22T15:50:45.611 INFO:tasks.workunit.client.1.smithi062.stdout:6/693: write d5/df/d40/d44/d76/fdb [621187,94581] 0 2024-02-22T15:50:45.612 INFO:tasks.workunit.client.1.smithi062.stdout:1/578: read d1/d6/d7/d2b/d47/d63/f68 [355136,35725] 0 2024-02-22T15:50:45.613 INFO:tasks.workunit.client.1.smithi062.stdout:7/490: dread d0/d14/d19/d30/f7f [0,4194304] 0 2024-02-22T15:50:45.614 INFO:tasks.workunit.client.1.smithi062.stdout:3/546: symlink d5/d16/d21/d84/lb3 0 2024-02-22T15:50:45.615 INFO:tasks.workunit.client.1.smithi062.stdout:6/694: mkdir d5/df/d40/dac/de1 0 2024-02-22T15:50:45.617 INFO:tasks.workunit.client.1.smithi062.stdout:7/491: truncate d0/d6/d2e/d4a/d5a/d5e/f6c 112398 0 2024-02-22T15:50:45.617 INFO:tasks.workunit.client.1.smithi062.stdout:3/547: creat d5/d26/d2d/fb4 x:0 0 0 2024-02-22T15:50:45.619 INFO:tasks.workunit.client.1.smithi062.stdout:1/579: rmdir d1/d6/d7/d2b/d94/db9 0 2024-02-22T15:50:45.621 INFO:tasks.workunit.client.1.smithi062.stdout:6/695: symlink d5/d8a/ddd/le2 0 2024-02-22T15:50:45.621 INFO:tasks.workunit.client.1.smithi062.stdout:7/492: mknod d0/d14/d19/d77/c9a 0 2024-02-22T15:50:45.626 INFO:tasks.workunit.client.1.smithi062.stdout:3/548: dwrite d5/d16/d21/fa5 [0,4194304] 0 2024-02-22T15:50:45.626 INFO:tasks.workunit.client.1.smithi062.stdout:6/696: creat d5/dd/fe3 x:0 0 0 2024-02-22T15:50:45.627 INFO:tasks.workunit.client.1.smithi062.stdout:7/493: truncate d0/d14/d19/d30/f8d 884578 0 2024-02-22T15:50:45.628 INFO:tasks.workunit.client.1.smithi062.stdout:7/494: readlink d0/d14/d19/l31 0 2024-02-22T15:50:45.629 INFO:tasks.workunit.client.1.smithi062.stdout:3/549: write d5/d16/d21/d36/d5a/d85/fb2 [1510473,7970] 0 2024-02-22T15:50:45.629 INFO:tasks.workunit.client.1.smithi062.stdout:7/495: chown d0/f89 29291631 1 2024-02-22T15:50:45.631 INFO:tasks.workunit.client.1.smithi062.stdout:3/550: truncate d5/d16/d88/f91 275287 0 2024-02-22T15:50:45.634 INFO:tasks.workunit.client.1.smithi062.stdout:1/580: write d1/d6/d7/f29 [667359,27705] 0 2024-02-22T15:50:45.638 INFO:tasks.workunit.client.1.smithi062.stdout:7/496: dwrite d0/d14/d19/f1a [0,4194304] 0 2024-02-22T15:50:45.640 INFO:tasks.workunit.client.1.smithi062.stdout:1/581: mknod d1/d6/d36/d37/cc2 0 2024-02-22T15:50:45.641 INFO:tasks.workunit.client.1.smithi062.stdout:3/551: link d5/d16/d21/d36/d5a/c81 d5/d16/cb5 0 2024-02-22T15:50:45.650 INFO:tasks.workunit.client.1.smithi062.stdout:1/582: mknod d1/d6/d7/cc3 0 2024-02-22T15:50:45.654 INFO:tasks.workunit.client.1.smithi062.stdout:7/497: dwrite d0/d14/d19/d30/d37/f95 [0,4194304] 0 2024-02-22T15:50:45.668 INFO:tasks.workunit.client.0.smithi019.stdout:9/809: sync 2024-02-22T15:50:45.669 INFO:tasks.workunit.client.0.smithi019.stdout:0/812: sync 2024-02-22T15:50:45.669 INFO:tasks.workunit.client.1.smithi062.stdout:3/552: mknod d5/d16/cb6 0 2024-02-22T15:50:45.670 INFO:tasks.workunit.client.0.smithi019.stdout:0/813: write d4/db/f43 [3945277,79104] 0 2024-02-22T15:50:45.671 INFO:tasks.workunit.client.0.smithi019.stdout:0/814: stat d4/db/d1b/d40/l10c 0 2024-02-22T15:50:45.671 INFO:tasks.workunit.client.1.smithi062.stdout:7/498: write d0/d14/d13/f4d [945088,61699] 0 2024-02-22T15:50:45.672 INFO:tasks.workunit.client.0.smithi019.stdout:9/810: rmdir d2/d1f/d2e/d57/d60/d8c 39 2024-02-22T15:50:45.673 INFO:tasks.workunit.client.0.smithi019.stdout:0/815: write d4/db/d1b/f1c [4472008,99696] 0 2024-02-22T15:50:45.673 INFO:tasks.workunit.client.0.smithi019.stdout:9/811: chown d2/d7/dc/d27/dbc/d110 374728238 1 2024-02-22T15:50:45.675 INFO:tasks.workunit.client.1.smithi062.stdout:1/583: creat d1/d6/d7/d2b/fc4 x:0 0 0 2024-02-22T15:50:45.677 INFO:tasks.workunit.client.0.smithi019.stdout:0/816: dread d4/db/d24/d80/ff9 [0,4194304] 0 2024-02-22T15:50:45.678 INFO:tasks.workunit.client.0.smithi019.stdout:0/817: stat d4/d32/c8a 0 2024-02-22T15:50:45.679 INFO:tasks.workunit.client.0.smithi019.stdout:9/812: getdents d2/d1f/d2e/d7d 0 2024-02-22T15:50:45.680 INFO:tasks.workunit.client.1.smithi062.stdout:1/584: creat d1/d6/d7/d2b/d47/d7e/fc5 x:0 0 0 2024-02-22T15:50:45.682 INFO:tasks.workunit.client.0.smithi019.stdout:9/813: rmdir d2/d7/dc/d27/dbc 39 2024-02-22T15:50:45.683 INFO:tasks.workunit.client.0.smithi019.stdout:0/818: symlink d4/db/d1b/d63/db0/da1/l118 0 2024-02-22T15:50:45.683 INFO:tasks.workunit.client.1.smithi062.stdout:1/585: readlink d1/d6/d7/d8a/l9c 0 2024-02-22T15:50:45.686 INFO:tasks.workunit.client.0.smithi019.stdout:0/819: creat d4/d22/d3d/f119 x:0 0 0 2024-02-22T15:50:45.687 INFO:tasks.workunit.client.0.smithi019.stdout:0/820: read f2 [3250863,128163] 0 2024-02-22T15:50:45.688 INFO:tasks.workunit.client.1.smithi062.stdout:1/586: creat d1/d6/d7/d2b/d47/d63/db6/fc6 x:0 0 0 2024-02-22T15:50:45.689 INFO:tasks.workunit.client.0.smithi019.stdout:9/814: dwrite d2/d1f/db5/fde [8388608,4194304] 0 2024-02-22T15:50:45.693 INFO:tasks.workunit.client.0.smithi019.stdout:9/815: symlink d2/d7/dc/d27/d68/l11e 0 2024-02-22T15:50:45.695 INFO:tasks.workunit.client.0.smithi019.stdout:9/816: creat d2/dc8/f11f x:0 0 0 2024-02-22T15:50:45.695 INFO:tasks.workunit.client.0.smithi019.stdout:0/821: truncate d4/f92 3873421 0 2024-02-22T15:50:45.696 INFO:tasks.workunit.client.1.smithi062.stdout:1/587: dread d1/d6/d36/d76/fa1 [0,4194304] 0 2024-02-22T15:50:45.697 INFO:tasks.workunit.client.0.smithi019.stdout:9/817: mknod d2/dc8/c120 0 2024-02-22T15:50:45.699 INFO:tasks.workunit.client.0.smithi019.stdout:0/822: mkdir d4/db/d1b/d40/d11a 0 2024-02-22T15:50:45.699 INFO:tasks.workunit.client.1.smithi062.stdout:7/499: dread d0/d91/f59 [0,4194304] 0 2024-02-22T15:50:45.699 INFO:tasks.workunit.client.1.smithi062.stdout:1/588: mknod d1/d6/de/d9/cc7 0 2024-02-22T15:50:45.700 INFO:tasks.workunit.client.1.smithi062.stdout:3/553: dwrite d5/d16/d56/f7e [0,4194304] 0 2024-02-22T15:50:45.701 INFO:tasks.workunit.client.1.smithi062.stdout:1/589: chown d1/d6/de/f17 17 1 2024-02-22T15:50:45.702 INFO:tasks.workunit.client.0.smithi019.stdout:9/818: fsync d2/d7/dc/d27/dbc/f82 0 2024-02-22T15:50:45.704 INFO:tasks.workunit.client.0.smithi019.stdout:9/819: symlink d2/d1f/d2e/d48/l121 0 2024-02-22T15:50:45.706 INFO:tasks.workunit.client.1.smithi062.stdout:3/554: write d5/d1a/f25 [2534696,55401] 0 2024-02-22T15:50:45.711 INFO:tasks.workunit.client.1.smithi062.stdout:1/590: truncate d1/d6/d7/d2b/d49/d74/f85 18567 0 2024-02-22T15:50:45.711 INFO:tasks.workunit.client.1.smithi062.stdout:3/555: read d5/d1a/f1e [3258365,102325] 0 2024-02-22T15:50:45.711 INFO:tasks.workunit.client.1.smithi062.stdout:1/591: chown d1/d6/d7/d4a/d65 1052163892 1 2024-02-22T15:50:45.712 INFO:tasks.workunit.client.0.smithi019.stdout:9/820: dwrite d2/d1f/d2e/d57/d60/fe9 [0,4194304] 0 2024-02-22T15:50:45.712 INFO:tasks.workunit.client.1.smithi062.stdout:3/556: write d5/d16/f48 [4973823,91455] 0 2024-02-22T15:50:45.713 INFO:tasks.workunit.client.0.smithi019.stdout:4/789: sync 2024-02-22T15:50:45.713 INFO:tasks.workunit.client.0.smithi019.stdout:5/842: sync 2024-02-22T15:50:45.713 INFO:tasks.workunit.client.0.smithi019.stdout:3/675: sync 2024-02-22T15:50:45.715 INFO:tasks.workunit.client.0.smithi019.stdout:5/843: write d8/dd/d19/d3a/d46/d7e/dbe/de8/ff0 [834068,645] 0 2024-02-22T15:50:45.715 INFO:tasks.workunit.client.1.smithi062.stdout:3/557: fsync d5/d16/d21/f34 0 2024-02-22T15:50:45.716 INFO:tasks.workunit.client.0.smithi019.stdout:4/790: dread d2/db/d14/d23/d33/f7d [0,4194304] 0 2024-02-22T15:50:45.717 INFO:tasks.workunit.client.1.smithi062.stdout:1/592: mknod d1/d6/d7/d2b/d96/cc8 0 2024-02-22T15:50:45.717 INFO:tasks.workunit.client.0.smithi019.stdout:9/821: truncate d2/d1f/d2e/d57/f94 804209 0 2024-02-22T15:50:45.719 INFO:tasks.workunit.client.0.smithi019.stdout:4/791: chown d2/db/d14/d23/d21/d42/dc4 1702 1 2024-02-22T15:50:45.719 INFO:tasks.workunit.client.0.smithi019.stdout:5/844: fsync d8/dd/d19/d3a/d46/d7e/dbe/fd7 0 2024-02-22T15:50:45.720 INFO:tasks.workunit.client.0.smithi019.stdout:9/822: unlink d2/dc8/fea 0 2024-02-22T15:50:45.722 INFO:tasks.workunit.client.1.smithi062.stdout:7/500: write d0/d6/d2e/d4a/f2d [413483,32931] 0 2024-02-22T15:50:45.723 INFO:tasks.workunit.client.0.smithi019.stdout:4/792: creat d2/db/d44/f101 x:0 0 0 2024-02-22T15:50:45.724 INFO:tasks.workunit.client.1.smithi062.stdout:1/593: write d1/d6/d7/d4a/d65/d83/f8c [1248712,93731] 0 2024-02-22T15:50:45.725 INFO:tasks.workunit.client.0.smithi019.stdout:4/793: dread - d2/d3f/ffa zero size 2024-02-22T15:50:45.726 INFO:tasks.workunit.client.0.smithi019.stdout:5/845: creat d8/d11/d33/f11f x:0 0 0 2024-02-22T15:50:45.728 INFO:tasks.workunit.client.0.smithi019.stdout:4/794: fsync d2/db/d92/ffc 0 2024-02-22T15:50:45.729 INFO:tasks.workunit.client.1.smithi062.stdout:1/594: creat d1/d6/d7/d2b/d96/fc9 x:0 0 0 2024-02-22T15:50:45.729 INFO:tasks.workunit.client.0.smithi019.stdout:5/846: stat d8/dd/d19/d77/c84 0 2024-02-22T15:50:45.730 INFO:tasks.workunit.client.1.smithi062.stdout:3/558: dwrite d5/d16/d21/d36/d5a/f8c [0,4194304] 0 2024-02-22T15:50:45.730 INFO:tasks.workunit.client.0.smithi019.stdout:9/823: unlink d2/d7/d52/ff2 0 2024-02-22T15:50:45.731 INFO:tasks.workunit.client.1.smithi062.stdout:1/595: readlink d1/d6/d7/d4a/d65/l97 0 2024-02-22T15:50:45.732 INFO:tasks.workunit.client.0.smithi019.stdout:0/823: truncate d4/db/d1b/f2f 395662 0 2024-02-22T15:50:45.733 INFO:tasks.workunit.client.0.smithi019.stdout:4/795: symlink d2/db/d14/d23/d67/l102 0 2024-02-22T15:50:45.735 INFO:tasks.workunit.client.1.smithi062.stdout:3/559: rename d5/d16/d21/d36/d5a/f8c to d5/d1a/d3c/d62/fb7 0 2024-02-22T15:50:45.739 INFO:tasks.workunit.client.0.smithi019.stdout:5/847: symlink d8/dd/d19/d10b/l120 0 2024-02-22T15:50:45.739 INFO:tasks.workunit.client.1.smithi062.stdout:1/596: rename d1/d6/d7/d2b/d94 to d1/d6/d7/d8a/dca 0 2024-02-22T15:50:45.740 INFO:tasks.workunit.client.1.smithi062.stdout:7/501: truncate d0/f10 765093 0 2024-02-22T15:50:45.740 INFO:tasks.workunit.client.0.smithi019.stdout:4/796: dread - d2/fb6 zero size 2024-02-22T15:50:45.741 INFO:tasks.workunit.client.0.smithi019.stdout:3/676: dread d9/d26/d32/f43 [0,4194304] 0 2024-02-22T15:50:45.742 INFO:tasks.workunit.client.1.smithi062.stdout:3/560: creat d5/d1a/d3c/d62/fb8 x:0 0 0 2024-02-22T15:50:45.742 INFO:tasks.workunit.client.1.smithi062.stdout:1/597: rmdir d1/d6/d7/d8a 39 2024-02-22T15:50:45.742 INFO:tasks.workunit.client.0.smithi019.stdout:5/848: truncate d8/d11/fbb 929401 0 2024-02-22T15:50:45.743 INFO:tasks.workunit.client.1.smithi062.stdout:7/502: symlink d0/d14/d19/d30/d37/l9b 0 2024-02-22T15:50:45.744 INFO:tasks.workunit.client.1.smithi062.stdout:4/690: sync 2024-02-22T15:50:45.744 INFO:tasks.workunit.client.1.smithi062.stdout:2/734: sync 2024-02-22T15:50:45.744 INFO:tasks.workunit.client.1.smithi062.stdout:8/779: sync 2024-02-22T15:50:45.745 INFO:tasks.workunit.client.1.smithi062.stdout:7/503: read d0/d14/d13/fc [136559,28493] 0 2024-02-22T15:50:45.746 INFO:tasks.workunit.client.0.smithi019.stdout:3/677: mkdir d9/d65/dd6 0 2024-02-22T15:50:45.748 INFO:tasks.workunit.client.0.smithi019.stdout:5/849: rmdir d8/dd/d19/d77/d7a 39 2024-02-22T15:50:45.750 INFO:tasks.workunit.client.0.smithi019.stdout:3/678: rmdir d9/d22/d27 39 2024-02-22T15:50:45.750 INFO:tasks.workunit.client.1.smithi062.stdout:3/561: dread - d5/d1a/d3b/f71 zero size 2024-02-22T15:50:45.751 INFO:tasks.workunit.client.1.smithi062.stdout:8/780: write d8/d9/d44/f56 [3020985,1382] 0 2024-02-22T15:50:45.753 INFO:tasks.workunit.client.0.smithi019.stdout:1/937: sync 2024-02-22T15:50:45.754 INFO:tasks.workunit.client.0.smithi019.stdout:2/880: sync 2024-02-22T15:50:45.754 INFO:tasks.workunit.client.0.smithi019.stdout:8/774: sync 2024-02-22T15:50:45.754 INFO:tasks.workunit.client.0.smithi019.stdout:0/824: write d4/d32/f46 [3195699,17665] 0 2024-02-22T15:50:45.754 INFO:tasks.workunit.client.0.smithi019.stdout:3/679: creat d9/d16/fd7 x:0 0 0 2024-02-22T15:50:45.754 INFO:tasks.workunit.client.1.smithi062.stdout:4/691: mknod d4/d2a/d2d/d2e/d6a/d94/cf0 0 2024-02-22T15:50:45.755 INFO:tasks.workunit.client.0.smithi019.stdout:5/850: mknod d8/dd/d19/d38/d47/ddb/dde/c121 0 2024-02-22T15:50:45.755 INFO:tasks.workunit.client.1.smithi062.stdout:2/735: mkdir d1/d28/df6 0 2024-02-22T15:50:45.757 INFO:tasks.workunit.client.0.smithi019.stdout:9/824: dread d2/d7/d52/faa [0,4194304] 0 2024-02-22T15:50:45.758 INFO:tasks.workunit.client.0.smithi019.stdout:1/938: stat d8/d15/d5b/dbc/l139 0 2024-02-22T15:50:45.758 INFO:tasks.workunit.client.0.smithi019.stdout:0/825: readlink d4/db/l84 0 2024-02-22T15:50:45.759 INFO:tasks.workunit.client.0.smithi019.stdout:2/881: chown d1/d46/d9d/l9f 616481 1 2024-02-22T15:50:45.760 INFO:tasks.workunit.client.1.smithi062.stdout:3/562: rename d5/d16/d21/f2b to d5/d16/d21/d55/fb9 0 2024-02-22T15:50:45.761 INFO:tasks.workunit.client.0.smithi019.stdout:3/680: dread - d9/d26/d3f/d59/dae/f81 zero size 2024-02-22T15:50:45.761 INFO:tasks.workunit.client.0.smithi019.stdout:0/826: chown d4/d22/d2c/fac 106878019 1 2024-02-22T15:50:45.761 INFO:tasks.workunit.client.1.smithi062.stdout:4/692: symlink d4/d2a/d2d/d46/lf1 0 2024-02-22T15:50:45.763 INFO:tasks.workunit.client.1.smithi062.stdout:2/736: symlink d1/d47/d39/d5d/lf7 0 2024-02-22T15:50:45.764 INFO:tasks.workunit.client.0.smithi019.stdout:0/827: chown d4/d32/f46 849102907 1 2024-02-22T15:50:45.765 INFO:tasks.workunit.client.0.smithi019.stdout:4/797: dread d2/fa [4194304,4194304] 0 2024-02-22T15:50:45.766 INFO:tasks.workunit.client.1.smithi062.stdout:3/563: write d5/d16/d21/d55/fa0 [810915,110587] 0 2024-02-22T15:50:45.767 INFO:tasks.workunit.client.1.smithi062.stdout:7/504: creat d0/f9c x:0 0 0 2024-02-22T15:50:45.768 INFO:tasks.workunit.client.1.smithi062.stdout:8/781: dwrite d8/d9/d2c/d49/d94/dc1/f91 [0,4194304] 0 2024-02-22T15:50:45.769 INFO:tasks.workunit.client.0.smithi019.stdout:8/775: dread d9/d14/d17/f50 [0,4194304] 0 2024-02-22T15:50:45.770 INFO:tasks.workunit.client.1.smithi062.stdout:4/693: symlink d4/d1c/dac/lf2 0 2024-02-22T15:50:45.771 INFO:tasks.workunit.client.0.smithi019.stdout:5/851: write d8/dd/d19/d38/d54/fb2 [1232452,54260] 0 2024-02-22T15:50:45.773 INFO:tasks.workunit.client.0.smithi019.stdout:9/825: write d2/d1f/d2e/d57/d60/f11b [6635850,117926] 0 2024-02-22T15:50:45.773 INFO:tasks.workunit.client.0.smithi019.stdout:1/939: dwrite d8/f53 [0,4194304] 0 2024-02-22T15:50:45.773 INFO:tasks.workunit.client.1.smithi062.stdout:1/598: getdents d1/d6 0 2024-02-22T15:50:45.776 INFO:tasks.workunit.client.1.smithi062.stdout:2/737: rmdir d1/d1e/d38/d55 39 2024-02-22T15:50:45.777 INFO:tasks.workunit.client.1.smithi062.stdout:8/782: read d8/f29 [1429002,104057] 0 2024-02-22T15:50:45.780 INFO:tasks.workunit.client.1.smithi062.stdout:2/738: chown d1/d47/d39/d5d/ce2 518688 1 2024-02-22T15:50:45.781 INFO:tasks.workunit.client.0.smithi019.stdout:2/882: dwrite d1/d17/d1c/d7f/fdf [0,4194304] 0 2024-02-22T15:50:45.782 INFO:tasks.workunit.client.1.smithi062.stdout:7/505: fdatasync d0/d6/d2e/d35/f94 0 2024-02-22T15:50:45.783 INFO:tasks.workunit.client.0.smithi019.stdout:9/826: mknod d2/d7/dee/c122 0 2024-02-22T15:50:45.784 INFO:tasks.workunit.client.1.smithi062.stdout:4/694: mknod d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd0/cf3 0 2024-02-22T15:50:45.784 INFO:tasks.workunit.client.1.smithi062.stdout:3/564: dwrite d5/d16/d21/d84/f9a [0,4194304] 0 2024-02-22T15:50:45.784 INFO:tasks.workunit.client.1.smithi062.stdout:8/783: chown d8/d9/d2c/c3f 5715065 1 2024-02-22T15:50:45.786 INFO:tasks.workunit.client.0.smithi019.stdout:3/681: dwrite d9/d26/d2d/d53/f56 [0,4194304] 0 2024-02-22T15:50:45.786 INFO:tasks.workunit.client.0.smithi019.stdout:8/776: creat d9/d14/d6f/d7e/d94/d9a/f105 x:0 0 0 2024-02-22T15:50:45.786 INFO:tasks.workunit.client.0.smithi019.stdout:0/828: link d4/db/d76/ceb d4/db/d1b/d40/d11a/c11b 0 2024-02-22T15:50:45.790 INFO:tasks.workunit.client.1.smithi062.stdout:1/599: symlink d1/d6/d7/d8a/dca/lcb 0 2024-02-22T15:50:45.793 INFO:tasks.workunit.client.0.smithi019.stdout:1/940: creat d8/d2a/f140 x:0 0 0 2024-02-22T15:50:45.796 INFO:tasks.workunit.client.0.smithi019.stdout:2/883: write d1/d17/d21/d71/f37 [2418803,82946] 0 2024-02-22T15:50:45.797 INFO:tasks.workunit.client.0.smithi019.stdout:4/798: dwrite d2/db/d14/f87 [0,4194304] 0 2024-02-22T15:50:45.797 INFO:tasks.workunit.client.0.smithi019.stdout:9/827: mkdir d2/d1f/d2e/d123 0 2024-02-22T15:50:45.798 INFO:tasks.workunit.client.1.smithi062.stdout:4/695: mknod d4/d1c/cf4 0 2024-02-22T15:50:45.799 INFO:tasks.workunit.client.1.smithi062.stdout:2/739: creat d1/d1e/ff8 x:0 0 0 2024-02-22T15:50:45.800 INFO:tasks.workunit.client.1.smithi062.stdout:1/600: mkdir d1/d6/d7/d2b/d49/dcc 0 2024-02-22T15:50:45.801 INFO:tasks.workunit.client.0.smithi019.stdout:9/828: fsync d2/d1f/d2e/d7d/f10b 0 2024-02-22T15:50:45.803 INFO:tasks.workunit.client.1.smithi062.stdout:7/506: dwrite d0/d14/f8b [0,4194304] 0 2024-02-22T15:50:45.804 INFO:tasks.workunit.client.0.smithi019.stdout:4/799: symlink d2/db/d14/d23/d21/dbf/l103 0 2024-02-22T15:50:45.805 INFO:tasks.workunit.client.0.smithi019.stdout:3/682: symlink d9/d26/d3f/d59/ld8 0 2024-02-22T15:50:45.806 INFO:tasks.workunit.client.0.smithi019.stdout:0/829: mkdir d4/db/d1b/d40/d44/d75/db7/d11c 0 2024-02-22T15:50:45.808 INFO:tasks.workunit.client.1.smithi062.stdout:3/565: link d5/d1a/l5d d5/d26/d2d/da2/dab/lba 0 2024-02-22T15:50:45.810 INFO:tasks.workunit.client.0.smithi019.stdout:1/941: symlink d8/d2a/d75/d81/l141 0 2024-02-22T15:50:45.810 INFO:tasks.workunit.client.0.smithi019.stdout:8/777: link d9/d14/d17/d1c/dd8/fe4 d9/d14/d6f/d7e/dff/f106 0 2024-02-22T15:50:45.812 INFO:tasks.workunit.client.1.smithi062.stdout:7/507: rename d0/d14/d19/d30/c43 to d0/d14/d19/d30/c9d 0 2024-02-22T15:50:45.812 INFO:tasks.workunit.client.1.smithi062.stdout:3/566: write d5/d26/f6c [11235,79916] 0 2024-02-22T15:50:45.813 INFO:tasks.workunit.client.0.smithi019.stdout:2/884: rename d1/de/d15/la0 to d1/d17/d1c/d128/db9/dd0/dee/d120/l142 0 2024-02-22T15:50:45.814 INFO:tasks.workunit.client.0.smithi019.stdout:3/683: truncate d9/d26/f55 485290 0 2024-02-22T15:50:45.814 INFO:tasks.workunit.client.0.smithi019.stdout:9/829: dwrite d2/d1f/d2e/d57/d60/fe9 [0,4194304] 0 2024-02-22T15:50:45.816 INFO:tasks.workunit.client.0.smithi019.stdout:4/800: creat d2/db/d14/d23/d21/d35/dc0/f104 x:0 0 0 2024-02-22T15:50:45.817 INFO:tasks.workunit.client.1.smithi062.stdout:3/567: write d5/d16/d97/fb1 [556447,12025] 0 2024-02-22T15:50:45.819 INFO:tasks.workunit.client.1.smithi062.stdout:9/640: sync 2024-02-22T15:50:45.819 INFO:tasks.workunit.client.1.smithi062.stdout:0/738: sync 2024-02-22T15:50:45.819 INFO:tasks.workunit.client.1.smithi062.stdout:5/534: sync 2024-02-22T15:50:45.821 INFO:tasks.workunit.client.0.smithi019.stdout:2/885: creat d1/d17/d67/dfa/d11e/f143 x:0 0 0 2024-02-22T15:50:45.821 INFO:tasks.workunit.client.0.smithi019.stdout:3/684: mknod d9/d26/d2d/d53/cd9 0 2024-02-22T15:50:45.822 INFO:tasks.workunit.client.0.smithi019.stdout:9/830: creat d2/d7/dc/d27/dbc/dcc/f124 x:0 0 0 2024-02-22T15:50:45.824 INFO:tasks.workunit.client.0.smithi019.stdout:8/778: creat d9/f107 x:0 0 0 2024-02-22T15:50:45.824 INFO:tasks.workunit.client.0.smithi019.stdout:2/886: write d1/d46/fab [2035946,66419] 0 2024-02-22T15:50:45.824 INFO:tasks.workunit.client.1.smithi062.stdout:1/601: dwrite d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:45.824 INFO:tasks.workunit.client.1.smithi062.stdout:4/696: dwrite d4/d1c/faa [4194304,4194304] 0 2024-02-22T15:50:45.831 INFO:tasks.workunit.client.0.smithi019.stdout:9/831: mknod d2/d7/dc/d27/dbc/d110/c125 0 2024-02-22T15:50:45.831 INFO:tasks.workunit.client.0.smithi019.stdout:2/887: mknod d1/de/d15/c144 0 2024-02-22T15:50:45.832 INFO:tasks.workunit.client.1.smithi062.stdout:3/568: creat d5/d26/d2d/d3f/fbb x:0 0 0 2024-02-22T15:50:45.832 INFO:tasks.workunit.client.1.smithi062.stdout:3/569: stat d5/d1a/d3b 0 2024-02-22T15:50:45.833 INFO:tasks.workunit.client.1.smithi062.stdout:4/697: truncate d4/d1c/d2c/db9/fba 469692 0 2024-02-22T15:50:45.835 INFO:tasks.workunit.client.1.smithi062.stdout:5/535: creat d0/d3/da/d13/d57/d79/fa4 x:0 0 0 2024-02-22T15:50:45.835 INFO:tasks.workunit.client.1.smithi062.stdout:7/508: dwrite d0/d6/d2e/d4a/f2d [0,4194304] 0 2024-02-22T15:50:45.835 INFO:tasks.workunit.client.1.smithi062.stdout:9/641: creat d4/d28/d50/da3/fcb x:0 0 0 2024-02-22T15:50:45.835 INFO:tasks.workunit.client.1.smithi062.stdout:3/570: chown d5/d1a/d3b/l52 659 1 2024-02-22T15:50:45.837 INFO:tasks.workunit.client.1.smithi062.stdout:5/536: readlink d0/l25 0 2024-02-22T15:50:45.837 INFO:tasks.workunit.client.1.smithi062.stdout:0/739: read d1/d13/dde/d76/d8c/f88 [349836,28000] 0 2024-02-22T15:50:45.838 INFO:tasks.workunit.client.0.smithi019.stdout:8/779: dwrite d9/d14/d1f/d5c/fc8 [0,4194304] 0 2024-02-22T15:50:45.839 INFO:tasks.workunit.client.1.smithi062.stdout:8/784: write d8/f39 [2157718,108469] 0 2024-02-22T15:50:45.840 INFO:tasks.workunit.client.1.smithi062.stdout:2/740: write d1/d47/d32/d52/d80/fb0 [454517,66384] 0 2024-02-22T15:50:45.840 INFO:tasks.workunit.client.1.smithi062.stdout:9/642: dread - d4/d15/d5e/d63/fb2 zero size 2024-02-22T15:50:45.841 INFO:tasks.workunit.client.0.smithi019.stdout:9/832: rmdir d2/dec 39 2024-02-22T15:50:45.842 INFO:tasks.workunit.client.1.smithi062.stdout:4/698: unlink d4/d1c/d2c/f87 0 2024-02-22T15:50:45.844 INFO:tasks.workunit.client.0.smithi019.stdout:9/833: readlink d2/d7/dc/dd6/lb8 0 2024-02-22T15:50:45.844 INFO:tasks.workunit.client.1.smithi062.stdout:5/537: write d0/d3/da/d13/d23/d4e/d60/f63 [569015,85948] 0 2024-02-22T15:50:45.845 INFO:tasks.workunit.client.1.smithi062.stdout:7/509: mknod d0/d14/d13/c9e 0 2024-02-22T15:50:45.847 INFO:tasks.workunit.client.0.smithi019.stdout:3/685: rename d9/d26/d3f/d59/f6b to d9/d22/d27/dd0/fda 0 2024-02-22T15:50:45.848 INFO:tasks.workunit.client.1.smithi062.stdout:3/571: chown d5/lb 162086 1 2024-02-22T15:50:45.850 INFO:tasks.workunit.client.0.smithi019.stdout:0/830: write d4/d22/d2c/d68/d6d/fc9 [3229,99132] 0 2024-02-22T15:50:45.853 INFO:tasks.workunit.client.1.smithi062.stdout:0/740: mknod d1/d13/dde/d76/d8c/d5d/dc1/cf5 0 2024-02-22T15:50:45.853 INFO:tasks.workunit.client.1.smithi062.stdout:9/643: truncate d4/d1c/f58 610326 0 2024-02-22T15:50:45.853 INFO:tasks.workunit.client.1.smithi062.stdout:6/697: sync 2024-02-22T15:50:45.853 INFO:tasks.workunit.client.1.smithi062.stdout:1/602: getdents d1/d6/d14 0 2024-02-22T15:50:45.854 INFO:tasks.workunit.client.1.smithi062.stdout:4/699: chown d4/d1c/dac/lf2 0 1 2024-02-22T15:50:45.855 INFO:tasks.workunit.client.1.smithi062.stdout:7/510: truncate d0/f9c 108348 0 2024-02-22T15:50:45.855 INFO:tasks.workunit.client.1.smithi062.stdout:3/572: mkdir d5/d16/d56/dbc 0 2024-02-22T15:50:45.857 INFO:tasks.workunit.client.0.smithi019.stdout:2/888: getdents d1/de/d9a 0 2024-02-22T15:50:45.857 INFO:tasks.workunit.client.0.smithi019.stdout:9/834: read d2/d7/dc/d27/dbc/f3f [3904209,117687] 0 2024-02-22T15:50:45.858 INFO:tasks.workunit.client.1.smithi062.stdout:2/741: read d1/d8/df/f31 [2666206,7865] 0 2024-02-22T15:50:45.858 INFO:tasks.workunit.client.0.smithi019.stdout:8/780: dwrite d9/d14/d1f/d5c/fa2 [0,4194304] 0 2024-02-22T15:50:45.858 INFO:tasks.workunit.client.0.smithi019.stdout:4/801: truncate d2/db/d14/d23/d21/d35/f38 3659730 0 2024-02-22T15:50:45.859 INFO:tasks.workunit.client.0.smithi019.stdout:0/831: mknod d4/db/d49/dc2/c11d 0 2024-02-22T15:50:45.860 INFO:tasks.workunit.client.1.smithi062.stdout:3/573: chown d5/d1a/d3c/f54 73 1 2024-02-22T15:50:45.862 INFO:tasks.workunit.client.1.smithi062.stdout:1/603: stat d1/d6/d36/d37/f86 0 2024-02-22T15:50:45.863 INFO:tasks.workunit.client.1.smithi062.stdout:5/538: creat d0/d3/da/d13/d57/d79/fa5 x:0 0 0 2024-02-22T15:50:45.863 INFO:tasks.workunit.client.1.smithi062.stdout:8/785: dwrite d8/f20 [0,4194304] 0 2024-02-22T15:50:45.865 INFO:tasks.workunit.client.1.smithi062.stdout:6/698: dread - d5/df/d40/d44/d76/d56/f65 zero size 2024-02-22T15:50:45.866 INFO:tasks.workunit.client.0.smithi019.stdout:4/802: creat d2/db/d14/d23/d21/d42/dc4/dd5/f105 x:0 0 0 2024-02-22T15:50:45.867 INFO:tasks.workunit.client.1.smithi062.stdout:4/700: write d4/d1c/f3d [2827417,26746] 0 2024-02-22T15:50:45.869 INFO:tasks.workunit.client.1.smithi062.stdout:9/644: symlink d4/d15/d5e/d63/da0/d92/lcc 0 2024-02-22T15:50:45.869 INFO:tasks.workunit.client.0.smithi019.stdout:9/835: rename d2/cd5 to d2/d7/dee/c126 0 2024-02-22T15:50:45.869 INFO:tasks.workunit.client.1.smithi062.stdout:1/604: rmdir d1/d6/d7/d2b/d47 39 2024-02-22T15:50:45.870 INFO:tasks.workunit.client.1.smithi062.stdout:5/539: write d0/d3/d2f/f3a [2201612,64173] 0 2024-02-22T15:50:45.871 INFO:tasks.workunit.client.0.smithi019.stdout:0/832: rmdir d4/da5 0 2024-02-22T15:50:45.871 INFO:tasks.workunit.client.0.smithi019.stdout:2/889: dwrite d1/de/d15/f8c [4194304,4194304] 0 2024-02-22T15:50:45.871 INFO:tasks.workunit.client.1.smithi062.stdout:1/605: stat d1/f27 0 2024-02-22T15:50:45.873 INFO:tasks.workunit.client.1.smithi062.stdout:0/741: dread - d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/fab zero size 2024-02-22T15:50:45.874 INFO:tasks.workunit.client.1.smithi062.stdout:9/645: chown d4/d28/d50/da3/fb7 1007232626 1 2024-02-22T15:50:45.874 INFO:tasks.workunit.client.1.smithi062.stdout:8/786: truncate d8/d9/d44/d4d/d51/f95 4338807 0 2024-02-22T15:50:45.878 INFO:tasks.workunit.client.1.smithi062.stdout:9/646: stat d4/d28/d50/f7a 0 2024-02-22T15:50:45.878 INFO:tasks.workunit.client.1.smithi062.stdout:8/787: dread - d8/d26/f9e zero size 2024-02-22T15:50:45.880 INFO:tasks.workunit.client.0.smithi019.stdout:3/686: write d9/f2f [524957,61022] 0 2024-02-22T15:50:45.880 INFO:tasks.workunit.client.1.smithi062.stdout:1/606: symlink d1/d6/de/lcd 0 2024-02-22T15:50:45.882 INFO:tasks.workunit.client.0.smithi019.stdout:3/687: write f6 [2716308,37699] 0 2024-02-22T15:50:45.885 INFO:tasks.workunit.client.0.smithi019.stdout:9/836: dwrite d2/d7/dc/d27/d68/f73 [0,4194304] 0 2024-02-22T15:50:45.885 INFO:tasks.workunit.client.0.smithi019.stdout:0/833: creat d4/db/d76/dae/f11e x:0 0 0 2024-02-22T15:50:45.886 INFO:tasks.workunit.client.0.smithi019.stdout:2/890: rename d1/d17/d1c/d128/l102 to d1/db4/l145 0 2024-02-22T15:50:45.887 INFO:tasks.workunit.client.1.smithi062.stdout:6/699: read d5/f6c [1108174,65961] 0 2024-02-22T15:50:45.888 INFO:tasks.workunit.client.1.smithi062.stdout:2/742: dread d1/d8/f6e [0,4194304] 0 2024-02-22T15:50:45.889 INFO:tasks.workunit.client.1.smithi062.stdout:4/701: dwrite d4/d1c/dac/fce [0,4194304] 0 2024-02-22T15:50:45.889 INFO:tasks.workunit.client.1.smithi062.stdout:9/647: write d4/d15/d5f/d75/d84/f59 [1565874,49266] 0 2024-02-22T15:50:45.890 INFO:tasks.workunit.client.0.smithi019.stdout:3/688: rename d9/d11/c62 to d9/dc8/cdb 0 2024-02-22T15:50:45.894 INFO:tasks.workunit.client.1.smithi062.stdout:1/607: mkdir d1/d6/d14/dce 0 2024-02-22T15:50:45.899 INFO:tasks.workunit.client.1.smithi062.stdout:5/540: dread d0/d3/da/d32/d33/f88 [0,4194304] 0 2024-02-22T15:50:45.899 INFO:tasks.workunit.client.1.smithi062.stdout:6/700: truncate d5/df/d40/fb2 414689 0 2024-02-22T15:50:45.900 INFO:tasks.workunit.client.0.smithi019.stdout:9/837: rmdir d2/d7/dc/dd9 0 2024-02-22T15:50:45.901 INFO:tasks.workunit.client.1.smithi062.stdout:9/648: write d4/d15/d5f/d75/d97/da7/fad [756766,104469] 0 2024-02-22T15:50:45.901 INFO:tasks.workunit.client.1.smithi062.stdout:2/743: fdatasync d1/d47/d39/fcd 0 2024-02-22T15:50:45.904 INFO:tasks.workunit.client.1.smithi062.stdout:0/742: dwrite d1/d38/f7b [0,4194304] 0 2024-02-22T15:50:45.906 INFO:tasks.workunit.client.1.smithi062.stdout:4/702: dread d4/d66/f6f [0,4194304] 0 2024-02-22T15:50:45.907 INFO:tasks.workunit.client.0.smithi019.stdout:9/838: dwrite d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:45.907 INFO:tasks.workunit.client.0.smithi019.stdout:0/834: getdents d4/db/d49/dc2 0 2024-02-22T15:50:45.908 INFO:tasks.workunit.client.1.smithi062.stdout:0/743: rmdir d1/d3 39 2024-02-22T15:50:45.910 INFO:tasks.workunit.client.0.smithi019.stdout:8/781: truncate d9/d14/d17/fbe 278252 0 2024-02-22T15:50:45.910 INFO:tasks.workunit.client.1.smithi062.stdout:6/701: mkdir d5/df/d40/dd2/de4 0 2024-02-22T15:50:45.913 INFO:tasks.workunit.client.0.smithi019.stdout:9/839: fdatasync d2/d1f/d2e/d57/d60/d8c/d66/fba 0 2024-02-22T15:50:45.913 INFO:tasks.workunit.client.1.smithi062.stdout:1/608: rename d1/d6/d36/d37/l39 to d1/d6/d7/d2b/lcf 0 2024-02-22T15:50:45.914 INFO:tasks.workunit.client.1.smithi062.stdout:3/574: truncate d5/d16/d21/f22 38474 0 2024-02-22T15:50:45.916 INFO:tasks.workunit.client.1.smithi062.stdout:4/703: rename d4 to d4/d1c/d30/da7/dcb/de2/df5 22 2024-02-22T15:50:45.916 INFO:tasks.workunit.client.1.smithi062.stdout:2/744: creat d1/d1e/d38/d76/dc5/dec/ff9 x:0 0 0 2024-02-22T15:50:45.918 INFO:tasks.workunit.client.0.smithi019.stdout:8/782: truncate d9/d14/d6f/d7e/d94/d9a/f105 730347 0 2024-02-22T15:50:45.920 INFO:tasks.workunit.client.1.smithi062.stdout:1/609: rmdir d1/d6/d7/d2b/d49 39 2024-02-22T15:50:45.922 INFO:tasks.workunit.client.1.smithi062.stdout:2/745: mknod d1/d47/d39/d5d/cfa 0 2024-02-22T15:50:45.923 INFO:tasks.workunit.client.0.smithi019.stdout:5/852: sync 2024-02-22T15:50:45.923 INFO:tasks.workunit.client.0.smithi019.stdout:9/840: creat d2/d1f/d2e/d57/d60/d8c/d66/f127 x:0 0 0 2024-02-22T15:50:45.925 INFO:tasks.workunit.client.1.smithi062.stdout:9/649: getdents d4/d15/d5f/d6a 0 2024-02-22T15:50:45.925 INFO:tasks.workunit.client.1.smithi062.stdout:8/788: write d8/d9/f18 [3847308,12199] 0 2024-02-22T15:50:45.926 INFO:tasks.workunit.client.1.smithi062.stdout:9/650: readlink d4/d15/d31/l61 0 2024-02-22T15:50:45.926 INFO:tasks.workunit.client.0.smithi019.stdout:5/853: rmdir d8/dd/d19/d38/d47/ddb/dde/df6 39 2024-02-22T15:50:45.928 INFO:tasks.workunit.client.0.smithi019.stdout:2/891: dwrite d1/d17/fb2 [0,4194304] 0 2024-02-22T15:50:45.930 INFO:tasks.workunit.client.1.smithi062.stdout:6/702: getdents d5/df/d40/d44/d4a/d41/d43/dc6 0 2024-02-22T15:50:45.931 INFO:tasks.workunit.client.1.smithi062.stdout:8/789: creat d8/d9/d2c/d49/d94/dc1/ff6 x:0 0 0 2024-02-22T15:50:45.931 INFO:tasks.workunit.client.1.smithi062.stdout:9/651: symlink d4/d15/d5e/d63/da0/d92/lcd 0 2024-02-22T15:50:45.932 INFO:tasks.workunit.client.1.smithi062.stdout:8/790: chown d8/d42/l5d 15534632 1 2024-02-22T15:50:45.933 INFO:tasks.workunit.client.0.smithi019.stdout:2/892: mknod d1/d17/d38/c146 0 2024-02-22T15:50:45.936 INFO:tasks.workunit.client.1.smithi062.stdout:2/746: creat d1/d8/d62/d73/dbd/d7d/ffb x:0 0 0 2024-02-22T15:50:45.936 INFO:tasks.workunit.client.1.smithi062.stdout:6/703: creat d5/df/dbc/fe5 x:0 0 0 2024-02-22T15:50:45.937 INFO:tasks.workunit.client.0.smithi019.stdout:2/893: chown d1/d17/d67/c8d 110419635 1 2024-02-22T15:50:45.938 INFO:tasks.workunit.client.0.smithi019.stdout:4/803: dread d2/db/d14/d23/d21/d35/f38 [0,4194304] 0 2024-02-22T15:50:45.940 INFO:tasks.workunit.client.0.smithi019.stdout:2/894: mkdir d1/def/d147 0 2024-02-22T15:50:45.940 INFO:tasks.workunit.client.1.smithi062.stdout:3/575: link d5/f8 d5/fbd 0 2024-02-22T15:50:45.941 INFO:tasks.workunit.client.1.smithi062.stdout:8/791: creat d8/d9/d2c/dd5/ff7 x:0 0 0 2024-02-22T15:50:45.945 INFO:tasks.workunit.client.1.smithi062.stdout:6/704: stat d5/df/d40/d44/d4a/d41/d43/dc6/lc9 0 2024-02-22T15:50:45.945 INFO:tasks.workunit.client.0.smithi019.stdout:2/895: write d1/d17/d67/dfa/d11e/d126/f138 [8962434,97307] 0 2024-02-22T15:50:45.946 INFO:tasks.workunit.client.0.smithi019.stdout:0/835: dwrite d4/f3e [0,4194304] 0 2024-02-22T15:50:45.949 INFO:tasks.workunit.client.1.smithi062.stdout:6/705: fdatasync d5/df/d40/d44/d76/d56/f61 0 2024-02-22T15:50:45.951 INFO:tasks.workunit.client.0.smithi019.stdout:9/841: write d2/d7/dc/d27/d87/fb1 [466554,35206] 0 2024-02-22T15:50:45.951 INFO:tasks.workunit.client.0.smithi019.stdout:5/854: truncate d8/dd/d19/d38/d54/fb2 1273310 0 2024-02-22T15:50:45.952 INFO:tasks.workunit.client.0.smithi019.stdout:8/783: dwrite d9/d14/d17/d1c/da0/fcc [0,4194304] 0 2024-02-22T15:50:45.952 INFO:tasks.workunit.client.1.smithi062.stdout:9/652: rename d4/d15/d5e/d63/f64 to d4/d15/d5f/d75/d97/fce 0 2024-02-22T15:50:45.957 INFO:tasks.workunit.client.0.smithi019.stdout:2/896: write d1/de/d9a/f113 [1979594,115215] 0 2024-02-22T15:50:45.959 INFO:tasks.workunit.client.1.smithi062.stdout:8/792: dwrite d8/f20 [0,4194304] 0 2024-02-22T15:50:45.960 INFO:tasks.workunit.client.0.smithi019.stdout:0/836: creat d4/db/d1b/d63/db0/da1/dbb/d7/f11f x:0 0 0 2024-02-22T15:50:45.961 INFO:tasks.workunit.client.1.smithi062.stdout:6/706: mkdir d5/d8a/ddd/de6 0 2024-02-22T15:50:45.963 INFO:tasks.workunit.client.0.smithi019.stdout:4/804: dwrite d2/db/d14/d23/d21/dbf/dd7/ff6 [0,4194304] 0 2024-02-22T15:50:45.967 INFO:tasks.workunit.client.1.smithi062.stdout:9/653: dread d4/d15/d5f/d75/d84/f59 [0,4194304] 0 2024-02-22T15:50:45.967 INFO:tasks.workunit.client.1.smithi062.stdout:8/793: write d8/d26/f34 [4311324,63028] 0 2024-02-22T15:50:45.968 INFO:tasks.workunit.client.1.smithi062.stdout:8/794: chown d8/d9/d44/d4d/d51 1 1 2024-02-22T15:50:45.968 INFO:tasks.workunit.client.1.smithi062.stdout:3/576: getdents d5/d16/d21/d36 0 2024-02-22T15:50:45.968 INFO:tasks.workunit.client.1.smithi062.stdout:6/707: creat d5/df/d40/dac/de1/fe7 x:0 0 0 2024-02-22T15:50:45.969 INFO:tasks.workunit.client.1.smithi062.stdout:9/654: mkdir d4/d28/d50/dcf 0 2024-02-22T15:50:45.969 INFO:tasks.workunit.client.0.smithi019.stdout:8/784: rename d9/d14/d1f/d6d/fc5 to d9/d14/d1f/d5c/f108 0 2024-02-22T15:50:45.971 INFO:tasks.workunit.client.1.smithi062.stdout:1/610: dread d1/d6/de/f72 [0,4194304] 0 2024-02-22T15:50:45.972 INFO:tasks.workunit.client.1.smithi062.stdout:0/744: dwrite d1/d3/de/f3a [0,4194304] 0 2024-02-22T15:50:45.973 INFO:tasks.workunit.client.0.smithi019.stdout:0/837: mknod d4/db/d49/dc2/c120 0 2024-02-22T15:50:45.973 INFO:tasks.workunit.client.1.smithi062.stdout:8/795: rmdir d8/d42/d87 39 2024-02-22T15:50:45.973 INFO:tasks.workunit.client.1.smithi062.stdout:8/796: stat d8/d28/f4c 0 2024-02-22T15:50:45.974 INFO:tasks.workunit.client.0.smithi019.stdout:4/805: mknod d2/db/d14/d23/d21/d35/d85/c106 0 2024-02-22T15:50:45.979 INFO:tasks.workunit.client.0.smithi019.stdout:0/838: write d4/db/d1b/d40/f5d [2890476,46221] 0 2024-02-22T15:50:45.980 INFO:tasks.workunit.client.0.smithi019.stdout:8/785: creat d9/d14/d6f/d7e/d94/daa/dea/f109 x:0 0 0 2024-02-22T15:50:45.981 INFO:tasks.workunit.client.0.smithi019.stdout:4/806: write d2/f32 [725330,24017] 0 2024-02-22T15:50:45.984 INFO:tasks.workunit.client.0.smithi019.stdout:9/842: dwrite d2/f4 [16777216,4194304] 0 2024-02-22T15:50:45.986 INFO:tasks.workunit.client.0.smithi019.stdout:2/897: dwrite d1/d17/fae [0,4194304] 0 2024-02-22T15:50:45.991 INFO:tasks.workunit.client.0.smithi019.stdout:8/786: creat d9/d14/d6f/d7e/d94/daa/dea/f10a x:0 0 0 2024-02-22T15:50:45.991 INFO:tasks.workunit.client.0.smithi019.stdout:2/898: write d1/d17/d21/d71/d60/d11b/f9b [3421860,17273] 0 2024-02-22T15:50:45.991 INFO:tasks.workunit.client.0.smithi019.stdout:0/839: creat d4/db/d1b/d40/d44/d75/df7/f121 x:0 0 0 2024-02-22T15:50:45.994 INFO:tasks.workunit.client.0.smithi019.stdout:9/843: getdents d2/d1f/d2e/d48/d89 0 2024-02-22T15:50:45.996 INFO:tasks.workunit.client.0.smithi019.stdout:9/844: write d2/d1f/d2e/d48/d36/ffb [836186,31886] 0 2024-02-22T15:50:45.997 INFO:tasks.workunit.client.0.smithi019.stdout:0/840: rmdir d4/db/d49/dc2/dc1 39 2024-02-22T15:50:45.997 INFO:tasks.workunit.client.0.smithi019.stdout:8/787: creat d9/d14/d17/d86/f10b x:0 0 0 2024-02-22T15:50:45.998 INFO:tasks.workunit.client.0.smithi019.stdout:2/899: dread d1/d17/d38/fce [0,4194304] 0 2024-02-22T15:50:45.998 INFO:tasks.workunit.client.0.smithi019.stdout:9/845: chown d2/d1f/d2e/d57/d60/d8c/l98 386714233 1 2024-02-22T15:50:45.999 INFO:tasks.workunit.client.0.smithi019.stdout:9/846: chown d2/d7/d52 15868439 1 2024-02-22T15:50:46.000 INFO:tasks.workunit.client.0.smithi019.stdout:0/841: mkdir d4/db/d24/d80/d122 0 2024-02-22T15:50:46.001 INFO:tasks.workunit.client.0.smithi019.stdout:2/900: mknod d1/d17/d1c/d128/c148 0 2024-02-22T15:50:46.002 INFO:tasks.workunit.client.0.smithi019.stdout:0/842: write d4/d22/d2c/fdd [941937,41895] 0 2024-02-22T15:50:46.003 INFO:tasks.workunit.client.0.smithi019.stdout:9/847: mknod d2/d7/dc/d27/d68/dca/c128 0 2024-02-22T15:50:46.005 INFO:tasks.workunit.client.0.smithi019.stdout:0/843: dread - d4/d23/d7d/f85 zero size 2024-02-22T15:50:46.008 INFO:tasks.workunit.client.0.smithi019.stdout:8/788: dwrite d9/d14/d1f/d5c/fc8 [0,4194304] 0 2024-02-22T15:50:46.010 INFO:tasks.workunit.client.0.smithi019.stdout:0/844: stat d4/db/d1b/d63/db0/da1/dbb/d7/cc0 0 2024-02-22T15:50:46.011 INFO:tasks.workunit.client.1.smithi062.stdout:3/577: symlink d5/d16/d21/d32/lbe 0 2024-02-22T15:50:46.012 INFO:tasks.workunit.client.0.smithi019.stdout:0/845: write d4/db/d1b/d40/f89 [13473,99632] 0 2024-02-22T15:50:46.014 INFO:tasks.workunit.client.0.smithi019.stdout:0/846: fsync d4/d22/f36 0 2024-02-22T15:50:46.015 INFO:tasks.workunit.client.1.smithi062.stdout:8/797: write d8/d86/fa6 [604577,52359] 0 2024-02-22T15:50:46.016 INFO:tasks.workunit.client.1.smithi062.stdout:1/611: truncate d1/f27 748248 0 2024-02-22T15:50:46.016 INFO:tasks.workunit.client.0.smithi019.stdout:8/789: stat d9/l11 0 2024-02-22T15:50:46.018 INFO:tasks.workunit.client.1.smithi062.stdout:9/655: dread d4/d15/d5f/d75/d84/f5b [0,4194304] 0 2024-02-22T15:50:46.019 INFO:tasks.workunit.client.0.smithi019.stdout:5/855: write d8/dd/d19/d3a/fab [3797706,54432] 0 2024-02-22T15:50:46.021 INFO:tasks.workunit.client.0.smithi019.stdout:8/790: creat d9/d14/d6f/d7e/d94/db8/f10c x:0 0 0 2024-02-22T15:50:46.022 INFO:tasks.workunit.client.0.smithi019.stdout:5/856: chown d8/dd/d19/d38/d54/f11a 6534 1 2024-02-22T15:50:46.022 INFO:tasks.workunit.client.0.smithi019.stdout:8/791: readlink d9/d14/d17/d3d/d3f/lf3 0 2024-02-22T15:50:46.023 INFO:tasks.workunit.client.1.smithi062.stdout:1/612: chown d1/f27 3 1 2024-02-22T15:50:46.024 INFO:tasks.workunit.client.0.smithi019.stdout:5/857: write d8/dd/d19/d3a/dc7/f117 [318909,52591] 0 2024-02-22T15:50:46.026 INFO:tasks.workunit.client.0.smithi019.stdout:8/792: chown d9/d14/d1f/d6d/c74 56794 1 2024-02-22T15:50:46.026 INFO:tasks.workunit.client.1.smithi062.stdout:8/798: mkdir d8/d9/d44/df8 0 2024-02-22T15:50:46.027 INFO:tasks.workunit.client.1.smithi062.stdout:1/613: rmdir d1/d6 39 2024-02-22T15:50:46.030 INFO:tasks.workunit.client.1.smithi062.stdout:3/578: dwrite d5/d26/d2d/fa1 [0,4194304] 0 2024-02-22T15:50:46.030 INFO:tasks.workunit.client.1.smithi062.stdout:6/708: write d5/df/d32/f35 [346052,2083] 0 2024-02-22T15:50:46.032 INFO:tasks.workunit.client.1.smithi062.stdout:8/799: read d8/f3a [3224579,95155] 0 2024-02-22T15:50:46.035 INFO:tasks.workunit.client.1.smithi062.stdout:9/656: rename d4/d1c/d83/d87/la6 to d4/d15/ld0 0 2024-02-22T15:50:46.036 INFO:tasks.workunit.client.1.smithi062.stdout:2/747: dread d1/d1e/d38/d76/f99 [0,4194304] 0 2024-02-22T15:50:46.038 INFO:tasks.workunit.client.1.smithi062.stdout:0/745: dwrite d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fa4 [0,4194304] 0 2024-02-22T15:50:46.040 INFO:tasks.workunit.client.1.smithi062.stdout:9/657: stat d4/f26 0 2024-02-22T15:50:46.043 INFO:tasks.workunit.client.1.smithi062.stdout:1/614: creat d1/d6/d14/dce/fd0 x:0 0 0 2024-02-22T15:50:46.044 INFO:tasks.workunit.client.1.smithi062.stdout:2/748: fdatasync d1/d8/df/f8b 0 2024-02-22T15:50:46.047 INFO:tasks.workunit.client.0.smithi019.stdout:9/848: dread d2/d1f/d2e/d48/d45/d5a/fdd [0,4194304] 0 2024-02-22T15:50:46.050 INFO:tasks.workunit.client.1.smithi062.stdout:2/749: fsync d1/d1e/d38/f54 0 2024-02-22T15:50:46.050 INFO:tasks.workunit.client.1.smithi062.stdout:8/800: mknod d8/d9/d44/d4d/d51/cf9 0 2024-02-22T15:50:46.050 INFO:tasks.workunit.client.0.smithi019.stdout:9/849: chown d2/d7/dc/d27/d68/dca/le8 14662023 1 2024-02-22T15:50:46.051 INFO:tasks.workunit.client.1.smithi062.stdout:9/658: truncate d4/d15/f24 717254 0 2024-02-22T15:50:46.053 INFO:tasks.workunit.client.1.smithi062.stdout:9/659: chown d4/d15/d5f/d75/d97/da7/fad 1720944 1 2024-02-22T15:50:46.053 INFO:tasks.workunit.client.0.smithi019.stdout:5/858: write d8/dd/d19/d38/d47/f68 [1005760,99964] 0 2024-02-22T15:50:46.055 INFO:tasks.workunit.client.1.smithi062.stdout:8/801: fdatasync d8/d9/fe 0 2024-02-22T15:50:46.055 INFO:tasks.workunit.client.0.smithi019.stdout:8/793: truncate d9/d14/d17/d3d/d83/fa6 1988502 0 2024-02-22T15:50:46.055 INFO:tasks.workunit.client.1.smithi062.stdout:9/660: truncate d4/d28/d50/f7a 720916 0 2024-02-22T15:50:46.055 INFO:tasks.workunit.client.0.smithi019.stdout:9/850: mkdir d2/d1f/d2e/d48/d45/d5a/d129 0 2024-02-22T15:50:46.057 INFO:tasks.workunit.client.0.smithi019.stdout:5/859: dread - d8/db1/fc6 zero size 2024-02-22T15:50:46.057 INFO:tasks.workunit.client.1.smithi062.stdout:0/746: getdents d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e 0 2024-02-22T15:50:46.057 INFO:tasks.workunit.client.1.smithi062.stdout:9/661: chown d4/d1c/c66 10 1 2024-02-22T15:50:46.060 INFO:tasks.workunit.client.0.smithi019.stdout:8/794: rmdir d9/d14/d6f/d7e/d94/daa 39 2024-02-22T15:50:46.060 INFO:tasks.workunit.client.0.smithi019.stdout:2/901: dwrite d1/d17/d1c/d128/db9/dd0/f11c [0,4194304] 0 2024-02-22T15:50:46.061 INFO:tasks.workunit.client.1.smithi062.stdout:6/709: write d5/df/d32/d8f/fa1 [761009,54746] 0 2024-02-22T15:50:46.062 INFO:tasks.workunit.client.1.smithi062.stdout:3/579: truncate d5/d1a/d3c/f54 514923 0 2024-02-22T15:50:46.062 INFO:tasks.workunit.client.0.smithi019.stdout:5/860: fsync d8/dd/d19/d38/d54/f95 0 2024-02-22T15:50:46.064 INFO:tasks.workunit.client.1.smithi062.stdout:3/580: readlink d5/d26/l69 0 2024-02-22T15:50:46.064 INFO:tasks.workunit.client.1.smithi062.stdout:9/662: rename d4/d28/f42 to d4/d1c/fd1 0 2024-02-22T15:50:46.065 INFO:tasks.workunit.client.0.smithi019.stdout:8/795: dread - d9/d14/d17/d3d/d3f/dd4/f101 zero size 2024-02-22T15:50:46.065 INFO:tasks.workunit.client.0.smithi019.stdout:2/902: mknod d1/d86/c149 0 2024-02-22T15:50:46.069 INFO:tasks.workunit.client.0.smithi019.stdout:2/903: chown d1/d17/d1c/d128/db9/dd0 1830836 1 2024-02-22T15:50:46.071 INFO:tasks.workunit.client.0.smithi019.stdout:9/851: dwrite d2/d1f/d2e/d48/d45/fd3 [4194304,4194304] 0 2024-02-22T15:50:46.072 INFO:tasks.workunit.client.1.smithi062.stdout:8/802: dwrite d8/d9/f37 [0,4194304] 0 2024-02-22T15:50:46.073 INFO:tasks.workunit.client.1.smithi062.stdout:0/747: getdents d1/d13/dde 0 2024-02-22T15:50:46.073 INFO:tasks.workunit.client.0.smithi019.stdout:5/861: unlink d8/dd/d19/d3a/dce/l106 0 2024-02-22T15:50:46.075 INFO:tasks.workunit.client.0.smithi019.stdout:8/796: unlink d9/d14/d17/d3d/ld1 0 2024-02-22T15:50:46.076 INFO:tasks.workunit.client.1.smithi062.stdout:9/663: truncate d4/d15/d5f/d75/fb8 283722 0 2024-02-22T15:50:46.077 INFO:tasks.workunit.client.1.smithi062.stdout:3/581: dread d5/d26/d2d/f90 [0,4194304] 0 2024-02-22T15:50:46.078 INFO:tasks.workunit.client.0.smithi019.stdout:5/862: truncate d8/d11/fdc 291273 0 2024-02-22T15:50:46.078 INFO:tasks.workunit.client.1.smithi062.stdout:6/710: rename d5/df/d40/dac/de1/fe7 to d5/dd/d33/d66/db9/fe8 0 2024-02-22T15:50:46.079 INFO:tasks.workunit.client.0.smithi019.stdout:9/852: write d2/dec/d79/fb2 [5357546,47887] 0 2024-02-22T15:50:46.084 INFO:tasks.workunit.client.0.smithi019.stdout:5/863: chown d8/dd/d19/d3a/c92 1 1 2024-02-22T15:50:46.084 INFO:tasks.workunit.client.1.smithi062.stdout:8/803: creat d8/d86/ffa x:0 0 0 2024-02-22T15:50:46.085 INFO:tasks.workunit.client.0.smithi019.stdout:2/904: dwrite d1/de/f5a [0,4194304] 0 2024-02-22T15:50:46.086 INFO:tasks.workunit.client.0.smithi019.stdout:9/853: mknod d2/d7/dc/dd6/c12a 0 2024-02-22T15:50:46.086 INFO:tasks.workunit.client.1.smithi062.stdout:0/748: stat d1/d13/dde/d1d/l9b 0 2024-02-22T15:50:46.088 INFO:tasks.workunit.client.1.smithi062.stdout:2/750: dwrite d1/d1e/d38/d55/f97 [0,4194304] 0 2024-02-22T15:50:46.088 INFO:tasks.workunit.client.0.smithi019.stdout:8/797: creat d9/d14/d1f/d6d/d103/f10d x:0 0 0 2024-02-22T15:50:46.089 INFO:tasks.workunit.client.1.smithi062.stdout:9/664: creat d4/d28/d50/da3/fd2 x:0 0 0 2024-02-22T15:50:46.089 INFO:tasks.workunit.client.1.smithi062.stdout:6/711: mkdir d5/df/dbc/dcd/de9 0 2024-02-22T15:50:46.089 INFO:tasks.workunit.client.0.smithi019.stdout:9/854: creat d2/d7/dee/f12b x:0 0 0 2024-02-22T15:50:46.090 INFO:tasks.workunit.client.1.smithi062.stdout:1/615: dwrite d1/d6/d7/d2b/d49/f51 [0,4194304] 0 2024-02-22T15:50:46.091 INFO:tasks.workunit.client.1.smithi062.stdout:3/582: truncate d5/d26/d2d/d3f/fbb 735754 0 2024-02-22T15:50:46.092 INFO:tasks.workunit.client.1.smithi062.stdout:8/804: chown d8/d9/d2c/d49/d94/da1/fbc 387 1 2024-02-22T15:50:46.092 INFO:tasks.workunit.client.0.smithi019.stdout:2/905: chown d1/d17/d1c/d128/db9/dd0/dee/d120 1932 1 2024-02-22T15:50:46.093 INFO:tasks.workunit.client.1.smithi062.stdout:9/665: symlink d4/d1c/d83/ld3 0 2024-02-22T15:50:46.094 INFO:tasks.workunit.client.0.smithi019.stdout:8/798: mknod d9/d14/d6f/d7e/d94/d9a/deb/c10e 0 2024-02-22T15:50:46.094 INFO:tasks.workunit.client.0.smithi019.stdout:9/855: fdatasync d2/d1f/d2e/d48/d36/f115 0 2024-02-22T15:50:46.096 INFO:tasks.workunit.client.1.smithi062.stdout:8/805: chown d8/d9/d2c/d49/d5e/c60 499 1 2024-02-22T15:50:46.097 INFO:tasks.workunit.client.1.smithi062.stdout:2/751: mkdir d1/d8/d62/d73/dbd/d7d/d84/dfc 0 2024-02-22T15:50:46.097 INFO:tasks.workunit.client.1.smithi062.stdout:6/712: truncate d5/df/d40/d44/d4a/f1c 1249767 0 2024-02-22T15:50:46.099 INFO:tasks.workunit.client.1.smithi062.stdout:0/749: getdents d1/d5e 0 2024-02-22T15:50:46.099 INFO:tasks.workunit.client.1.smithi062.stdout:9/666: write d4/d1c/fc1 [615767,16199] 0 2024-02-22T15:50:46.100 INFO:tasks.workunit.client.1.smithi062.stdout:1/616: symlink d1/d6/d7/d2b/d49/da9/ld1 0 2024-02-22T15:50:46.103 INFO:tasks.workunit.client.1.smithi062.stdout:8/806: mknod d8/d28/d70/cfb 0 2024-02-22T15:50:46.103 INFO:tasks.workunit.client.1.smithi062.stdout:3/583: symlink d5/d16/d56/dbc/lbf 0 2024-02-22T15:50:46.104 INFO:tasks.workunit.client.0.smithi019.stdout:5/864: dwrite d8/d11/fed [0,4194304] 0 2024-02-22T15:50:46.105 INFO:tasks.workunit.client.1.smithi062.stdout:6/713: mknod d5/d6e/cea 0 2024-02-22T15:50:46.105 INFO:tasks.workunit.client.0.smithi019.stdout:2/906: unlink d1/db4/l10f 0 2024-02-22T15:50:46.107 INFO:tasks.workunit.client.1.smithi062.stdout:9/667: dread - d4/d15/d31/f4e zero size 2024-02-22T15:50:46.107 INFO:tasks.workunit.client.1.smithi062.stdout:0/750: mknod d1/d13/dde/d76/da5/cf6 0 2024-02-22T15:50:46.109 INFO:tasks.workunit.client.1.smithi062.stdout:8/807: mkdir d8/d28/d70/dfc 0 2024-02-22T15:50:46.111 INFO:tasks.workunit.client.1.smithi062.stdout:1/617: symlink d1/d6/d7/d2b/d47/ld2 0 2024-02-22T15:50:46.111 INFO:tasks.workunit.client.0.smithi019.stdout:8/799: rmdir d9/d14/d6f/d7e/d94/d9a/dd3/d104 0 2024-02-22T15:50:46.112 INFO:tasks.workunit.client.0.smithi019.stdout:5/865: rename d8/dd/d19/d38/d47/ddb to d8/dd/d19/d38/d54/db0/d10d/d122 0 2024-02-22T15:50:46.112 INFO:tasks.workunit.client.1.smithi062.stdout:3/584: write d5/d1a/fa9 [949927,13328] 0 2024-02-22T15:50:46.113 INFO:tasks.workunit.client.0.smithi019.stdout:2/907: write d1/d17/d21/d71/d60/d11b/f7b [357892,120568] 0 2024-02-22T15:50:46.114 INFO:tasks.workunit.client.1.smithi062.stdout:2/752: dwrite d1/d1e/d38/f74 [0,4194304] 0 2024-02-22T15:50:46.114 INFO:tasks.workunit.client.0.smithi019.stdout:8/800: creat d9/d14/d6f/d7e/f10f x:0 0 0 2024-02-22T15:50:46.115 INFO:tasks.workunit.client.0.smithi019.stdout:5/866: truncate d8/dd/d19/d3a/ff7 662571 0 2024-02-22T15:50:46.116 INFO:tasks.workunit.client.1.smithi062.stdout:0/751: mknod d1/d3/d4/d1e/cf7 0 2024-02-22T15:50:46.116 INFO:tasks.workunit.client.0.smithi019.stdout:5/867: chown d8/dd/d19/d38/d54/db0/d10d/d122/dde 26 1 2024-02-22T15:50:46.117 INFO:tasks.workunit.client.0.smithi019.stdout:8/801: truncate d9/d14/d17/d3d/d3f/f9e 974488 0 2024-02-22T15:50:46.118 INFO:tasks.workunit.client.0.smithi019.stdout:5/868: write d8/d11/fb6 [4341306,123524] 0 2024-02-22T15:50:46.120 INFO:tasks.workunit.client.1.smithi062.stdout:3/585: chown d5/d16/d56/fb0 242214 1 2024-02-22T15:50:46.122 INFO:tasks.workunit.client.0.smithi019.stdout:5/869: mknod d8/dd/d19/d3a/dce/c123 0 2024-02-22T15:50:46.123 INFO:tasks.workunit.client.1.smithi062.stdout:3/586: chown d5/d26/d2d/d3f/f7c 162 1 2024-02-22T15:50:46.123 INFO:tasks.workunit.client.0.smithi019.stdout:8/802: truncate d9/d14/d1f/fa5 407131 0 2024-02-22T15:50:46.125 INFO:tasks.workunit.client.1.smithi062.stdout:1/618: mknod d1/d6/d36/d37/cd3 0 2024-02-22T15:50:46.125 INFO:tasks.workunit.client.1.smithi062.stdout:6/714: dwrite d5/df/d40/d44/d76/d56/fca [0,4194304] 0 2024-02-22T15:50:46.126 INFO:tasks.workunit.client.0.smithi019.stdout:5/870: creat d8/dd/f124 x:0 0 0 2024-02-22T15:50:46.127 INFO:tasks.workunit.client.0.smithi019.stdout:1/942: sync 2024-02-22T15:50:46.128 INFO:tasks.workunit.client.1.smithi062.stdout:2/753: write d1/d8/d62/d73/da6/fb1 [693235,104904] 0 2024-02-22T15:50:46.129 INFO:tasks.workunit.client.1.smithi062.stdout:6/715: dread - d5/dd/f94 zero size 2024-02-22T15:50:46.130 INFO:tasks.workunit.client.1.smithi062.stdout:9/668: dread d4/d15/d5f/d75/f65 [0,4194304] 0 2024-02-22T15:50:46.130 INFO:tasks.workunit.client.1.smithi062.stdout:0/752: link d1/d3/d5f/c52 d1/d3/d5f/cf8 0 2024-02-22T15:50:46.132 INFO:tasks.workunit.client.1.smithi062.stdout:1/619: write d1/d6/d7/d2b/d47/d7e/fc5 [263613,121210] 0 2024-02-22T15:50:46.132 INFO:tasks.workunit.client.1.smithi062.stdout:2/754: mknod d1/d8/d62/cfd 0 2024-02-22T15:50:46.134 INFO:tasks.workunit.client.0.smithi019.stdout:8/803: dwrite d9/f107 [0,4194304] 0 2024-02-22T15:50:46.136 INFO:tasks.workunit.client.1.smithi062.stdout:0/753: creat d1/d5e/ff9 x:0 0 0 2024-02-22T15:50:46.137 INFO:tasks.workunit.client.1.smithi062.stdout:9/669: mknod d4/d15/d5f/d75/d97/cd4 0 2024-02-22T15:50:46.139 INFO:tasks.workunit.client.1.smithi062.stdout:0/754: dread - d1/d13/dde/d76/d8c/d92/f96 zero size 2024-02-22T15:50:46.140 INFO:tasks.workunit.client.1.smithi062.stdout:1/620: symlink d1/d6/d7/d2b/d49/d74/ld4 0 2024-02-22T15:50:46.142 INFO:tasks.workunit.client.0.smithi019.stdout:5/871: getdents d8/dd/d19/d77 0 2024-02-22T15:50:46.144 INFO:tasks.workunit.client.1.smithi062.stdout:0/755: chown d1/d13/ce7 12351150 1 2024-02-22T15:50:46.144 INFO:tasks.workunit.client.0.smithi019.stdout:8/804: link d9/f107 d9/d14/d6f/d7e/f110 0 2024-02-22T15:50:46.146 INFO:tasks.workunit.client.0.smithi019.stdout:8/805: fsync d9/d14/d17/d1c/f9d 0 2024-02-22T15:50:46.146 INFO:tasks.workunit.client.1.smithi062.stdout:1/621: readlink d1/d6/d7/d2b/lcf 0 2024-02-22T15:50:46.147 INFO:tasks.workunit.client.1.smithi062.stdout:2/755: getdents d1/d8/d62/d73/dbd/dbb/dc4 0 2024-02-22T15:50:46.147 INFO:tasks.workunit.client.0.smithi019.stdout:2/908: dread d1/de/f4e [0,4194304] 0 2024-02-22T15:50:46.147 INFO:tasks.workunit.client.0.smithi019.stdout:8/806: stat d9/d14/d17/d3d/d3f/dd4/f101 0 2024-02-22T15:50:46.148 INFO:tasks.workunit.client.0.smithi019.stdout:2/909: stat d1/d17/d1c/d128/d13b 0 2024-02-22T15:50:46.149 INFO:tasks.workunit.client.1.smithi062.stdout:1/622: chown d1/lb3 2 1 2024-02-22T15:50:46.149 INFO:tasks.workunit.client.0.smithi019.stdout:8/807: write d9/fc3 [2331306,5726] 0 2024-02-22T15:50:46.149 INFO:tasks.workunit.client.0.smithi019.stdout:5/872: creat d8/dd/d19/d3a/d46/d7e/dbe/dc3/f125 x:0 0 0 2024-02-22T15:50:46.150 INFO:tasks.workunit.client.1.smithi062.stdout:9/670: dwrite d4/d15/f5c [0,4194304] 0 2024-02-22T15:50:46.150 INFO:tasks.workunit.client.1.smithi062.stdout:8/808: truncate d8/d9/f18 614368 0 2024-02-22T15:50:46.150 INFO:tasks.workunit.client.0.smithi019.stdout:5/873: readlink d8/dd/lae 0 2024-02-22T15:50:46.151 INFO:tasks.workunit.client.1.smithi062.stdout:3/587: write d5/d1a/d3b/f71 [611183,43554] 0 2024-02-22T15:50:46.152 INFO:tasks.workunit.client.1.smithi062.stdout:2/756: dread - d1/d8/d62/d73/dbd/d7d/d84/fcc zero size 2024-02-22T15:50:46.154 INFO:tasks.workunit.client.1.smithi062.stdout:1/623: read d1/d6/d7/f28 [1367617,32860] 0 2024-02-22T15:50:46.154 INFO:tasks.workunit.client.1.smithi062.stdout:3/588: fsync d5/d26/d2d/d3f/fa7 0 2024-02-22T15:50:46.155 INFO:tasks.workunit.client.1.smithi062.stdout:9/671: rmdir d4/d28 39 2024-02-22T15:50:46.156 INFO:tasks.workunit.client.1.smithi062.stdout:8/809: dread d8/d9/d44/d4d/f9f [0,4194304] 0 2024-02-22T15:50:46.157 INFO:tasks.workunit.client.1.smithi062.stdout:0/756: dread d1/d3/f39 [0,4194304] 0 2024-02-22T15:50:46.158 INFO:tasks.workunit.client.0.smithi019.stdout:2/910: mkdir d1/da4/d132/d14a 0 2024-02-22T15:50:46.158 INFO:tasks.workunit.client.0.smithi019.stdout:5/874: fsync d8/f28 0 2024-02-22T15:50:46.160 INFO:tasks.workunit.client.1.smithi062.stdout:3/589: unlink d5/d1a/f33 0 2024-02-22T15:50:46.161 INFO:tasks.workunit.client.0.smithi019.stdout:9/856: write d2/dec/f106 [1136846,23073] 0 2024-02-22T15:50:46.162 INFO:tasks.workunit.client.1.smithi062.stdout:2/757: dread d1/f82 [0,4194304] 0 2024-02-22T15:50:46.162 INFO:tasks.workunit.client.1.smithi062.stdout:8/810: creat d8/d86/db6/ffd x:0 0 0 2024-02-22T15:50:46.164 INFO:tasks.workunit.client.0.smithi019.stdout:9/857: creat d2/d1f/f12c x:0 0 0 2024-02-22T15:50:46.165 INFO:tasks.workunit.client.1.smithi062.stdout:1/624: getdents d1/d6/d7/d2b/d49 0 2024-02-22T15:50:46.166 INFO:tasks.workunit.client.0.smithi019.stdout:9/858: chown d2/d7/dc/f29 5 1 2024-02-22T15:50:46.166 INFO:tasks.workunit.client.0.smithi019.stdout:2/911: getdents d1/d46/d9d 0 2024-02-22T15:50:46.168 INFO:tasks.workunit.client.1.smithi062.stdout:3/590: dread d5/d16/d88/f91 [0,4194304] 0 2024-02-22T15:50:46.168 INFO:tasks.workunit.client.0.smithi019.stdout:9/859: rmdir d2/d7/d52/dc0/dcd 39 2024-02-22T15:50:46.169 INFO:tasks.workunit.client.1.smithi062.stdout:8/811: dread - d8/d9/d2c/d49/d94/dc1/f76 zero size 2024-02-22T15:50:46.169 INFO:tasks.workunit.client.0.smithi019.stdout:2/912: truncate d1/d46/d9d/fe2 284536 0 2024-02-22T15:50:46.170 INFO:tasks.workunit.client.1.smithi062.stdout:2/758: fdatasync d1/d8/fdd 0 2024-02-22T15:50:46.170 INFO:tasks.workunit.client.0.smithi019.stdout:9/860: fsync d2/d1f/f47 0 2024-02-22T15:50:46.171 INFO:tasks.workunit.client.1.smithi062.stdout:9/672: dwrite d4/d1c/f56 [0,4194304] 0 2024-02-22T15:50:46.172 INFO:tasks.workunit.client.0.smithi019.stdout:2/913: chown d1/d17/d21/d51/f77 91217 1 2024-02-22T15:50:46.172 INFO:tasks.workunit.client.1.smithi062.stdout:0/757: getdents d1/d13/dde/d76/d8c/d5d/dc1 0 2024-02-22T15:50:46.174 INFO:tasks.workunit.client.1.smithi062.stdout:8/812: chown d8/d9/f2a 1100 1 2024-02-22T15:50:46.174 INFO:tasks.workunit.client.0.smithi019.stdout:9/861: truncate d2/d1f/d2e/d57/fc7 1357486 0 2024-02-22T15:50:46.175 INFO:tasks.workunit.client.0.smithi019.stdout:1/943: dread d8/d2a/d75/d81/d8e/fec [0,4194304] 0 2024-02-22T15:50:46.175 INFO:tasks.workunit.client.1.smithi062.stdout:1/625: symlink d1/d6/d7/d2b/d47/d63/db2/ld5 0 2024-02-22T15:50:46.176 INFO:tasks.workunit.client.1.smithi062.stdout:2/759: write d1/fb8 [1832228,10009] 0 2024-02-22T15:50:46.178 INFO:tasks.workunit.client.1.smithi062.stdout:0/758: mkdir d1/d38/dfa 0 2024-02-22T15:50:46.180 INFO:tasks.workunit.client.1.smithi062.stdout:1/626: mknod d1/cd6 0 2024-02-22T15:50:46.182 INFO:tasks.workunit.client.0.smithi019.stdout:2/914: link d1/d17/d21/d71/d60/d11b/f4a d1/d17/d21/d71/d60/d11b/f14b 0 2024-02-22T15:50:46.182 INFO:tasks.workunit.client.1.smithi062.stdout:0/759: fsync d1/d3/d4/d1e/fd8 0 2024-02-22T15:50:46.182 INFO:tasks.workunit.client.1.smithi062.stdout:3/591: truncate d5/d16/d21/d36/d5a/d85/f9b 410442 0 2024-02-22T15:50:46.183 INFO:tasks.workunit.client.1.smithi062.stdout:6/716: dread d5/df/d40/d44/d4a/fad [0,4194304] 0 2024-02-22T15:50:46.184 INFO:tasks.workunit.client.0.smithi019.stdout:2/915: chown d1/d17/d1c/d128/db9/dd0/df4/f12d 13304 1 2024-02-22T15:50:46.184 INFO:tasks.workunit.client.1.smithi062.stdout:9/673: rename d4/d15/d5f/d75/d97/da7/cc9 to d4/d15/d5e/d63/da0/d92/cd5 0 2024-02-22T15:50:46.186 INFO:tasks.workunit.client.1.smithi062.stdout:3/592: stat d5/d16/d21/f4e 0 2024-02-22T15:50:46.187 INFO:tasks.workunit.client.1.smithi062.stdout:2/760: mkdir d1/d8/d62/d73/dbd/dbb/dc4/dfe 0 2024-02-22T15:50:46.187 INFO:tasks.workunit.client.0.smithi019.stdout:2/916: creat d1/d17/d1c/d7f/f14c x:0 0 0 2024-02-22T15:50:46.188 INFO:tasks.workunit.client.1.smithi062.stdout:6/717: write d5/f1a [546209,22782] 0 2024-02-22T15:50:46.189 INFO:tasks.workunit.client.1.smithi062.stdout:1/627: unlink d1/d6/d7/d2b/d47/f79 0 2024-02-22T15:50:46.190 INFO:tasks.workunit.client.1.smithi062.stdout:0/760: mknod d1/d5e/cfb 0 2024-02-22T15:50:46.192 INFO:tasks.workunit.client.1.smithi062.stdout:8/813: dwrite d8/d9/d44/d4d/f82 [0,4194304] 0 2024-02-22T15:50:46.194 INFO:tasks.workunit.client.0.smithi019.stdout:5/875: write d8/db1/fc6 [612716,45731] 0 2024-02-22T15:50:46.196 INFO:tasks.workunit.client.1.smithi062.stdout:3/593: stat d5/d16/d21/d36/d5a/c81 0 2024-02-22T15:50:46.197 INFO:tasks.workunit.client.1.smithi062.stdout:1/628: unlink d1/d6/d7/d8a/cb5 0 2024-02-22T15:50:46.198 INFO:tasks.workunit.client.1.smithi062.stdout:3/594: write d5/d16/d21/d36/d5a/d85/f93 [702767,81773] 0 2024-02-22T15:50:46.200 INFO:tasks.workunit.client.1.smithi062.stdout:2/761: symlink d1/de1/lff 0 2024-02-22T15:50:46.200 INFO:tasks.workunit.client.1.smithi062.stdout:3/595: fdatasync d5/d26/f61 0 2024-02-22T15:50:46.203 INFO:tasks.workunit.client.0.smithi019.stdout:9/862: truncate d2/d1f/d2e/d57/d60/f75 1461065 0 2024-02-22T15:50:46.203 INFO:tasks.workunit.client.0.smithi019.stdout:5/876: write d8/dd/d19/d3a/dc7/f117 [907827,50262] 0 2024-02-22T15:50:46.204 INFO:tasks.workunit.client.1.smithi062.stdout:0/761: rename d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/fbc to d1/d3/de/ffc 0 2024-02-22T15:50:46.206 INFO:tasks.workunit.client.0.smithi019.stdout:8/808: dwrite d9/d14/d1f/d5c/f108 [0,4194304] 0 2024-02-22T15:50:46.206 INFO:tasks.workunit.client.1.smithi062.stdout:3/596: symlink d5/d26/d2d/d3f/lc0 0 2024-02-22T15:50:46.207 INFO:tasks.workunit.client.0.smithi019.stdout:1/944: dread d8/d15/d34/fac [0,4194304] 0 2024-02-22T15:50:46.208 INFO:tasks.workunit.client.0.smithi019.stdout:9/863: write d2/d1f/d2e/d7d/f108 [1481756,117608] 0 2024-02-22T15:50:46.208 INFO:tasks.workunit.client.0.smithi019.stdout:2/917: dwrite d1/d17/d67/dfa/f12a [0,4194304] 0 2024-02-22T15:50:46.209 INFO:tasks.workunit.client.0.smithi019.stdout:8/809: chown d9/c44 2442 1 2024-02-22T15:50:46.211 INFO:tasks.workunit.client.1.smithi062.stdout:1/629: dwrite d1/d6/d7/d4a/d65/d83/faa [0,4194304] 0 2024-02-22T15:50:46.213 INFO:tasks.workunit.client.1.smithi062.stdout:2/762: symlink d1/d8/d62/d73/dbd/dbb/dc4/dfe/l100 0 2024-02-22T15:50:46.213 INFO:tasks.workunit.client.0.smithi019.stdout:8/810: chown d9/d14/d6f/d7e/d94/db8/lc0 4477 1 2024-02-22T15:50:46.218 INFO:tasks.workunit.client.0.smithi019.stdout:9/864: fdatasync d2/f30 0 2024-02-22T15:50:46.219 INFO:tasks.workunit.client.1.smithi062.stdout:1/630: rmdir d1/d6/d7/d2b/d49/da9 39 2024-02-22T15:50:46.224 INFO:tasks.workunit.client.1.smithi062.stdout:2/763: symlink d1/l101 0 2024-02-22T15:50:46.225 INFO:tasks.workunit.client.0.smithi019.stdout:8/811: read d9/ff [479490,91875] 0 2024-02-22T15:50:46.225 INFO:tasks.workunit.client.0.smithi019.stdout:9/865: write d2/dc8/fa2 [1200895,62128] 0 2024-02-22T15:50:46.226 INFO:tasks.workunit.client.1.smithi062.stdout:2/764: write d1/d8/f1a [536255,21918] 0 2024-02-22T15:50:46.226 INFO:tasks.workunit.client.0.smithi019.stdout:2/918: dwrite d1/f18 [0,4194304] 0 2024-02-22T15:50:46.227 INFO:tasks.workunit.client.0.smithi019.stdout:8/812: chown d9/d14/d17/d1c/da0/c96 7 1 2024-02-22T15:50:46.228 INFO:tasks.workunit.client.1.smithi062.stdout:9/674: truncate d4/d1c/f27 2095384 0 2024-02-22T15:50:46.234 INFO:tasks.workunit.client.0.smithi019.stdout:1/945: dread d8/d15/d5b/dbc/f109 [0,4194304] 0 2024-02-22T15:50:46.235 INFO:tasks.workunit.client.0.smithi019.stdout:2/919: dread d1/d17/d21/d71/d60/d11b/f131 [0,4194304] 0 2024-02-22T15:50:46.235 INFO:tasks.workunit.client.1.smithi062.stdout:0/762: write d1/d13/dde/d1d/fb3 [427067,9699] 0 2024-02-22T15:50:46.235 INFO:tasks.workunit.client.0.smithi019.stdout:8/813: mknod d9/d14/d17/d3d/de5/c111 0 2024-02-22T15:50:46.237 INFO:tasks.workunit.client.1.smithi062.stdout:9/675: rename d4/d28/d50/d5a/lbc to d4/d15/d5e/d63/da0/d92/ld6 0 2024-02-22T15:50:46.237 INFO:tasks.workunit.client.0.smithi019.stdout:9/866: getdents d2/d1f/d2e 0 2024-02-22T15:50:46.238 INFO:tasks.workunit.client.0.smithi019.stdout:8/814: readlink d9/d14/d17/d1c/le0 0 2024-02-22T15:50:46.238 INFO:tasks.workunit.client.0.smithi019.stdout:3/689: sync 2024-02-22T15:50:46.238 INFO:tasks.workunit.client.0.smithi019.stdout:9/867: chown d2/f30 4 1 2024-02-22T15:50:46.239 INFO:tasks.workunit.client.1.smithi062.stdout:9/676: write d4/d28/d50/f7a [1741047,103077] 0 2024-02-22T15:50:46.239 INFO:tasks.workunit.client.1.smithi062.stdout:6/718: dread d5/df/d32/f9f [0,4194304] 0 2024-02-22T15:50:46.240 INFO:tasks.workunit.client.1.smithi062.stdout:0/763: getdents d1/d3/de/df0 0 2024-02-22T15:50:46.245 INFO:tasks.workunit.client.1.smithi062.stdout:6/719: write d5/dd/d33/d66/db9/fe8 [663094,90888] 0 2024-02-22T15:50:46.246 INFO:tasks.workunit.client.0.smithi019.stdout:2/920: getdents d1/d17/d1c/d128/db9/dd0 0 2024-02-22T15:50:46.246 INFO:tasks.workunit.client.0.smithi019.stdout:8/815: mknod d9/d14/d1f/dca/dfb/c112 0 2024-02-22T15:50:46.247 INFO:tasks.workunit.client.1.smithi062.stdout:9/677: getdents d4/d15/d31 0 2024-02-22T15:50:46.248 INFO:tasks.workunit.client.1.smithi062.stdout:0/764: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/dfd 0 2024-02-22T15:50:46.249 INFO:tasks.workunit.client.0.smithi019.stdout:1/946: write d8/d18/d85/f9d [771438,77398] 0 2024-02-22T15:50:46.250 INFO:tasks.workunit.client.0.smithi019.stdout:3/690: truncate d9/d22/f50 1105196 0 2024-02-22T15:50:46.251 INFO:tasks.workunit.client.0.smithi019.stdout:2/921: read d1/d17/d21/d51/f6a [3992168,11470] 0 2024-02-22T15:50:46.252 INFO:tasks.workunit.client.0.smithi019.stdout:9/868: dwrite d2/dc8/f11f [0,4194304] 0 2024-02-22T15:50:46.252 INFO:tasks.workunit.client.1.smithi062.stdout:9/678: rename d4/d15/lbb to d4/d28/d50/ld7 0 2024-02-22T15:50:46.253 INFO:tasks.workunit.client.0.smithi019.stdout:1/947: stat d8/d2a/d116/f13f 0 2024-02-22T15:50:46.253 INFO:tasks.workunit.client.0.smithi019.stdout:8/816: rmdir d9/d14/d6f/d7e/d94/d9a/dd3 39 2024-02-22T15:50:46.253 INFO:tasks.workunit.client.1.smithi062.stdout:0/765: truncate d1/d13/dde/d76/d8c/d92/f96 686823 0 2024-02-22T15:50:46.254 INFO:tasks.workunit.client.1.smithi062.stdout:0/766: chown d1/d3/d5f 33752 1 2024-02-22T15:50:46.255 INFO:tasks.workunit.client.0.smithi019.stdout:3/691: stat d9/d26/d2d/dc7 0 2024-02-22T15:50:46.256 INFO:tasks.workunit.client.1.smithi062.stdout:0/767: read d1/fe6 [2537630,90686] 0 2024-02-22T15:50:46.256 INFO:tasks.workunit.client.1.smithi062.stdout:2/765: truncate d1/d8/d62/d73/dbd/f46 2127075 0 2024-02-22T15:50:46.258 INFO:tasks.workunit.client.1.smithi062.stdout:9/679: rename d4/d28/d50/da3/fd2 to d4/d35/fd8 0 2024-02-22T15:50:46.261 INFO:tasks.workunit.client.0.smithi019.stdout:8/817: dread - d9/d14/d6f/d7e/d94/d9c/fcf zero size 2024-02-22T15:50:46.262 INFO:tasks.workunit.client.1.smithi062.stdout:2/766: creat d1/d47/d39/d5d/f102 x:0 0 0 2024-02-22T15:50:46.262 INFO:tasks.workunit.client.0.smithi019.stdout:5/877: dwrite d8/dd/d19/d38/d54/fb2 [0,4194304] 0 2024-02-22T15:50:46.263 INFO:tasks.workunit.client.0.smithi019.stdout:3/692: unlink d9/d26/d2d/d39/ca2 0 2024-02-22T15:50:46.264 INFO:tasks.workunit.client.0.smithi019.stdout:8/818: chown d9/le7 147294678 1 2024-02-22T15:50:46.264 INFO:tasks.workunit.client.1.smithi062.stdout:9/680: write d4/d28/d50/da3/fcb [341635,59411] 0 2024-02-22T15:50:46.264 INFO:tasks.workunit.client.0.smithi019.stdout:9/869: creat d2/d7/dc/d27/dbc/d110/f12d x:0 0 0 2024-02-22T15:50:46.265 INFO:tasks.workunit.client.1.smithi062.stdout:6/720: dread d5/df/f26 [0,4194304] 0 2024-02-22T15:50:46.265 INFO:tasks.workunit.client.0.smithi019.stdout:1/948: truncate d8/d15/d34/d9e/d126/f11a 384868 0 2024-02-22T15:50:46.267 INFO:tasks.workunit.client.1.smithi062.stdout:9/681: write d4/d1c/f56 [820907,36726] 0 2024-02-22T15:50:46.269 INFO:tasks.workunit.client.1.smithi062.stdout:9/682: readlink d4/d1c/d83/ld3 0 2024-02-22T15:50:46.269 INFO:tasks.workunit.client.1.smithi062.stdout:0/768: dwrite d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fc9 [0,4194304] 0 2024-02-22T15:50:46.269 INFO:tasks.workunit.client.0.smithi019.stdout:5/878: write d8/d11/fa8 [264999,24822] 0 2024-02-22T15:50:46.271 INFO:tasks.workunit.client.0.smithi019.stdout:1/949: chown d8/d23/d33/c6d 367088 1 2024-02-22T15:50:46.273 INFO:tasks.workunit.client.1.smithi062.stdout:9/683: write d4/d15/d5e/d63/da0/f25 [77272,74112] 0 2024-02-22T15:50:46.273 INFO:tasks.workunit.client.1.smithi062.stdout:0/769: mkdir d1/d13/dde/d1d/ddb/dfe 0 2024-02-22T15:50:46.275 INFO:tasks.workunit.client.0.smithi019.stdout:2/922: dwrite d1/f18 [4194304,4194304] 0 2024-02-22T15:50:46.277 INFO:tasks.workunit.client.0.smithi019.stdout:1/950: mknod d8/d15/da4/c142 0 2024-02-22T15:50:46.280 INFO:tasks.workunit.client.0.smithi019.stdout:0/847: sync 2024-02-22T15:50:46.280 INFO:tasks.workunit.client.0.smithi019.stdout:4/807: sync 2024-02-22T15:50:46.280 INFO:tasks.workunit.client.0.smithi019.stdout:9/870: truncate d2/d7/dc/d27/dbc/f105 449709 0 2024-02-22T15:50:46.281 INFO:tasks.workunit.client.1.smithi062.stdout:0/770: dwrite d1/d5e/feb [0,4194304] 0 2024-02-22T15:50:46.289 INFO:tasks.workunit.client.0.smithi019.stdout:4/808: write d2/d3f/d48/d76/fd8 [615734,69345] 0 2024-02-22T15:50:46.290 INFO:tasks.workunit.client.0.smithi019.stdout:5/879: mknod d8/dd/d19/d77/d7a/dfc/c126 0 2024-02-22T15:50:46.291 INFO:tasks.workunit.client.0.smithi019.stdout:0/848: truncate d4/d22/fa9 4526568 0 2024-02-22T15:50:46.291 INFO:tasks.workunit.client.1.smithi062.stdout:9/684: truncate d4/d15/d5f/d75/f89 261417 0 2024-02-22T15:50:46.296 INFO:tasks.workunit.client.0.smithi019.stdout:0/849: write d4/f5c [4060651,48390] 0 2024-02-22T15:50:46.300 INFO:tasks.workunit.client.0.smithi019.stdout:2/923: dwrite d1/d17/d67/dfa/f12a [0,4194304] 0 2024-02-22T15:50:46.300 INFO:tasks.workunit.client.0.smithi019.stdout:3/693: link d9/d14/d19/c90 d9/d22/d58/cdc 0 2024-02-22T15:50:46.300 INFO:tasks.workunit.client.0.smithi019.stdout:5/880: chown d8/dd/d19/d3a/d46/c88 0 1 2024-02-22T15:50:46.302 INFO:tasks.workunit.client.0.smithi019.stdout:8/819: dwrite d9/d14/d6f/d7e/d94/daa/dd0/ff0 [0,4194304] 0 2024-02-22T15:50:46.309 INFO:tasks.workunit.client.0.smithi019.stdout:5/881: creat d8/f127 x:0 0 0 2024-02-22T15:50:46.309 INFO:tasks.workunit.client.0.smithi019.stdout:8/820: mknod d9/d88/c113 0 2024-02-22T15:50:46.312 INFO:tasks.workunit.client.0.smithi019.stdout:2/924: dwrite d1/d17/d21/d71/dd1/d106/f13e [0,4194304] 0 2024-02-22T15:50:46.318 INFO:tasks.workunit.client.0.smithi019.stdout:1/951: dread d8/d44/d5d/fc4 [0,4194304] 0 2024-02-22T15:50:46.334 INFO:tasks.workunit.client.1.smithi062.stdout:0/771: truncate d1/d13/dde/d1d/fb3 188059 0 2024-02-22T15:50:46.342 INFO:tasks.workunit.client.1.smithi062.stdout:0/772: truncate d1/d13/dde/d76/d8c/d5d/dc3/f48 3708429 0 2024-02-22T15:50:46.349 INFO:tasks.workunit.client.1.smithi062.stdout:0/773: truncate d1/d3/d4/fe5 1363584 0 2024-02-22T15:50:46.365 INFO:tasks.workunit.client.0.smithi019.stdout:4/809: mkdir d2/db/d14/d23/d33/db2/d107 0 2024-02-22T15:50:46.366 INFO:tasks.workunit.client.0.smithi019.stdout:3/694: symlink d9/d22/d49/ldd 0 2024-02-22T15:50:46.370 INFO:tasks.workunit.client.0.smithi019.stdout:8/821: dread d9/d14/d6f/d7e/d94/daa/fb6 [0,4194304] 0 2024-02-22T15:50:46.373 INFO:tasks.workunit.client.0.smithi019.stdout:5/882: creat d8/dd/d8e/db3/f128 x:0 0 0 2024-02-22T15:50:46.374 INFO:tasks.workunit.client.1.smithi062.stdout:7/511: sync 2024-02-22T15:50:46.377 INFO:tasks.workunit.client.1.smithi062.stdout:7/512: rename d0/d6/d2e/d4a/d5a/d5e/l6e to d0/d91/l9f 0 2024-02-22T15:50:46.377 INFO:tasks.workunit.client.1.smithi062.stdout:7/513: chown d0/d14/d19/d30 0 1 2024-02-22T15:50:46.378 INFO:tasks.workunit.client.0.smithi019.stdout:1/952: dwrite d8/d18/d85/f8a [0,4194304] 0 2024-02-22T15:50:46.379 INFO:tasks.workunit.client.0.smithi019.stdout:5/883: rmdir d8/dd/d19/d3a/d46/d7e/dbe/dc3 39 2024-02-22T15:50:46.380 INFO:tasks.workunit.client.1.smithi062.stdout:7/514: link d0/d14/d13/f36 d0/d6/d2e/d55/d80/d8a/fa0 0 2024-02-22T15:50:46.382 INFO:tasks.workunit.client.1.smithi062.stdout:7/515: truncate d0/d6/d2e/d55/f87 520129 0 2024-02-22T15:50:46.383 INFO:tasks.workunit.client.0.smithi019.stdout:1/953: truncate d8/d18/d64/d7f/de3/ffa 954407 0 2024-02-22T15:50:46.384 INFO:tasks.workunit.client.1.smithi062.stdout:7/516: creat d0/d14/d19/d30/d37/fa1 x:0 0 0 2024-02-22T15:50:46.384 INFO:tasks.workunit.client.0.smithi019.stdout:9/871: truncate d2/d1f/d2e/d48/d45/d5a/f63 2156798 0 2024-02-22T15:50:46.385 INFO:tasks.workunit.client.0.smithi019.stdout:5/884: fsync d8/d11/f3b 0 2024-02-22T15:50:46.387 INFO:tasks.workunit.client.0.smithi019.stdout:4/810: getdents d2/db/d14/d23/d21/d35/dc0 0 2024-02-22T15:50:46.388 INFO:tasks.workunit.client.0.smithi019.stdout:3/695: getdents d9/d26/d2d/d39 0 2024-02-22T15:50:46.389 INFO:tasks.workunit.client.0.smithi019.stdout:5/885: write d8/dd/d19/d38/d54/fe0 [766875,35109] 0 2024-02-22T15:50:46.389 INFO:tasks.workunit.client.1.smithi062.stdout:7/517: dread d0/d14/d13/f48 [0,4194304] 0 2024-02-22T15:50:46.390 INFO:tasks.workunit.client.1.smithi062.stdout:7/518: fsync d0/f24 0 2024-02-22T15:50:46.393 INFO:tasks.workunit.client.0.smithi019.stdout:1/954: symlink d8/d2a/d75/def/l143 0 2024-02-22T15:50:46.395 INFO:tasks.workunit.client.0.smithi019.stdout:2/925: write d1/d17/f30 [429609,71128] 0 2024-02-22T15:50:46.397 INFO:tasks.workunit.client.0.smithi019.stdout:9/872: rename d2/d7/dc/d27/dbc/l3a to d2/d7/dc/d27/dbc/d110/l12e 0 2024-02-22T15:50:46.399 INFO:tasks.workunit.client.0.smithi019.stdout:5/886: dwrite d8/dd/f75 [0,4194304] 0 2024-02-22T15:50:46.399 INFO:tasks.workunit.client.0.smithi019.stdout:9/873: chown d2/d1f/d2e/d48/d45/d5a/daf 10 1 2024-02-22T15:50:46.400 INFO:tasks.workunit.client.0.smithi019.stdout:3/696: creat d9/d65/da7/fde x:0 0 0 2024-02-22T15:50:46.401 INFO:tasks.workunit.client.0.smithi019.stdout:8/822: link d9/d14/d6f/cab d9/d14/d17/d1c/c114 0 2024-02-22T15:50:46.405 INFO:tasks.workunit.client.1.smithi062.stdout:7/519: dwrite d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:46.406 INFO:tasks.workunit.client.0.smithi019.stdout:3/697: truncate d9/d26/d2d/d39/fcd 940932 0 2024-02-22T15:50:46.408 INFO:tasks.workunit.client.0.smithi019.stdout:1/955: rename d8/d15/f20 to d8/d15/da4/f144 0 2024-02-22T15:50:46.409 INFO:tasks.workunit.client.0.smithi019.stdout:9/874: rmdir d2 39 2024-02-22T15:50:46.413 INFO:tasks.workunit.client.1.smithi062.stdout:7/520: dread d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:46.415 INFO:tasks.workunit.client.1.smithi062.stdout:7/521: mkdir d0/d6/d2e/da2 0 2024-02-22T15:50:46.416 INFO:tasks.workunit.client.0.smithi019.stdout:3/698: mknod d9/d65/da7/cdf 0 2024-02-22T15:50:46.417 INFO:tasks.workunit.client.0.smithi019.stdout:4/811: write d2/db/d14/d23/f2f [2571241,120009] 0 2024-02-22T15:50:46.419 INFO:tasks.workunit.client.0.smithi019.stdout:5/887: symlink d8/dd/d19/d38/d54/db0/d10d/d122/dde/l129 0 2024-02-22T15:50:46.422 INFO:tasks.workunit.client.0.smithi019.stdout:8/823: truncate d9/d14/f6a 306829 0 2024-02-22T15:50:46.423 INFO:tasks.workunit.client.0.smithi019.stdout:3/699: unlink d9/d26/d3f/d59/d66/f85 0 2024-02-22T15:50:46.425 INFO:tasks.workunit.client.0.smithi019.stdout:9/875: mkdir d2/d12f 0 2024-02-22T15:50:46.428 INFO:tasks.workunit.client.0.smithi019.stdout:9/876: chown d2/f4 19811434 1 2024-02-22T15:50:46.429 INFO:tasks.workunit.client.1.smithi062.stdout:7/522: creat d0/d6/d2e/fa3 x:0 0 0 2024-02-22T15:50:46.429 INFO:tasks.workunit.client.0.smithi019.stdout:5/888: dread d8/f9b [0,4194304] 0 2024-02-22T15:50:46.430 INFO:tasks.workunit.client.0.smithi019.stdout:9/877: chown d2/d12f 2 1 2024-02-22T15:50:46.431 INFO:tasks.workunit.client.1.smithi062.stdout:7/523: fsync d0/d14/d13/f2a 0 2024-02-22T15:50:46.432 INFO:tasks.workunit.client.0.smithi019.stdout:4/812: dwrite d2/db/d14/d23/d21/d35/f64 [0,4194304] 0 2024-02-22T15:50:46.434 INFO:tasks.workunit.client.1.smithi062.stdout:7/524: symlink d0/d6/la4 0 2024-02-22T15:50:46.438 INFO:tasks.workunit.client.0.smithi019.stdout:9/878: unlink d2/d1f/d2e/c101 0 2024-02-22T15:50:46.439 INFO:tasks.workunit.client.0.smithi019.stdout:1/956: dread d8/d2a/db3/f123 [0,4194304] 0 2024-02-22T15:50:46.440 INFO:tasks.workunit.client.0.smithi019.stdout:5/889: write d8/dd/d19/d3a/d46/d7e/dbe/de8/ff0 [751950,72518] 0 2024-02-22T15:50:46.443 INFO:tasks.workunit.client.0.smithi019.stdout:2/926: dwrite d1/d86/f9e [0,4194304] 0 2024-02-22T15:50:46.444 INFO:tasks.workunit.client.0.smithi019.stdout:3/700: rename d9/d26/d3f/d59/dae/f81 to d9/d22/fe0 0 2024-02-22T15:50:46.444 INFO:tasks.workunit.client.0.smithi019.stdout:4/813: fdatasync d2/db/d14/d23/d21/fb3 0 2024-02-22T15:50:46.448 INFO:tasks.workunit.client.0.smithi019.stdout:8/824: dwrite d9/d14/d17/f50 [0,4194304] 0 2024-02-22T15:50:46.449 INFO:tasks.workunit.client.0.smithi019.stdout:9/879: mkdir d2/d7/dc/d27/dbc/d130 0 2024-02-22T15:50:46.450 INFO:tasks.workunit.client.0.smithi019.stdout:5/890: fsync d8/f89 0 2024-02-22T15:50:46.453 INFO:tasks.workunit.client.1.smithi062.stdout:7/525: dwrite d0/d6/d2e/d55/d80/d8a/fa0 [4194304,4194304] 0 2024-02-22T15:50:46.454 INFO:tasks.workunit.client.0.smithi019.stdout:2/927: unlink d1/d17/d21/c8a 0 2024-02-22T15:50:46.456 INFO:tasks.workunit.client.0.smithi019.stdout:1/957: fsync d8/d18/d64/d7f/de3/f132 0 2024-02-22T15:50:46.457 INFO:tasks.workunit.client.0.smithi019.stdout:4/814: creat d2/db/d14/d23/d33/db2/f108 x:0 0 0 2024-02-22T15:50:46.462 INFO:tasks.workunit.client.0.smithi019.stdout:5/891: unlink d8/d11/d33/f74 0 2024-02-22T15:50:46.464 INFO:tasks.workunit.client.0.smithi019.stdout:3/701: link d9/d26/d2d/d53/da0/fcb d9/d26/d2d/d53/fe1 0 2024-02-22T15:50:46.465 INFO:tasks.workunit.client.0.smithi019.stdout:8/825: truncate d9/d14/d6f/d7e/d94/f9b 431364 0 2024-02-22T15:50:46.465 INFO:tasks.workunit.client.0.smithi019.stdout:5/892: dread - d8/dd/d19/d3a/d46/d7e/dbe/fd7 zero size 2024-02-22T15:50:46.467 INFO:tasks.workunit.client.0.smithi019.stdout:2/928: dwrite d1/d17/d38/fd9 [0,4194304] 0 2024-02-22T15:50:46.467 INFO:tasks.workunit.client.0.smithi019.stdout:4/815: mknod d2/db/d14/d23/d67/d8f/c109 0 2024-02-22T15:50:46.467 INFO:tasks.workunit.client.1.smithi062.stdout:7/526: dread d0/d14/f38 [4194304,4194304] 0 2024-02-22T15:50:46.468 INFO:tasks.workunit.client.1.smithi062.stdout:7/527: write d0/f24 [1080559,20735] 0 2024-02-22T15:50:46.471 INFO:tasks.workunit.client.1.smithi062.stdout:7/528: mknod d0/d6/d2e/d2f/ca5 0 2024-02-22T15:50:46.472 INFO:tasks.workunit.client.1.smithi062.stdout:7/529: chown d0/d6/d2e/d2f/c62 18762831 1 2024-02-22T15:50:46.473 INFO:tasks.workunit.client.0.smithi019.stdout:1/958: creat d8/d15/d34/d9e/f145 x:0 0 0 2024-02-22T15:50:46.474 INFO:tasks.workunit.client.0.smithi019.stdout:9/880: rename d2/d1f/d2e/d7d/cd7 to d2/d7/c131 0 2024-02-22T15:50:46.474 INFO:tasks.workunit.client.1.smithi062.stdout:7/530: symlink d0/d14/d19/d30/d37/la6 0 2024-02-22T15:50:46.475 INFO:tasks.workunit.client.0.smithi019.stdout:3/702: creat d9/d22/d27/d6d/fe2 x:0 0 0 2024-02-22T15:50:46.476 INFO:tasks.workunit.client.0.smithi019.stdout:9/881: stat d2/f6 0 2024-02-22T15:50:46.477 INFO:tasks.workunit.client.0.smithi019.stdout:2/929: write d1/d46/d9d/fe2 [42327,83050] 0 2024-02-22T15:50:46.478 INFO:tasks.workunit.client.1.smithi062.stdout:7/531: unlink d0/d14/d13/c27 0 2024-02-22T15:50:46.486 INFO:tasks.workunit.client.1.smithi062.stdout:7/532: dwrite d0/d14/d19/d30/f8d [0,4194304] 0 2024-02-22T15:50:46.499 INFO:tasks.workunit.client.1.smithi062.stdout:5/541: sync 2024-02-22T15:50:46.499 INFO:tasks.workunit.client.1.smithi062.stdout:4/704: sync 2024-02-22T15:50:46.499 INFO:tasks.workunit.client.1.smithi062.stdout:8/814: sync 2024-02-22T15:50:46.500 INFO:tasks.workunit.client.0.smithi019.stdout:5/893: creat d8/dd/d19/d3a/dce/f12a x:0 0 0 2024-02-22T15:50:46.500 INFO:tasks.workunit.client.0.smithi019.stdout:4/816: rmdir d2/db/d54 39 2024-02-22T15:50:46.500 INFO:tasks.workunit.client.0.smithi019.stdout:0/850: sync 2024-02-22T15:50:46.500 INFO:tasks.workunit.client.0.smithi019.stdout:8/826: mknod d9/d14/d17/c115 0 2024-02-22T15:50:46.500 INFO:tasks.workunit.client.0.smithi019.stdout:3/703: rmdir d9/d26/d2d 39 2024-02-22T15:50:46.501 INFO:tasks.workunit.client.1.smithi062.stdout:5/542: readlink d0/d3/da/d13/l1f 0 2024-02-22T15:50:46.502 INFO:tasks.workunit.client.0.smithi019.stdout:1/959: creat d8/d15/d5b/dbc/f146 x:0 0 0 2024-02-22T15:50:46.502 INFO:tasks.workunit.client.1.smithi062.stdout:4/705: dread - d4/d1c/d2c/f81 zero size 2024-02-22T15:50:46.502 INFO:tasks.workunit.client.0.smithi019.stdout:3/704: read d9/d14/f1b [352960,24521] 0 2024-02-22T15:50:46.503 INFO:tasks.workunit.client.1.smithi062.stdout:5/543: dread - d0/d3/da/d13/d57/d79/fa5 zero size 2024-02-22T15:50:46.503 INFO:tasks.workunit.client.0.smithi019.stdout:0/851: symlink d4/db/d1b/d40/d8c/l123 0 2024-02-22T15:50:46.504 INFO:tasks.workunit.client.1.smithi062.stdout:7/533: dwrite d0/d14/f29 [0,4194304] 0 2024-02-22T15:50:46.505 INFO:tasks.workunit.client.0.smithi019.stdout:2/930: symlink d1/db4/d118/l14d 0 2024-02-22T15:50:46.508 INFO:tasks.workunit.client.0.smithi019.stdout:4/817: rename d2/db/d14/d23/d21/dbf/dd7 to d2/db/d14/d23/d9e/d10a 0 2024-02-22T15:50:46.509 INFO:tasks.workunit.client.0.smithi019.stdout:0/852: truncate d4/db/d1b/d40/d44/d75/db7/f90 5566861 0 2024-02-22T15:50:46.511 INFO:tasks.workunit.client.0.smithi019.stdout:5/894: mknod d8/dd/d19/c12b 0 2024-02-22T15:50:46.511 INFO:tasks.workunit.client.0.smithi019.stdout:1/960: dread d8/d44/f9f [0,4194304] 0 2024-02-22T15:50:46.511 INFO:tasks.workunit.client.0.smithi019.stdout:0/853: fdatasync d4/db/d24/d80/fa0 0 2024-02-22T15:50:46.511 INFO:tasks.workunit.client.1.smithi062.stdout:4/706: mknod d4/d1c/d2c/d6b/d9b/db1/dc6/dd5/cf6 0 2024-02-22T15:50:46.514 INFO:tasks.workunit.client.1.smithi062.stdout:4/707: mkdir d4/d2a/d2d/d2e/d6a/df7 0 2024-02-22T15:50:46.516 INFO:tasks.workunit.client.0.smithi019.stdout:4/818: write d2/db/d44/fa2 [3503314,76517] 0 2024-02-22T15:50:46.516 INFO:tasks.workunit.client.0.smithi019.stdout:9/882: creat d2/d1f/d2e/d57/d60/f132 x:0 0 0 2024-02-22T15:50:46.516 INFO:tasks.workunit.client.0.smithi019.stdout:0/854: write d4/d22/d3d/f119 [796606,104441] 0 2024-02-22T15:50:46.517 INFO:tasks.workunit.client.1.smithi062.stdout:4/708: fdatasync d4/d1c/d2c/f56 0 2024-02-22T15:50:46.517 INFO:tasks.workunit.client.0.smithi019.stdout:4/819: read - d2/d3f/d48/d76/fcb zero size 2024-02-22T15:50:46.520 INFO:tasks.workunit.client.0.smithi019.stdout:3/705: symlink d9/d22/d27/d96/dba/le3 0 2024-02-22T15:50:46.520 INFO:tasks.workunit.client.0.smithi019.stdout:2/931: dwrite d1/d17/d67/dfa/d11e/f134 [0,4194304] 0 2024-02-22T15:50:46.521 INFO:tasks.workunit.client.1.smithi062.stdout:7/534: dwrite d0/d6/d2e/d55/d80/d8a/fa0 [0,4194304] 0 2024-02-22T15:50:46.525 INFO:tasks.workunit.client.1.smithi062.stdout:7/535: chown d0/d6/d2e/d4a/f1f 76 1 2024-02-22T15:50:46.527 INFO:tasks.workunit.client.1.smithi062.stdout:4/709: dwrite d4/d1c/d2c/f56 [4194304,4194304] 0 2024-02-22T15:50:46.528 INFO:tasks.workunit.client.0.smithi019.stdout:5/895: mknod d8/dd/d19/d38/d54/c12c 0 2024-02-22T15:50:46.530 INFO:tasks.workunit.client.1.smithi062.stdout:7/536: rmdir d0/d14/d13 39 2024-02-22T15:50:46.533 INFO:tasks.workunit.client.1.smithi062.stdout:4/710: symlink d4/d2a/d2d/d2e/d6a/d9e/da4/lf8 0 2024-02-22T15:50:46.533 INFO:tasks.workunit.client.1.smithi062.stdout:7/537: write d0/d14/d19/f1a [2812283,66428] 0 2024-02-22T15:50:46.535 INFO:tasks.workunit.client.1.smithi062.stdout:7/538: readlink d0/d6/d2e/d2f/l68 0 2024-02-22T15:50:46.535 INFO:tasks.workunit.client.0.smithi019.stdout:3/706: rmdir d9 39 2024-02-22T15:50:46.537 INFO:tasks.workunit.client.1.smithi062.stdout:4/711: creat d4/d2a/d2d/ff9 x:0 0 0 2024-02-22T15:50:46.537 INFO:tasks.workunit.client.1.smithi062.stdout:7/539: symlink d0/d14/d19/d30/d37/la7 0 2024-02-22T15:50:46.538 INFO:tasks.workunit.client.0.smithi019.stdout:2/932: rmdir d1/d17/d21/d71/d60/d11b 39 2024-02-22T15:50:46.539 INFO:tasks.workunit.client.1.smithi062.stdout:4/712: mkdir d4/d66/dfa 0 2024-02-22T15:50:46.540 INFO:tasks.workunit.client.0.smithi019.stdout:8/827: getdents d9/d14/d17/d5d/de6 0 2024-02-22T15:50:46.540 INFO:tasks.workunit.client.1.smithi062.stdout:7/540: creat d0/d6/d2e/d55/d80/d8a/fa8 x:0 0 0 2024-02-22T15:50:46.541 INFO:tasks.workunit.client.1.smithi062.stdout:4/713: chown d4/d1c/d2c/d6b/d70/l78 6384451 1 2024-02-22T15:50:46.541 INFO:tasks.workunit.client.0.smithi019.stdout:0/855: dwrite f2 [8388608,4194304] 0 2024-02-22T15:50:46.542 INFO:tasks.workunit.client.1.smithi062.stdout:7/541: symlink d0/d14/d19/d30/la9 0 2024-02-22T15:50:46.543 INFO:tasks.workunit.client.1.smithi062.stdout:5/544: dread d0/d3/da/f53 [0,4194304] 0 2024-02-22T15:50:46.543 INFO:tasks.workunit.client.0.smithi019.stdout:4/820: dread d2/d3f/d48/d76/fa8 [0,4194304] 0 2024-02-22T15:50:46.545 INFO:tasks.workunit.client.1.smithi062.stdout:4/714: read d4/d1c/d2c/d6b/d9b/db1/dc6/fef [2991044,106081] 0 2024-02-22T15:50:46.546 INFO:tasks.workunit.client.0.smithi019.stdout:3/707: read - d9/d22/d27/fb6 zero size 2024-02-22T15:50:46.546 INFO:tasks.workunit.client.1.smithi062.stdout:5/545: symlink d0/d3/d2f/d59/d67/d69/d3f/la6 0 2024-02-22T15:50:46.547 INFO:tasks.workunit.client.0.smithi019.stdout:4/821: read d2/db/d14/d23/d33/f7d [35512,3747] 0 2024-02-22T15:50:46.547 INFO:tasks.workunit.client.0.smithi019.stdout:2/933: unlink d1/def/cb6 0 2024-02-22T15:50:46.548 INFO:tasks.workunit.client.0.smithi019.stdout:4/822: stat d2/d3f/d6e 0 2024-02-22T15:50:46.548 INFO:tasks.workunit.client.1.smithi062.stdout:4/715: creat d4/d2a/d2d/d2e/ffb x:0 0 0 2024-02-22T15:50:46.549 INFO:tasks.workunit.client.0.smithi019.stdout:5/896: mknod d8/dd/d19/d3a/d46/d7e/dbe/dc3/c12d 0 2024-02-22T15:50:46.550 INFO:tasks.workunit.client.0.smithi019.stdout:5/897: readlink d8/dd/d19/d3a/la7 0 2024-02-22T15:50:46.551 INFO:tasks.workunit.client.0.smithi019.stdout:8/828: mknod d9/d14/d17/d3d/d3f/dd4/c116 0 2024-02-22T15:50:46.552 INFO:tasks.workunit.client.0.smithi019.stdout:0/856: symlink d4/db/d1b/d40/d8c/l124 0 2024-02-22T15:50:46.553 INFO:tasks.workunit.client.1.smithi062.stdout:4/716: mkdir d4/d1c/d2c/d6b/d70/de8/dfc 0 2024-02-22T15:50:46.555 INFO:tasks.workunit.client.1.smithi062.stdout:5/546: rename d0/c4 to d0/d3/da/d13/d23/d31/ca7 0 2024-02-22T15:50:46.556 INFO:tasks.workunit.client.0.smithi019.stdout:9/883: rename d2/d7/dc/d27/d68/f6b to d2/d1f/d2e/d57/d60/f133 0 2024-02-22T15:50:46.557 INFO:tasks.workunit.client.1.smithi062.stdout:4/717: getdents d4/d34 0 2024-02-22T15:50:46.558 INFO:tasks.workunit.client.0.smithi019.stdout:2/934: symlink d1/d17/d21/d71/dd1/d106/l14e 0 2024-02-22T15:50:46.559 INFO:tasks.workunit.client.0.smithi019.stdout:5/898: fdatasync d8/f9b 0 2024-02-22T15:50:46.559 INFO:tasks.workunit.client.1.smithi062.stdout:5/547: dread d0/d3/f14 [0,4194304] 0 2024-02-22T15:50:46.560 INFO:tasks.workunit.client.0.smithi019.stdout:1/961: dread d8/d18/d85/f98 [0,4194304] 0 2024-02-22T15:50:46.560 INFO:tasks.workunit.client.0.smithi019.stdout:0/857: symlink d4/d22/d2c/d68/d6d/l125 0 2024-02-22T15:50:46.561 INFO:tasks.workunit.client.1.smithi062.stdout:4/718: rename d4/df/l1e to d4/d2a/d2d/d2e/d6a/d9e/lfd 0 2024-02-22T15:50:46.562 INFO:tasks.workunit.client.0.smithi019.stdout:3/708: symlink d9/d22/d27/le4 0 2024-02-22T15:50:46.564 INFO:tasks.workunit.client.0.smithi019.stdout:9/884: mknod d2/d1f/d2e/d7d/c134 0 2024-02-22T15:50:46.564 INFO:tasks.workunit.client.0.smithi019.stdout:4/823: rmdir d2/db/d54 39 2024-02-22T15:50:46.565 INFO:tasks.workunit.client.1.smithi062.stdout:4/719: unlink d4/d2a/d2d/d2e/d6a/d9e/da4/db6/fc5 0 2024-02-22T15:50:46.568 INFO:tasks.workunit.client.1.smithi062.stdout:4/720: mkdir d4/d2a/d2d/d2e/d6a/d94/dfe 0 2024-02-22T15:50:46.568 INFO:tasks.workunit.client.0.smithi019.stdout:1/962: mkdir d8/d2a/d75/d81/d8e/d147 0 2024-02-22T15:50:46.571 INFO:tasks.workunit.client.0.smithi019.stdout:8/829: rename d9/d14/d17/fbe to d9/d14/d1f/d6d/d103/f117 0 2024-02-22T15:50:46.573 INFO:tasks.workunit.client.0.smithi019.stdout:3/709: dread d9/d11/f8a [0,4194304] 0 2024-02-22T15:50:46.573 INFO:tasks.workunit.client.0.smithi019.stdout:0/858: symlink d4/db/l126 0 2024-02-22T15:50:46.573 INFO:tasks.workunit.client.1.smithi062.stdout:7/542: truncate d0/d6/f23 712973 0 2024-02-22T15:50:46.574 INFO:tasks.workunit.client.1.smithi062.stdout:7/543: chown d0/d14/d76 1 1 2024-02-22T15:50:46.574 INFO:tasks.workunit.client.0.smithi019.stdout:4/824: mknod d2/db/d14/d23/d21/d35/d3d/c10b 0 2024-02-22T15:50:46.574 INFO:tasks.workunit.client.0.smithi019.stdout:0/859: stat d4/db/d49/dc2 0 2024-02-22T15:50:46.576 INFO:tasks.workunit.client.0.smithi019.stdout:2/935: link d1/d17/d21/d71/d60/fba d1/d46/ddb/d10d/d12c/d13c/f14f 0 2024-02-22T15:50:46.577 INFO:tasks.workunit.client.1.smithi062.stdout:5/548: write d0/d3/ff [4209000,47770] 0 2024-02-22T15:50:46.578 INFO:tasks.workunit.client.0.smithi019.stdout:8/830: fdatasync d9/d14/d6f/d7e/d94/d9c/fa3 0 2024-02-22T15:50:46.580 INFO:tasks.workunit.client.0.smithi019.stdout:3/710: rmdir d9/d65 39 2024-02-22T15:50:46.581 INFO:tasks.workunit.client.1.smithi062.stdout:7/544: rename d0/d14/d19/d30/f8d to d0/d6/d5b/faa 0 2024-02-22T15:50:46.582 INFO:tasks.workunit.client.0.smithi019.stdout:4/825: symlink d2/d3f/d6e/l10c 0 2024-02-22T15:50:46.582 INFO:tasks.workunit.client.1.smithi062.stdout:5/549: creat d0/fa8 x:0 0 0 2024-02-22T15:50:46.584 INFO:tasks.workunit.client.0.smithi019.stdout:8/831: chown d9/d14/d6f/d7e/d94/daa/fbd 298 1 2024-02-22T15:50:46.584 INFO:tasks.workunit.client.1.smithi062.stdout:4/721: write d4/df/fbb [3702594,19262] 0 2024-02-22T15:50:46.587 INFO:tasks.workunit.client.0.smithi019.stdout:4/826: dread - d2/db/d14/d23/d67/fe5 zero size 2024-02-22T15:50:46.590 INFO:tasks.workunit.client.0.smithi019.stdout:2/936: dwrite d1/d17/d21/d71/d60/f141 [0,4194304] 0 2024-02-22T15:50:46.591 INFO:tasks.workunit.client.1.smithi062.stdout:5/550: dwrite d0/d3/da/d13/f2a [0,4194304] 0 2024-02-22T15:50:46.591 INFO:tasks.workunit.client.1.smithi062.stdout:4/722: creat d4/d1c/d2c/d6b/d70/de8/dfc/fff x:0 0 0 2024-02-22T15:50:46.593 INFO:tasks.workunit.client.1.smithi062.stdout:5/551: fsync d0/f39 0 2024-02-22T15:50:46.596 INFO:tasks.workunit.client.0.smithi019.stdout:1/963: creat d8/d18/f148 x:0 0 0 2024-02-22T15:50:46.596 INFO:tasks.workunit.client.0.smithi019.stdout:8/832: mknod d9/d14/d17/d5d/c118 0 2024-02-22T15:50:46.597 INFO:tasks.workunit.client.1.smithi062.stdout:4/723: fdatasync d4/d1c/d30/d37/fa3 0 2024-02-22T15:50:46.597 INFO:tasks.workunit.client.0.smithi019.stdout:9/885: dread d2/dec/d79/fb2 [0,4194304] 0 2024-02-22T15:50:46.598 INFO:tasks.workunit.client.1.smithi062.stdout:5/552: mkdir d0/d3/d2f/d59/d67/d69/d3f/d66/da9 0 2024-02-22T15:50:46.598 INFO:tasks.workunit.client.0.smithi019.stdout:3/711: truncate d9/d26/d2d/d39/f5f 521359 0 2024-02-22T15:50:46.601 INFO:tasks.workunit.client.0.smithi019.stdout:2/937: fsync d1/de/d15/fa8 0 2024-02-22T15:50:46.601 INFO:tasks.workunit.client.0.smithi019.stdout:4/827: readlink d2/db/d14/d23/d21/d35/l41 0 2024-02-22T15:50:46.603 INFO:tasks.workunit.client.1.smithi062.stdout:5/553: creat d0/d3/da/d13/d23/faa x:0 0 0 2024-02-22T15:50:46.605 INFO:tasks.workunit.client.0.smithi019.stdout:2/938: write d1/d17/d21/d71/f37 [3549325,14909] 0 2024-02-22T15:50:46.605 INFO:tasks.workunit.client.1.smithi062.stdout:5/554: write d0/d3/d2f/d59/d67/d69/d3f/f51 [4990717,14755] 0 2024-02-22T15:50:46.607 INFO:tasks.workunit.client.0.smithi019.stdout:1/964: rename d8/d2a/d75/def/df8/f117 to d8/d18/dd2/f149 0 2024-02-22T15:50:46.610 INFO:tasks.workunit.client.0.smithi019.stdout:3/712: mknod d9/d22/ce5 0 2024-02-22T15:50:46.610 INFO:tasks.workunit.client.1.smithi062.stdout:5/555: rename d0/d3/da/d13/d23/d78/f7d to d0/d3/d30/d71/d7c/fab 0 2024-02-22T15:50:46.611 INFO:tasks.workunit.client.0.smithi019.stdout:0/860: dread d4/d22/fa9 [0,4194304] 0 2024-02-22T15:50:46.612 INFO:tasks.workunit.client.1.smithi062.stdout:5/556: write d0/d3/da/d32/f8d [763225,121542] 0 2024-02-22T15:50:46.613 INFO:tasks.workunit.client.0.smithi019.stdout:4/828: mknod d2/d3f/c10d 0 2024-02-22T15:50:46.614 INFO:tasks.workunit.client.1.smithi062.stdout:7/545: dread d0/d6/d2e/d4a/f1f [4194304,4194304] 0 2024-02-22T15:50:46.614 INFO:tasks.workunit.client.1.smithi062.stdout:5/557: fdatasync d0/d3/da/d13/d23/d31/f43 0 2024-02-22T15:50:46.615 INFO:tasks.workunit.client.0.smithi019.stdout:3/713: read - d9/d26/d32/fbf zero size 2024-02-22T15:50:46.615 INFO:tasks.workunit.client.1.smithi062.stdout:7/546: chown d0/d6/d2e/da2 4 1 2024-02-22T15:50:46.615 INFO:tasks.workunit.client.0.smithi019.stdout:9/886: dread d2/d1f/d2e/d48/d36/f43 [0,4194304] 0 2024-02-22T15:50:46.617 INFO:tasks.workunit.client.1.smithi062.stdout:7/547: write d0/d14/d19/f1a [2333586,94267] 0 2024-02-22T15:50:46.623 INFO:tasks.workunit.client.1.smithi062.stdout:5/558: truncate d0/d3/da/f53 21052 0 2024-02-22T15:50:46.625 INFO:tasks.workunit.client.1.smithi062.stdout:5/559: fdatasync d0/d3/da/d32/d42/f65 0 2024-02-22T15:50:46.625 INFO:tasks.workunit.client.1.smithi062.stdout:3/597: sync 2024-02-22T15:50:46.627 INFO:tasks.workunit.client.1.smithi062.stdout:5/560: stat d0/d3/da/d13/d23/d4e/d6d/f93 0 2024-02-22T15:50:46.627 INFO:tasks.workunit.client.1.smithi062.stdout:3/598: fsync d5/d26/d2d/d3f/fbb 0 2024-02-22T15:50:46.627 INFO:tasks.workunit.client.0.smithi019.stdout:2/939: dwrite d1/d17/d67/dfa/d11e/d126/f13a [0,4194304] 0 2024-02-22T15:50:46.629 INFO:tasks.workunit.client.1.smithi062.stdout:5/561: mkdir d0/d3/da/d13/d23/d31/dac 0 2024-02-22T15:50:46.629 INFO:tasks.workunit.client.0.smithi019.stdout:3/714: rename d9/fc1 to d9/d26/d3f/d5c/fe6 0 2024-02-22T15:50:46.633 INFO:tasks.workunit.client.0.smithi019.stdout:2/940: read - d1/d17/d67/dfa/d11e/f13d zero size 2024-02-22T15:50:46.633 INFO:tasks.workunit.client.0.smithi019.stdout:9/887: dread d2/d1f/d2e/d48/d45/d5a/fdd [0,4194304] 0 2024-02-22T15:50:46.636 INFO:tasks.workunit.client.0.smithi019.stdout:1/965: dwrite d8/d15/d5b/dbc/fe0 [0,4194304] 0 2024-02-22T15:50:46.638 INFO:tasks.workunit.client.0.smithi019.stdout:4/829: rename d2/db/d44/l60 to d2/db/d14/d23/d33/db2/l10e 0 2024-02-22T15:50:46.640 INFO:tasks.workunit.client.1.smithi062.stdout:4/724: dread d4/d1c/faa [0,4194304] 0 2024-02-22T15:50:46.640 INFO:tasks.workunit.client.0.smithi019.stdout:0/861: dwrite d4/d22/d2c/f9a [0,4194304] 0 2024-02-22T15:50:46.643 INFO:tasks.workunit.client.0.smithi019.stdout:3/715: read d9/d26/d32/f9c [903077,129783] 0 2024-02-22T15:50:46.643 INFO:tasks.workunit.client.0.smithi019.stdout:9/888: unlink d2/f19 0 2024-02-22T15:50:46.649 INFO:tasks.workunit.client.0.smithi019.stdout:4/830: unlink d2/d3f/d48/d76/lb4 0 2024-02-22T15:50:46.650 INFO:tasks.workunit.client.0.smithi019.stdout:1/966: creat d8/d2a/d75/d136/f14a x:0 0 0 2024-02-22T15:50:46.651 INFO:tasks.workunit.client.0.smithi019.stdout:9/889: fsync d2/f30 0 2024-02-22T15:50:46.652 INFO:tasks.workunit.client.0.smithi019.stdout:1/967: rmdir d8/d2a/dde 39 2024-02-22T15:50:46.653 INFO:tasks.workunit.client.0.smithi019.stdout:9/890: chown d2/d1f/d2e/d48/d36/f71 11409 1 2024-02-22T15:50:46.654 INFO:tasks.workunit.client.0.smithi019.stdout:2/941: write d1/f14 [2853172,6162] 0 2024-02-22T15:50:46.656 INFO:tasks.workunit.client.0.smithi019.stdout:9/891: chown d2/dec/f106 6707 1 2024-02-22T15:50:46.656 INFO:tasks.workunit.client.0.smithi019.stdout:4/831: read d2/db/d59/ff9 [900,6162] 0 2024-02-22T15:50:46.656 INFO:tasks.workunit.client.0.smithi019.stdout:3/716: link d9/d65/da7/fde d9/d22/d27/d96/fe7 0 2024-02-22T15:50:46.658 INFO:tasks.workunit.client.0.smithi019.stdout:3/717: chown d9/d26/c45 1442801 1 2024-02-22T15:50:46.659 INFO:tasks.workunit.client.0.smithi019.stdout:1/968: rename d8/d15/da4/f144 to d8/d18/d64/d7f/d94/f14b 0 2024-02-22T15:50:46.660 INFO:tasks.workunit.client.1.smithi062.stdout:7/548: dread d0/d14/f67 [0,4194304] 0 2024-02-22T15:50:46.661 INFO:tasks.workunit.client.0.smithi019.stdout:2/942: truncate d1/d17/d1c/d128/db9/dd0/dee/ffc 462345 0 2024-02-22T15:50:46.661 INFO:tasks.workunit.client.0.smithi019.stdout:4/832: write d2/db/d14/d23/d21/d35/f64 [2312496,128816] 0 2024-02-22T15:50:46.664 INFO:tasks.workunit.client.0.smithi019.stdout:9/892: dread d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:46.665 INFO:tasks.workunit.client.0.smithi019.stdout:2/943: fsync d1/de/f4e 0 2024-02-22T15:50:46.667 INFO:tasks.workunit.client.0.smithi019.stdout:2/944: write d1/d17/d21/d51/f72 [1812430,116975] 0 2024-02-22T15:50:46.671 INFO:tasks.workunit.client.0.smithi019.stdout:4/833: getdents d2/db/d59 0 2024-02-22T15:50:46.672 INFO:tasks.workunit.client.1.smithi062.stdout:4/725: dread d4/fd [0,4194304] 0 2024-02-22T15:50:46.672 INFO:tasks.workunit.client.0.smithi019.stdout:4/834: stat d2/fb6 0 2024-02-22T15:50:46.673 INFO:tasks.workunit.client.1.smithi062.stdout:7/549: link d0/d6/d2e/d55/d80/c97 d0/d6/d2e/d58/cab 0 2024-02-22T15:50:46.674 INFO:tasks.workunit.client.1.smithi062.stdout:4/726: truncate d4/d13/d20/fc2 913850 0 2024-02-22T15:50:46.674 INFO:tasks.workunit.client.0.smithi019.stdout:3/718: write d9/d22/d27/f3d [3782468,29357] 0 2024-02-22T15:50:46.675 INFO:tasks.workunit.client.0.smithi019.stdout:2/945: dwrite d1/d17/d21/d51/f72 [0,4194304] 0 2024-02-22T15:50:46.675 INFO:tasks.workunit.client.0.smithi019.stdout:4/835: write d2/d3f/d48/d76/fcb [118429,87450] 0 2024-02-22T15:50:46.676 INFO:tasks.workunit.client.1.smithi062.stdout:7/550: truncate d0/d6/f23 160620 0 2024-02-22T15:50:46.680 INFO:tasks.workunit.client.1.smithi062.stdout:7/551: unlink d0/d14/d19/d30/d37/f53 0 2024-02-22T15:50:46.683 INFO:tasks.workunit.client.1.smithi062.stdout:7/552: truncate d0/d6/f93 761781 0 2024-02-22T15:50:46.685 INFO:tasks.workunit.client.0.smithi019.stdout:3/719: getdents d9/d65/da7 0 2024-02-22T15:50:46.685 INFO:tasks.workunit.client.0.smithi019.stdout:4/836: dwrite d2/db/d14/d23/d4b/f86 [0,4194304] 0 2024-02-22T15:50:46.688 INFO:tasks.workunit.client.0.smithi019.stdout:4/837: chown d2/c8d 478294386 1 2024-02-22T15:50:46.689 INFO:tasks.workunit.client.0.smithi019.stdout:3/720: symlink d9/d26/d2d/dc7/le8 0 2024-02-22T15:50:46.691 INFO:tasks.workunit.client.0.smithi019.stdout:4/838: dread - d2/db/d59/feb zero size 2024-02-22T15:50:46.694 INFO:tasks.workunit.client.0.smithi019.stdout:4/839: write d2/db/d59/fe3 [214737,6464] 0 2024-02-22T15:50:46.696 INFO:tasks.workunit.client.0.smithi019.stdout:3/721: rename d9/d14/d19/l23 to d9/d22/d27/dd0/le9 0 2024-02-22T15:50:46.697 INFO:tasks.workunit.client.0.smithi019.stdout:4/840: write d2/db/d14/d61/f73 [1576927,41180] 0 2024-02-22T15:50:46.699 INFO:tasks.workunit.client.0.smithi019.stdout:2/946: dwrite d1/d17/d1c/fe7 [0,4194304] 0 2024-02-22T15:50:46.700 INFO:tasks.workunit.client.0.smithi019.stdout:3/722: fsync d9/d16/f42 0 2024-02-22T15:50:46.701 INFO:tasks.workunit.client.1.smithi062.stdout:4/727: dwrite d4/d1c/d2c/d6b/d70/f76 [0,4194304] 0 2024-02-22T15:50:46.705 INFO:tasks.workunit.client.0.smithi019.stdout:3/723: link f4 d9/d26/d3f/d59/d66/fea 0 2024-02-22T15:50:46.706 INFO:tasks.workunit.client.1.smithi062.stdout:4/728: creat d4/d1c/d2c/d6b/f100 x:0 0 0 2024-02-22T15:50:46.708 INFO:tasks.workunit.client.0.smithi019.stdout:4/841: rename d2/db/d14/d23/d21/d42/c94 to d2/db/d14/d23/c10f 0 2024-02-22T15:50:46.708 INFO:tasks.workunit.client.0.smithi019.stdout:3/724: fsync d9/d26/d2d/d53/f56 0 2024-02-22T15:50:46.710 INFO:tasks.workunit.client.1.smithi062.stdout:4/729: unlink d4/d2a/d2d/d64/fb5 0 2024-02-22T15:50:46.711 INFO:tasks.workunit.client.0.smithi019.stdout:2/947: dwrite d1/d17/d21/d71/f37 [0,4194304] 0 2024-02-22T15:50:46.716 INFO:tasks.workunit.client.0.smithi019.stdout:4/842: fsync d2/db/d14/d23/d21/f29 0 2024-02-22T15:50:46.716 INFO:tasks.workunit.client.1.smithi062.stdout:4/730: creat d4/d1c/d2c/f101 x:0 0 0 2024-02-22T15:50:46.719 INFO:tasks.workunit.client.0.smithi019.stdout:4/843: creat d2/f110 x:0 0 0 2024-02-22T15:50:46.720 INFO:tasks.workunit.client.0.smithi019.stdout:4/844: fdatasync d2/db/d14/d23/d21/dbf/fdd 0 2024-02-22T15:50:46.722 INFO:tasks.workunit.client.0.smithi019.stdout:4/845: truncate d2/db/d59/fe3 1319887 0 2024-02-22T15:50:46.722 INFO:tasks.workunit.client.1.smithi062.stdout:7/553: dread d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:46.723 INFO:tasks.workunit.client.1.smithi062.stdout:4/731: write d4/d1c/d2c/d6b/d70/f76 [2800266,114266] 0 2024-02-22T15:50:46.725 INFO:tasks.workunit.client.1.smithi062.stdout:7/554: mknod d0/d14/d19/d30/d45/cac 0 2024-02-22T15:50:46.725 INFO:tasks.workunit.client.0.smithi019.stdout:4/846: creat d2/db/d14/d23/d21/d42/dc4/f111 x:0 0 0 2024-02-22T15:50:46.727 INFO:tasks.workunit.client.1.smithi062.stdout:7/555: mknod d0/d6/d2e/d4a/d5a/cad 0 2024-02-22T15:50:46.728 INFO:tasks.workunit.client.1.smithi062.stdout:7/556: readlink d0/d6/la4 0 2024-02-22T15:50:46.729 INFO:tasks.workunit.client.1.smithi062.stdout:7/557: write d0/d6/d5b/faa [4919337,86350] 0 2024-02-22T15:50:46.730 INFO:tasks.workunit.client.1.smithi062.stdout:7/558: stat d0/d14/d19 0 2024-02-22T15:50:46.732 INFO:tasks.workunit.client.1.smithi062.stdout:7/559: write d0/d14/d13/f3e [3456042,34681] 0 2024-02-22T15:50:46.739 INFO:tasks.workunit.client.1.smithi062.stdout:9/685: sync 2024-02-22T15:50:46.740 INFO:tasks.workunit.client.1.smithi062.stdout:6/721: sync 2024-02-22T15:50:46.740 INFO:tasks.workunit.client.1.smithi062.stdout:8/815: sync 2024-02-22T15:50:46.740 INFO:tasks.workunit.client.1.smithi062.stdout:0/774: sync 2024-02-22T15:50:46.740 INFO:tasks.workunit.client.1.smithi062.stdout:1/631: sync 2024-02-22T15:50:46.740 INFO:tasks.workunit.client.1.smithi062.stdout:2/767: sync 2024-02-22T15:50:46.743 INFO:tasks.workunit.client.1.smithi062.stdout:8/816: readlink d8/d9/d2c/d49/d94/da1/ld2 0 2024-02-22T15:50:46.743 INFO:tasks.workunit.client.0.smithi019.stdout:2/948: write d1/d46/f94 [685756,1626] 0 2024-02-22T15:50:46.744 INFO:tasks.workunit.client.0.smithi019.stdout:3/725: truncate d9/d22/d27/f3d 4698853 0 2024-02-22T15:50:46.744 INFO:tasks.workunit.client.0.smithi019.stdout:2/949: read - d1/d17/f119 zero size 2024-02-22T15:50:46.745 INFO:tasks.workunit.client.1.smithi062.stdout:9/686: fdatasync d4/d35/f8a 0 2024-02-22T15:50:46.749 INFO:tasks.workunit.client.0.smithi019.stdout:2/950: mkdir d1/d17/d1c/d128/db9/dd0/d150 0 2024-02-22T15:50:46.750 INFO:tasks.workunit.client.1.smithi062.stdout:8/817: mknod d8/d9/d48/cfe 0 2024-02-22T15:50:46.751 INFO:tasks.workunit.client.0.smithi019.stdout:2/951: fsync d1/d46/f12f 0 2024-02-22T15:50:46.752 INFO:tasks.workunit.client.0.smithi019.stdout:3/726: write d9/d22/d27/f46 [7530539,111278] 0 2024-02-22T15:50:46.752 INFO:tasks.workunit.client.0.smithi019.stdout:4/847: dwrite d2/db/d14/d23/f28 [4194304,4194304] 0 2024-02-22T15:50:46.754 INFO:tasks.workunit.client.1.smithi062.stdout:0/775: symlink d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/df4/lff 0 2024-02-22T15:50:46.754 INFO:tasks.workunit.client.0.smithi019.stdout:2/952: readlink d1/d17/d1c/l117 0 2024-02-22T15:50:46.755 INFO:tasks.workunit.client.1.smithi062.stdout:2/768: rmdir d1/d1e/d38/d55/dc1 0 2024-02-22T15:50:46.756 INFO:tasks.workunit.client.1.smithi062.stdout:4/732: dwrite d4/d2a/d2d/d2e/d6a/d9e/fc3 [0,4194304] 0 2024-02-22T15:50:46.760 INFO:tasks.workunit.client.0.smithi019.stdout:4/848: truncate d2/db/d14/f5e 701853 0 2024-02-22T15:50:46.761 INFO:tasks.workunit.client.1.smithi062.stdout:9/687: rename d4/c8d to d4/cd9 0 2024-02-22T15:50:46.762 INFO:tasks.workunit.client.0.smithi019.stdout:2/953: creat d1/d17/d21/f151 x:0 0 0 2024-02-22T15:50:46.767 INFO:tasks.workunit.client.1.smithi062.stdout:7/560: dwrite d0/d14/d13/f2a [0,4194304] 0 2024-02-22T15:50:46.768 INFO:tasks.workunit.client.0.smithi019.stdout:4/849: rename d2/db/d14/d23/c10f to d2/db/d14/d23/c112 0 2024-02-22T15:50:46.771 INFO:tasks.workunit.client.1.smithi062.stdout:8/818: creat d8/d9/d2c/d74/df5/fff x:0 0 0 2024-02-22T15:50:46.771 INFO:tasks.workunit.client.1.smithi062.stdout:1/632: dread d1/d6/de/f9a [0,4194304] 0 2024-02-22T15:50:46.771 INFO:tasks.workunit.client.0.smithi019.stdout:4/850: creat d2/db/d14/d23/d67/d8f/f113 x:0 0 0 2024-02-22T15:50:46.775 INFO:tasks.workunit.client.1.smithi062.stdout:6/722: dread d5/df/d40/d44/d4a/d41/f55 [0,4194304] 0 2024-02-22T15:50:46.777 INFO:tasks.workunit.client.1.smithi062.stdout:0/776: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d100 0 2024-02-22T15:50:46.777 INFO:tasks.workunit.client.1.smithi062.stdout:8/819: truncate d8/d86/ffa 825201 0 2024-02-22T15:50:46.781 INFO:tasks.workunit.client.1.smithi062.stdout:1/633: creat d1/d6/d36/d76/fd7 x:0 0 0 2024-02-22T15:50:46.781 INFO:tasks.workunit.client.1.smithi062.stdout:4/733: link d4/d2a/d2d/d64/ca9 d4/d1c/d2c/d6b/d70/de8/dfc/c102 0 2024-02-22T15:50:46.782 INFO:tasks.workunit.client.1.smithi062.stdout:2/769: dwrite d1/f50 [0,4194304] 0 2024-02-22T15:50:46.782 INFO:tasks.workunit.client.0.smithi019.stdout:4/851: write d2/db/d14/f81 [1135457,102024] 0 2024-02-22T15:50:46.784 INFO:tasks.workunit.client.1.smithi062.stdout:7/561: symlink d0/d6/d2e/d55/d80/d8a/lae 0 2024-02-22T15:50:46.784 INFO:tasks.workunit.client.1.smithi062.stdout:6/723: chown d5/df/d40/d44/d76/d56/f65 16 1 2024-02-22T15:50:46.784 INFO:tasks.workunit.client.1.smithi062.stdout:8/820: read d8/d86/f96 [142207,68477] 0 2024-02-22T15:50:46.785 INFO:tasks.workunit.client.1.smithi062.stdout:0/777: write d1/d3/d4/d1e/d74/f70 [2495139,16709] 0 2024-02-22T15:50:46.786 INFO:tasks.workunit.client.0.smithi019.stdout:3/727: dwrite d9/d16/f8f [0,4194304] 0 2024-02-22T15:50:46.789 INFO:tasks.workunit.client.1.smithi062.stdout:2/770: rmdir d1/d1e/d38/d55 39 2024-02-22T15:50:46.790 INFO:tasks.workunit.client.1.smithi062.stdout:8/821: dread - d8/d86/db6/ffd zero size 2024-02-22T15:50:46.791 INFO:tasks.workunit.client.1.smithi062.stdout:0/778: chown d1/d13/dde/d76/d8c/d5d/dc1 1548260 1 2024-02-22T15:50:46.793 INFO:tasks.workunit.client.1.smithi062.stdout:7/562: write d0/d14/f38 [2545296,34864] 0 2024-02-22T15:50:46.793 INFO:tasks.workunit.client.0.smithi019.stdout:3/728: dread d9/d22/d27/f47 [0,4194304] 0 2024-02-22T15:50:46.794 INFO:tasks.workunit.client.1.smithi062.stdout:1/634: read d1/d6/d10/f1c [3788826,61691] 0 2024-02-22T15:50:46.796 INFO:tasks.workunit.client.0.smithi019.stdout:3/729: dread d9/d22/d27/f47 [0,4194304] 0 2024-02-22T15:50:46.796 INFO:tasks.workunit.client.1.smithi062.stdout:4/734: getdents d4/d2a/d2d/d2e/d6a 0 2024-02-22T15:50:46.798 INFO:tasks.workunit.client.1.smithi062.stdout:6/724: truncate d5/df/d40/d44/fbf 486818 0 2024-02-22T15:50:46.798 INFO:tasks.workunit.client.1.smithi062.stdout:7/563: fdatasync d0/d6/d5b/faa 0 2024-02-22T15:50:46.800 INFO:tasks.workunit.client.1.smithi062.stdout:8/822: creat d8/d9/d44/df8/f100 x:0 0 0 2024-02-22T15:50:46.800 INFO:tasks.workunit.client.0.smithi019.stdout:3/730: creat d9/d26/d3f/d5c/feb x:0 0 0 2024-02-22T15:50:46.801 INFO:tasks.workunit.client.0.smithi019.stdout:2/954: dread d1/d17/d21/d71/ff1 [0,4194304] 0 2024-02-22T15:50:46.801 INFO:tasks.workunit.client.1.smithi062.stdout:1/635: dread - d1/d6/d7/d2b/d47/d63/db6/fc6 zero size 2024-02-22T15:50:46.802 INFO:tasks.workunit.client.1.smithi062.stdout:0/779: creat d1/d3/d4/d1e/f101 x:0 0 0 2024-02-22T15:50:46.804 INFO:tasks.workunit.client.0.smithi019.stdout:3/731: symlink d9/d26/d3f/d59/lec 0 2024-02-22T15:50:46.805 INFO:tasks.workunit.client.1.smithi062.stdout:4/735: stat d4/d13/l22 0 2024-02-22T15:50:46.805 INFO:tasks.workunit.client.0.smithi019.stdout:4/852: dwrite d2/db/d14/d23/d9e/fab [0,4194304] 0 2024-02-22T15:50:46.805 INFO:tasks.workunit.client.0.smithi019.stdout:2/955: rmdir d1/d17/d1c/d128/db9/dd0/dee 39 2024-02-22T15:50:46.807 INFO:tasks.workunit.client.1.smithi062.stdout:6/725: rmdir d5/d8a/ddd 39 2024-02-22T15:50:46.807 INFO:tasks.workunit.client.0.smithi019.stdout:4/853: fsync d2/db/d14/d23/d4b/f82 0 2024-02-22T15:50:46.808 INFO:tasks.workunit.client.0.smithi019.stdout:2/956: chown d1/d17/d21/d71/dd1/d106/f10a 90990537 1 2024-02-22T15:50:46.809 INFO:tasks.workunit.client.0.smithi019.stdout:3/732: creat d9/dc8/fed x:0 0 0 2024-02-22T15:50:46.811 INFO:tasks.workunit.client.0.smithi019.stdout:4/854: chown d2/db/d14/d23/d21/d35/ccf 23 1 2024-02-22T15:50:46.811 INFO:tasks.workunit.client.1.smithi062.stdout:4/736: fsync d4/d2a/f8b 0 2024-02-22T15:50:46.811 INFO:tasks.workunit.client.1.smithi062.stdout:1/636: rmdir d1/d6/d7/d2b/d47/d24 39 2024-02-22T15:50:46.813 INFO:tasks.workunit.client.1.smithi062.stdout:9/688: dwrite d4/d15/f24 [0,4194304] 0 2024-02-22T15:50:46.814 INFO:tasks.workunit.client.1.smithi062.stdout:1/637: fsync d1/f27 0 2024-02-22T15:50:46.815 INFO:tasks.workunit.client.1.smithi062.stdout:0/780: mkdir d1/d13/dde/d76/d102 0 2024-02-22T15:50:46.819 INFO:tasks.workunit.client.0.smithi019.stdout:2/957: dwrite d1/da4/fe6 [4194304,4194304] 0 2024-02-22T15:50:46.819 INFO:tasks.workunit.client.1.smithi062.stdout:2/771: dwrite d1/d8/d62/d73/dbd/dbb/dc4/fe7 [0,4194304] 0 2024-02-22T15:50:46.822 INFO:tasks.workunit.client.1.smithi062.stdout:6/726: rename d5/df/d32/d47/l4f to d5/df/d32/d8f/leb 0 2024-02-22T15:50:46.823 INFO:tasks.workunit.client.1.smithi062.stdout:9/689: readlink d4/d15/d5e/d63/da0/d92/lcd 0 2024-02-22T15:50:46.823 INFO:tasks.workunit.client.1.smithi062.stdout:1/638: write d1/d6/de/f42 [2826780,32278] 0 2024-02-22T15:50:46.824 INFO:tasks.workunit.client.0.smithi019.stdout:2/958: write d1/d46/f95 [2360649,74232] 0 2024-02-22T15:50:46.825 INFO:tasks.workunit.client.1.smithi062.stdout:2/772: stat d1/d1e/d38/d76/dc5/dec 0 2024-02-22T15:50:46.826 INFO:tasks.workunit.client.0.smithi019.stdout:4/855: write d2/db/f6a [522952,68350] 0 2024-02-22T15:50:46.828 INFO:tasks.workunit.client.0.smithi019.stdout:2/959: symlink d1/d17/d21/d71/d60/l152 0 2024-02-22T15:50:46.830 INFO:tasks.workunit.client.1.smithi062.stdout:0/781: symlink d1/d13/dde/d76/d8c/d5d/dc3/d100/l103 0 2024-02-22T15:50:46.833 INFO:tasks.workunit.client.1.smithi062.stdout:9/690: truncate d4/d1c/d83/fca 667299 0 2024-02-22T15:50:46.835 INFO:tasks.workunit.client.1.smithi062.stdout:2/773: write d1/d8/d62/d73/dbd/dbb/dc4/fe7 [4953671,13308] 0 2024-02-22T15:50:46.836 INFO:tasks.workunit.client.0.smithi019.stdout:2/960: rmdir d1/de 39 2024-02-22T15:50:46.836 INFO:tasks.workunit.client.0.smithi019.stdout:2/961: readlink d1/db4/d118/l14d 0 2024-02-22T15:50:46.836 INFO:tasks.workunit.client.0.smithi019.stdout:3/733: dwrite d9/d26/d2d/d53/da0/fcb [0,4194304] 0 2024-02-22T15:50:46.837 INFO:tasks.workunit.client.1.smithi062.stdout:4/737: dwrite d4/d2a/d2d/d2e/ffb [0,4194304] 0 2024-02-22T15:50:46.838 INFO:tasks.workunit.client.0.smithi019.stdout:4/856: dread d2/db/d14/d23/d21/d35/f64 [0,4194304] 0 2024-02-22T15:50:46.839 INFO:tasks.workunit.client.0.smithi019.stdout:3/734: mknod d9/d26/d2d/cee 0 2024-02-22T15:50:46.843 INFO:tasks.workunit.client.1.smithi062.stdout:1/639: rename d1/d6/d7/d2b/d47/d63/db2/ld5 to d1/d6/d7/d4a/ld8 0 2024-02-22T15:50:46.845 INFO:tasks.workunit.client.0.smithi019.stdout:3/735: rename d9/d26/d2d/f91 to d9/d26/d2d/d53/da0/fef 0 2024-02-22T15:50:46.845 INFO:tasks.workunit.client.1.smithi062.stdout:2/774: symlink d1/d8/d62/dcb/l103 0 2024-02-22T15:50:46.846 INFO:tasks.workunit.client.0.smithi019.stdout:3/736: chown d9/d26/d2d/cee 3469 1 2024-02-22T15:50:46.846 INFO:tasks.workunit.client.0.smithi019.stdout:3/737: stat d9/d65/f7d 0 2024-02-22T15:50:46.848 INFO:tasks.workunit.client.1.smithi062.stdout:9/691: rename d4/d15/d31/f8f to d4/d28/d50/da3/fda 0 2024-02-22T15:50:46.849 INFO:tasks.workunit.client.1.smithi062.stdout:1/640: fsync d1/d6/de/f42 0 2024-02-22T15:50:46.849 INFO:tasks.workunit.client.0.smithi019.stdout:3/738: link d9/f6a d9/d22/d58/ff0 0 2024-02-22T15:50:46.850 INFO:tasks.workunit.client.1.smithi062.stdout:4/738: truncate d4/d13/d20/fe9 508584 0 2024-02-22T15:50:46.850 INFO:tasks.workunit.client.0.smithi019.stdout:3/739: fdatasync d9/d26/d3f/d5c/f6e 0 2024-02-22T15:50:46.850 INFO:tasks.workunit.client.1.smithi062.stdout:8/823: dread d8/d9/f2a [0,4194304] 0 2024-02-22T15:50:46.853 INFO:tasks.workunit.client.0.smithi019.stdout:3/740: getdents d9/d22/d27/d96 0 2024-02-22T15:50:46.854 INFO:tasks.workunit.client.1.smithi062.stdout:2/775: mkdir d1/d47/d32/d104 0 2024-02-22T15:50:46.855 INFO:tasks.workunit.client.0.smithi019.stdout:5/899: sync 2024-02-22T15:50:46.855 INFO:tasks.workunit.client.1.smithi062.stdout:6/727: dwrite d5/df/d40/d44/fdf [0,4194304] 0 2024-02-22T15:50:46.856 INFO:tasks.workunit.client.1.smithi062.stdout:0/782: dwrite d1/d5e/ff9 [0,4194304] 0 2024-02-22T15:50:46.858 INFO:tasks.workunit.client.0.smithi019.stdout:5/900: rmdir d8/dd/d8e/d2c 39 2024-02-22T15:50:46.860 INFO:tasks.workunit.client.0.smithi019.stdout:3/741: dwrite d9/d11/fb4 [8388608,4194304] 0 2024-02-22T15:50:46.861 INFO:tasks.workunit.client.1.smithi062.stdout:1/641: read - d1/d6/d7/d2b/d47/d63/db2/fbe zero size 2024-02-22T15:50:46.862 INFO:tasks.workunit.client.0.smithi019.stdout:5/901: dread - d8/dd/d19/d3a/d46/d7e/dbe/dc3/fea zero size 2024-02-22T15:50:46.862 INFO:tasks.workunit.client.1.smithi062.stdout:6/728: unlink d5/df/d40/d44/d78/l7b 0 2024-02-22T15:50:46.864 INFO:tasks.workunit.client.1.smithi062.stdout:0/783: fdatasync d1/d3/d4/f99 0 2024-02-22T15:50:46.868 INFO:tasks.workunit.client.0.smithi019.stdout:5/902: truncate d8/dd/d19/d3a/f5e 1126509 0 2024-02-22T15:50:46.869 INFO:tasks.workunit.client.1.smithi062.stdout:6/729: chown d5/df/d40/d44/d76/d56/l60 1013 1 2024-02-22T15:50:46.869 INFO:tasks.workunit.client.1.smithi062.stdout:2/776: rename d1/d47/d39/d5d/cfa to d1/d8/d62/d73/dbd/d65/dd4/c105 0 2024-02-22T15:50:46.872 INFO:tasks.workunit.client.1.smithi062.stdout:8/824: dwrite d8/d9/d44/f9b [0,4194304] 0 2024-02-22T15:50:46.873 INFO:tasks.workunit.client.0.smithi019.stdout:5/903: creat d8/dd/f12e x:0 0 0 2024-02-22T15:50:46.875 INFO:tasks.workunit.client.0.smithi019.stdout:5/904: truncate d8/dd/d19/d38/f11b 9124 0 2024-02-22T15:50:46.878 INFO:tasks.workunit.client.0.smithi019.stdout:3/742: dread d9/d22/f4f [0,4194304] 0 2024-02-22T15:50:46.879 INFO:tasks.workunit.client.0.smithi019.stdout:3/743: chown d9/dc8/fed 7514953 1 2024-02-22T15:50:46.880 INFO:tasks.workunit.client.0.smithi019.stdout:5/905: dread d8/d11/fed [0,4194304] 0 2024-02-22T15:50:46.883 INFO:tasks.workunit.client.1.smithi062.stdout:0/784: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dd7/d104 0 2024-02-22T15:50:46.883 INFO:tasks.workunit.client.1.smithi062.stdout:8/825: rename d8/d26/f47 to d8/d9/d2c/d49/d5e/f101 0 2024-02-22T15:50:46.883 INFO:tasks.workunit.client.1.smithi062.stdout:4/739: write d4/df/fda [357896,31641] 0 2024-02-22T15:50:46.883 INFO:tasks.workunit.client.0.smithi019.stdout:5/906: mknod d8/dd/d19/d38/d54/db0/d10d/d122/dde/c12f 0 2024-02-22T15:50:46.884 INFO:tasks.workunit.client.0.smithi019.stdout:5/907: fsync d8/dd/d19/d3a/d46/d7e/fd1 0 2024-02-22T15:50:46.885 INFO:tasks.workunit.client.1.smithi062.stdout:2/777: creat d1/d1e/f106 x:0 0 0 2024-02-22T15:50:46.885 INFO:tasks.workunit.client.1.smithi062.stdout:8/826: stat d8/l6e 0 2024-02-22T15:50:46.886 INFO:tasks.workunit.client.0.smithi019.stdout:5/908: symlink d8/dd/d19/d77/l130 0 2024-02-22T15:50:46.888 INFO:tasks.workunit.client.1.smithi062.stdout:4/740: rmdir d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc 39 2024-02-22T15:50:46.889 INFO:tasks.workunit.client.0.smithi019.stdout:5/909: symlink d8/d11/d33/l131 0 2024-02-22T15:50:46.890 INFO:tasks.workunit.client.1.smithi062.stdout:0/785: link d1/d13/dde/d76/l7a d1/d38/dfa/l105 0 2024-02-22T15:50:46.894 INFO:tasks.workunit.client.1.smithi062.stdout:4/741: creat d4/d34/f103 x:0 0 0 2024-02-22T15:50:46.894 INFO:tasks.workunit.client.1.smithi062.stdout:6/730: write d5/df/d32/d47/f50 [4720476,17759] 0 2024-02-22T15:50:46.895 INFO:tasks.workunit.client.1.smithi062.stdout:2/778: getdents d1/d1e/dda/de8 0 2024-02-22T15:50:46.897 INFO:tasks.workunit.client.1.smithi062.stdout:0/786: symlink d1/d13/dde/d76/d8c/d2c/db5/l106 0 2024-02-22T15:50:46.898 INFO:tasks.workunit.client.1.smithi062.stdout:2/779: creat d1/d47/f107 x:0 0 0 2024-02-22T15:50:46.900 INFO:tasks.workunit.client.0.smithi019.stdout:3/744: truncate d9/d26/f35 2449251 0 2024-02-22T15:50:46.901 INFO:tasks.workunit.client.1.smithi062.stdout:2/780: chown d1/d8/f3d 362329609 1 2024-02-22T15:50:46.904 INFO:tasks.workunit.client.1.smithi062.stdout:1/642: dread d1/d6/d7/f54 [0,4194304] 0 2024-02-22T15:50:46.905 INFO:tasks.workunit.client.1.smithi062.stdout:4/742: truncate d4/d1c/f27 4135314 0 2024-02-22T15:50:46.910 INFO:tasks.workunit.client.1.smithi062.stdout:4/743: write d4/d2a/f8b [4676029,70472] 0 2024-02-22T15:50:46.911 INFO:tasks.workunit.client.1.smithi062.stdout:6/731: dwrite d5/df/d40/d44/f7c [4194304,4194304] 0 2024-02-22T15:50:46.911 INFO:tasks.workunit.client.0.smithi019.stdout:5/910: dwrite d8/dd/d8e/d2c/f5d [0,4194304] 0 2024-02-22T15:50:46.911 INFO:tasks.workunit.client.0.smithi019.stdout:3/745: dwrite d9/d26/d3f/d59/d66/fb3 [0,4194304] 0 2024-02-22T15:50:46.912 INFO:tasks.workunit.client.1.smithi062.stdout:0/787: getdents d1/d13/dde/d76/d8c/d5d/dc3 0 2024-02-22T15:50:46.914 INFO:tasks.workunit.client.0.smithi019.stdout:3/746: truncate d9/d22/d27/d6d/fe2 156717 0 2024-02-22T15:50:46.915 INFO:tasks.workunit.client.1.smithi062.stdout:8/827: dread d8/d9/d2c/d49/f90 [0,4194304] 0 2024-02-22T15:50:46.915 INFO:tasks.workunit.client.1.smithi062.stdout:2/781: getdents d1/d28 0 2024-02-22T15:50:46.916 INFO:tasks.workunit.client.0.smithi019.stdout:3/747: fsync d9/d26/d3f/f4a 0 2024-02-22T15:50:46.917 INFO:tasks.workunit.client.1.smithi062.stdout:2/782: write d1/d8/f15 [272032,54918] 0 2024-02-22T15:50:46.918 INFO:tasks.workunit.client.0.smithi019.stdout:5/911: getdents d8/dd/d19/d3a/d46/d7e/dbe/dc3 0 2024-02-22T15:50:46.919 INFO:tasks.workunit.client.0.smithi019.stdout:3/748: symlink d9/d26/d3f/d59/lf1 0 2024-02-22T15:50:46.923 INFO:tasks.workunit.client.0.smithi019.stdout:3/749: chown d9/d65/da7/fde 310 1 2024-02-22T15:50:46.924 INFO:tasks.workunit.client.0.smithi019.stdout:3/750: chown d9/d65/da7/fc9 3 1 2024-02-22T15:50:46.926 INFO:tasks.workunit.client.1.smithi062.stdout:0/788: write d1/d13/dde/d76/d8c/f88 [1223129,56940] 0 2024-02-22T15:50:46.927 INFO:tasks.workunit.client.0.smithi019.stdout:5/912: dwrite d8/d11/d33/fc4 [0,4194304] 0 2024-02-22T15:50:46.929 INFO:tasks.workunit.client.1.smithi062.stdout:8/828: creat d8/d9/d2c/d49/d94/dc1/de9/dd1/f102 x:0 0 0 2024-02-22T15:50:46.933 INFO:tasks.workunit.client.1.smithi062.stdout:8/829: mkdir d8/d9/d48/d103 0 2024-02-22T15:50:46.933 INFO:tasks.workunit.client.1.smithi062.stdout:0/789: dread - d1/d13/dde/d76/fe2 zero size 2024-02-22T15:50:46.934 INFO:tasks.workunit.client.0.smithi019.stdout:5/913: getdents d8/dd/d19/d38/d54 0 2024-02-22T15:50:46.936 INFO:tasks.workunit.client.1.smithi062.stdout:8/830: fsync d8/d9/d48/fa4 0 2024-02-22T15:50:46.937 INFO:tasks.workunit.client.0.smithi019.stdout:5/914: creat d8/dd/d19/d38/d47/f132 x:0 0 0 2024-02-22T15:50:46.937 INFO:tasks.workunit.client.1.smithi062.stdout:0/790: readlink d1/d3/d4/l75 0 2024-02-22T15:50:46.938 INFO:tasks.workunit.client.0.smithi019.stdout:5/915: write d8/dd/d8e/d2c/f5d [735032,67853] 0 2024-02-22T15:50:46.940 INFO:tasks.workunit.client.1.smithi062.stdout:4/744: dwrite d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/feb [0,4194304] 0 2024-02-22T15:50:46.942 INFO:tasks.workunit.client.0.smithi019.stdout:5/916: creat d8/dd/d8e/d2c/d10e/f133 x:0 0 0 2024-02-22T15:50:46.944 INFO:tasks.workunit.client.1.smithi062.stdout:4/745: symlink d4/d1c/d30/da5/l104 0 2024-02-22T15:50:46.944 INFO:tasks.workunit.client.0.smithi019.stdout:5/917: mknod d8/dd/d19/d3a/d46/d7e/c134 0 2024-02-22T15:50:46.946 INFO:tasks.workunit.client.0.smithi019.stdout:3/751: dread d9/d26/d32/f33 [0,4194304] 0 2024-02-22T15:50:46.946 INFO:tasks.workunit.client.0.smithi019.stdout:5/918: chown d8/dd/d19/l104 5830115 1 2024-02-22T15:50:46.947 INFO:tasks.workunit.client.1.smithi062.stdout:1/643: dread d1/d6/fbb [0,4194304] 0 2024-02-22T15:50:46.948 INFO:tasks.workunit.client.1.smithi062.stdout:8/831: rmdir d8/d9/d2c/d49/dab 0 2024-02-22T15:50:46.948 INFO:tasks.workunit.client.1.smithi062.stdout:1/644: write d1/d6/d7/d2b/d96/f9e [2395717,7243] 0 2024-02-22T15:50:46.951 INFO:tasks.workunit.client.1.smithi062.stdout:1/645: write d1/d66/f75 [2678437,107224] 0 2024-02-22T15:50:46.952 INFO:tasks.workunit.client.0.smithi019.stdout:3/752: rename d9/d26/d3f/d59/d66/fd2 to d9/d22/d27/d96/ff2 0 2024-02-22T15:50:46.953 INFO:tasks.workunit.client.1.smithi062.stdout:8/832: rename d8/f29 to d8/d26/db5/f104 0 2024-02-22T15:50:46.954 INFO:tasks.workunit.client.0.smithi019.stdout:5/919: getdents d8/d11 0 2024-02-22T15:50:46.954 INFO:tasks.workunit.client.1.smithi062.stdout:8/833: readlink d8/d9/d44/d4d/l4f 0 2024-02-22T15:50:46.954 INFO:tasks.workunit.client.1.smithi062.stdout:4/746: creat d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f105 x:0 0 0 2024-02-22T15:50:46.955 INFO:tasks.workunit.client.0.smithi019.stdout:3/753: write d9/d22/d27/d6d/fe2 [454022,124302] 0 2024-02-22T15:50:46.957 INFO:tasks.workunit.client.0.smithi019.stdout:5/920: symlink d8/dd/d19/d3a/l135 0 2024-02-22T15:50:46.958 INFO:tasks.workunit.client.1.smithi062.stdout:4/747: mknod d4/d1c/d2c/d6b/d70/de8/c106 0 2024-02-22T15:50:46.959 INFO:tasks.workunit.client.1.smithi062.stdout:8/834: getdents d8/d9/d2c/dd5 0 2024-02-22T15:50:46.959 INFO:tasks.workunit.client.1.smithi062.stdout:4/748: write d4/f3f [2604694,34349] 0 2024-02-22T15:50:46.961 INFO:tasks.workunit.client.1.smithi062.stdout:8/835: truncate d8/d26/fd7 707409 0 2024-02-22T15:50:46.962 INFO:tasks.workunit.client.0.smithi019.stdout:5/921: link d8/dd/d19/d77/l130 d8/dd/d19/d38/d54/db0/l136 0 2024-02-22T15:50:46.963 INFO:tasks.workunit.client.0.smithi019.stdout:5/922: chown d8/dd/d19/d38/d54/db0/d10d/d122/dde/c121 53825147 1 2024-02-22T15:50:46.965 INFO:tasks.workunit.client.0.smithi019.stdout:5/923: rmdir d8/dd/d19/d38/d54 39 2024-02-22T15:50:46.965 INFO:tasks.workunit.client.1.smithi062.stdout:2/783: write d1/d47/d39/fac [672311,54065] 0 2024-02-22T15:50:46.968 INFO:tasks.workunit.client.1.smithi062.stdout:4/749: dread d4/d1c/f23 [0,4194304] 0 2024-02-22T15:50:46.969 INFO:tasks.workunit.client.0.smithi019.stdout:5/924: symlink d8/dd/d19/d38/d47/l137 0 2024-02-22T15:50:46.969 INFO:tasks.workunit.client.1.smithi062.stdout:0/791: write d1/d13/dde/d76/d8c/d5d/dc3/f47 [1338722,8547] 0 2024-02-22T15:50:46.970 INFO:tasks.workunit.client.1.smithi062.stdout:4/750: write d4/d13/d20/fea [671090,38280] 0 2024-02-22T15:50:46.971 INFO:tasks.workunit.client.0.smithi019.stdout:5/925: readlink d8/dd/d19/d38/d54/l116 0 2024-02-22T15:50:46.971 INFO:tasks.workunit.client.1.smithi062.stdout:2/784: symlink d1/d8/d62/d73/dbd/dbb/l108 0 2024-02-22T15:50:46.971 INFO:tasks.workunit.client.1.smithi062.stdout:0/792: chown d1/d3/d4/d1e/f79 30980793 1 2024-02-22T15:50:46.973 INFO:tasks.workunit.client.1.smithi062.stdout:2/785: chown d1/d8/df/c2f 2520 1 2024-02-22T15:50:46.974 INFO:tasks.workunit.client.1.smithi062.stdout:0/793: chown d1/d3/d4/d1e/d74/d68/c71 284160 1 2024-02-22T15:50:46.976 INFO:tasks.workunit.client.1.smithi062.stdout:2/786: mkdir d1/d8/df/d109 0 2024-02-22T15:50:46.980 INFO:tasks.workunit.client.1.smithi062.stdout:0/794: read d1/d13/dde/d1d/fb3 [63849,30992] 0 2024-02-22T15:50:46.980 INFO:tasks.workunit.client.1.smithi062.stdout:4/751: link d4/d1c/d2c/d6b/d70/f76 d4/d2a/d2d/d2e/d6a/d9e/db4/f107 0 2024-02-22T15:50:46.981 INFO:tasks.workunit.client.1.smithi062.stdout:8/836: dwrite d8/f40 [4194304,4194304] 0 2024-02-22T15:50:46.982 INFO:tasks.workunit.client.1.smithi062.stdout:2/787: mknod d1/d47/d32/d52/d80/c10a 0 2024-02-22T15:50:46.982 INFO:tasks.workunit.client.1.smithi062.stdout:0/795: write d1/d38/f7b [4630564,114540] 0 2024-02-22T15:50:46.984 INFO:tasks.workunit.client.1.smithi062.stdout:2/788: dread - d1/d47/d32/fd3 zero size 2024-02-22T15:50:46.985 INFO:tasks.workunit.client.0.smithi019.stdout:3/754: dread d9/d14/f95 [4194304,4194304] 0 2024-02-22T15:50:46.985 INFO:tasks.workunit.client.1.smithi062.stdout:8/837: truncate d8/d9/f71 2288461 0 2024-02-22T15:50:46.986 INFO:tasks.workunit.client.0.smithi019.stdout:3/755: chown d9/d26/f35 1186584 1 2024-02-22T15:50:46.987 INFO:tasks.workunit.client.1.smithi062.stdout:2/789: fsync d1/f50 0 2024-02-22T15:50:46.990 INFO:tasks.workunit.client.0.smithi019.stdout:3/756: mknod d9/d22/d27/cf3 0 2024-02-22T15:50:46.990 INFO:tasks.workunit.client.1.smithi062.stdout:8/838: truncate d8/d9/d2c/d74/ff4 705390 0 2024-02-22T15:50:46.990 INFO:tasks.workunit.client.1.smithi062.stdout:0/796: creat d1/d13/dde/d76/d8c/d2c/d84/f107 x:0 0 0 2024-02-22T15:50:46.991 INFO:tasks.workunit.client.1.smithi062.stdout:4/752: dwrite d4/d2a/d2d/ff9 [0,4194304] 0 2024-02-22T15:50:46.992 INFO:tasks.workunit.client.0.smithi019.stdout:3/757: mkdir d9/d26/d2d/df4 0 2024-02-22T15:50:46.993 INFO:tasks.workunit.client.0.smithi019.stdout:5/926: dread d8/f2d [0,4194304] 0 2024-02-22T15:50:46.993 INFO:tasks.workunit.client.1.smithi062.stdout:2/790: creat d1/d1e/dda/de8/f10b x:0 0 0 2024-02-22T15:50:46.994 INFO:tasks.workunit.client.0.smithi019.stdout:3/758: chown d9/d22/d27/d96/fca 2212514 1 2024-02-22T15:50:46.994 INFO:tasks.workunit.client.0.smithi019.stdout:5/927: chown d8/dd/d19/d3a/c71 16261 1 2024-02-22T15:50:46.996 INFO:tasks.workunit.client.0.smithi019.stdout:3/759: mknod d9/cf5 0 2024-02-22T15:50:46.996 INFO:tasks.workunit.client.0.smithi019.stdout:5/928: dread d8/f9b [0,4194304] 0 2024-02-22T15:50:46.998 INFO:tasks.workunit.client.0.smithi019.stdout:5/929: write d8/dd/fc8 [1374342,35319] 0 2024-02-22T15:50:46.999 INFO:tasks.workunit.client.1.smithi062.stdout:0/797: creat d1/d13/dde/d76/d8c/d5d/f108 x:0 0 0 2024-02-22T15:50:47.000 INFO:tasks.workunit.client.0.smithi019.stdout:3/760: symlink d9/lf6 0 2024-02-22T15:50:47.000 INFO:tasks.workunit.client.0.smithi019.stdout:5/930: chown d8/dd/d19/d38/d47/fd8 704258862 1 2024-02-22T15:50:47.000 INFO:tasks.workunit.client.0.smithi019.stdout:3/761: chown d9/d16/dcf 1 1 2024-02-22T15:50:47.003 INFO:tasks.workunit.client.0.smithi019.stdout:3/762: rmdir d9/d65 39 2024-02-22T15:50:47.004 INFO:tasks.workunit.client.1.smithi062.stdout:8/839: dwrite d8/f39 [0,4194304] 0 2024-02-22T15:50:47.004 INFO:tasks.workunit.client.0.smithi019.stdout:3/763: chown d9/d65/dd6 44 1 2024-02-22T15:50:47.006 INFO:tasks.workunit.client.0.smithi019.stdout:5/931: link d8/dd/d19/d3a/d46/c52 d8/d11/d33/c138 0 2024-02-22T15:50:47.006 INFO:tasks.workunit.client.1.smithi062.stdout:4/753: dwrite d4/d1c/d2c/d6b/d70/f93 [0,4194304] 0 2024-02-22T15:50:47.007 INFO:tasks.workunit.client.0.smithi019.stdout:5/932: dread - d8/d11/fe4 zero size 2024-02-22T15:50:47.008 INFO:tasks.workunit.client.0.smithi019.stdout:5/933: write d8/dd/d19/d3a/dc7/f117 [278612,38477] 0 2024-02-22T15:50:47.008 INFO:tasks.workunit.client.1.smithi062.stdout:0/798: creat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/f109 x:0 0 0 2024-02-22T15:50:47.009 INFO:tasks.workunit.client.0.smithi019.stdout:3/764: link d9/d65/da7/fde d9/d26/d2d/dc7/ff7 0 2024-02-22T15:50:47.013 INFO:tasks.workunit.client.1.smithi062.stdout:0/799: chown d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fc9 72040 1 2024-02-22T15:50:47.013 INFO:tasks.workunit.client.0.smithi019.stdout:3/765: symlink d9/d26/d2d/lf8 0 2024-02-22T15:50:47.015 INFO:tasks.workunit.client.0.smithi019.stdout:3/766: mknod d9/d22/d49/cf9 0 2024-02-22T15:50:47.015 INFO:tasks.workunit.client.1.smithi062.stdout:4/754: mknod d4/d1c/d2c/d6b/d9b/c108 0 2024-02-22T15:50:47.016 INFO:tasks.workunit.client.0.smithi019.stdout:3/767: rmdir d9/d11 39 2024-02-22T15:50:47.019 INFO:tasks.workunit.client.1.smithi062.stdout:0/800: read - d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/f109 zero size 2024-02-22T15:50:47.019 INFO:tasks.workunit.client.1.smithi062.stdout:8/840: dwrite d8/d9/d2c/d74/fcc [0,4194304] 0 2024-02-22T15:50:47.019 INFO:tasks.workunit.client.0.smithi019.stdout:3/768: mkdir d9/d16/dcf/dfa 0 2024-02-22T15:50:47.020 INFO:tasks.workunit.client.0.smithi019.stdout:3/769: dread - d9/d14/fce zero size 2024-02-22T15:50:47.021 INFO:tasks.workunit.client.1.smithi062.stdout:0/801: stat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/fe4 0 2024-02-22T15:50:47.027 INFO:tasks.workunit.client.1.smithi062.stdout:8/841: dwrite d8/f53 [4194304,4194304] 0 2024-02-22T15:50:47.039 INFO:tasks.workunit.client.1.smithi062.stdout:2/791: dread d1/f5 [0,4194304] 0 2024-02-22T15:50:47.042 INFO:tasks.workunit.client.1.smithi062.stdout:2/792: getdents d1/d1e/d38/d76 0 2024-02-22T15:50:47.046 INFO:tasks.workunit.client.1.smithi062.stdout:2/793: creat d1/d1e/d38/f10c x:0 0 0 2024-02-22T15:50:47.047 INFO:tasks.workunit.client.1.smithi062.stdout:0/802: write d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fd2 [828010,42302] 0 2024-02-22T15:50:47.050 INFO:tasks.workunit.client.1.smithi062.stdout:2/794: fdatasync d1/d1e/d38/d76/f99 0 2024-02-22T15:50:47.050 INFO:tasks.workunit.client.0.smithi019.stdout:3/770: write f4 [4400190,67427] 0 2024-02-22T15:50:47.052 INFO:tasks.workunit.client.1.smithi062.stdout:0/803: truncate d1/dd/f6c 34373 0 2024-02-22T15:50:47.053 INFO:tasks.workunit.client.0.smithi019.stdout:3/771: creat d9/dc8/ffb x:0 0 0 2024-02-22T15:50:47.053 INFO:tasks.workunit.client.0.smithi019.stdout:5/934: dwrite d8/dd/d19/d3a/d46/f53 [0,4194304] 0 2024-02-22T15:50:47.055 INFO:tasks.workunit.client.1.smithi062.stdout:0/804: readlink d1/dd/l89 0 2024-02-22T15:50:47.056 INFO:tasks.workunit.client.1.smithi062.stdout:2/795: rename d1/d1e/f94 to d1/d1e/d38/d76/df5/f10d 0 2024-02-22T15:50:47.056 INFO:tasks.workunit.client.0.smithi019.stdout:8/833: sync 2024-02-22T15:50:47.057 INFO:tasks.workunit.client.0.smithi019.stdout:0/862: sync 2024-02-22T15:50:47.057 INFO:tasks.workunit.client.1.smithi062.stdout:4/755: dwrite d4/f53 [0,4194304] 0 2024-02-22T15:50:47.061 INFO:tasks.workunit.client.0.smithi019.stdout:0/863: readlink d4/d22/d2c/d68/d6d/l125 0 2024-02-22T15:50:47.062 INFO:tasks.workunit.client.0.smithi019.stdout:0/864: stat d4/db/d76/lc4 0 2024-02-22T15:50:47.063 INFO:tasks.workunit.client.0.smithi019.stdout:3/772: dwrite d9/d16/f8f [0,4194304] 0 2024-02-22T15:50:47.064 INFO:tasks.workunit.client.1.smithi062.stdout:8/842: dread d8/f24 [0,4194304] 0 2024-02-22T15:50:47.065 INFO:tasks.workunit.client.0.smithi019.stdout:5/935: rename d8/dd/d19/d3a/d46/d7e/dbe/dc3/c12d to d8/dd/d19/d3a/d46/d7e/dbe/c139 0 2024-02-22T15:50:47.068 INFO:tasks.workunit.client.1.smithi062.stdout:2/796: fsync d1/d1e/d38/f54 0 2024-02-22T15:50:47.069 INFO:tasks.workunit.client.1.smithi062.stdout:4/756: mknod d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd7/c109 0 2024-02-22T15:50:47.072 INFO:tasks.workunit.client.0.smithi019.stdout:3/773: rename d9/d16/f8f to d9/d26/d2d/df4/ffc 0 2024-02-22T15:50:47.074 INFO:tasks.workunit.client.1.smithi062.stdout:8/843: rename d8/d28 to d8/d9/d44/df8/d105 0 2024-02-22T15:50:47.074 INFO:tasks.workunit.client.1.smithi062.stdout:5/562: sync 2024-02-22T15:50:47.075 INFO:tasks.workunit.client.1.smithi062.stdout:3/599: sync 2024-02-22T15:50:47.077 INFO:tasks.workunit.client.1.smithi062.stdout:0/805: mknod d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/ddc/c10a 0 2024-02-22T15:50:47.077 INFO:tasks.workunit.client.0.smithi019.stdout:3/774: dread d9/d26/d32/f33 [0,4194304] 0 2024-02-22T15:50:47.078 INFO:tasks.workunit.client.0.smithi019.stdout:5/936: dwrite d8/dd/d19/d77/dd2/f101 [0,4194304] 0 2024-02-22T15:50:47.078 INFO:tasks.workunit.client.0.smithi019.stdout:8/834: write d9/d14/f36 [3674920,30627] 0 2024-02-22T15:50:47.081 INFO:tasks.workunit.client.0.smithi019.stdout:3/775: symlink d9/d26/d2d/df4/lfd 0 2024-02-22T15:50:47.081 INFO:tasks.workunit.client.1.smithi062.stdout:0/806: write d1/d3/de/f3a [448439,24929] 0 2024-02-22T15:50:47.081 INFO:tasks.workunit.client.1.smithi062.stdout:2/797: creat d1/d1e/d38/d76/df5/f10e x:0 0 0 2024-02-22T15:50:47.082 INFO:tasks.workunit.client.1.smithi062.stdout:3/600: chown d5/d1a/d3c/f54 24540 1 2024-02-22T15:50:47.082 INFO:tasks.workunit.client.1.smithi062.stdout:3/601: readlink d5/l45 0 2024-02-22T15:50:47.083 INFO:tasks.workunit.client.0.smithi019.stdout:8/835: chown d9/d14/d1f/d4c/f7f 877 1 2024-02-22T15:50:47.083 INFO:tasks.workunit.client.0.smithi019.stdout:3/776: chown d9/d26/d3f/d5c/feb 20383 1 2024-02-22T15:50:47.083 INFO:tasks.workunit.client.0.smithi019.stdout:1/969: sync 2024-02-22T15:50:47.083 INFO:tasks.workunit.client.0.smithi019.stdout:9/893: sync 2024-02-22T15:50:47.083 INFO:tasks.workunit.client.0.smithi019.stdout:2/962: sync 2024-02-22T15:50:47.083 INFO:tasks.workunit.client.0.smithi019.stdout:4/857: sync 2024-02-22T15:50:47.084 INFO:tasks.workunit.client.1.smithi062.stdout:4/757: symlink d4/d66/dfa/l10a 0 2024-02-22T15:50:47.084 INFO:tasks.workunit.client.1.smithi062.stdout:8/844: mkdir d8/d9/d44/df8/d105/dc8/d106 0 2024-02-22T15:50:47.085 INFO:tasks.workunit.client.0.smithi019.stdout:8/836: stat d9/d14/d1f/dca/dfb 0 2024-02-22T15:50:47.085 INFO:tasks.workunit.client.0.smithi019.stdout:5/937: rename d8/dd/d19/d38/d47/f68 to d8/dd/d19/d38/d54/f13a 0 2024-02-22T15:50:47.086 INFO:tasks.workunit.client.1.smithi062.stdout:0/807: truncate d1/d13/dde/f34 8977469 0 2024-02-22T15:50:47.086 INFO:tasks.workunit.client.1.smithi062.stdout:2/798: stat d1/l34 0 2024-02-22T15:50:47.090 INFO:tasks.workunit.client.0.smithi019.stdout:4/858: readlink d2/db/d14/d23/l30 0 2024-02-22T15:50:47.090 INFO:tasks.workunit.client.0.smithi019.stdout:5/938: write d8/dd/d19/d38/d47/fd8 [297841,72295] 0 2024-02-22T15:50:47.091 INFO:tasks.workunit.client.1.smithi062.stdout:8/845: rmdir d8/d9/d2c/d74/df5 39 2024-02-22T15:50:47.093 INFO:tasks.workunit.client.0.smithi019.stdout:4/859: creat d2/db/d44/f114 x:0 0 0 2024-02-22T15:50:47.095 INFO:tasks.workunit.client.1.smithi062.stdout:4/758: mknod d4/d1c/d2c/d6b/d9b/db1/dc6/dd5/c10b 0 2024-02-22T15:50:47.095 INFO:tasks.workunit.client.0.smithi019.stdout:1/970: creat d8/f14c x:0 0 0 2024-02-22T15:50:47.095 INFO:tasks.workunit.client.0.smithi019.stdout:2/963: mkdir d1/de/d15/d153 0 2024-02-22T15:50:47.098 INFO:tasks.workunit.client.1.smithi062.stdout:5/563: dwrite d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:47.098 INFO:tasks.workunit.client.0.smithi019.stdout:5/939: symlink d8/d11/l13b 0 2024-02-22T15:50:47.100 INFO:tasks.workunit.client.0.smithi019.stdout:3/777: dwrite d9/d11/f4d [0,4194304] 0 2024-02-22T15:50:47.101 INFO:tasks.workunit.client.1.smithi062.stdout:8/846: mkdir d8/d9/d2c/d74/d107 0 2024-02-22T15:50:47.101 INFO:tasks.workunit.client.0.smithi019.stdout:8/837: getdents d9/d14/d6f/d7e/d94/d9a 0 2024-02-22T15:50:47.102 INFO:tasks.workunit.client.0.smithi019.stdout:2/964: readlink d1/d17/d21/l7a 0 2024-02-22T15:50:47.105 INFO:tasks.workunit.client.1.smithi062.stdout:0/808: dwrite d1/d3/d4/d1e/d74/f70 [0,4194304] 0 2024-02-22T15:50:47.106 INFO:tasks.workunit.client.0.smithi019.stdout:0/865: dread d4/f56 [0,4194304] 0 2024-02-22T15:50:47.108 INFO:tasks.workunit.client.0.smithi019.stdout:3/778: write d9/dc8/ffb [309749,121144] 0 2024-02-22T15:50:47.108 INFO:tasks.workunit.client.0.smithi019.stdout:9/894: dwrite d2/d1f/d2e/d48/d36/f83 [0,4194304] 0 2024-02-22T15:50:47.108 INFO:tasks.workunit.client.1.smithi062.stdout:8/847: unlink d8/d9/d44/df8/d105/d4e/c46 0 2024-02-22T15:50:47.109 INFO:tasks.workunit.client.0.smithi019.stdout:2/965: read d1/d17/d1c/d128/db9/dd0/ff7 [2303525,11971] 0 2024-02-22T15:50:47.110 INFO:tasks.workunit.client.0.smithi019.stdout:3/779: readlink d9/d26/d3f/d59/l73 0 2024-02-22T15:50:47.111 INFO:tasks.workunit.client.0.smithi019.stdout:5/940: rmdir d8/dd/d19/d3a/d46/d7e/dbe/de8 39 2024-02-22T15:50:47.117 INFO:tasks.workunit.client.0.smithi019.stdout:4/860: write d2/db/f26 [1837415,106282] 0 2024-02-22T15:50:47.118 INFO:tasks.workunit.client.0.smithi019.stdout:8/838: write d9/f12 [1612561,23276] 0 2024-02-22T15:50:47.122 INFO:tasks.workunit.client.0.smithi019.stdout:3/780: dwrite d9/d26/d2d/d53/f56 [0,4194304] 0 2024-02-22T15:50:47.122 INFO:tasks.workunit.client.1.smithi062.stdout:4/759: dread d4/d2a/d2d/d2e/ffb [0,4194304] 0 2024-02-22T15:50:47.123 INFO:tasks.workunit.client.0.smithi019.stdout:5/941: mknod d8/dd/d19/d77/dd2/dec/c13c 0 2024-02-22T15:50:47.125 INFO:tasks.workunit.client.1.smithi062.stdout:2/799: truncate d1/fb8 2891832 0 2024-02-22T15:50:47.125 INFO:tasks.workunit.client.1.smithi062.stdout:3/602: write d5/d16/d21/d32/d6a/d7b/f5f [616994,28689] 0 2024-02-22T15:50:47.127 INFO:tasks.workunit.client.0.smithi019.stdout:9/895: mknod d2/d7/dc/d27/d68/c135 0 2024-02-22T15:50:47.127 INFO:tasks.workunit.client.0.smithi019.stdout:0/866: symlink d4/db/d1b/d40/d8c/dda/l127 0 2024-02-22T15:50:47.128 INFO:tasks.workunit.client.1.smithi062.stdout:0/809: fdatasync d1/d3/f39 0 2024-02-22T15:50:47.128 INFO:tasks.workunit.client.1.smithi062.stdout:5/564: rename d0/d3/da/d32/d4f/fa2 to d0/d3/da/d13/d57/fad 0 2024-02-22T15:50:47.129 INFO:tasks.workunit.client.1.smithi062.stdout:5/565: chown d0/d3/f14 44825 1 2024-02-22T15:50:47.131 INFO:tasks.workunit.client.1.smithi062.stdout:4/760: unlink d4/d1c/d30/da5/l104 0 2024-02-22T15:50:47.132 INFO:tasks.workunit.client.0.smithi019.stdout:9/896: mknod d2/d1f/d2e/d48/d45/d5a/daf/c136 0 2024-02-22T15:50:47.133 INFO:tasks.workunit.client.1.smithi062.stdout:5/566: creat d0/d3/d30/d71/d7c/fae x:0 0 0 2024-02-22T15:50:47.134 INFO:tasks.workunit.client.1.smithi062.stdout:2/800: mkdir d1/d1e/d38/d76/dc5/df3/d10f 0 2024-02-22T15:50:47.135 INFO:tasks.workunit.client.1.smithi062.stdout:3/603: dwrite d5/d16/f48 [0,4194304] 0 2024-02-22T15:50:47.135 INFO:tasks.workunit.client.0.smithi019.stdout:3/781: creat d9/d16/ffe x:0 0 0 2024-02-22T15:50:47.136 INFO:tasks.workunit.client.0.smithi019.stdout:1/971: truncate d8/d15/f68 5611845 0 2024-02-22T15:50:47.139 INFO:tasks.workunit.client.1.smithi062.stdout:3/604: truncate d5/d16/d97/fb1 1081380 0 2024-02-22T15:50:47.140 INFO:tasks.workunit.client.1.smithi062.stdout:5/567: rmdir d0/d3/da/d32/d42 39 2024-02-22T15:50:47.140 INFO:tasks.workunit.client.0.smithi019.stdout:9/897: chown d2/d7/dc/d27/dbc/d110/l11a 4855844 1 2024-02-22T15:50:47.142 INFO:tasks.workunit.client.0.smithi019.stdout:3/782: rename d9/d26/d2d/d39/l52 to d9/d26/d2d/d53/lff 0 2024-02-22T15:50:47.144 INFO:tasks.workunit.client.0.smithi019.stdout:5/942: dwrite d8/d11/fdc [0,4194304] 0 2024-02-22T15:50:47.146 INFO:tasks.workunit.client.0.smithi019.stdout:1/972: getdents d8/d2a/d75/d81/d8e/d147 0 2024-02-22T15:50:47.146 INFO:tasks.workunit.client.1.smithi062.stdout:5/568: creat d0/d3/da/d13/d23/d31/faf x:0 0 0 2024-02-22T15:50:47.146 INFO:tasks.workunit.client.1.smithi062.stdout:4/761: dwrite d4/d1c/d30/d3b/d49/f95 [0,4194304] 0 2024-02-22T15:50:47.149 INFO:tasks.workunit.client.1.smithi062.stdout:2/801: link d1/d8/d62/d73/dbd/dbb/dc4/dfe/l100 d1/d1e/d38/d76/dc5/df3/d10f/l110 0 2024-02-22T15:50:47.149 INFO:tasks.workunit.client.0.smithi019.stdout:3/783: creat d9/d22/d49/f100 x:0 0 0 2024-02-22T15:50:47.150 INFO:tasks.workunit.client.1.smithi062.stdout:3/605: truncate d5/f6f 736786 0 2024-02-22T15:50:47.151 INFO:tasks.workunit.client.0.smithi019.stdout:9/898: getdents d2/d1f/db5 0 2024-02-22T15:50:47.151 INFO:tasks.workunit.client.0.smithi019.stdout:4/861: dread d2/db/f26 [0,4194304] 0 2024-02-22T15:50:47.152 INFO:tasks.workunit.client.0.smithi019.stdout:1/973: fsync d8/d23/d33/f135 0 2024-02-22T15:50:47.153 INFO:tasks.workunit.client.1.smithi062.stdout:5/569: read d0/d3/da/d13/d23/d4e/f5d [793796,10676] 0 2024-02-22T15:50:47.155 INFO:tasks.workunit.client.1.smithi062.stdout:4/762: mkdir d4/df/daf/d10c 0 2024-02-22T15:50:47.155 INFO:tasks.workunit.client.0.smithi019.stdout:5/943: getdents d8/dd/d19/d38/d54/db0/d10d 0 2024-02-22T15:50:47.155 INFO:tasks.workunit.client.0.smithi019.stdout:1/974: write d8/d18/d64/d7f/d94/f121 [1155324,102493] 0 2024-02-22T15:50:47.156 INFO:tasks.workunit.client.0.smithi019.stdout:4/862: write d2/db/d14/d23/d67/d8f/f113 [660188,35765] 0 2024-02-22T15:50:47.157 INFO:tasks.workunit.client.1.smithi062.stdout:4/763: chown d4/d1c/d30/d3b/c71 0 1 2024-02-22T15:50:47.159 INFO:tasks.workunit.client.1.smithi062.stdout:5/570: mknod d0/d3/d2f/d59/d67/d69/d3f/d66/cb0 0 2024-02-22T15:50:47.160 INFO:tasks.workunit.client.0.smithi019.stdout:9/899: creat d2/d7/f137 x:0 0 0 2024-02-22T15:50:47.160 INFO:tasks.workunit.client.1.smithi062.stdout:4/764: creat d4/d2a/d2d/d2e/d6a/d8a/ddb/f10d x:0 0 0 2024-02-22T15:50:47.160 INFO:tasks.workunit.client.0.smithi019.stdout:1/975: creat d8/d15/d34/d9e/d126/f14d x:0 0 0 2024-02-22T15:50:47.161 INFO:tasks.workunit.client.0.smithi019.stdout:5/944: symlink d8/dd/d19/d3a/dc7/l13d 0 2024-02-22T15:50:47.161 INFO:tasks.workunit.client.0.smithi019.stdout:9/900: readlink d2/d7/dc/d27/d68/l10e 0 2024-02-22T15:50:47.163 INFO:tasks.workunit.client.0.smithi019.stdout:3/784: getdents d9/d26/d3f/d59/dae 0 2024-02-22T15:50:47.163 INFO:tasks.workunit.client.1.smithi062.stdout:5/571: creat d0/d3/da/d13/fb1 x:0 0 0 2024-02-22T15:50:47.163 INFO:tasks.workunit.client.0.smithi019.stdout:1/976: chown d8/d2a/d75/d81/f10d 23776 1 2024-02-22T15:50:47.164 INFO:tasks.workunit.client.1.smithi062.stdout:4/765: symlink d4/d2a/d2d/d2e/d6a/d8a/ddb/l10e 0 2024-02-22T15:50:47.164 INFO:tasks.workunit.client.1.smithi062.stdout:2/802: dwrite d1/d8/df/f31 [0,4194304] 0 2024-02-22T15:50:47.166 INFO:tasks.workunit.client.0.smithi019.stdout:3/785: fdatasync d9/d26/d76/f77 0 2024-02-22T15:50:47.166 INFO:tasks.workunit.client.0.smithi019.stdout:5/945: write d8/d11/fed [4445159,4568] 0 2024-02-22T15:50:47.167 INFO:tasks.workunit.client.1.smithi062.stdout:5/572: rmdir d0/d3/da/d32/d4f/d72 39 2024-02-22T15:50:47.167 INFO:tasks.workunit.client.1.smithi062.stdout:2/803: fdatasync d1/d8/d62/d73/dbd/d7d/ffb 0 2024-02-22T15:50:47.169 INFO:tasks.workunit.client.0.smithi019.stdout:4/863: rename d2/db/dea/ded/lee to d2/db/d14/d23/d67/d8f/l115 0 2024-02-22T15:50:47.169 INFO:tasks.workunit.client.0.smithi019.stdout:1/977: unlink d8/d23/lc5 0 2024-02-22T15:50:47.170 INFO:tasks.workunit.client.1.smithi062.stdout:4/766: unlink d4/d1c/d2c/d6b/cb7 0 2024-02-22T15:50:47.171 INFO:tasks.workunit.client.1.smithi062.stdout:5/573: symlink d0/d3/da/d13/d23/d78/lb2 0 2024-02-22T15:50:47.171 INFO:tasks.workunit.client.0.smithi019.stdout:2/966: dwrite d1/d17/d21/d71/d60/fba [0,4194304] 0 2024-02-22T15:50:47.172 INFO:tasks.workunit.client.1.smithi062.stdout:4/767: write d4/d2a/d2d/d2e/d6a/d9e/fc3 [460507,15954] 0 2024-02-22T15:50:47.174 INFO:tasks.workunit.client.1.smithi062.stdout:5/574: write d0/f7 [8164260,69679] 0 2024-02-22T15:50:47.174 INFO:tasks.workunit.client.0.smithi019.stdout:3/786: chown d9/d26/d3f/d59/d66/fb3 55 1 2024-02-22T15:50:47.175 INFO:tasks.workunit.client.0.smithi019.stdout:5/946: creat d8/dd/d19/d77/dd2/dd9/f13e x:0 0 0 2024-02-22T15:50:47.176 INFO:tasks.workunit.client.1.smithi062.stdout:4/768: write d4/f3f [4089814,21284] 0 2024-02-22T15:50:47.177 INFO:tasks.workunit.client.0.smithi019.stdout:9/901: rename d2/d1f/d2e/d57 to d2/d7/dc/d27/dbc/d110/d138 0 2024-02-22T15:50:47.177 INFO:tasks.workunit.client.0.smithi019.stdout:2/967: chown d1/d17/d21/d51/fc2 255914 1 2024-02-22T15:50:47.177 INFO:tasks.workunit.client.1.smithi062.stdout:2/804: dread d1/d1e/f66 [0,4194304] 0 2024-02-22T15:50:47.180 INFO:tasks.workunit.client.0.smithi019.stdout:4/864: stat d2/d3f/d48/de9 0 2024-02-22T15:50:47.182 INFO:tasks.workunit.client.0.smithi019.stdout:3/787: creat d9/d22/d27/d96/f101 x:0 0 0 2024-02-22T15:50:47.182 INFO:tasks.workunit.client.1.smithi062.stdout:2/805: readlink d1/d47/d32/d52/d80/la4 0 2024-02-22T15:50:47.183 INFO:tasks.workunit.client.0.smithi019.stdout:5/947: dread - d8/d11/d33/fe2 zero size 2024-02-22T15:50:47.183 INFO:tasks.workunit.client.0.smithi019.stdout:2/968: creat d1/f154 x:0 0 0 2024-02-22T15:50:47.185 INFO:tasks.workunit.client.1.smithi062.stdout:4/769: rename d4/d2a/d2d/c8c to d4/d2a/d2d/d2e/d6a/c10f 0 2024-02-22T15:50:47.185 INFO:tasks.workunit.client.1.smithi062.stdout:2/806: write d1/d8/df/f31 [3939671,35637] 0 2024-02-22T15:50:47.188 INFO:tasks.workunit.client.0.smithi019.stdout:1/978: dwrite d8/d18/ffb [0,4194304] 0 2024-02-22T15:50:47.188 INFO:tasks.workunit.client.0.smithi019.stdout:8/839: write d9/d14/d6f/d7e/d94/d9a/fba [134109,65729] 0 2024-02-22T15:50:47.189 INFO:tasks.workunit.client.1.smithi062.stdout:4/770: write d4/d1c/d2c/d6b/d70/f93 [1363505,71083] 0 2024-02-22T15:50:47.189 INFO:tasks.workunit.client.0.smithi019.stdout:0/867: truncate d4/d9/f26 4626471 0 2024-02-22T15:50:47.190 INFO:tasks.workunit.client.0.smithi019.stdout:8/840: chown d9/d14/d1f/d5c 141042667 1 2024-02-22T15:50:47.191 INFO:tasks.workunit.client.0.smithi019.stdout:3/788: unlink d9/d14/f3c 0 2024-02-22T15:50:47.192 INFO:tasks.workunit.client.1.smithi062.stdout:5/575: dwrite d0/d3/da/d13/d57/d79/fa5 [0,4194304] 0 2024-02-22T15:50:47.193 INFO:tasks.workunit.client.0.smithi019.stdout:2/969: symlink d1/d86/l155 0 2024-02-22T15:50:47.194 INFO:tasks.workunit.client.0.smithi019.stdout:8/841: stat d9/d14/d1f/d5c/c67 0 2024-02-22T15:50:47.196 INFO:tasks.workunit.client.1.smithi062.stdout:4/771: mknod d4/d66/c110 0 2024-02-22T15:50:47.197 INFO:tasks.workunit.client.0.smithi019.stdout:4/865: mkdir d2/db/d116 0 2024-02-22T15:50:47.198 INFO:tasks.workunit.client.0.smithi019.stdout:1/979: dread - d8/d2a/d75/d81/d8e/f138 zero size 2024-02-22T15:50:47.200 INFO:tasks.workunit.client.0.smithi019.stdout:9/902: dwrite d2/d7/dc/d27/dbc/dcc/f124 [0,4194304] 0 2024-02-22T15:50:47.204 INFO:tasks.workunit.client.0.smithi019.stdout:9/903: chown d2/d1f/d2e/d48/d36/f83 0 1 2024-02-22T15:50:47.204 INFO:tasks.workunit.client.0.smithi019.stdout:2/970: truncate d1/de/d9a/f113 5012651 0 2024-02-22T15:50:47.204 INFO:tasks.workunit.client.0.smithi019.stdout:5/948: mknod d8/dd/d8e/d2c/d10e/c13f 0 2024-02-22T15:50:47.205 INFO:tasks.workunit.client.1.smithi062.stdout:2/807: creat d1/d8/d62/d73/dbd/d7d/f111 x:0 0 0 2024-02-22T15:50:47.206 INFO:tasks.workunit.client.1.smithi062.stdout:2/808: readlink d1/d8/d62/dcb/l103 0 2024-02-22T15:50:47.207 INFO:tasks.workunit.client.0.smithi019.stdout:8/842: fsync d9/d14/d6f/d7e/d94/d9a/ff1 0 2024-02-22T15:50:47.207 INFO:tasks.workunit.client.0.smithi019.stdout:2/971: write d1/d17/d21/d71/d60/fba [2046103,13304] 0 2024-02-22T15:50:47.208 INFO:tasks.workunit.client.0.smithi019.stdout:3/789: creat d9/d26/d32/f102 x:0 0 0 2024-02-22T15:50:47.209 INFO:tasks.workunit.client.0.smithi019.stdout:8/843: chown d9/d14/f38 177 1 2024-02-22T15:50:47.210 INFO:tasks.workunit.client.0.smithi019.stdout:9/904: truncate d2/d7/dc/f49 4607854 0 2024-02-22T15:50:47.212 INFO:tasks.workunit.client.0.smithi019.stdout:3/790: fdatasync d9/d26/d2d/d53/f56 0 2024-02-22T15:50:47.213 INFO:tasks.workunit.client.0.smithi019.stdout:5/949: mknod d8/dd/d8e/db3/c140 0 2024-02-22T15:50:47.220 INFO:tasks.workunit.client.0.smithi019.stdout:5/950: mkdir d8/dd/d19/d3a/dc7/d141 0 2024-02-22T15:50:47.221 INFO:tasks.workunit.client.1.smithi062.stdout:4/772: rmdir d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc 39 2024-02-22T15:50:47.222 INFO:tasks.workunit.client.1.smithi062.stdout:5/576: truncate d0/d3/d30/f46 3828116 0 2024-02-22T15:50:47.223 INFO:tasks.workunit.client.0.smithi019.stdout:9/905: dwrite d2/dc8/f11f [0,4194304] 0 2024-02-22T15:50:47.226 INFO:tasks.workunit.client.1.smithi062.stdout:2/809: write d1/d8/df/f61 [649470,111376] 0 2024-02-22T15:50:47.227 INFO:tasks.workunit.client.0.smithi019.stdout:3/791: mkdir d9/d26/d2d/d103 0 2024-02-22T15:50:47.227 INFO:tasks.workunit.client.0.smithi019.stdout:2/972: mknod d1/d17/d1c/d128/db9/dd0/dee/d120/c156 0 2024-02-22T15:50:47.230 INFO:tasks.workunit.client.0.smithi019.stdout:8/844: creat d9/f119 x:0 0 0 2024-02-22T15:50:47.230 INFO:tasks.workunit.client.0.smithi019.stdout:2/973: dread - d1/d17/d1c/d7f/fa2 zero size 2024-02-22T15:50:47.231 INFO:tasks.workunit.client.1.smithi062.stdout:2/810: write d1/d1e/d38/d55/f97 [4005279,88607] 0 2024-02-22T15:50:47.231 INFO:tasks.workunit.client.0.smithi019.stdout:3/792: mkdir d9/d22/d104 0 2024-02-22T15:50:47.233 INFO:tasks.workunit.client.1.smithi062.stdout:2/811: fsync d1/d8/d62/d73/da6/fb1 0 2024-02-22T15:50:47.235 INFO:tasks.workunit.client.1.smithi062.stdout:4/773: truncate d4/df/fda 497832 0 2024-02-22T15:50:47.237 INFO:tasks.workunit.client.0.smithi019.stdout:4/866: truncate d2/fd 2788631 0 2024-02-22T15:50:47.237 INFO:tasks.workunit.client.0.smithi019.stdout:1/980: truncate d8/d2a/d75/d81/d8e/f118 858919 0 2024-02-22T15:50:47.237 INFO:tasks.workunit.client.1.smithi062.stdout:2/812: mkdir d1/d1e/dda/de8/d112 0 2024-02-22T15:50:47.239 INFO:tasks.workunit.client.0.smithi019.stdout:3/793: creat d9/d26/d3f/d5c/f105 x:0 0 0 2024-02-22T15:50:47.239 INFO:tasks.workunit.client.1.smithi062.stdout:2/813: unlink d1/d8/d8f/f96 0 2024-02-22T15:50:47.240 INFO:tasks.workunit.client.1.smithi062.stdout:4/774: symlink d4/d2a/d2d/d2e/d6a/d9e/da4/db6/l111 0 2024-02-22T15:50:47.241 INFO:tasks.workunit.client.0.smithi019.stdout:1/981: chown d8/d15/l3d 48 1 2024-02-22T15:50:47.242 INFO:tasks.workunit.client.0.smithi019.stdout:9/906: dwrite d2/d1f/d2e/d48/d36/ff1 [0,4194304] 0 2024-02-22T15:50:47.242 INFO:tasks.workunit.client.1.smithi062.stdout:5/577: write d0/d3/d30/f4a [1073581,110363] 0 2024-02-22T15:50:47.244 INFO:tasks.workunit.client.1.smithi062.stdout:2/814: creat d1/d1e/d38/f113 x:0 0 0 2024-02-22T15:50:47.244 INFO:tasks.workunit.client.0.smithi019.stdout:8/845: symlink d9/d14/d1f/d6d/db5/l11a 0 2024-02-22T15:50:47.245 INFO:tasks.workunit.client.0.smithi019.stdout:5/951: write d8/dd/d19/d3a/d46/d7e/fcc [901170,62886] 0 2024-02-22T15:50:47.249 INFO:tasks.workunit.client.0.smithi019.stdout:3/794: write d9/d22/d27/f46 [3150178,130926] 0 2024-02-22T15:50:47.249 INFO:tasks.workunit.client.1.smithi062.stdout:4/775: dwrite d4/f3f [0,4194304] 0 2024-02-22T15:50:47.249 INFO:tasks.workunit.client.1.smithi062.stdout:5/578: mkdir d0/d3/d2f/d59/d67/d69/d3f/d66/db3 0 2024-02-22T15:50:47.251 INFO:tasks.workunit.client.1.smithi062.stdout:2/815: rename d1/d47/d39/l64 to d1/d28/l114 0 2024-02-22T15:50:47.251 INFO:tasks.workunit.client.0.smithi019.stdout:2/974: creat d1/d17/d1c/f157 x:0 0 0 2024-02-22T15:50:47.252 INFO:tasks.workunit.client.1.smithi062.stdout:5/579: stat d0/d3/c2b 0 2024-02-22T15:50:47.255 INFO:tasks.workunit.client.1.smithi062.stdout:5/580: stat d0/d3/da/d32/d4f/c9c 0 2024-02-22T15:50:47.255 INFO:tasks.workunit.client.0.smithi019.stdout:5/952: write d8/d11/d33/fc4 [2452474,18575] 0 2024-02-22T15:50:47.255 INFO:tasks.workunit.client.0.smithi019.stdout:4/867: dwrite d2/db/d44/f114 [0,4194304] 0 2024-02-22T15:50:47.259 INFO:tasks.workunit.client.0.smithi019.stdout:3/795: readlink d9/d26/d2d/d53/da0/lac 0 2024-02-22T15:50:47.261 INFO:tasks.workunit.client.1.smithi062.stdout:5/581: rmdir d0/d3/da/d13/d57/d79 39 2024-02-22T15:50:47.262 INFO:tasks.workunit.client.1.smithi062.stdout:4/776: link d4/la d4/d34/l112 0 2024-02-22T15:50:47.264 INFO:tasks.workunit.client.1.smithi062.stdout:4/777: fsync d4/d1c/f23 0 2024-02-22T15:50:47.265 INFO:tasks.workunit.client.1.smithi062.stdout:5/582: creat d0/d3/da/d13/d23/d78/d81/fb4 x:0 0 0 2024-02-22T15:50:47.267 INFO:tasks.workunit.client.1.smithi062.stdout:2/816: dread d1/d47/f48 [0,4194304] 0 2024-02-22T15:50:47.270 INFO:tasks.workunit.client.1.smithi062.stdout:4/778: truncate d4/d2a/d2d/d2e/d6a/d8a/ddb/f10d 491370 0 2024-02-22T15:50:47.271 INFO:tasks.workunit.client.1.smithi062.stdout:5/583: mkdir d0/d3/da/d13/d23/d4e/d60/db5 0 2024-02-22T15:50:47.271 INFO:tasks.workunit.client.1.smithi062.stdout:4/779: dread - d4/d1c/d2c/f101 zero size 2024-02-22T15:50:47.273 INFO:tasks.workunit.client.1.smithi062.stdout:2/817: dread d1/d8/fa [4194304,4194304] 0 2024-02-22T15:50:47.277 INFO:tasks.workunit.client.1.smithi062.stdout:2/818: dread d1/d47/d39/fac [0,4194304] 0 2024-02-22T15:50:47.278 INFO:tasks.workunit.client.1.smithi062.stdout:4/780: rename d4/d2a/d2d/d64/ca9 to d4/d2a/d2d/d46/c113 0 2024-02-22T15:50:47.280 INFO:tasks.workunit.client.1.smithi062.stdout:2/819: fdatasync d1/d47/f48 0 2024-02-22T15:50:47.282 INFO:tasks.workunit.client.0.smithi019.stdout:5/953: fdatasync d8/dd/f5b 0 2024-02-22T15:50:47.283 INFO:tasks.workunit.client.1.smithi062.stdout:2/820: chown d1/d47/d32/fd0 32341204 1 2024-02-22T15:50:47.285 INFO:tasks.workunit.client.0.smithi019.stdout:2/975: rename d1/d17/d21/d71/d60/c7d to d1/da4/d132/d14a/c158 0 2024-02-22T15:50:47.288 INFO:tasks.workunit.client.1.smithi062.stdout:4/781: dwrite d4/d2a/d2d/d46/fad [0,4194304] 0 2024-02-22T15:50:47.289 INFO:tasks.workunit.client.0.smithi019.stdout:1/982: dwrite d8/d2a/d116/f13f [0,4194304] 0 2024-02-22T15:50:47.289 INFO:tasks.workunit.client.0.smithi019.stdout:2/976: write d1/d46/f95 [2374705,124641] 0 2024-02-22T15:50:47.290 INFO:tasks.workunit.client.1.smithi062.stdout:2/821: rename d1/d8/df/ce4 to d1/d8/d62/d73/dbd/d7d/d84/c115 0 2024-02-22T15:50:47.292 INFO:tasks.workunit.client.0.smithi019.stdout:1/983: readlink d8/d2a/d75/ld0 0 2024-02-22T15:50:47.294 INFO:tasks.workunit.client.1.smithi062.stdout:5/584: truncate d0/d3/ff 3623357 0 2024-02-22T15:50:47.295 INFO:tasks.workunit.client.1.smithi062.stdout:2/822: mkdir d1/d1e/d38/d76/dc5/dec/d116 0 2024-02-22T15:50:47.296 INFO:tasks.workunit.client.0.smithi019.stdout:1/984: write d8/d18/d64/f83 [1345754,30293] 0 2024-02-22T15:50:47.299 INFO:tasks.workunit.client.1.smithi062.stdout:2/823: mknod d1/de1/c117 0 2024-02-22T15:50:47.300 INFO:tasks.workunit.client.1.smithi062.stdout:5/585: symlink d0/d3/da/d13/d23/d78/d8c/lb6 0 2024-02-22T15:50:47.300 INFO:tasks.workunit.client.1.smithi062.stdout:2/824: chown d1/d1e/d38/d76/dc5/dec 11 1 2024-02-22T15:50:47.301 INFO:tasks.workunit.client.0.smithi019.stdout:2/977: unlink d1/d17/d21/d71/d60/l152 0 2024-02-22T15:50:47.301 INFO:tasks.workunit.client.0.smithi019.stdout:3/796: write d9/d26/fb5 [1028429,65201] 0 2024-02-22T15:50:47.301 INFO:tasks.workunit.client.0.smithi019.stdout:4/868: write d2/db/fdc [4694480,52721] 0 2024-02-22T15:50:47.303 INFO:tasks.workunit.client.1.smithi062.stdout:5/586: rename d0/d3/da/d13/d23/c64 to d0/d3/cb7 0 2024-02-22T15:50:47.304 INFO:tasks.workunit.client.0.smithi019.stdout:4/869: chown d2/d3f/d48/d76/cb8 1133 1 2024-02-22T15:50:47.304 INFO:tasks.workunit.client.0.smithi019.stdout:2/978: stat d1/d17/d21/d51/l6b 0 2024-02-22T15:50:47.305 INFO:tasks.workunit.client.0.smithi019.stdout:1/985: mknod d8/d2a/d97/d13a/c14e 0 2024-02-22T15:50:47.306 INFO:tasks.workunit.client.0.smithi019.stdout:9/907: dread d2/d7/f1b [0,4194304] 0 2024-02-22T15:50:47.306 INFO:tasks.workunit.client.0.smithi019.stdout:3/797: dread - d9/dc8/fed zero size 2024-02-22T15:50:47.308 INFO:tasks.workunit.client.1.smithi062.stdout:2/825: creat d1/d8/d62/d73/dbd/f118 x:0 0 0 2024-02-22T15:50:47.308 INFO:tasks.workunit.client.1.smithi062.stdout:5/587: mkdir d0/d3/d2f/d59/d91/db8 0 2024-02-22T15:50:47.310 INFO:tasks.workunit.client.1.smithi062.stdout:5/588: stat d0/d3/d2f/d59/d67/d69/d3f/f5c 0 2024-02-22T15:50:47.311 INFO:tasks.workunit.client.0.smithi019.stdout:8/846: dread d9/d14/d17/d1c/f9d [0,4194304] 0 2024-02-22T15:50:47.312 INFO:tasks.workunit.client.0.smithi019.stdout:9/908: rmdir d2/dc8 39 2024-02-22T15:50:47.312 INFO:tasks.workunit.client.1.smithi062.stdout:4/782: dwrite d4/d1c/d2c/f81 [0,4194304] 0 2024-02-22T15:50:47.312 INFO:tasks.workunit.client.0.smithi019.stdout:3/798: readlink d9/d11/l10 0 2024-02-22T15:50:47.313 INFO:tasks.workunit.client.1.smithi062.stdout:5/589: creat d0/d3/fb9 x:0 0 0 2024-02-22T15:50:47.314 INFO:tasks.workunit.client.0.smithi019.stdout:1/986: symlink d8/d2a/d75/d81/l14f 0 2024-02-22T15:50:47.314 INFO:tasks.workunit.client.1.smithi062.stdout:4/783: mknod d4/d1c/d30/c114 0 2024-02-22T15:50:47.316 INFO:tasks.workunit.client.1.smithi062.stdout:4/784: mknod d4/d66/dfa/c115 0 2024-02-22T15:50:47.318 INFO:tasks.workunit.client.1.smithi062.stdout:5/590: creat d0/d3/da/d13/d23/d31/fba x:0 0 0 2024-02-22T15:50:47.321 INFO:tasks.workunit.client.0.smithi019.stdout:4/870: truncate d2/db/d14/d23/d21/d35/d3d/fe0 416563 0 2024-02-22T15:50:47.321 INFO:tasks.workunit.client.1.smithi062.stdout:5/591: creat d0/d3/d30/fbb x:0 0 0 2024-02-22T15:50:47.324 INFO:tasks.workunit.client.1.smithi062.stdout:5/592: write d0/d3/f83 [675837,23415] 0 2024-02-22T15:50:47.324 INFO:tasks.workunit.client.0.smithi019.stdout:2/979: dwrite d1/de/f5a [4194304,4194304] 0 2024-02-22T15:50:47.326 INFO:tasks.workunit.client.0.smithi019.stdout:5/954: dread d8/dd/d19/d38/d54/f13a [0,4194304] 0 2024-02-22T15:50:47.327 INFO:tasks.workunit.client.1.smithi062.stdout:4/785: dwrite d4/d1c/f3d [4194304,4194304] 0 2024-02-22T15:50:47.327 INFO:tasks.workunit.client.0.smithi019.stdout:3/799: rename d9/dc8 to d9/d65/d106 0 2024-02-22T15:50:47.328 INFO:tasks.workunit.client.0.smithi019.stdout:2/980: write d1/d46/ddb/d10d/d12c/d13c/f14f [578695,82434] 0 2024-02-22T15:50:47.328 INFO:tasks.workunit.client.0.smithi019.stdout:1/987: symlink d8/d18/d64/l150 0 2024-02-22T15:50:47.328 INFO:tasks.workunit.client.1.smithi062.stdout:5/593: mkdir d0/d3/da/d32/d33/dbc 0 2024-02-22T15:50:47.331 INFO:tasks.workunit.client.1.smithi062.stdout:4/786: readlink d4/d2a/d2d/d2e/d6a/d9e/lfd 0 2024-02-22T15:50:47.335 INFO:tasks.workunit.client.1.smithi062.stdout:5/594: mkdir d0/d3/da/d13/d23/d78/d8c/dbd 0 2024-02-22T15:50:47.339 INFO:tasks.workunit.client.1.smithi062.stdout:5/595: symlink d0/d3/d2f/d59/d67/d69/d3f/d66/da9/lbe 0 2024-02-22T15:50:47.339 INFO:tasks.workunit.client.0.smithi019.stdout:3/800: mkdir d9/d16/d107 0 2024-02-22T15:50:47.339 INFO:tasks.workunit.client.0.smithi019.stdout:4/871: readlink d2/db/d54/d8c/lcd 0 2024-02-22T15:50:47.340 INFO:tasks.workunit.client.0.smithi019.stdout:1/988: rename d8/d15/d5b/dbc/f12a to d8/d18/d85/f151 0 2024-02-22T15:50:47.343 INFO:tasks.workunit.client.1.smithi062.stdout:5/596: rmdir d0/d3/d2f/d59/d67/d69/d3f/d66/db3 0 2024-02-22T15:50:47.343 INFO:tasks.workunit.client.1.smithi062.stdout:5/597: write d0/d3/f94 [367721,32673] 0 2024-02-22T15:50:47.345 INFO:tasks.workunit.client.0.smithi019.stdout:5/955: truncate d8/dd/d19/d38/d54/db0/d10d/d122/dde/deb/fd4 606727 0 2024-02-22T15:50:47.346 INFO:tasks.workunit.client.0.smithi019.stdout:2/981: dwrite d1/d46/f95 [0,4194304] 0 2024-02-22T15:50:47.346 INFO:tasks.workunit.client.0.smithi019.stdout:5/956: stat d8/dd/d19/d3a/l135 0 2024-02-22T15:50:47.349 INFO:tasks.workunit.client.0.smithi019.stdout:2/982: dread - d1/d17/d67/dfa/d11e/f13d zero size 2024-02-22T15:50:47.351 INFO:tasks.workunit.client.1.smithi062.stdout:5/598: dwrite d0/f16 [8388608,4194304] 0 2024-02-22T15:50:47.358 INFO:tasks.workunit.client.0.smithi019.stdout:3/801: rename d9/d22/d27/dd0/fda to d9/d26/d76/f108 0 2024-02-22T15:50:47.358 INFO:tasks.workunit.client.1.smithi062.stdout:5/599: creat d0/d3/da/d13/d57/fbf x:0 0 0 2024-02-22T15:50:47.359 INFO:tasks.workunit.client.1.smithi062.stdout:5/600: readlink d0/d3/d2f/d59/d67/d69/d3f/d66/l6e 0 2024-02-22T15:50:47.360 INFO:tasks.workunit.client.1.smithi062.stdout:4/787: truncate d4/df/f14 3844366 0 2024-02-22T15:50:47.361 INFO:tasks.workunit.client.0.smithi019.stdout:2/983: symlink d1/d46/ddb/d10d/d12c/l159 0 2024-02-22T15:50:47.363 INFO:tasks.workunit.client.0.smithi019.stdout:3/802: rmdir d9/d26/d2d/d39 39 2024-02-22T15:50:47.363 INFO:tasks.workunit.client.1.smithi062.stdout:9/692: sync 2024-02-22T15:50:47.363 INFO:tasks.workunit.client.1.smithi062.stdout:7/564: sync 2024-02-22T15:50:47.363 INFO:tasks.workunit.client.1.smithi062.stdout:5/601: mknod d0/d3/da/d13/d23/cc0 0 2024-02-22T15:50:47.365 INFO:tasks.workunit.client.0.smithi019.stdout:1/989: dread d8/d2a/f78 [0,4194304] 0 2024-02-22T15:50:47.365 INFO:tasks.workunit.client.1.smithi062.stdout:4/788: truncate d4/d1c/d2c/d6b/d70/de8/dfc/fff 711814 0 2024-02-22T15:50:47.366 INFO:tasks.workunit.client.1.smithi062.stdout:5/602: dread - d0/d3/da/d13/fb1 zero size 2024-02-22T15:50:47.366 INFO:tasks.workunit.client.0.smithi019.stdout:9/909: truncate d2/d1f/db5/fde 2243338 0 2024-02-22T15:50:47.367 INFO:tasks.workunit.client.0.smithi019.stdout:2/984: symlink d1/d17/d67/l15a 0 2024-02-22T15:50:47.369 INFO:tasks.workunit.client.0.smithi019.stdout:1/990: fdatasync d8/d15/d34/d9e/d126/f107 0 2024-02-22T15:50:47.369 INFO:tasks.workunit.client.0.smithi019.stdout:4/872: write d2/d3f/d48/f5b [1939020,7322] 0 2024-02-22T15:50:47.369 INFO:tasks.workunit.client.0.smithi019.stdout:5/957: rmdir d8 39 2024-02-22T15:50:47.370 INFO:tasks.workunit.client.1.smithi062.stdout:5/603: dread - d0/d3/d2f/d59/d67/f95 zero size 2024-02-22T15:50:47.370 INFO:tasks.workunit.client.0.smithi019.stdout:2/985: rmdir d1/d17 39 2024-02-22T15:50:47.371 INFO:tasks.workunit.client.1.smithi062.stdout:4/789: creat d4/d2a/d2d/f116 x:0 0 0 2024-02-22T15:50:47.373 INFO:tasks.workunit.client.0.smithi019.stdout:4/873: dread - d2/d3f/ffa zero size 2024-02-22T15:50:47.373 INFO:tasks.workunit.client.0.smithi019.stdout:2/986: stat d1/da4/fe6 0 2024-02-22T15:50:47.373 INFO:tasks.workunit.client.0.smithi019.stdout:9/910: creat d2/d1f/d2e/d48/d45/d5a/f139 x:0 0 0 2024-02-22T15:50:47.374 INFO:tasks.workunit.client.1.smithi062.stdout:9/693: dwrite d4/d28/f72 [4194304,4194304] 0 2024-02-22T15:50:47.375 INFO:tasks.workunit.client.1.smithi062.stdout:9/694: readlink d4/l91 0 2024-02-22T15:50:47.376 INFO:tasks.workunit.client.1.smithi062.stdout:4/790: mknod d4/d2a/d2d/d2e/d6a/d8a/c117 0 2024-02-22T15:50:47.377 INFO:tasks.workunit.client.0.smithi019.stdout:3/803: dwrite d9/d65/d106/fed [0,4194304] 0 2024-02-22T15:50:47.377 INFO:tasks.workunit.client.0.smithi019.stdout:2/987: mkdir d1/d46/d15b 0 2024-02-22T15:50:47.378 INFO:tasks.workunit.client.1.smithi062.stdout:9/695: dread - d4/d15/d5f/f9b zero size 2024-02-22T15:50:47.381 INFO:tasks.workunit.client.1.smithi062.stdout:9/696: read - d4/d15/d31/fa5 zero size 2024-02-22T15:50:47.382 INFO:tasks.workunit.client.0.smithi019.stdout:4/874: readlink d2/db/d14/d23/d9e/dac/ld9 0 2024-02-22T15:50:47.382 INFO:tasks.workunit.client.1.smithi062.stdout:7/565: write d0/d91/f59 [2589956,18051] 0 2024-02-22T15:50:47.386 INFO:tasks.workunit.client.0.smithi019.stdout:9/911: creat d2/d1f/d2e/f13a x:0 0 0 2024-02-22T15:50:47.386 INFO:tasks.workunit.client.0.smithi019.stdout:2/988: fsync d1/d86/fe8 0 2024-02-22T15:50:47.387 INFO:tasks.workunit.client.0.smithi019.stdout:8/847: dwrite d9/d14/d6f/d7e/d94/db8/fa1 [0,4194304] 0 2024-02-22T15:50:47.388 INFO:tasks.workunit.client.1.smithi062.stdout:5/604: dwrite d0/d3/da/d32/d4f/d72/f9a [0,4194304] 0 2024-02-22T15:50:47.388 INFO:tasks.workunit.client.1.smithi062.stdout:7/566: readlink d0/d91/l9f 0 2024-02-22T15:50:47.388 INFO:tasks.workunit.client.1.smithi062.stdout:9/697: rmdir d4/d15/d5e/d63/da0/d92 39 2024-02-22T15:50:47.388 INFO:tasks.workunit.client.1.smithi062.stdout:7/567: chown d0/d14/d19/d30/d37/la6 16 1 2024-02-22T15:50:47.391 INFO:tasks.workunit.client.0.smithi019.stdout:1/991: link d8/d2a/dde/f130 d8/d23/f152 0 2024-02-22T15:50:47.398 INFO:tasks.workunit.client.1.smithi062.stdout:5/605: dread d0/d3/d30/f4a [0,4194304] 0 2024-02-22T15:50:47.400 INFO:tasks.workunit.client.1.smithi062.stdout:4/791: dwrite d4/d1c/d30/f65 [0,4194304] 0 2024-02-22T15:50:47.401 INFO:tasks.workunit.client.0.smithi019.stdout:3/804: rmdir d9/d65/dd6 0 2024-02-22T15:50:47.401 INFO:tasks.workunit.client.1.smithi062.stdout:7/568: fsync d0/d6/d2e/d55/f81 0 2024-02-22T15:50:47.403 INFO:tasks.workunit.client.1.smithi062.stdout:5/606: fdatasync d0/d3/d2f/f3a 0 2024-02-22T15:50:47.404 INFO:tasks.workunit.client.1.smithi062.stdout:4/792: fsync d4/d2a/fd1 0 2024-02-22T15:50:47.406 INFO:tasks.workunit.client.0.smithi019.stdout:9/912: read d2/d7/f44 [3920870,42350] 0 2024-02-22T15:50:47.406 INFO:tasks.workunit.client.0.smithi019.stdout:4/875: dwrite d2/db/d14/d23/d21/dbf/fdd [0,4194304] 0 2024-02-22T15:50:47.406 INFO:tasks.workunit.client.1.smithi062.stdout:5/607: write d0/f26 [694915,114992] 0 2024-02-22T15:50:47.407 INFO:tasks.workunit.client.0.smithi019.stdout:9/913: write d2/d1f/f12c [575653,113571] 0 2024-02-22T15:50:47.408 INFO:tasks.workunit.client.0.smithi019.stdout:8/848: creat d9/d14/d6f/d7e/d94/d9c/dc6/f11b x:0 0 0 2024-02-22T15:50:47.411 INFO:tasks.workunit.client.0.smithi019.stdout:8/849: stat d9/d14/d6f/d7e/d94/daa/dea/f100 0 2024-02-22T15:50:47.412 INFO:tasks.workunit.client.0.smithi019.stdout:2/989: getdents d1/d86/d11f 0 2024-02-22T15:50:47.413 INFO:tasks.workunit.client.0.smithi019.stdout:9/914: truncate d2/d1f/d2e/d48/d36/f115 233404 0 2024-02-22T15:50:47.414 INFO:tasks.workunit.client.0.smithi019.stdout:1/992: rename d8/d23/d33/l62 to d8/d2a/d75/d136/l153 0 2024-02-22T15:50:47.417 INFO:tasks.workunit.client.1.smithi062.stdout:5/608: dwrite d0/fa8 [0,4194304] 0 2024-02-22T15:50:47.419 INFO:tasks.workunit.client.1.smithi062.stdout:4/793: dwrite d4/d2a/d2d/d2e/d6a/d9e/fc3 [0,4194304] 0 2024-02-22T15:50:47.420 INFO:tasks.workunit.client.1.smithi062.stdout:5/609: chown d0/d3/da/d13/l29 721680 1 2024-02-22T15:50:47.420 INFO:tasks.workunit.client.0.smithi019.stdout:2/990: read d1/de/f4e [2076924,70042] 0 2024-02-22T15:50:47.422 INFO:tasks.workunit.client.0.smithi019.stdout:9/915: fdatasync d2/d7/dc/d27/dbc/d110/d138/d60/f133 0 2024-02-22T15:50:47.424 INFO:tasks.workunit.client.1.smithi062.stdout:4/794: unlink d4/d2a/d2d/d2e/d6a/d9e/da4/db6/l111 0 2024-02-22T15:50:47.426 INFO:tasks.workunit.client.1.smithi062.stdout:4/795: readlink d4/d2a/d2d/d2e/d6a/d9e/lfd 0 2024-02-22T15:50:47.428 INFO:tasks.workunit.client.1.smithi062.stdout:5/610: dwrite d0/fa8 [0,4194304] 0 2024-02-22T15:50:47.430 INFO:tasks.workunit.client.0.smithi019.stdout:4/876: dwrite d2/db/fd1 [0,4194304] 0 2024-02-22T15:50:47.432 INFO:tasks.workunit.client.1.smithi062.stdout:4/796: rename d4/d2a/d2d/d2e/d6a/d8a/f9a to d4/d1c/d2c/f118 0 2024-02-22T15:50:47.434 INFO:tasks.workunit.client.1.smithi062.stdout:5/611: mknod d0/d3/da/d13/d23/d31/dac/cc1 0 2024-02-22T15:50:47.439 INFO:tasks.workunit.client.1.smithi062.stdout:5/612: rename d0/d3/da/d13/d23/d31/f43 to d0/d3/d2f/d59/fc2 0 2024-02-22T15:50:47.442 INFO:tasks.workunit.client.1.smithi062.stdout:4/797: link d4/d2a/d2d/d2e/d6a/c10f d4/d2a/d75/c119 0 2024-02-22T15:50:47.443 INFO:tasks.workunit.client.0.smithi019.stdout:2/991: getdents d1/d17/d1c 0 2024-02-22T15:50:47.445 INFO:tasks.workunit.client.0.smithi019.stdout:2/992: stat d1/de/lbb 0 2024-02-22T15:50:47.446 INFO:tasks.workunit.client.1.smithi062.stdout:5/613: rename d0/d3/d30/d71/ca1 to d0/d3/d2f/d59/d67/d69/d3f/d66/cc3 0 2024-02-22T15:50:47.446 INFO:tasks.workunit.client.1.smithi062.stdout:4/798: readlink d4/d2a/d2d/d2e/d6a/ldf 0 2024-02-22T15:50:47.448 INFO:tasks.workunit.client.0.smithi019.stdout:4/877: mkdir d2/d3f/d48/de9/d117 0 2024-02-22T15:50:47.448 INFO:tasks.workunit.client.1.smithi062.stdout:4/799: creat d4/d66/f11a x:0 0 0 2024-02-22T15:50:47.449 INFO:tasks.workunit.client.0.smithi019.stdout:8/850: dwrite d9/d14/d6f/d7e/d94/db8/f10c [0,4194304] 0 2024-02-22T15:50:47.450 INFO:tasks.workunit.client.1.smithi062.stdout:5/614: creat d0/d3/da/d32/d33/dbc/fc4 x:0 0 0 2024-02-22T15:50:47.451 INFO:tasks.workunit.client.0.smithi019.stdout:3/805: dwrite d9/d22/d27/f7a [0,4194304] 0 2024-02-22T15:50:47.454 INFO:tasks.workunit.client.0.smithi019.stdout:2/993: mkdir d1/d17/d1c/d128/db9/dd0/dee/d15c 0 2024-02-22T15:50:47.454 INFO:tasks.workunit.client.0.smithi019.stdout:9/916: dread d2/d7/dc/d27/dbc/d110/d138/d60/f133 [0,4194304] 0 2024-02-22T15:50:47.455 INFO:tasks.workunit.client.1.smithi062.stdout:5/615: chown d0/d3/da/d32/d42 1230 1 2024-02-22T15:50:47.455 INFO:tasks.workunit.client.0.smithi019.stdout:8/851: fsync d9/d14/d1f/fa5 0 2024-02-22T15:50:47.458 INFO:tasks.workunit.client.1.smithi062.stdout:9/698: dwrite d4/f10 [0,4194304] 0 2024-02-22T15:50:47.461 INFO:tasks.workunit.client.1.smithi062.stdout:5/616: creat d0/d3/da/d13/d23/d4e/d60/db5/fc5 x:0 0 0 2024-02-22T15:50:47.465 INFO:tasks.workunit.client.1.smithi062.stdout:7/569: dwrite d0/d6/d2e/d4a/f1f [4194304,4194304] 0 2024-02-22T15:50:47.469 INFO:tasks.workunit.client.1.smithi062.stdout:9/699: read d4/d28/d50/f52 [6310,45186] 0 2024-02-22T15:50:47.471 INFO:tasks.workunit.client.1.smithi062.stdout:9/700: readlink d4/d15/l16 0 2024-02-22T15:50:47.472 INFO:tasks.workunit.client.1.smithi062.stdout:7/570: write d0/d6/d2e/d35/f70 [798006,27441] 0 2024-02-22T15:50:47.473 INFO:tasks.workunit.client.1.smithi062.stdout:5/617: dwrite d0/f39 [0,4194304] 0 2024-02-22T15:50:47.474 INFO:tasks.workunit.client.0.smithi019.stdout:2/994: dwrite d1/d46/ddb/d10d/d12c/d13c/f14f [0,4194304] 0 2024-02-22T15:50:47.477 INFO:tasks.workunit.client.1.smithi062.stdout:9/701: rmdir d4/d15/d5f/d75/d97/da7 39 2024-02-22T15:50:47.479 INFO:tasks.workunit.client.1.smithi062.stdout:9/702: write d4/d15/d5e/d63/f86 [949727,47004] 0 2024-02-22T15:50:47.480 INFO:tasks.workunit.client.1.smithi062.stdout:5/618: getdents d0/d3/da/d13/d23/d4e 0 2024-02-22T15:50:47.482 INFO:tasks.workunit.client.1.smithi062.stdout:5/619: write d0/d3/f94 [882363,26122] 0 2024-02-22T15:50:47.485 INFO:tasks.workunit.client.1.smithi062.stdout:9/703: symlink d4/d15/d5f/d75/d97/ldb 0 2024-02-22T15:50:47.486 INFO:tasks.workunit.client.1.smithi062.stdout:9/704: chown d4/d15/d5e 3 1 2024-02-22T15:50:47.489 INFO:tasks.workunit.client.1.smithi062.stdout:9/705: creat d4/d1c/d83/fdc x:0 0 0 2024-02-22T15:50:47.490 INFO:tasks.workunit.client.1.smithi062.stdout:5/620: creat d0/d3/da/d13/d23/d78/d8c/dbd/fc6 x:0 0 0 2024-02-22T15:50:47.495 INFO:tasks.workunit.client.1.smithi062.stdout:9/706: mknod d4/d15/d5e/d63/da0/d92/cdd 0 2024-02-22T15:50:47.496 INFO:tasks.workunit.client.1.smithi062.stdout:5/621: mkdir d0/d3/da/d13/d57/dc7 0 2024-02-22T15:50:47.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:47 smithi019 ceph-mon[67418]: pgmap v39: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 132 MiB/s rd, 397 MiB/s wr, 756 op/s 2024-02-22T15:50:47.498 INFO:tasks.workunit.client.1.smithi062.stdout:9/707: read d4/d15/d5f/d75/d84/f3e [3859224,5402] 0 2024-02-22T15:50:47.500 INFO:tasks.workunit.client.1.smithi062.stdout:5/622: creat d0/d3/d2f/fc8 x:0 0 0 2024-02-22T15:50:47.505 INFO:tasks.workunit.client.1.smithi062.stdout:5/623: mkdir d0/d3/da/d13/d23/d78/d8c/dbd/dc9 0 2024-02-22T15:50:47.507 INFO:tasks.workunit.client.1.smithi062.stdout:9/708: dwrite d4/f10 [0,4194304] 0 2024-02-22T15:50:47.508 INFO:tasks.workunit.client.1.smithi062.stdout:4/800: write d4/d1c/d2c/d6b/d70/f76 [1557054,40977] 0 2024-02-22T15:50:47.510 INFO:tasks.workunit.client.1.smithi062.stdout:4/801: stat d4/d1c/d30/d37 0 2024-02-22T15:50:47.511 INFO:tasks.workunit.client.1.smithi062.stdout:5/624: mkdir d0/d3/da/d32/d4f/d72/dca 0 2024-02-22T15:50:47.512 INFO:tasks.workunit.client.1.smithi062.stdout:9/709: chown d4/d15/d5f/d75/d97/da7 424762 1 2024-02-22T15:50:47.514 INFO:tasks.workunit.client.1.smithi062.stdout:4/802: chown d4/d1c/d2c/d6b/d70/f93 280567616 1 2024-02-22T15:50:47.514 INFO:tasks.workunit.client.1.smithi062.stdout:5/625: creat d0/d3/d30/d71/fcb x:0 0 0 2024-02-22T15:50:47.515 INFO:tasks.workunit.client.1.smithi062.stdout:9/710: mkdir d4/d35/dde 0 2024-02-22T15:50:47.515 INFO:tasks.workunit.client.1.smithi062.stdout:4/803: fdatasync d4/d1c/d2c/d6b/d9b/fc0 0 2024-02-22T15:50:47.516 INFO:tasks.workunit.client.1.smithi062.stdout:9/711: stat d4/f26 0 2024-02-22T15:50:47.516 INFO:tasks.workunit.client.0.smithi019.stdout:3/806: dread d9/d22/d27/f46 [4194304,4194304] 0 2024-02-22T15:50:47.519 INFO:tasks.workunit.client.1.smithi062.stdout:9/712: creat d4/d28/d50/fdf x:0 0 0 2024-02-22T15:50:47.519 INFO:tasks.workunit.client.1.smithi062.stdout:4/804: truncate d4/f51 117394 0 2024-02-22T15:50:47.521 INFO:tasks.workunit.client.1.smithi062.stdout:5/626: mkdir d0/d3/da/d13/d57/dcc 0 2024-02-22T15:50:47.522 INFO:tasks.workunit.client.1.smithi062.stdout:9/713: symlink d4/d15/d5e/d63/da0/d92/le0 0 2024-02-22T15:50:47.523 INFO:tasks.workunit.client.1.smithi062.stdout:4/805: mkdir d4/d1c/d2c/d6b/d70/de8/d11b 0 2024-02-22T15:50:47.525 INFO:tasks.workunit.client.1.smithi062.stdout:9/714: truncate d4/d15/d5f/d75/fb8 1096189 0 2024-02-22T15:50:47.528 INFO:tasks.workunit.client.0.smithi019.stdout:8/852: dread d9/d14/d6f/d7e/f110 [0,4194304] 0 2024-02-22T15:50:47.528 INFO:tasks.workunit.client.1.smithi062.stdout:5/627: rename d0/d3/da/d13/d23/d31/ca7 to d0/ccd 0 2024-02-22T15:50:47.529 INFO:tasks.workunit.client.1.smithi062.stdout:9/715: rmdir d4/d28/d50/da3 39 2024-02-22T15:50:47.532 INFO:tasks.workunit.client.1.smithi062.stdout:5/628: truncate d0/d3/d30/d71/d7c/fae 319618 0 2024-02-22T15:50:47.532 INFO:tasks.workunit.client.1.smithi062.stdout:7/571: write d0/d6/d2e/d2f/f82 [1176857,73947] 0 2024-02-22T15:50:47.536 INFO:tasks.workunit.client.1.smithi062.stdout:9/716: creat d4/fe1 x:0 0 0 2024-02-22T15:50:47.536 INFO:tasks.workunit.client.0.smithi019.stdout:2/995: dread d1/d17/d1c/d7f/fdf [0,4194304] 0 2024-02-22T15:50:47.537 INFO:tasks.workunit.client.1.smithi062.stdout:9/717: write d4/d1c/d83/fdc [973947,51159] 0 2024-02-22T15:50:47.537 INFO:tasks.workunit.client.1.smithi062.stdout:7/572: chown d0/d14/f2c 1 1 2024-02-22T15:50:47.539 INFO:tasks.workunit.client.0.smithi019.stdout:2/996: truncate d1/def/f140 325664 0 2024-02-22T15:50:47.539 INFO:tasks.workunit.client.1.smithi062.stdout:5/629: creat d0/d3/d2f/d59/d67/d69/d3f/d66/fce x:0 0 0 2024-02-22T15:50:47.541 INFO:tasks.workunit.client.1.smithi062.stdout:9/718: read - d4/d28/d50/da3/fb7 zero size 2024-02-22T15:50:47.542 INFO:tasks.workunit.client.1.smithi062.stdout:7/573: chown d0/d14/d19/d30/d37/la7 12133 1 2024-02-22T15:50:47.542 INFO:tasks.workunit.client.0.smithi019.stdout:9/917: rename d2/d7/d52/dc0/de4 to d2/d7/dc/d27/dbc/d13b 0 2024-02-22T15:50:47.544 INFO:tasks.workunit.client.1.smithi062.stdout:9/719: creat d4/d1c/fe2 x:0 0 0 2024-02-22T15:50:47.545 INFO:tasks.workunit.client.1.smithi062.stdout:5/630: creat d0/d3/da/d13/d23/d4e/d5a/fcf x:0 0 0 2024-02-22T15:50:47.545 INFO:tasks.workunit.client.1.smithi062.stdout:6/732: sync 2024-02-22T15:50:47.546 INFO:tasks.workunit.client.1.smithi062.stdout:1/646: sync 2024-02-22T15:50:47.548 INFO:tasks.workunit.client.1.smithi062.stdout:9/720: mknod d4/d15/d5f/d6a/ce3 0 2024-02-22T15:50:47.552 INFO:tasks.workunit.client.1.smithi062.stdout:5/631: unlink d0/d3/d2f/d59/d67/d69/l84 0 2024-02-22T15:50:47.554 INFO:tasks.workunit.client.0.smithi019.stdout:1/993: dwrite d8/d18/fda [0,4194304] 0 2024-02-22T15:50:47.554 INFO:tasks.workunit.client.1.smithi062.stdout:6/733: symlink d5/df/d32/lec 0 2024-02-22T15:50:47.555 INFO:tasks.workunit.client.1.smithi062.stdout:7/574: dwrite d0/d6/d2e/f86 [0,4194304] 0 2024-02-22T15:50:47.556 INFO:tasks.workunit.client.1.smithi062.stdout:4/806: truncate d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/feb 2264706 0 2024-02-22T15:50:47.556 INFO:tasks.workunit.client.1.smithi062.stdout:5/632: dread - d0/d3/da/d32/d33/dbc/fc4 zero size 2024-02-22T15:50:47.559 INFO:tasks.workunit.client.1.smithi062.stdout:9/721: creat d4/d15/d31/fe4 x:0 0 0 2024-02-22T15:50:47.560 INFO:tasks.workunit.client.0.smithi019.stdout:9/918: dwrite d2/dec/d79/fff [0,4194304] 0 2024-02-22T15:50:47.563 INFO:tasks.workunit.client.1.smithi062.stdout:5/633: fsync d0/d3/d30/d71/f77 0 2024-02-22T15:50:47.564 INFO:tasks.workunit.client.1.smithi062.stdout:7/575: creat d0/d6/d2e/d35/faf x:0 0 0 2024-02-22T15:50:47.566 INFO:tasks.workunit.client.1.smithi062.stdout:9/722: fdatasync d4/d28/d50/d5a/f85 0 2024-02-22T15:50:47.567 INFO:tasks.workunit.client.1.smithi062.stdout:1/647: dread d1/d6/d10/f4b [0,4194304] 0 2024-02-22T15:50:47.567 INFO:tasks.workunit.client.1.smithi062.stdout:4/807: truncate d4/df/fda 130576 0 2024-02-22T15:50:47.569 INFO:tasks.workunit.client.1.smithi062.stdout:1/648: read d1/d6/d10/f4b [2358478,127689] 0 2024-02-22T15:50:47.572 INFO:tasks.workunit.client.1.smithi062.stdout:9/723: dread - d4/d15/d5f/d6a/fa8 zero size 2024-02-22T15:50:47.572 INFO:tasks.workunit.client.1.smithi062.stdout:1/649: readlink d1/d6/d7/l22 0 2024-02-22T15:50:47.572 INFO:tasks.workunit.client.1.smithi062.stdout:6/734: link d5/df/d40/d44/d4a/d41/ca5 d5/dd/d33/d66/db9/ced 0 2024-02-22T15:50:47.574 INFO:tasks.workunit.client.1.smithi062.stdout:9/724: readlink d4/d28/d50/d5a/l69 0 2024-02-22T15:50:47.575 INFO:tasks.workunit.client.1.smithi062.stdout:5/634: mkdir d0/d3/da/d13/d57/d79/dd0 0 2024-02-22T15:50:47.575 INFO:tasks.workunit.client.1.smithi062.stdout:9/725: readlink d4/d15/d31/l3b 0 2024-02-22T15:50:47.576 INFO:tasks.workunit.client.1.smithi062.stdout:4/808: rename d4/d2a/d2d/d2e/c7f to d4/d2a/d2d/d2e/d6a/d7b/c11c 0 2024-02-22T15:50:47.579 INFO:tasks.workunit.client.1.smithi062.stdout:6/735: creat d5/df/d40/dac/fee x:0 0 0 2024-02-22T15:50:47.581 INFO:tasks.workunit.client.1.smithi062.stdout:8/848: sync 2024-02-22T15:50:47.581 INFO:tasks.workunit.client.1.smithi062.stdout:0/810: sync 2024-02-22T15:50:47.581 INFO:tasks.workunit.client.1.smithi062.stdout:3/606: sync 2024-02-22T15:50:47.585 INFO:tasks.workunit.client.0.smithi019.stdout:0/868: sync 2024-02-22T15:50:47.587 INFO:tasks.workunit.client.1.smithi062.stdout:1/650: dwrite d1/d6/d7/d8a/fa5 [0,4194304] 0 2024-02-22T15:50:47.591 INFO:tasks.workunit.client.1.smithi062.stdout:3/607: write d5/d16/d56/f7e [4204280,114033] 0 2024-02-22T15:50:47.591 INFO:tasks.workunit.client.1.smithi062.stdout:4/809: getdents d4/d2a/d2d/d2e/d6a/d9e 0 2024-02-22T15:50:47.591 INFO:tasks.workunit.client.1.smithi062.stdout:6/736: write d5/df/d40/d44/d4a/d41/d43/dc6/fd4 [916775,122087] 0 2024-02-22T15:50:47.597 INFO:tasks.workunit.client.1.smithi062.stdout:1/651: symlink d1/d6/d36/d37/ld9 0 2024-02-22T15:50:47.597 INFO:tasks.workunit.client.1.smithi062.stdout:3/608: readlink d5/d26/l69 0 2024-02-22T15:50:47.597 INFO:tasks.workunit.client.1.smithi062.stdout:5/635: truncate d0/d3/d30/d71/f77 809368 0 2024-02-22T15:50:47.599 INFO:tasks.workunit.client.1.smithi062.stdout:9/726: dwrite d4/fe1 [0,4194304] 0 2024-02-22T15:50:47.601 INFO:tasks.workunit.client.1.smithi062.stdout:4/810: creat d4/d2a/f11d x:0 0 0 2024-02-22T15:50:47.605 INFO:tasks.workunit.client.1.smithi062.stdout:3/609: chown d5/f1d 25006 1 2024-02-22T15:50:47.609 INFO:tasks.workunit.client.1.smithi062.stdout:4/811: fsync d4/d1c/d2c/d6b/d70/f76 0 2024-02-22T15:50:47.616 INFO:tasks.workunit.client.1.smithi062.stdout:1/652: rename d1/d6/d7/d4a/f7f to d1/d6/d7/fda 0 2024-02-22T15:50:47.617 INFO:tasks.workunit.client.1.smithi062.stdout:7/576: dwrite d0/f10 [0,4194304] 0 2024-02-22T15:50:47.622 INFO:tasks.workunit.client.1.smithi062.stdout:9/727: getdents d4/d1c/d83/d87 0 2024-02-22T15:50:47.622 INFO:tasks.workunit.client.1.smithi062.stdout:6/737: rename d5/df/d40/d44/d76/c29 to d5/df/d40/dd2/cef 0 2024-02-22T15:50:47.623 INFO:tasks.workunit.client.1.smithi062.stdout:5/636: dwrite d0/d3/d30/d71/d7c/fab [0,4194304] 0 2024-02-22T15:50:47.623 INFO:tasks.workunit.client.0.smithi019.stdout:8/853: creat d9/d14/d6f/d7e/d94/db8/f11c x:0 0 0 2024-02-22T15:50:47.624 INFO:tasks.workunit.client.1.smithi062.stdout:3/610: dread d5/d16/d21/d32/f66 [0,4194304] 0 2024-02-22T15:50:47.628 INFO:tasks.workunit.client.0.smithi019.stdout:8/854: chown d9/d14/d1f/d20 68622 1 2024-02-22T15:50:47.629 INFO:tasks.workunit.client.1.smithi062.stdout:8/849: dread d8/f3a [0,4194304] 0 2024-02-22T15:50:47.630 INFO:tasks.workunit.client.1.smithi062.stdout:3/611: write d5/d16/d21/d32/f8b [601992,89584] 0 2024-02-22T15:50:47.630 INFO:tasks.workunit.client.1.smithi062.stdout:2/826: sync 2024-02-22T15:50:47.631 INFO:tasks.workunit.client.1.smithi062.stdout:7/577: mknod d0/d6/d5b/cb0 0 2024-02-22T15:50:47.631 INFO:tasks.workunit.client.1.smithi062.stdout:6/738: truncate d5/df/d32/d8f/fa1 1549377 0 2024-02-22T15:50:47.632 INFO:tasks.workunit.client.0.smithi019.stdout:1/994: chown d8/l1f 8797 1 2024-02-22T15:50:47.633 INFO:tasks.workunit.client.1.smithi062.stdout:3/612: chown d5/d16/d21/f4e 1318035 1 2024-02-22T15:50:47.635 INFO:tasks.workunit.client.1.smithi062.stdout:4/812: mknod d4/d2a/d2d/d2e/d6a/df7/c11e 0 2024-02-22T15:50:47.637 INFO:tasks.workunit.client.0.smithi019.stdout:3/807: fdatasync d9/d22/d27/d96/fe7 0 2024-02-22T15:50:47.638 INFO:tasks.workunit.client.1.smithi062.stdout:6/739: fsync d5/df/d40/d44/f7c 0 2024-02-22T15:50:47.640 INFO:tasks.workunit.client.0.smithi019.stdout:2/997: rename d1/d17/d21/d71/d60/d11b/f4a to d1/da4/d132/d14a/f15d 0 2024-02-22T15:50:47.640 INFO:tasks.workunit.client.1.smithi062.stdout:2/827: creat d1/d1e/dda/de8/f119 x:0 0 0 2024-02-22T15:50:47.640 INFO:tasks.workunit.client.1.smithi062.stdout:1/653: creat d1/d6/d7/d4a/fdb x:0 0 0 2024-02-22T15:50:47.641 INFO:tasks.workunit.client.1.smithi062.stdout:7/578: mknod d0/d6/d2e/d4a/d5a/cb1 0 2024-02-22T15:50:47.641 INFO:tasks.workunit.client.1.smithi062.stdout:4/813: fsync d4/d66/f6f 0 2024-02-22T15:50:47.641 INFO:tasks.workunit.client.0.smithi019.stdout:8/855: mknod d9/d88/c11d 0 2024-02-22T15:50:47.643 INFO:tasks.workunit.client.0.smithi019.stdout:1/995: fdatasync d8/d44/f9f 0 2024-02-22T15:50:47.643 INFO:tasks.workunit.client.1.smithi062.stdout:6/740: readlink d5/df/d32/lba 0 2024-02-22T15:50:47.644 INFO:tasks.workunit.client.0.smithi019.stdout:2/998: read d1/d17/d21/d71/d60/f141 [3361169,18604] 0 2024-02-22T15:50:47.644 INFO:tasks.workunit.client.0.smithi019.stdout:9/919: mknod d2/d1f/d2e/d48/d36/c13c 0 2024-02-22T15:50:47.645 INFO:tasks.workunit.client.1.smithi062.stdout:0/811: dread d1/d3/d4/d1e/fb2 [0,4194304] 0 2024-02-22T15:50:47.645 INFO:tasks.workunit.client.1.smithi062.stdout:5/637: dwrite d0/d3/d2f/d59/d67/d69/d3f/f51 [0,4194304] 0 2024-02-22T15:50:47.646 INFO:tasks.workunit.client.0.smithi019.stdout:2/999: stat d1/d17/d67/c10b 0 2024-02-22T15:50:47.648 INFO:tasks.workunit.client.0.smithi019.stdout:9/920: truncate d2/d7/f107 992216 0 2024-02-22T15:50:47.649 INFO:tasks.workunit.client.1.smithi062.stdout:3/613: truncate d5/d26/f38 452261 0 2024-02-22T15:50:47.653 INFO:tasks.workunit.client.0.smithi019.stdout:8/856: creat d9/d14/d6f/d7e/d94/daa/f11e x:0 0 0 2024-02-22T15:50:47.654 INFO:tasks.workunit.client.1.smithi062.stdout:6/741: chown d5/df/d40/db3/cd9 28 1 2024-02-22T15:50:47.656 INFO:tasks.workunit.client.0.smithi019.stdout:0/869: dread d4/d22/d2c/fac [0,4194304] 0 2024-02-22T15:50:47.656 INFO:tasks.workunit.client.0.smithi019.stdout:0/870: dread - d4/d67/f115 zero size 2024-02-22T15:50:47.658 INFO:tasks.workunit.client.1.smithi062.stdout:1/654: dwrite d1/d6/d7/d2b/fae [0,4194304] 0 2024-02-22T15:50:47.660 INFO:tasks.workunit.client.0.smithi019.stdout:0/871: write d4/d32/ffe [965461,67339] 0 2024-02-22T15:50:47.668 INFO:tasks.workunit.client.1.smithi062.stdout:4/814: mknod d4/d2a/d2d/d2e/d6a/d9e/c11f 0 2024-02-22T15:50:47.671 INFO:tasks.workunit.client.1.smithi062.stdout:8/850: write d8/f22 [4656011,46037] 0 2024-02-22T15:50:47.671 INFO:tasks.workunit.client.1.smithi062.stdout:5/638: rename d0/d3/da/c1c to d0/d3/d2f/d59/d67/d69/cd1 0 2024-02-22T15:50:47.672 INFO:tasks.workunit.client.0.smithi019.stdout:8/857: dwrite d9/d14/d6f/d7e/d94/d9c/fdc [0,4194304] 0 2024-02-22T15:50:47.673 INFO:tasks.workunit.client.0.smithi019.stdout:8/858: stat d9/d14/d17/d1c/da0/c96 0 2024-02-22T15:50:47.674 INFO:tasks.workunit.client.1.smithi062.stdout:8/851: write d8/d9/d2c/d49/d94/dc1/ff6 [912288,36316] 0 2024-02-22T15:50:47.676 INFO:tasks.workunit.client.0.smithi019.stdout:8/859: symlink d9/d14/d6f/d7e/l11f 0 2024-02-22T15:50:47.677 INFO:tasks.workunit.client.1.smithi062.stdout:0/812: write d1/d3/d4/d1e/fb2 [1530440,104437] 0 2024-02-22T15:50:47.678 INFO:tasks.workunit.client.0.smithi019.stdout:1/996: dread d8/d23/d33/f93 [0,4194304] 0 2024-02-22T15:50:47.683 INFO:tasks.workunit.client.0.smithi019.stdout:1/997: rmdir d8/d44/d5d 39 2024-02-22T15:50:47.684 INFO:tasks.workunit.client.0.smithi019.stdout:1/998: symlink d8/d15/d5b/l154 0 2024-02-22T15:50:47.685 INFO:tasks.workunit.client.0.smithi019.stdout:1/999: fsync d8/fe1 0 2024-02-22T15:50:47.686 INFO:tasks.workunit.client.0.smithi019.stdout:8/860: rename d9/d14/d17/f50 to d9/f120 0 2024-02-22T15:50:47.687 INFO:tasks.workunit.client.1.smithi062.stdout:8/852: dwrite d8/d26/fec [0,4194304] 0 2024-02-22T15:50:47.687 INFO:tasks.workunit.client.1.smithi062.stdout:4/815: creat d4/d1c/d2c/d6b/d9b/f120 x:0 0 0 2024-02-22T15:50:47.687 INFO:tasks.workunit.client.0.smithi019.stdout:8/861: write d9/d14/d1f/ffd [486774,81626] 0 2024-02-22T15:50:47.689 INFO:tasks.workunit.client.1.smithi062.stdout:8/853: readlink d8/d9/d2c/lad 0 2024-02-22T15:50:47.690 INFO:tasks.workunit.client.1.smithi062.stdout:3/614: truncate d5/f6 1540045 0 2024-02-22T15:50:47.692 INFO:tasks.workunit.client.1.smithi062.stdout:7/579: creat d0/d6/d2e/d55/d72/fb2 x:0 0 0 2024-02-22T15:50:47.693 INFO:tasks.workunit.client.1.smithi062.stdout:2/828: getdents d1/d8/d62/d73/dbd/d7d/d84 0 2024-02-22T15:50:47.694 INFO:tasks.workunit.client.1.smithi062.stdout:6/742: creat d5/df/d40/d44/d4a/d41/d85/dd1/ff0 x:0 0 0 2024-02-22T15:50:47.696 INFO:tasks.workunit.client.1.smithi062.stdout:3/615: mkdir d5/d1a/d3b/dc1 0 2024-02-22T15:50:47.696 INFO:tasks.workunit.client.1.smithi062.stdout:5/639: dwrite d0/d3/d2f/d59/f9f [0,4194304] 0 2024-02-22T15:50:47.701 INFO:tasks.workunit.client.1.smithi062.stdout:4/816: creat d4/df/daf/d10c/f121 x:0 0 0 2024-02-22T15:50:47.703 INFO:tasks.workunit.client.0.smithi019.stdout:9/921: creat d2/d1f/d2e/f13d x:0 0 0 2024-02-22T15:50:47.706 INFO:tasks.workunit.client.1.smithi062.stdout:8/854: mkdir d8/d42/d87/d108 0 2024-02-22T15:50:47.706 INFO:tasks.workunit.client.1.smithi062.stdout:5/640: chown d0/d3/da/d13/d23/d31/dac/cc1 6 1 2024-02-22T15:50:47.706 INFO:tasks.workunit.client.1.smithi062.stdout:4/817: dread - d4/df/daf/d10c/f121 zero size 2024-02-22T15:50:47.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:47 smithi062 ceph-mon[48130]: pgmap v39: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 132 MiB/s rd, 397 MiB/s wr, 756 op/s 2024-02-22T15:50:47.707 INFO:tasks.workunit.client.1.smithi062.stdout:7/580: dread d0/d14/d13/fc [0,4194304] 0 2024-02-22T15:50:47.708 INFO:tasks.workunit.client.0.smithi019.stdout:3/808: creat d9/d26/f109 x:0 0 0 2024-02-22T15:50:47.709 INFO:tasks.workunit.client.0.smithi019.stdout:3/809: fdatasync d9/d26/d3f/d5c/f6e 0 2024-02-22T15:50:47.709 INFO:tasks.workunit.client.1.smithi062.stdout:2/829: mkdir d1/d1e/d38/d11a 0 2024-02-22T15:50:47.710 INFO:tasks.workunit.client.1.smithi062.stdout:0/813: dread d1/d3/d4/f23 [0,4194304] 0 2024-02-22T15:50:47.710 INFO:tasks.workunit.client.0.smithi019.stdout:0/872: symlink d4/db/d49/l128 0 2024-02-22T15:50:47.710 INFO:tasks.workunit.client.1.smithi062.stdout:3/616: mkdir d5/d16/d21/d32/d9e/dc2 0 2024-02-22T15:50:47.714 INFO:tasks.workunit.client.0.smithi019.stdout:0/873: mkdir d4/db/d1b/d40/d44/d75/df7/d129 0 2024-02-22T15:50:47.715 INFO:tasks.workunit.client.1.smithi062.stdout:7/581: mkdir d0/d6/d2e/d4a/d5a/db3 0 2024-02-22T15:50:47.715 INFO:tasks.workunit.client.1.smithi062.stdout:8/855: symlink d8/d9/d2c/d49/d94/dc1/de9/dd1/l109 0 2024-02-22T15:50:47.716 INFO:tasks.workunit.client.1.smithi062.stdout:1/655: dread d1/d6/d10/f8f [0,4194304] 0 2024-02-22T15:50:47.716 INFO:tasks.workunit.client.1.smithi062.stdout:5/641: rmdir d0/d3/d2f/d59/d91/db8 0 2024-02-22T15:50:47.717 INFO:tasks.workunit.client.1.smithi062.stdout:4/818: dread d4/d1c/d2c/d6b/d70/de8/dfc/fff [0,4194304] 0 2024-02-22T15:50:47.717 INFO:tasks.workunit.client.0.smithi019.stdout:0/874: truncate d4/d22/d2c/fac 443671 0 2024-02-22T15:50:47.719 INFO:tasks.workunit.client.0.smithi019.stdout:0/875: mkdir d4/db/d1b/d63/db0/da1/dbb/dc/d12a 0 2024-02-22T15:50:47.720 INFO:tasks.workunit.client.0.smithi019.stdout:0/876: truncate d4/db/d24/d80/fa0 946534 0 2024-02-22T15:50:47.720 INFO:tasks.workunit.client.1.smithi062.stdout:2/830: creat d1/d8/d62/d73/dbd/dbb/dc4/dfe/f11b x:0 0 0 2024-02-22T15:50:47.721 INFO:tasks.workunit.client.1.smithi062.stdout:5/642: readlink d0/d3/da/d32/d4f/d72/l82 0 2024-02-22T15:50:47.722 INFO:tasks.workunit.client.1.smithi062.stdout:3/617: getdents d5/d16/d21/d96 0 2024-02-22T15:50:47.725 INFO:tasks.workunit.client.0.smithi019.stdout:8/862: write d9/d14/d1f/f21 [757273,53695] 0 2024-02-22T15:50:47.726 INFO:tasks.workunit.client.0.smithi019.stdout:9/922: write d2/d1f/d2e/d48/d36/f71 [6788109,113086] 0 2024-02-22T15:50:47.726 INFO:tasks.workunit.client.0.smithi019.stdout:8/863: readlink d9/d14/d6f/lc7 0 2024-02-22T15:50:47.727 INFO:tasks.workunit.client.0.smithi019.stdout:3/810: write d9/d16/f42 [5077347,1624] 0 2024-02-22T15:50:47.727 INFO:tasks.workunit.client.1.smithi062.stdout:1/656: creat d1/d6/d36/d76/fdc x:0 0 0 2024-02-22T15:50:47.728 INFO:tasks.workunit.client.0.smithi019.stdout:3/811: fdatasync d9/d26/d3f/f4a 0 2024-02-22T15:50:47.730 INFO:tasks.workunit.client.0.smithi019.stdout:9/923: write d2/d1f/d2e/d48/d36/f115 [5117,79903] 0 2024-02-22T15:50:47.730 INFO:tasks.workunit.client.0.smithi019.stdout:8/864: mkdir d9/d14/d17/d3d/d3f/dd4/d121 0 2024-02-22T15:50:47.733 INFO:tasks.workunit.client.0.smithi019.stdout:3/812: creat d9/d26/d2d/d103/f10a x:0 0 0 2024-02-22T15:50:47.733 INFO:tasks.workunit.client.0.smithi019.stdout:8/865: mknod d9/d14/d1f/d6d/c122 0 2024-02-22T15:50:47.734 INFO:tasks.workunit.client.0.smithi019.stdout:0/877: dread d4/d22/f51 [4194304,4194304] 0 2024-02-22T15:50:47.740 INFO:tasks.workunit.client.0.smithi019.stdout:3/813: rename d9/d26/d3f/d59 to d9/d26/d3f/d10b 0 2024-02-22T15:50:47.741 INFO:tasks.workunit.client.1.smithi062.stdout:0/814: dwrite d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fdd [0,4194304] 0 2024-02-22T15:50:47.741 INFO:tasks.workunit.client.0.smithi019.stdout:0/878: rename d4/db/d49/d65 to d4/db/d76/d77/d12b 0 2024-02-22T15:50:47.744 INFO:tasks.workunit.client.0.smithi019.stdout:8/866: rmdir d9/d14/d17/d3d/d3f/dd4/d121 0 2024-02-22T15:50:47.744 INFO:tasks.workunit.client.1.smithi062.stdout:0/815: truncate d1/ff3 998500 0 2024-02-22T15:50:47.748 INFO:tasks.workunit.client.1.smithi062.stdout:7/582: dwrite d0/d6/d2e/d55/d80/d8a/f8f [0,4194304] 0 2024-02-22T15:50:47.749 INFO:tasks.workunit.client.0.smithi019.stdout:0/879: symlink d4/db/d1b/d63/db0/da1/dbb/d7/l12c 0 2024-02-22T15:50:47.749 INFO:tasks.workunit.client.0.smithi019.stdout:8/867: write d9/d14/d6f/d7e/d94/daa/dea/f109 [1043094,99216] 0 2024-02-22T15:50:47.750 INFO:tasks.workunit.client.1.smithi062.stdout:4/819: dwrite d4/d2a/d2d/d46/fad [0,4194304] 0 2024-02-22T15:50:47.751 INFO:tasks.workunit.client.1.smithi062.stdout:2/831: creat d1/d8/d62/d73/dbd/d65/f11c x:0 0 0 2024-02-22T15:50:47.752 INFO:tasks.workunit.client.0.smithi019.stdout:0/880: fsync d4/db/d24/d80/ff9 0 2024-02-22T15:50:47.752 INFO:tasks.workunit.client.1.smithi062.stdout:8/856: dwrite d8/d9/f2a [4194304,4194304] 0 2024-02-22T15:50:47.754 INFO:tasks.workunit.client.0.smithi019.stdout:0/881: write d4/d22/d2c/f9a [108416,97569] 0 2024-02-22T15:50:47.755 INFO:tasks.workunit.client.0.smithi019.stdout:8/868: getdents d9/d14/d17/d3d/de5 0 2024-02-22T15:50:47.763 INFO:tasks.workunit.client.1.smithi062.stdout:3/618: getdents d5/d1a/d3c 0 2024-02-22T15:50:47.763 INFO:tasks.workunit.client.0.smithi019.stdout:3/814: dread d9/d26/d2d/f4b [0,4194304] 0 2024-02-22T15:50:47.764 INFO:tasks.workunit.client.0.smithi019.stdout:0/882: dwrite d4/db/d1b/f1c [0,4194304] 0 2024-02-22T15:50:47.766 INFO:tasks.workunit.client.0.smithi019.stdout:8/869: creat d9/d14/d6f/d7e/d94/d9c/f123 x:0 0 0 2024-02-22T15:50:47.767 INFO:tasks.workunit.client.1.smithi062.stdout:4/820: mknod d4/d2a/d2d/d2e/d6a/d9e/c122 0 2024-02-22T15:50:47.767 INFO:tasks.workunit.client.0.smithi019.stdout:3/815: fdatasync d9/d26/d3f/f8c 0 2024-02-22T15:50:47.768 INFO:tasks.workunit.client.1.smithi062.stdout:1/657: link d1/d6/d7/c99 d1/d6/d7/d2b/d47/d63/db2/cdd 0 2024-02-22T15:50:47.770 INFO:tasks.workunit.client.0.smithi019.stdout:5/958: sync 2024-02-22T15:50:47.770 INFO:tasks.workunit.client.1.smithi062.stdout:6/743: dwrite d5/df/d40/d44/d78/f31 [0,4194304] 0 2024-02-22T15:50:47.772 INFO:tasks.workunit.client.0.smithi019.stdout:3/816: creat d9/d14/d19/f10c x:0 0 0 2024-02-22T15:50:47.773 INFO:tasks.workunit.client.1.smithi062.stdout:8/857: dread d8/d9/d44/df8/d105/d4e/f8e [0,4194304] 0 2024-02-22T15:50:47.775 INFO:tasks.workunit.client.0.smithi019.stdout:0/883: creat d4/db/d49/dc2/f12d x:0 0 0 2024-02-22T15:50:47.776 INFO:tasks.workunit.client.1.smithi062.stdout:0/816: dwrite d1/d38/f98 [0,4194304] 0 2024-02-22T15:50:47.779 INFO:tasks.workunit.client.1.smithi062.stdout:4/821: unlink d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/le7 0 2024-02-22T15:50:47.779 INFO:tasks.workunit.client.1.smithi062.stdout:0/817: truncate d1/fc2 4489952 0 2024-02-22T15:50:47.779 INFO:tasks.workunit.client.0.smithi019.stdout:8/870: dwrite d9/d14/d6f/d7e/d94/d9c/dc6/f11b [0,4194304] 0 2024-02-22T15:50:47.779 INFO:tasks.workunit.client.0.smithi019.stdout:5/959: mknod d8/dd/d19/d38/d47/c142 0 2024-02-22T15:50:47.781 INFO:tasks.workunit.client.0.smithi019.stdout:3/817: truncate d9/d26/d3f/d10b/d66/fb3 4762354 0 2024-02-22T15:50:47.782 INFO:tasks.workunit.client.0.smithi019.stdout:3/818: stat d9/d65/da7/fc9 0 2024-02-22T15:50:47.785 INFO:tasks.workunit.client.1.smithi062.stdout:1/658: creat d1/d6/d7/d4a/fde x:0 0 0 2024-02-22T15:50:47.785 INFO:tasks.workunit.client.0.smithi019.stdout:8/871: mknod d9/d14/d6f/d7e/d94/d9a/deb/c124 0 2024-02-22T15:50:47.787 INFO:tasks.workunit.client.1.smithi062.stdout:6/744: mkdir d5/d8a/ddd/de6/df1 0 2024-02-22T15:50:47.787 INFO:tasks.workunit.client.1.smithi062.stdout:5/643: dread d0/d3/d30/f47 [0,4194304] 0 2024-02-22T15:50:47.788 INFO:tasks.workunit.client.0.smithi019.stdout:5/960: creat d8/dd/d19/d77/f143 x:0 0 0 2024-02-22T15:50:47.791 INFO:tasks.workunit.client.1.smithi062.stdout:3/619: getdents d5/d16/d21/d36/d4a 0 2024-02-22T15:50:47.791 INFO:tasks.workunit.client.0.smithi019.stdout:0/884: dwrite d4/db/d1b/d63/db0/da1/dbb/d52/dee/ff4 [0,4194304] 0 2024-02-22T15:50:47.793 INFO:tasks.workunit.client.1.smithi062.stdout:4/822: dread d4/d13/d20/fea [0,4194304] 0 2024-02-22T15:50:47.793 INFO:tasks.workunit.client.1.smithi062.stdout:1/659: creat d1/d6/d7/d8a/dca/fdf x:0 0 0 2024-02-22T15:50:47.794 INFO:tasks.workunit.client.1.smithi062.stdout:6/745: unlink d5/df/d32/d8f/f91 0 2024-02-22T15:50:47.794 INFO:tasks.workunit.client.0.smithi019.stdout:5/961: dread d8/dd/d19/d38/d54/f108 [0,4194304] 0 2024-02-22T15:50:47.794 INFO:tasks.workunit.client.0.smithi019.stdout:8/872: creat d9/d14/d6f/d7e/d94/d9a/f125 x:0 0 0 2024-02-22T15:50:47.797 INFO:tasks.workunit.client.1.smithi062.stdout:6/746: chown d5/df/d40/d44/d4a/d41 24 1 2024-02-22T15:50:47.797 INFO:tasks.workunit.client.1.smithi062.stdout:6/747: readlink d5/la 0 2024-02-22T15:50:47.797 INFO:tasks.workunit.client.1.smithi062.stdout:1/660: mkdir d1/d6/d7/d2b/d47/d63/db2/de0 0 2024-02-22T15:50:47.800 INFO:tasks.workunit.client.1.smithi062.stdout:4/823: creat d4/d1c/d30/d37/f123 x:0 0 0 2024-02-22T15:50:47.801 INFO:tasks.workunit.client.1.smithi062.stdout:5/644: dread d0/d3/d2f/d59/d67/d69/d3f/f4b [0,4194304] 0 2024-02-22T15:50:47.801 INFO:tasks.workunit.client.0.smithi019.stdout:5/962: symlink d8/dd/d19/d77/d7a/l144 0 2024-02-22T15:50:47.802 INFO:tasks.workunit.client.1.smithi062.stdout:3/620: getdents d5/d16/d21/d36/d5a 0 2024-02-22T15:50:47.804 INFO:tasks.workunit.client.1.smithi062.stdout:4/824: unlink d4/d13/d20/fe9 0 2024-02-22T15:50:47.804 INFO:tasks.workunit.client.0.smithi019.stdout:5/963: getdents d8/dd/d19/d77/dd2 0 2024-02-22T15:50:47.805 INFO:tasks.workunit.client.0.smithi019.stdout:5/964: chown d8/dd/d19/d77 35767060 1 2024-02-22T15:50:47.806 INFO:tasks.workunit.client.1.smithi062.stdout:6/748: rename d5/dd/l81 to d5/d8a/lf2 0 2024-02-22T15:50:47.808 INFO:tasks.workunit.client.1.smithi062.stdout:5/645: mkdir d0/d3/da/d13/d23/d4e/d6d/dd2 0 2024-02-22T15:50:47.808 INFO:tasks.workunit.client.1.smithi062.stdout:3/621: creat d5/d1a/d3b/dc1/fc3 x:0 0 0 2024-02-22T15:50:47.810 INFO:tasks.workunit.client.1.smithi062.stdout:5/646: readlink d0/d3/da/d13/d57/l90 0 2024-02-22T15:50:47.813 INFO:tasks.workunit.client.1.smithi062.stdout:4/825: rename d4/d1c/dac/fce to d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f124 0 2024-02-22T15:50:47.813 INFO:tasks.workunit.client.1.smithi062.stdout:3/622: truncate d5/d26/d2d/da2/dab/faf 894787 0 2024-02-22T15:50:47.814 INFO:tasks.workunit.client.1.smithi062.stdout:5/647: symlink d0/d3/d30/ld3 0 2024-02-22T15:50:47.815 INFO:tasks.workunit.client.1.smithi062.stdout:3/623: fdatasync d5/d16/d21/f60 0 2024-02-22T15:50:47.817 INFO:tasks.workunit.client.1.smithi062.stdout:4/826: creat d4/d1c/d30/da7/f125 x:0 0 0 2024-02-22T15:50:47.819 INFO:tasks.workunit.client.1.smithi062.stdout:5/648: symlink d0/d3/d30/d71/d7c/ld4 0 2024-02-22T15:50:47.820 INFO:tasks.workunit.client.0.smithi019.stdout:3/819: dread d9/d26/d32/f9c [0,4194304] 0 2024-02-22T15:50:47.820 INFO:tasks.workunit.client.1.smithi062.stdout:3/624: truncate d5/d16/d21/d32/f51 1137289 0 2024-02-22T15:50:47.820 INFO:tasks.workunit.client.1.smithi062.stdout:6/749: link d5/df/d40/dac/db6/dc0/cc3 d5/df/d40/dac/de1/cf3 0 2024-02-22T15:50:47.821 INFO:tasks.workunit.client.0.smithi019.stdout:3/820: write d9/d14/f86 [51578,73973] 0 2024-02-22T15:50:47.823 INFO:tasks.workunit.client.1.smithi062.stdout:4/827: creat d4/d2a/d2d/d2e/d6a/f126 x:0 0 0 2024-02-22T15:50:47.824 INFO:tasks.workunit.client.1.smithi062.stdout:6/750: write d5/dd/d33/f48 [4984395,34208] 0 2024-02-22T15:50:47.825 INFO:tasks.workunit.client.1.smithi062.stdout:5/649: mknod d0/d3/da/d13/d57/cd5 0 2024-02-22T15:50:47.826 INFO:tasks.workunit.client.1.smithi062.stdout:1/661: dread d1/d6/d7/d4a/d65/d83/faa [0,4194304] 0 2024-02-22T15:50:47.830 INFO:tasks.workunit.client.1.smithi062.stdout:5/650: chown d0/d3/d2f/d59/d67/d69/d3f/d66/cb0 0 1 2024-02-22T15:50:47.831 INFO:tasks.workunit.client.1.smithi062.stdout:1/662: dread - d1/d6/d7/d2b/d47/d24/f9f zero size 2024-02-22T15:50:47.832 INFO:tasks.workunit.client.1.smithi062.stdout:4/828: dwrite d4/f53 [4194304,4194304] 0 2024-02-22T15:50:47.834 INFO:tasks.workunit.client.1.smithi062.stdout:4/829: chown d4/d2a/d2d/d46 6 1 2024-02-22T15:50:47.834 INFO:tasks.workunit.client.1.smithi062.stdout:6/751: rename d5/df/d40/d44/d4a/d41/c93 to d5/df/d40/d44/d4a/d41/d43/dc6/cf4 0 2024-02-22T15:50:47.835 INFO:tasks.workunit.client.1.smithi062.stdout:5/651: read d0/d3/da/d32/d42/f65 [3807350,16996] 0 2024-02-22T15:50:47.838 INFO:tasks.workunit.client.1.smithi062.stdout:4/830: chown d4/d66/dfa/l10a 592 1 2024-02-22T15:50:47.840 INFO:tasks.workunit.client.1.smithi062.stdout:5/652: fsync d0/d3/da/f96 0 2024-02-22T15:50:47.843 INFO:tasks.workunit.client.1.smithi062.stdout:1/663: dwrite d1/d6/d7/d8a/fa5 [0,4194304] 0 2024-02-22T15:50:47.845 INFO:tasks.workunit.client.0.smithi019.stdout:0/885: truncate d4/db/d1b/d63/db0/da1/dbb/d52/f62 3951573 0 2024-02-22T15:50:47.847 INFO:tasks.workunit.client.1.smithi062.stdout:5/653: truncate d0/d3/da/d32/d4f/d72/f99 772283 0 2024-02-22T15:50:47.849 INFO:tasks.workunit.client.1.smithi062.stdout:4/831: dwrite d4/f3f [0,4194304] 0 2024-02-22T15:50:47.851 INFO:tasks.workunit.client.0.smithi019.stdout:0/886: unlink d4/db/d1b/c112 0 2024-02-22T15:50:47.856 INFO:tasks.workunit.client.0.smithi019.stdout:8/873: dwrite d9/d14/d17/f85 [0,4194304] 0 2024-02-22T15:50:47.857 INFO:tasks.workunit.client.1.smithi062.stdout:5/654: read d0/d3/da/d13/d23/f3b [35001,34805] 0 2024-02-22T15:50:47.858 INFO:tasks.workunit.client.0.smithi019.stdout:5/965: dwrite d8/dd/d19/d3a/f5e [0,4194304] 0 2024-02-22T15:50:47.859 INFO:tasks.workunit.client.0.smithi019.stdout:0/887: unlink d4/db/d76/d77/d12b/cb5 0 2024-02-22T15:50:47.860 INFO:tasks.workunit.client.0.smithi019.stdout:3/821: dwrite d9/d22/d27/d96/fe7 [0,4194304] 0 2024-02-22T15:50:47.861 INFO:tasks.workunit.client.1.smithi062.stdout:6/752: link d5/df/d32/c51 d5/df/d40/d44/d78/cf5 0 2024-02-22T15:50:47.861 INFO:tasks.workunit.client.1.smithi062.stdout:4/832: fsync d4/d2a/d2d/d2e/d6a/d9e/fc3 0 2024-02-22T15:50:47.864 INFO:tasks.workunit.client.0.smithi019.stdout:8/874: unlink d9/d14/d6f/d7e/d94/db8/fe2 0 2024-02-22T15:50:47.867 INFO:tasks.workunit.client.1.smithi062.stdout:2/832: dwrite d1/d8/df/f4a [0,4194304] 0 2024-02-22T15:50:47.869 INFO:tasks.workunit.client.1.smithi062.stdout:0/818: dwrite d1/d13/f65 [4194304,4194304] 0 2024-02-22T15:50:47.871 INFO:tasks.workunit.client.0.smithi019.stdout:3/822: creat d9/d22/d49/f10d x:0 0 0 2024-02-22T15:50:47.872 INFO:tasks.workunit.client.0.smithi019.stdout:0/888: creat d4/db/d24/d80/d122/f12e x:0 0 0 2024-02-22T15:50:47.873 INFO:tasks.workunit.client.0.smithi019.stdout:5/966: mknod d8/dd/d19/d3a/d46/d7e/dbe/dc3/d119/c145 0 2024-02-22T15:50:47.873 INFO:tasks.workunit.client.0.smithi019.stdout:8/875: read d9/d14/d1f/f33 [1849614,127991] 0 2024-02-22T15:50:47.874 INFO:tasks.workunit.client.0.smithi019.stdout:3/823: symlink d9/d65/d106/l10e 0 2024-02-22T15:50:47.875 INFO:tasks.workunit.client.0.smithi019.stdout:0/889: write d4/db/d24/d80/d122/f12e [1000825,130937] 0 2024-02-22T15:50:47.877 INFO:tasks.workunit.client.1.smithi062.stdout:2/833: write d1/d1e/ff8 [210033,39450] 0 2024-02-22T15:50:47.879 INFO:tasks.workunit.client.0.smithi019.stdout:5/967: rmdir d8/dd/d19/d38/d54/db0/d10d/d122/dde/df6 39 2024-02-22T15:50:47.880 INFO:tasks.workunit.client.1.smithi062.stdout:5/655: dread d0/d3/da/f53 [0,4194304] 0 2024-02-22T15:50:47.882 INFO:tasks.workunit.client.0.smithi019.stdout:5/968: chown d8/d11/c57 27 1 2024-02-22T15:50:47.884 INFO:tasks.workunit.client.1.smithi062.stdout:6/753: creat d5/dd/d33/ff6 x:0 0 0 2024-02-22T15:50:47.884 INFO:tasks.workunit.client.1.smithi062.stdout:0/819: chown d1/d13/dde/d76/fe2 28975 1 2024-02-22T15:50:47.884 INFO:tasks.workunit.client.0.smithi019.stdout:0/890: mkdir d4/db/d1b/d63/db0/da1/dbb/da3/de1/d12f 0 2024-02-22T15:50:47.886 INFO:tasks.workunit.client.0.smithi019.stdout:5/969: write d8/dd/f5b [2080993,16490] 0 2024-02-22T15:50:47.887 INFO:tasks.workunit.client.1.smithi062.stdout:8/858: dwrite d8/d26/db5/f104 [0,4194304] 0 2024-02-22T15:50:47.888 INFO:tasks.workunit.client.0.smithi019.stdout:3/824: dwrite d9/d11/fb4 [0,4194304] 0 2024-02-22T15:50:47.889 INFO:tasks.workunit.client.0.smithi019.stdout:5/970: dread - d8/f127 zero size 2024-02-22T15:50:47.889 INFO:tasks.workunit.client.1.smithi062.stdout:2/834: unlink d1/d1e/d38/d76/df5/f10d 0 2024-02-22T15:50:47.891 INFO:tasks.workunit.client.1.smithi062.stdout:6/754: mknod d5/df/d32/d47/cf7 0 2024-02-22T15:50:47.892 INFO:tasks.workunit.client.1.smithi062.stdout:0/820: mknod d1/d3/d4/d1e/d74/c10b 0 2024-02-22T15:50:47.893 INFO:tasks.workunit.client.0.smithi019.stdout:3/825: chown d9/lb9 9 1 2024-02-22T15:50:47.895 INFO:tasks.workunit.client.0.smithi019.stdout:5/971: chown d8/d11/f93 200903 1 2024-02-22T15:50:47.895 INFO:tasks.workunit.client.0.smithi019.stdout:0/891: unlink d4/db/d1b/d40/d44/d75/df7/f121 0 2024-02-22T15:50:47.896 INFO:tasks.workunit.client.1.smithi062.stdout:5/656: getdents d0/d3/da/d13/d57/dc7 0 2024-02-22T15:50:47.896 INFO:tasks.workunit.client.1.smithi062.stdout:8/859: dread - d8/d9/d44/df8/d105/d4e/fcf zero size 2024-02-22T15:50:47.897 INFO:tasks.workunit.client.1.smithi062.stdout:6/755: chown d5/df/d40/d44/d4a/d41/d43/dc6/fd4 1649271213 1 2024-02-22T15:50:47.897 INFO:tasks.workunit.client.0.smithi019.stdout:0/892: chown d4/d23/d7d/fff 6953469 1 2024-02-22T15:50:47.898 INFO:tasks.workunit.client.0.smithi019.stdout:8/876: dwrite d9/d14/d17/d3d/d3f/dd4/f101 [0,4194304] 0 2024-02-22T15:50:47.903 INFO:tasks.workunit.client.0.smithi019.stdout:8/877: mkdir d9/d14/d6f/d7e/d94/daa/d126 0 2024-02-22T15:50:47.903 INFO:tasks.workunit.client.1.smithi062.stdout:6/756: creat d5/df/dbc/dcd/ff8 x:0 0 0 2024-02-22T15:50:47.904 INFO:tasks.workunit.client.1.smithi062.stdout:3/625: dwrite d5/d16/d21/d36/d4a/f4f [0,4194304] 0 2024-02-22T15:50:47.908 INFO:tasks.workunit.client.1.smithi062.stdout:2/835: creat d1/d8/d62/d73/dbd/d7d/f11d x:0 0 0 2024-02-22T15:50:47.908 INFO:tasks.workunit.client.1.smithi062.stdout:5/657: mkdir d0/d3/da/d13/d23/d4e/d5a/dd6 0 2024-02-22T15:50:47.909 INFO:tasks.workunit.client.1.smithi062.stdout:0/821: mknod d1/d13/dde/d76/d8c/d2c/d84/c10c 0 2024-02-22T15:50:47.909 INFO:tasks.workunit.client.1.smithi062.stdout:6/757: unlink d5/dd/d33/f49 0 2024-02-22T15:50:47.909 INFO:tasks.workunit.client.1.smithi062.stdout:5/658: write d0/f7 [7459691,9140] 0 2024-02-22T15:50:47.911 INFO:tasks.workunit.client.0.smithi019.stdout:8/878: rename d9/d14/d17/d3d/d3f/dd4/c116 to d9/d14/d1f/d4c/c127 0 2024-02-22T15:50:47.911 INFO:tasks.workunit.client.0.smithi019.stdout:0/893: dwrite d4/db/d1b/d63/db0/da1/dbb/fe [0,4194304] 0 2024-02-22T15:50:47.911 INFO:tasks.workunit.client.1.smithi062.stdout:8/860: link d8/d86/l98 d8/d9/d2c/d49/d94/dc1/de9/dd1/l10a 0 2024-02-22T15:50:47.914 INFO:tasks.workunit.client.1.smithi062.stdout:6/758: symlink d5/df/d40/dac/lf9 0 2024-02-22T15:50:47.915 INFO:tasks.workunit.client.1.smithi062.stdout:8/861: write d8/d9/d2c/fe6 [1475631,121672] 0 2024-02-22T15:50:47.917 INFO:tasks.workunit.client.1.smithi062.stdout:5/659: creat d0/d3/d2f/d59/d67/d69/fd7 x:0 0 0 2024-02-22T15:50:47.918 INFO:tasks.workunit.client.0.smithi019.stdout:0/894: symlink d4/d23/df5/l130 0 2024-02-22T15:50:47.918 INFO:tasks.workunit.client.0.smithi019.stdout:8/879: chown d9/d14/c6e 6388 1 2024-02-22T15:50:47.921 INFO:tasks.workunit.client.1.smithi062.stdout:0/822: symlink d1/d13/dde/d76/d8c/d2c/l10d 0 2024-02-22T15:50:47.922 INFO:tasks.workunit.client.1.smithi062.stdout:2/836: dwrite d1/d1e/d38/f81 [0,4194304] 0 2024-02-22T15:50:47.925 INFO:tasks.workunit.client.1.smithi062.stdout:8/862: unlink d8/d9/d2c/f78 0 2024-02-22T15:50:47.925 INFO:tasks.workunit.client.1.smithi062.stdout:1/664: dread d1/f30 [0,4194304] 0 2024-02-22T15:50:47.927 INFO:tasks.workunit.client.0.smithi019.stdout:8/880: dwrite d9/d14/d1f/fda [0,4194304] 0 2024-02-22T15:50:47.930 INFO:tasks.workunit.client.1.smithi062.stdout:6/759: mknod d5/d8a/ddd/cfa 0 2024-02-22T15:50:47.933 INFO:tasks.workunit.client.1.smithi062.stdout:2/837: rmdir d1/d8/d62 39 2024-02-22T15:50:47.935 INFO:tasks.workunit.client.0.smithi019.stdout:8/881: rename d9/d14/d6f/d7e/d94/d9a to d9/d14/d17/d3d/d128 0 2024-02-22T15:50:47.935 INFO:tasks.workunit.client.1.smithi062.stdout:1/665: creat d1/d6/d36/fe1 x:0 0 0 2024-02-22T15:50:47.937 INFO:tasks.workunit.client.0.smithi019.stdout:3/826: dread d9/d22/d27/f46 [0,4194304] 0 2024-02-22T15:50:47.937 INFO:tasks.workunit.client.1.smithi062.stdout:0/823: read d1/d13/dde/d1d/fa7 [4174609,102905] 0 2024-02-22T15:50:47.937 INFO:tasks.workunit.client.0.smithi019.stdout:8/882: truncate d9/d14/d1f/d6d/d103/f10d 6193 0 2024-02-22T15:50:47.939 INFO:tasks.workunit.client.0.smithi019.stdout:3/827: mknod d9/d26/d3f/d10b/c10f 0 2024-02-22T15:50:47.940 INFO:tasks.workunit.client.0.smithi019.stdout:8/883: chown d9/d14/d17/d3d/d128/cf4 0 1 2024-02-22T15:50:47.940 INFO:tasks.workunit.client.1.smithi062.stdout:8/863: dwrite d8/d9/d2c/d49/d94/dc1/de9/dd1/f102 [0,4194304] 0 2024-02-22T15:50:47.941 INFO:tasks.workunit.client.1.smithi062.stdout:5/660: getdents d0/d3/da/d13/d23/d31/dac 0 2024-02-22T15:50:47.941 INFO:tasks.workunit.client.1.smithi062.stdout:2/838: readlink d1/d47/d39/d5d/lb3 0 2024-02-22T15:50:47.943 INFO:tasks.workunit.client.1.smithi062.stdout:8/864: chown d8/d26/l33 16039 1 2024-02-22T15:50:47.943 INFO:tasks.workunit.client.0.smithi019.stdout:8/884: fsync d9/d14/f6a 0 2024-02-22T15:50:47.944 INFO:tasks.workunit.client.1.smithi062.stdout:1/666: mknod d1/d6/d7/d2b/ce2 0 2024-02-22T15:50:47.945 INFO:tasks.workunit.client.0.smithi019.stdout:3/828: rename d9/d22/cbc to d9/d11/c110 0 2024-02-22T15:50:47.947 INFO:tasks.workunit.client.1.smithi062.stdout:6/760: link d5/df/d40/d44/d4a/l64 d5/df/dbc/dce/lfb 0 2024-02-22T15:50:47.948 INFO:tasks.workunit.client.0.smithi019.stdout:8/885: creat d9/d14/d1f/d4c/f129 x:0 0 0 2024-02-22T15:50:47.949 INFO:tasks.workunit.client.1.smithi062.stdout:0/824: link d1/d13/dde/d76/d8c/d2c/lb0 d1/dd/l10e 0 2024-02-22T15:50:47.951 INFO:tasks.workunit.client.1.smithi062.stdout:6/761: creat d5/df/d32/d47/ffc x:0 0 0 2024-02-22T15:50:47.952 INFO:tasks.workunit.client.0.smithi019.stdout:8/886: getdents d9/d14/d17/d5d 0 2024-02-22T15:50:47.952 INFO:tasks.workunit.client.1.smithi062.stdout:5/661: creat d0/d3/d2f/d59/d67/fd8 x:0 0 0 2024-02-22T15:50:47.953 INFO:tasks.workunit.client.1.smithi062.stdout:0/825: mkdir d1/d13/dde/d76/d102/d10f 0 2024-02-22T15:50:47.954 INFO:tasks.workunit.client.1.smithi062.stdout:5/662: fdatasync d0/d3/da/d32/d4f/d72/f99 0 2024-02-22T15:50:47.955 INFO:tasks.workunit.client.0.smithi019.stdout:8/887: mkdir d9/d14/d17/d3d/d128/dd3/d12a 0 2024-02-22T15:50:47.955 INFO:tasks.workunit.client.1.smithi062.stdout:6/762: creat d5/dd/d33/d71/ffd x:0 0 0 2024-02-22T15:50:47.956 INFO:tasks.workunit.client.1.smithi062.stdout:5/663: chown d0/fa8 13 1 2024-02-22T15:50:47.957 INFO:tasks.workunit.client.1.smithi062.stdout:0/826: rmdir d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d 39 2024-02-22T15:50:47.958 INFO:tasks.workunit.client.1.smithi062.stdout:6/763: creat d5/d8a/ddd/dd5/ffe x:0 0 0 2024-02-22T15:50:47.960 INFO:tasks.workunit.client.1.smithi062.stdout:5/664: mkdir d0/d3/da/d13/d23/dd9 0 2024-02-22T15:50:47.961 INFO:tasks.workunit.client.1.smithi062.stdout:0/827: mknod d1/d13/dde/d76/d8c/d2c/db5/c110 0 2024-02-22T15:50:47.962 INFO:tasks.workunit.client.0.smithi019.stdout:8/888: dwrite d9/d14/d6f/d7e/f10f [0,4194304] 0 2024-02-22T15:50:47.964 INFO:tasks.workunit.client.1.smithi062.stdout:0/828: creat d1/d13/f111 x:0 0 0 2024-02-22T15:50:47.967 INFO:tasks.workunit.client.1.smithi062.stdout:6/764: link d5/df/d32/lec d5/df/d40/d44/d4a/lff 0 2024-02-22T15:50:47.968 INFO:tasks.workunit.client.1.smithi062.stdout:6/765: write d5/dd/d33/d71/ffd [357873,95787] 0 2024-02-22T15:50:47.968 INFO:tasks.workunit.client.1.smithi062.stdout:0/829: truncate d1/d13/dde/d1d/fa7 3373232 0 2024-02-22T15:50:47.973 INFO:tasks.workunit.client.1.smithi062.stdout:1/667: dread d1/d66/f75 [0,4194304] 0 2024-02-22T15:50:47.978 INFO:tasks.workunit.client.1.smithi062.stdout:6/766: dwrite d5/df/d40/d44/d4a/d41/d43/dc6/fd4 [0,4194304] 0 2024-02-22T15:50:47.979 INFO:tasks.workunit.client.1.smithi062.stdout:1/668: dread d1/f30 [0,4194304] 0 2024-02-22T15:50:47.982 INFO:tasks.workunit.client.0.smithi019.stdout:4/878: sync 2024-02-22T15:50:47.983 INFO:tasks.workunit.client.1.smithi062.stdout:1/669: dread d1/d6/d7/d2b/fae [0,4194304] 0 2024-02-22T15:50:47.985 INFO:tasks.workunit.client.1.smithi062.stdout:1/670: fdatasync d1/d66/f75 0 2024-02-22T15:50:47.986 INFO:tasks.workunit.client.0.smithi019.stdout:4/879: rename d2/db/d14/d23/c4c to d2/db/d14/d61/c118 0 2024-02-22T15:50:47.988 INFO:tasks.workunit.client.1.smithi062.stdout:1/671: creat d1/d6/d7/d2b/d56/d95/fe3 x:0 0 0 2024-02-22T15:50:47.989 INFO:tasks.workunit.client.0.smithi019.stdout:4/880: symlink d2/db/d14/d23/d33/db2/d107/l119 0 2024-02-22T15:50:47.991 INFO:tasks.workunit.client.0.smithi019.stdout:4/881: creat d2/db/d14/d23/d67/f11a x:0 0 0 2024-02-22T15:50:47.992 INFO:tasks.workunit.client.0.smithi019.stdout:4/882: write d2/d3f/f51 [1294567,53894] 0 2024-02-22T15:50:47.993 INFO:tasks.workunit.client.0.smithi019.stdout:4/883: symlink d2/d3f/d48/l11b 0 2024-02-22T15:50:47.995 INFO:tasks.workunit.client.1.smithi062.stdout:1/672: dwrite d1/d6/d7/d2b/d47/d63/db2/fbe [0,4194304] 0 2024-02-22T15:50:48.000 INFO:tasks.workunit.client.0.smithi019.stdout:4/884: dwrite d2/d3f/d48/d76/fa8 [0,4194304] 0 2024-02-22T15:50:48.022 INFO:tasks.workunit.client.0.smithi019.stdout:4/885: dread d2/db/d14/d23/f65 [0,4194304] 0 2024-02-22T15:50:48.025 INFO:tasks.workunit.client.0.smithi019.stdout:9/924: sync 2024-02-22T15:50:48.039 INFO:tasks.workunit.client.1.smithi062.stdout:4/833: write d4/d2a/d2d/f85 [4055434,83997] 0 2024-02-22T15:50:48.040 INFO:tasks.workunit.client.0.smithi019.stdout:5/972: write d8/d11/f1c [4625716,102813] 0 2024-02-22T15:50:48.041 INFO:tasks.workunit.client.0.smithi019.stdout:3/829: dread f6 [0,4194304] 0 2024-02-22T15:50:48.042 INFO:tasks.workunit.client.1.smithi062.stdout:4/834: mknod d4/d1c/d2c/d6b/d70/de8/c127 0 2024-02-22T15:50:48.042 INFO:tasks.workunit.client.1.smithi062.stdout:4/835: dread - d4/d1c/d2c/f101 zero size 2024-02-22T15:50:48.043 INFO:tasks.workunit.client.1.smithi062.stdout:4/836: stat d4/d2a/d2d/d2e/d6a/d9e/c11f 0 2024-02-22T15:50:48.044 INFO:tasks.workunit.client.1.smithi062.stdout:4/837: dread - d4/d1c/d30/da7/dcb/fd4 zero size 2024-02-22T15:50:48.048 INFO:tasks.workunit.client.0.smithi019.stdout:0/895: dwrite d4/d22/fa9 [0,4194304] 0 2024-02-22T15:50:48.049 INFO:tasks.workunit.client.0.smithi019.stdout:8/889: truncate d9/d14/d17/d3d/d128/fba 105189 0 2024-02-22T15:50:48.053 INFO:tasks.workunit.client.0.smithi019.stdout:8/890: dread - d9/d14/d17/ff2 zero size 2024-02-22T15:50:48.054 INFO:tasks.workunit.client.0.smithi019.stdout:3/830: dwrite d9/d26/d3f/d10b/d66/f7f [0,4194304] 0 2024-02-22T15:50:48.056 INFO:tasks.workunit.client.0.smithi019.stdout:0/896: mknod d4/db/d1b/d40/d11a/c131 0 2024-02-22T15:50:48.056 INFO:tasks.workunit.client.1.smithi062.stdout:4/838: creat d4/d1c/d30/da5/f128 x:0 0 0 2024-02-22T15:50:48.057 INFO:tasks.workunit.client.0.smithi019.stdout:0/897: chown d4/d22/d2c/d68/d6d/fc9 167334232 1 2024-02-22T15:50:48.059 INFO:tasks.workunit.client.1.smithi062.stdout:3/626: truncate d5/f8a 1238044 0 2024-02-22T15:50:48.059 INFO:tasks.workunit.client.0.smithi019.stdout:3/831: chown d9/d26/d2d/c78 4 1 2024-02-22T15:50:48.062 INFO:tasks.workunit.client.1.smithi062.stdout:4/839: creat d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f129 x:0 0 0 2024-02-22T15:50:48.063 INFO:tasks.workunit.client.1.smithi062.stdout:3/627: write d5/d1a/d3c/d62/fb7 [3982575,121991] 0 2024-02-22T15:50:48.065 INFO:tasks.workunit.client.1.smithi062.stdout:3/628: symlink d5/d16/d21/d36/d5a/lc4 0 2024-02-22T15:50:48.066 INFO:tasks.workunit.client.1.smithi062.stdout:3/629: write d5/d16/d21/d32/d6a/d7b/f5f [1046255,37392] 0 2024-02-22T15:50:48.068 INFO:tasks.workunit.client.1.smithi062.stdout:3/630: write d5/d16/d21/d32/f51 [164145,49969] 0 2024-02-22T15:50:48.069 INFO:tasks.workunit.client.1.smithi062.stdout:9/728: sync 2024-02-22T15:50:48.073 INFO:tasks.workunit.client.1.smithi062.stdout:9/729: mknod d4/d15/d5f/d75/ce5 0 2024-02-22T15:50:48.074 INFO:tasks.workunit.client.1.smithi062.stdout:9/730: write d4/f1b [4418972,84610] 0 2024-02-22T15:50:48.075 INFO:tasks.workunit.client.1.smithi062.stdout:9/731: creat d4/d1c/fe6 x:0 0 0 2024-02-22T15:50:48.076 INFO:tasks.workunit.client.0.smithi019.stdout:5/973: dread d8/dd/d19/d3a/d46/f61 [0,4194304] 0 2024-02-22T15:50:48.077 INFO:tasks.workunit.client.1.smithi062.stdout:3/631: dread d5/d16/d21/d55/fa0 [0,4194304] 0 2024-02-22T15:50:48.078 INFO:tasks.workunit.client.1.smithi062.stdout:3/632: rename d5/d1a/d3c to d5/d1a/d3c/dc5 22 2024-02-22T15:50:48.078 INFO:tasks.workunit.client.1.smithi062.stdout:9/732: creat d4/d15/fe7 x:0 0 0 2024-02-22T15:50:48.079 INFO:tasks.workunit.client.0.smithi019.stdout:5/974: creat d8/dd/d19/d77/f146 x:0 0 0 2024-02-22T15:50:48.080 INFO:tasks.workunit.client.1.smithi062.stdout:9/733: mkdir d4/d28/d50/da3/de8 0 2024-02-22T15:50:48.081 INFO:tasks.workunit.client.0.smithi019.stdout:5/975: rmdir d8/dd/d19/d3a/d46/d7e/dbe/dc3 39 2024-02-22T15:50:48.081 INFO:tasks.workunit.client.1.smithi062.stdout:9/734: truncate d4/d35/fd8 364416 0 2024-02-22T15:50:48.082 INFO:tasks.workunit.client.0.smithi019.stdout:5/976: chown d8/dd/c16 1981277 1 2024-02-22T15:50:48.083 INFO:tasks.workunit.client.0.smithi019.stdout:5/977: mknod d8/dd/d8e/d2c/d10e/c147 0 2024-02-22T15:50:48.085 INFO:tasks.workunit.client.1.smithi062.stdout:9/735: getdents d4/d15/d5e/d63/da0/d92 0 2024-02-22T15:50:48.086 INFO:tasks.workunit.client.0.smithi019.stdout:5/978: creat d8/dd/d19/d38/d54/f148 x:0 0 0 2024-02-22T15:50:48.087 INFO:tasks.workunit.client.0.smithi019.stdout:4/886: write d2/db/d54/d8c/fd6 [729169,1947] 0 2024-02-22T15:50:48.088 INFO:tasks.workunit.client.0.smithi019.stdout:9/925: write d2/d1f/d2e/d48/d45/d5a/daf/fbd [243300,101542] 0 2024-02-22T15:50:48.091 INFO:tasks.workunit.client.0.smithi019.stdout:9/926: readlink d2/d7/dc/lf 0 2024-02-22T15:50:48.092 INFO:tasks.workunit.client.1.smithi062.stdout:4/840: dread d4/df/f14 [0,4194304] 0 2024-02-22T15:50:48.092 INFO:tasks.workunit.client.0.smithi019.stdout:8/891: write d9/f29 [107314,129117] 0 2024-02-22T15:50:48.094 INFO:tasks.workunit.client.1.smithi062.stdout:9/736: dread d4/d15/d5e/d63/da0/f54 [0,4194304] 0 2024-02-22T15:50:48.095 INFO:tasks.workunit.client.0.smithi019.stdout:4/887: unlink d2/db/d14/d23/d9e/dac/dc2/dc9/lf7 0 2024-02-22T15:50:48.095 INFO:tasks.workunit.client.0.smithi019.stdout:9/927: write d2/dec/d79/fff [622400,94671] 0 2024-02-22T15:50:48.098 INFO:tasks.workunit.client.0.smithi019.stdout:3/832: write d9/f6a [1236948,128458] 0 2024-02-22T15:50:48.098 INFO:tasks.workunit.client.1.smithi062.stdout:2/839: write d1/d8/d62/d73/f75 [2809956,85544] 0 2024-02-22T15:50:48.099 INFO:tasks.workunit.client.1.smithi062.stdout:9/737: fsync d4/d1c/f1d 0 2024-02-22T15:50:48.100 INFO:tasks.workunit.client.0.smithi019.stdout:5/979: creat d8/dd/d19/d3a/d46/d7e/dbe/de8/f149 x:0 0 0 2024-02-22T15:50:48.102 INFO:tasks.workunit.client.0.smithi019.stdout:0/898: dwrite d4/d9/f26 [0,4194304] 0 2024-02-22T15:50:48.103 INFO:tasks.workunit.client.1.smithi062.stdout:9/738: chown d4/d1c/fc1 0 1 2024-02-22T15:50:48.103 INFO:tasks.workunit.client.1.smithi062.stdout:4/841: dwrite d4/d1c/d2c/f101 [0,4194304] 0 2024-02-22T15:50:48.103 INFO:tasks.workunit.client.0.smithi019.stdout:8/892: creat d9/d14/d17/d3d/d3f/dd4/f12b x:0 0 0 2024-02-22T15:50:48.104 INFO:tasks.workunit.client.0.smithi019.stdout:4/888: write d2/db/d92/ffc [4412909,70471] 0 2024-02-22T15:50:48.107 INFO:tasks.workunit.client.0.smithi019.stdout:5/980: rmdir d8/dd/d19/d38/d47 39 2024-02-22T15:50:48.107 INFO:tasks.workunit.client.0.smithi019.stdout:9/928: creat d2/d1f/d2e/d48/f13e x:0 0 0 2024-02-22T15:50:48.108 INFO:tasks.workunit.client.0.smithi019.stdout:0/899: chown d4/db/d24/d80/daf/f114 165615285 1 2024-02-22T15:50:48.110 INFO:tasks.workunit.client.0.smithi019.stdout:5/981: dread - d8/dd/d19/d3a/d46/d7e/dbe/fd7 zero size 2024-02-22T15:50:48.111 INFO:tasks.workunit.client.1.smithi062.stdout:2/840: rename d1/d8/df/f85 to d1/d47/d32/d52/d80/f11e 0 2024-02-22T15:50:48.112 INFO:tasks.workunit.client.1.smithi062.stdout:5/665: truncate d0/d3/da/d32/d33/f88 3318494 0 2024-02-22T15:50:48.112 INFO:tasks.workunit.client.0.smithi019.stdout:8/893: creat d9/d14/d6f/d7e/dff/f12c x:0 0 0 2024-02-22T15:50:48.114 INFO:tasks.workunit.client.1.smithi062.stdout:3/633: dread d5/d16/d21/d36/d5a/d85/fb2 [0,4194304] 0 2024-02-22T15:50:48.114 INFO:tasks.workunit.client.0.smithi019.stdout:3/833: mkdir d9/d26/d3f/d111 0 2024-02-22T15:50:48.115 INFO:tasks.workunit.client.0.smithi019.stdout:0/900: dread d4/d32/ffe [0,4194304] 0 2024-02-22T15:50:48.116 INFO:tasks.workunit.client.1.smithi062.stdout:2/841: chown d1/d8/d62/c6a 165 1 2024-02-22T15:50:48.116 INFO:tasks.workunit.client.1.smithi062.stdout:5/666: truncate d0/f39 4660818 0 2024-02-22T15:50:48.117 INFO:tasks.workunit.client.1.smithi062.stdout:9/739: creat d4/d15/d5f/d75/d84/fe9 x:0 0 0 2024-02-22T15:50:48.118 INFO:tasks.workunit.client.1.smithi062.stdout:0/830: truncate d1/d13/dde/d76/d8c/f4c 961331 0 2024-02-22T15:50:48.120 INFO:tasks.workunit.client.0.smithi019.stdout:3/834: symlink d9/d26/d76/l112 0 2024-02-22T15:50:48.120 INFO:tasks.workunit.client.1.smithi062.stdout:4/842: dwrite d4/d1c/d30/da7/f125 [0,4194304] 0 2024-02-22T15:50:48.121 INFO:tasks.workunit.client.0.smithi019.stdout:0/901: creat d4/db/d76/dc6/f132 x:0 0 0 2024-02-22T15:50:48.123 INFO:tasks.workunit.client.0.smithi019.stdout:8/894: dwrite d9/d14/d1f/d5c/f108 [0,4194304] 0 2024-02-22T15:50:48.124 INFO:tasks.workunit.client.1.smithi062.stdout:1/673: truncate d1/d6/d7/d2b/d47/d63/db2/fbe 1549326 0 2024-02-22T15:50:48.124 INFO:tasks.workunit.client.1.smithi062.stdout:0/831: stat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e 0 2024-02-22T15:50:48.124 INFO:tasks.workunit.client.0.smithi019.stdout:8/895: stat d9/d14/d17/f85 0 2024-02-22T15:50:48.125 INFO:tasks.workunit.client.1.smithi062.stdout:4/843: write d4/d1c/fe4 [603638,67334] 0 2024-02-22T15:50:48.129 INFO:tasks.workunit.client.1.smithi062.stdout:3/634: rename d5/d16/f48 to d5/d16/d21/d36/d5a/d85/fc6 0 2024-02-22T15:50:48.129 INFO:tasks.workunit.client.0.smithi019.stdout:0/902: mkdir d4/db/d1b/d63/db0/da1/dbb/d133 0 2024-02-22T15:50:48.130 INFO:tasks.workunit.client.1.smithi062.stdout:5/667: truncate d0/d3/da/f18 44147 0 2024-02-22T15:50:48.132 INFO:tasks.workunit.client.1.smithi062.stdout:2/842: truncate d1/d28/fc2 1206714 0 2024-02-22T15:50:48.133 INFO:tasks.workunit.client.1.smithi062.stdout:0/832: chown d1/d13/dde/d76/d8c/d5d/dc3/f24 39878166 1 2024-02-22T15:50:48.134 INFO:tasks.workunit.client.0.smithi019.stdout:0/903: chown d4/db/d76/ceb 10362869 1 2024-02-22T15:50:48.135 INFO:tasks.workunit.client.0.smithi019.stdout:4/889: write d2/f6 [5611734,63455] 0 2024-02-22T15:50:48.135 INFO:tasks.workunit.client.1.smithi062.stdout:9/740: dwrite d4/d1c/f56 [4194304,4194304] 0 2024-02-22T15:50:48.136 INFO:tasks.workunit.client.0.smithi019.stdout:9/929: write d2/d1f/d2e/d48/d45/d5a/f72 [1559423,68621] 0 2024-02-22T15:50:48.137 INFO:tasks.workunit.client.0.smithi019.stdout:0/904: rmdir d4/d67 39 2024-02-22T15:50:48.138 INFO:tasks.workunit.client.1.smithi062.stdout:4/844: rename d4/d2a/c62 to d4/d2a/d75/c12a 0 2024-02-22T15:50:48.138 INFO:tasks.workunit.client.0.smithi019.stdout:8/896: dwrite d9/d14/d1f/ffd [0,4194304] 0 2024-02-22T15:50:48.142 INFO:tasks.workunit.client.0.smithi019.stdout:0/905: write d4/db/d1b/f1f [3702290,77979] 0 2024-02-22T15:50:48.143 INFO:tasks.workunit.client.1.smithi062.stdout:1/674: chown d1/d6/d7/d2b/d49/da9/fb8 13011 1 2024-02-22T15:50:48.144 INFO:tasks.workunit.client.0.smithi019.stdout:9/930: chown d2/d7/dc/d27/dbc/d110/d138/d60/d8c/d66/fc6 40615138 1 2024-02-22T15:50:48.146 INFO:tasks.workunit.client.1.smithi062.stdout:1/675: fdatasync d1/d6/d7/d2b/d47/d7e/fc5 0 2024-02-22T15:50:48.146 INFO:tasks.workunit.client.1.smithi062.stdout:3/635: unlink d5/d16/d21/fa5 0 2024-02-22T15:50:48.148 INFO:tasks.workunit.client.0.smithi019.stdout:8/897: unlink d9/d14/d17/d1c/da0/c96 0 2024-02-22T15:50:48.149 INFO:tasks.workunit.client.1.smithi062.stdout:4/845: creat d4/d2a/f12b x:0 0 0 2024-02-22T15:50:48.150 INFO:tasks.workunit.client.1.smithi062.stdout:0/833: rename d1/d13/dde/d1d/d3b to d1/d13/dde/d1d/ddb/dfe/d112 0 2024-02-22T15:50:48.152 INFO:tasks.workunit.client.1.smithi062.stdout:0/834: write d1/d13/dde/d76/fd6 [53581,111660] 0 2024-02-22T15:50:48.153 INFO:tasks.workunit.client.1.smithi062.stdout:5/668: mkdir d0/d3/da/d13/d57/dc7/dda 0 2024-02-22T15:50:48.156 INFO:tasks.workunit.client.0.smithi019.stdout:0/906: write d4/d22/d2c/fdd [992879,26227] 0 2024-02-22T15:50:48.157 INFO:tasks.workunit.client.0.smithi019.stdout:5/982: dwrite d8/dd/d19/d38/fb9 [4194304,4194304] 0 2024-02-22T15:50:48.159 INFO:tasks.workunit.client.1.smithi062.stdout:3/636: rmdir d5/d1a/d3b/dc1 39 2024-02-22T15:50:48.159 INFO:tasks.workunit.client.0.smithi019.stdout:5/983: dread - d8/dd/d19/d77/dd2/dd9/fff zero size 2024-02-22T15:50:48.160 INFO:tasks.workunit.client.0.smithi019.stdout:0/907: rmdir d4/d9 39 2024-02-22T15:50:48.162 INFO:tasks.workunit.client.1.smithi062.stdout:4/846: mkdir d4/d2a/d7d/d12c 0 2024-02-22T15:50:48.162 INFO:tasks.workunit.client.0.smithi019.stdout:5/984: fsync d8/dd/d19/d3a/d46/d7e/dbe/dc3/fea 0 2024-02-22T15:50:48.163 INFO:tasks.workunit.client.1.smithi062.stdout:4/847: chown d4/d66/f11a 4 1 2024-02-22T15:50:48.163 INFO:tasks.workunit.client.0.smithi019.stdout:0/908: getdents d4/db/d1b/d40/d44 0 2024-02-22T15:50:48.163 INFO:tasks.workunit.client.0.smithi019.stdout:5/985: readlink d8/dd/d19/d3a/d46/ldf 0 2024-02-22T15:50:48.164 INFO:tasks.workunit.client.1.smithi062.stdout:9/741: link d4/d15/d5e/d63/da0/f25 d4/d1c/d83/d87/fea 0 2024-02-22T15:50:48.166 INFO:tasks.workunit.client.1.smithi062.stdout:2/843: rmdir d1/d28/df6 0 2024-02-22T15:50:48.169 INFO:tasks.workunit.client.1.smithi062.stdout:3/637: creat d5/d16/d21/d32/d6a/fc7 x:0 0 0 2024-02-22T15:50:48.169 INFO:tasks.workunit.client.0.smithi019.stdout:5/986: symlink d8/dd/d19/d3a/d46/d7e/dbe/l14a 0 2024-02-22T15:50:48.171 INFO:tasks.workunit.client.0.smithi019.stdout:0/909: dwrite d4/db/d24/d80/d122/f12e [0,4194304] 0 2024-02-22T15:50:48.172 INFO:tasks.workunit.client.0.smithi019.stdout:5/987: symlink d8/dd/d8e/d2c/d10e/l14b 0 2024-02-22T15:50:48.173 INFO:tasks.workunit.client.1.smithi062.stdout:4/848: dwrite d4/f45 [8388608,4194304] 0 2024-02-22T15:50:48.173 INFO:tasks.workunit.client.0.smithi019.stdout:0/910: mknod d4/db/d49/c134 0 2024-02-22T15:50:48.174 INFO:tasks.workunit.client.0.smithi019.stdout:5/988: mkdir d8/dd/d19/d38/d54/d14c 0 2024-02-22T15:50:48.174 INFO:tasks.workunit.client.0.smithi019.stdout:0/911: fdatasync d4/d23/d7d/f85 0 2024-02-22T15:50:48.175 INFO:tasks.workunit.client.0.smithi019.stdout:0/912: chown d4/db/d24/d80/daf 0 1 2024-02-22T15:50:48.176 INFO:tasks.workunit.client.1.smithi062.stdout:7/583: sync 2024-02-22T15:50:48.177 INFO:tasks.workunit.client.0.smithi019.stdout:0/913: truncate d4/d22/d2c/d68/f6a 4435815 0 2024-02-22T15:50:48.178 INFO:tasks.workunit.client.0.smithi019.stdout:5/989: creat d8/dd/d19/d77/f14d x:0 0 0 2024-02-22T15:50:48.179 INFO:tasks.workunit.client.0.smithi019.stdout:0/914: fdatasync d4/d9/ffd 0 2024-02-22T15:50:48.182 INFO:tasks.workunit.client.0.smithi019.stdout:0/915: creat d4/db/d76/d77/f135 x:0 0 0 2024-02-22T15:50:48.182 INFO:tasks.workunit.client.1.smithi062.stdout:0/835: rename d1/d13/dde/lee to d1/l113 0 2024-02-22T15:50:48.183 INFO:tasks.workunit.client.1.smithi062.stdout:4/849: write d4/d2a/d2d/d2e/d6a/f126 [182101,80624] 0 2024-02-22T15:50:48.187 INFO:tasks.workunit.client.1.smithi062.stdout:7/584: creat d0/d6/d2e/d55/d80/fb4 x:0 0 0 2024-02-22T15:50:48.188 INFO:tasks.workunit.client.0.smithi019.stdout:4/890: dread d2/db/d92/ffc [0,4194304] 0 2024-02-22T15:50:48.190 INFO:tasks.workunit.client.1.smithi062.stdout:4/850: mknod d4/d2a/d2d/d2e/d6a/d9e/c12d 0 2024-02-22T15:50:48.191 INFO:tasks.workunit.client.1.smithi062.stdout:2/844: dread d1/d8/f6e [0,4194304] 0 2024-02-22T15:50:48.198 INFO:tasks.workunit.client.0.smithi019.stdout:9/931: write d2/d1f/db5/fde [2818529,59628] 0 2024-02-22T15:50:48.204 INFO:tasks.workunit.client.1.smithi062.stdout:4/851: creat d4/d2a/d2d/d2e/d6a/d94/dfe/f12e x:0 0 0 2024-02-22T15:50:48.204 INFO:tasks.workunit.client.1.smithi062.stdout:0/836: dwrite d1/d3/de/ff [4194304,4194304] 0 2024-02-22T15:50:48.206 INFO:tasks.workunit.client.0.smithi019.stdout:9/932: dwrite d2/d7/dc/f49 [0,4194304] 0 2024-02-22T15:50:48.210 INFO:tasks.workunit.client.1.smithi062.stdout:4/852: mkdir d4/d1c/d2c/d6b/d70/de8/dfc/d12f 0 2024-02-22T15:50:48.211 INFO:tasks.workunit.client.1.smithi062.stdout:2/845: dwrite d1/d8/d62/d73/dbd/dbb/dc4/dfe/f11b [0,4194304] 0 2024-02-22T15:50:48.211 INFO:tasks.workunit.client.0.smithi019.stdout:9/933: unlink d2/d7/dc/d27/d68/dca/c128 0 2024-02-22T15:50:48.212 INFO:tasks.workunit.client.0.smithi019.stdout:9/934: write d2/d7/dc/f29 [141292,26432] 0 2024-02-22T15:50:48.212 INFO:tasks.workunit.client.1.smithi062.stdout:9/742: dread d4/d1c/f27 [0,4194304] 0 2024-02-22T15:50:48.217 INFO:tasks.workunit.client.0.smithi019.stdout:9/935: truncate d2/dc8/fb0 4593607 0 2024-02-22T15:50:48.218 INFO:tasks.workunit.client.1.smithi062.stdout:4/853: dread d4/d66/f6f [0,4194304] 0 2024-02-22T15:50:48.223 INFO:tasks.workunit.client.1.smithi062.stdout:4/854: mkdir d4/d1c/d2c/d6b/d70/d130 0 2024-02-22T15:50:48.225 INFO:tasks.workunit.client.1.smithi062.stdout:3/638: dread d5/d16/d21/d32/d6a/f78 [0,4194304] 0 2024-02-22T15:50:48.226 INFO:tasks.workunit.client.0.smithi019.stdout:9/936: dwrite d2/d1f/d2e/d48/f13e [0,4194304] 0 2024-02-22T15:50:48.226 INFO:tasks.workunit.client.1.smithi062.stdout:9/743: rename d4/d15/d5e/l82 to d4/d15/d5f/d75/d84/leb 0 2024-02-22T15:50:48.228 INFO:tasks.workunit.client.1.smithi062.stdout:7/585: dread d0/d14/d13/f36 [0,4194304] 0 2024-02-22T15:50:48.229 INFO:tasks.workunit.client.1.smithi062.stdout:3/639: chown d5/d16/c8d 1655473 1 2024-02-22T15:50:48.229 INFO:tasks.workunit.client.0.smithi019.stdout:9/937: mknod d2/d7/dc/d97/c13f 0 2024-02-22T15:50:48.231 INFO:tasks.workunit.client.0.smithi019.stdout:9/938: creat d2/d1f/d2e/d48/d45/d5a/f140 x:0 0 0 2024-02-22T15:50:48.231 INFO:tasks.workunit.client.1.smithi062.stdout:3/640: dread d5/d16/d21/f60 [0,4194304] 0 2024-02-22T15:50:48.232 INFO:tasks.workunit.client.1.smithi062.stdout:2/846: rename d1/d8/d62/d73/dbd/dbb/dc4/lf4 to d1/d47/d32/d52/d80/l11f 0 2024-02-22T15:50:48.233 INFO:tasks.workunit.client.1.smithi062.stdout:4/855: truncate d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/feb 1973611 0 2024-02-22T15:50:48.235 INFO:tasks.workunit.client.1.smithi062.stdout:2/847: readlink d1/d47/d39/l49 0 2024-02-22T15:50:48.236 INFO:tasks.workunit.client.1.smithi062.stdout:3/641: mknod d5/d1a/d3c/d62/cc8 0 2024-02-22T15:50:48.237 INFO:tasks.workunit.client.0.smithi019.stdout:9/939: dwrite d2/f6 [8388608,4194304] 0 2024-02-22T15:50:48.238 INFO:tasks.workunit.client.1.smithi062.stdout:2/848: chown d1/c30 1040306656 1 2024-02-22T15:50:48.240 INFO:tasks.workunit.client.0.smithi019.stdout:9/940: mknod d2/dec/d79/c141 0 2024-02-22T15:50:48.242 INFO:tasks.workunit.client.0.smithi019.stdout:9/941: truncate d2/d1f/d2e/d48/d45/f61 1355439 0 2024-02-22T15:50:48.244 INFO:tasks.workunit.client.0.smithi019.stdout:9/942: rmdir d2/dec/dd4 39 2024-02-22T15:50:48.246 INFO:tasks.workunit.client.0.smithi019.stdout:9/943: truncate d2/d1f/d2e/d48/d36/f83 4380989 0 2024-02-22T15:50:48.248 INFO:tasks.workunit.client.0.smithi019.stdout:9/944: mknod d2/d1f/c142 0 2024-02-22T15:50:48.248 INFO:tasks.workunit.client.0.smithi019.stdout:9/945: chown d2/d7/dc/dd6 1841 1 2024-02-22T15:50:48.248 INFO:tasks.workunit.client.1.smithi062.stdout:7/586: dread d0/d6/f23 [0,4194304] 0 2024-02-22T15:50:48.248 INFO:tasks.workunit.client.1.smithi062.stdout:4/856: dwrite d4/d2a/d2d/d2e/d6a/d8a/ddb/fe0 [0,4194304] 0 2024-02-22T15:50:48.249 INFO:tasks.workunit.client.0.smithi019.stdout:9/946: chown d2/d7/l17 0 1 2024-02-22T15:50:48.256 INFO:tasks.workunit.client.1.smithi062.stdout:4/857: creat d4/d1c/d30/da5/f131 x:0 0 0 2024-02-22T15:50:48.257 INFO:tasks.workunit.client.1.smithi062.stdout:4/858: write d4/d34/f103 [154656,91071] 0 2024-02-22T15:50:48.258 INFO:tasks.workunit.client.1.smithi062.stdout:7/587: link d0/d6/d2e/d58/f5c d0/d14/d19/d77/fb5 0 2024-02-22T15:50:48.262 INFO:tasks.workunit.client.1.smithi062.stdout:4/859: mkdir d4/d1c/d2c/d6b/d70/d130/d132 0 2024-02-22T15:50:48.265 INFO:tasks.workunit.client.1.smithi062.stdout:4/860: symlink d4/d2a/d7d/d12c/l133 0 2024-02-22T15:50:48.267 INFO:tasks.workunit.client.1.smithi062.stdout:7/588: read d0/d14/d19/d30/d37/f95 [1957593,44762] 0 2024-02-22T15:50:48.267 INFO:tasks.workunit.client.1.smithi062.stdout:2/849: dread d1/d8/df/f31 [0,4194304] 0 2024-02-22T15:50:48.270 INFO:tasks.workunit.client.1.smithi062.stdout:7/589: creat d0/d14/d19/d30/fb6 x:0 0 0 2024-02-22T15:50:48.276 INFO:tasks.workunit.client.1.smithi062.stdout:7/590: write d0/f89 [1650096,51654] 0 2024-02-22T15:50:48.278 INFO:tasks.workunit.client.1.smithi062.stdout:2/850: dwrite d1/d8/df/f25 [4194304,4194304] 0 2024-02-22T15:50:48.284 INFO:tasks.workunit.client.1.smithi062.stdout:7/591: truncate d0/d14/d19/f98 1112337 0 2024-02-22T15:50:48.285 INFO:tasks.workunit.client.1.smithi062.stdout:7/592: fdatasync d0/d6/d2e/d4a/f2d 0 2024-02-22T15:50:48.306 INFO:tasks.workunit.client.0.smithi019.stdout:5/990: rmdir d8/dd/d19/d38/d54 39 2024-02-22T15:50:48.307 INFO:tasks.workunit.client.0.smithi019.stdout:5/991: chown d8/d11/d33/ca5 3760 1 2024-02-22T15:50:48.308 INFO:tasks.workunit.client.0.smithi019.stdout:4/891: write d2/db/d14/d23/fa7 [673146,32392] 0 2024-02-22T15:50:48.311 INFO:tasks.workunit.client.0.smithi019.stdout:5/992: chown d8/dd/d19/d77/dd2/def/lf1 2258 1 2024-02-22T15:50:48.312 INFO:tasks.workunit.client.0.smithi019.stdout:4/892: symlink d2/db/d14/d23/d9e/d10a/l11c 0 2024-02-22T15:50:48.315 INFO:tasks.workunit.client.0.smithi019.stdout:0/916: dwrite d4/d22/d2c/fac [0,4194304] 0 2024-02-22T15:50:48.315 INFO:tasks.workunit.client.0.smithi019.stdout:5/993: dread - d8/dd/d8e/d2c/d10e/f133 zero size 2024-02-22T15:50:48.316 INFO:tasks.workunit.client.0.smithi019.stdout:4/893: chown d2/db/d14/d61/f73 93017 1 2024-02-22T15:50:48.318 INFO:tasks.workunit.client.1.smithi062.stdout:5/669: truncate d0/d3/d30/d71/d7c/fab 2648463 0 2024-02-22T15:50:48.319 INFO:tasks.workunit.client.0.smithi019.stdout:0/917: write d4/d67/de5/fef [1118563,148] 0 2024-02-22T15:50:48.320 INFO:tasks.workunit.client.0.smithi019.stdout:5/994: write d8/dd/d19/d38/d54/f108 [12422,107892] 0 2024-02-22T15:50:48.321 INFO:tasks.workunit.client.1.smithi062.stdout:5/670: fdatasync d0/d3/da/d13/d23/d4e/d5a/fcf 0 2024-02-22T15:50:48.322 INFO:tasks.workunit.client.0.smithi019.stdout:0/918: rmdir d4/db/d49/dc2 39 2024-02-22T15:50:48.323 INFO:tasks.workunit.client.0.smithi019.stdout:9/947: write d2/dc8/f9e [575629,126513] 0 2024-02-22T15:50:48.324 INFO:tasks.workunit.client.1.smithi062.stdout:5/671: readlink d0/d3/d2f/d59/d67/d69/l48 0 2024-02-22T15:50:48.326 INFO:tasks.workunit.client.0.smithi019.stdout:8/898: sync 2024-02-22T15:50:48.326 INFO:tasks.workunit.client.0.smithi019.stdout:3/835: sync 2024-02-22T15:50:48.326 INFO:tasks.workunit.client.0.smithi019.stdout:0/919: write d4/db/d1b/d40/d44/d75/db7/f90 [266843,33058] 0 2024-02-22T15:50:48.327 INFO:tasks.workunit.client.0.smithi019.stdout:4/894: rename d2/db/d14/d23/d21/d42/fa9 to d2/db/d14/d23/d9e/dac/dc2/dc9/f11d 0 2024-02-22T15:50:48.328 INFO:tasks.workunit.client.0.smithi019.stdout:3/836: fdatasync d9/d14/d19/f10c 0 2024-02-22T15:50:48.332 INFO:tasks.workunit.client.0.smithi019.stdout:8/899: dread - d9/d14/d6f/d7e/d94/daa/dea/f10a zero size 2024-02-22T15:50:48.332 INFO:tasks.workunit.client.0.smithi019.stdout:3/837: chown d9/d22/d27 5509244 1 2024-02-22T15:50:48.333 INFO:tasks.workunit.client.0.smithi019.stdout:5/995: dwrite d8/dd/fc8 [0,4194304] 0 2024-02-22T15:50:48.338 INFO:tasks.workunit.client.0.smithi019.stdout:5/996: read d8/dd/d19/d77/dd2/f101 [2499228,61409] 0 2024-02-22T15:50:48.338 INFO:tasks.workunit.client.0.smithi019.stdout:3/838: creat d9/d26/d2d/df4/f113 x:0 0 0 2024-02-22T15:50:48.340 INFO:tasks.workunit.client.0.smithi019.stdout:5/997: stat d8/dd/f124 0 2024-02-22T15:50:48.340 INFO:tasks.workunit.client.0.smithi019.stdout:8/900: rmdir d9/d14/d1f/d20 39 2024-02-22T15:50:48.341 INFO:tasks.workunit.client.0.smithi019.stdout:3/839: truncate d9/d22/d27/d96/ff2 1332880 0 2024-02-22T15:50:48.341 INFO:tasks.workunit.client.0.smithi019.stdout:0/920: dread d4/d23/d7d/f88 [0,4194304] 0 2024-02-22T15:50:48.342 INFO:tasks.workunit.client.1.smithi062.stdout:0/837: dwrite d1/d13/dde/d1d/fa0 [0,4194304] 0 2024-02-22T15:50:48.344 INFO:tasks.workunit.client.0.smithi019.stdout:4/895: getdents d2/db/d14/d23/d21/d42/dc4/dd5 0 2024-02-22T15:50:48.344 INFO:tasks.workunit.client.1.smithi062.stdout:2/851: truncate d1/d8/d62/d73/f75 2535877 0 2024-02-22T15:50:48.344 INFO:tasks.workunit.client.1.smithi062.stdout:7/593: dread d0/d14/d19/d30/d37/f95 [0,4194304] 0 2024-02-22T15:50:48.344 INFO:tasks.workunit.client.0.smithi019.stdout:9/948: read d2/d7/dc/d27/dbc/fd8 [485064,130348] 0 2024-02-22T15:50:48.345 INFO:tasks.workunit.client.0.smithi019.stdout:3/840: chown d9/d14/d19/c90 4658 1 2024-02-22T15:50:48.347 INFO:tasks.workunit.client.0.smithi019.stdout:4/896: creat d2/d3f/d48/d76/f11e x:0 0 0 2024-02-22T15:50:48.350 INFO:tasks.workunit.client.0.smithi019.stdout:4/897: dread - d2/db/d14/d23/d21/d42/dc4/f111 zero size 2024-02-22T15:50:48.353 INFO:tasks.workunit.client.1.smithi062.stdout:3/642: dwrite d5/f6f [0,4194304] 0 2024-02-22T15:50:48.354 INFO:tasks.workunit.client.1.smithi062.stdout:0/838: creat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/df4/f114 x:0 0 0 2024-02-22T15:50:48.354 INFO:tasks.workunit.client.0.smithi019.stdout:8/901: rename d9/d14/d1f/d4c/l71 to d9/d14/d1f/l12d 0 2024-02-22T15:50:48.354 INFO:tasks.workunit.client.0.smithi019.stdout:5/998: link d8/dd/d19/d38/d54/db0/d10d/d122/dde/c12f d8/dd/d19/d3a/d46/d7e/c14e 0 2024-02-22T15:50:48.357 INFO:tasks.workunit.client.1.smithi062.stdout:0/839: read d1/d38/f98 [3216424,116897] 0 2024-02-22T15:50:48.359 INFO:tasks.workunit.client.0.smithi019.stdout:5/999: dread - d8/dd/f12e zero size 2024-02-22T15:50:48.362 INFO:tasks.workunit.client.1.smithi062.stdout:3/643: getdents d5/d16/d21/d32/d75 0 2024-02-22T15:50:48.363 INFO:tasks.workunit.client.0.smithi019.stdout:9/949: dwrite d2/d7/dc/d27/dbc/d110/d138/d60/d8c/d66/fef [0,4194304] 0 2024-02-22T15:50:48.363 INFO:tasks.workunit.client.1.smithi062.stdout:5/672: dread d0/d3/da/d32/d42/f65 [0,4194304] 0 2024-02-22T15:50:48.363 INFO:tasks.workunit.client.0.smithi019.stdout:3/841: dwrite d9/d26/d3f/d5c/f105 [0,4194304] 0 2024-02-22T15:50:48.364 INFO:tasks.workunit.client.1.smithi062.stdout:0/840: rename d1/d13/d43/f53 to d1/d13/dde/d1d/ddb/f115 0 2024-02-22T15:50:48.365 INFO:tasks.workunit.client.1.smithi062.stdout:3/644: creat d5/da6/fc9 x:0 0 0 2024-02-22T15:50:48.366 INFO:tasks.workunit.client.0.smithi019.stdout:9/950: dread - d2/d7/dc/d27/dbc/d110/f12d zero size 2024-02-22T15:50:48.367 INFO:tasks.workunit.client.1.smithi062.stdout:3/645: chown d5/d16/d21/d32/d9e 2 1 2024-02-22T15:50:48.367 INFO:tasks.workunit.client.1.smithi062.stdout:0/841: chown d1/d13/dde/d76/d8c/d2c/d87/fb9 237831 1 2024-02-22T15:50:48.368 INFO:tasks.workunit.client.1.smithi062.stdout:5/673: creat d0/d3/d2f/d59/d67/d69/d3f/d5f/fdb x:0 0 0 2024-02-22T15:50:48.369 INFO:tasks.workunit.client.1.smithi062.stdout:3/646: write d5/d1a/f25 [2791072,53685] 0 2024-02-22T15:50:48.372 INFO:tasks.workunit.client.0.smithi019.stdout:9/951: symlink d2/d7/dc/d27/d68/l143 0 2024-02-22T15:50:48.377 INFO:tasks.workunit.client.0.smithi019.stdout:9/952: creat d2/d7/dc/d27/dbc/d110/d138/f144 x:0 0 0 2024-02-22T15:50:48.379 INFO:tasks.workunit.client.1.smithi062.stdout:3/647: read d5/d26/d2d/f90 [184912,61957] 0 2024-02-22T15:50:48.379 INFO:tasks.workunit.client.0.smithi019.stdout:9/953: mkdir d2/d12f/d145 0 2024-02-22T15:50:48.380 INFO:tasks.workunit.client.0.smithi019.stdout:9/954: chown d2/d7/dee/c122 6 1 2024-02-22T15:50:48.383 INFO:tasks.workunit.client.1.smithi062.stdout:5/674: unlink d0/d3/d2f/d59/d67/d69/d3f/d5f/f89 0 2024-02-22T15:50:48.383 INFO:tasks.workunit.client.0.smithi019.stdout:9/955: fdatasync d2/d7/f137 0 2024-02-22T15:50:48.383 INFO:tasks.workunit.client.0.smithi019.stdout:9/956: mkdir d2/d1f/db5/d146 0 2024-02-22T15:50:48.386 INFO:tasks.workunit.client.0.smithi019.stdout:3/842: dread d9/d26/d2d/d39/f5f [0,4194304] 0 2024-02-22T15:50:48.386 INFO:tasks.workunit.client.1.smithi062.stdout:3/648: creat d5/d16/d21/d84/fca x:0 0 0 2024-02-22T15:50:48.387 INFO:tasks.workunit.client.0.smithi019.stdout:3/843: fsync d9/f2f 0 2024-02-22T15:50:48.387 INFO:tasks.workunit.client.1.smithi062.stdout:5/675: mknod d0/d3/da/d13/d23/d78/d8c/dbd/cdc 0 2024-02-22T15:50:48.387 INFO:tasks.workunit.client.1.smithi062.stdout:8/865: sync 2024-02-22T15:50:48.389 INFO:tasks.workunit.client.0.smithi019.stdout:9/957: read d2/dec/d79/fb2 [516163,70628] 0 2024-02-22T15:50:48.392 INFO:tasks.workunit.client.0.smithi019.stdout:9/958: mkdir d2/d7/dee/d147 0 2024-02-22T15:50:48.393 INFO:tasks.workunit.client.1.smithi062.stdout:3/649: mkdir d5/d16/d56/dbc/dcb 0 2024-02-22T15:50:48.395 INFO:tasks.workunit.client.1.smithi062.stdout:7/594: dwrite d0/d6/f52 [0,4194304] 0 2024-02-22T15:50:48.397 INFO:tasks.workunit.client.0.smithi019.stdout:3/844: dwrite d9/d26/d3f/f8c [0,4194304] 0 2024-02-22T15:50:48.397 INFO:tasks.workunit.client.0.smithi019.stdout:8/902: dread f3 [0,4194304] 0 2024-02-22T15:50:48.399 INFO:tasks.workunit.client.1.smithi062.stdout:2/852: dwrite d1/d1e/d38/f4f [0,4194304] 0 2024-02-22T15:50:48.399 INFO:tasks.workunit.client.0.smithi019.stdout:9/959: creat d2/d7/dc/f148 x:0 0 0 2024-02-22T15:50:48.399 INFO:tasks.workunit.client.0.smithi019.stdout:4/898: truncate d2/db/d14/d23/d4b/f86 3864462 0 2024-02-22T15:50:48.400 INFO:tasks.workunit.client.0.smithi019.stdout:8/903: creat d9/d88/f12e x:0 0 0 2024-02-22T15:50:48.404 INFO:tasks.workunit.client.0.smithi019.stdout:9/960: symlink d2/d7/l149 0 2024-02-22T15:50:48.405 INFO:tasks.workunit.client.1.smithi062.stdout:8/866: link d8/d26/fec d8/d9/d2c/d49/d94/dc1/f10b 0 2024-02-22T15:50:48.406 INFO:tasks.workunit.client.0.smithi019.stdout:4/899: mkdir d2/db/d14/d23/d21/d42/dc4/d11f 0 2024-02-22T15:50:48.408 INFO:tasks.workunit.client.0.smithi019.stdout:8/904: creat d9/d14/d1f/d20/f12f x:0 0 0 2024-02-22T15:50:48.409 INFO:tasks.workunit.client.0.smithi019.stdout:8/905: fdatasync d9/d14/d17/f45 0 2024-02-22T15:50:48.411 INFO:tasks.workunit.client.0.smithi019.stdout:9/961: mkdir d2/d7/d14a 0 2024-02-22T15:50:48.411 INFO:tasks.workunit.client.0.smithi019.stdout:3/845: link d9/d26/d2d/d39/l8d d9/d26/d3f/d10b/dae/l114 0 2024-02-22T15:50:48.413 INFO:tasks.workunit.client.0.smithi019.stdout:8/906: mknod d9/d14/d1f/d5c/c130 0 2024-02-22T15:50:48.414 INFO:tasks.workunit.client.0.smithi019.stdout:9/962: chown d2/d1f/d2e/l5b 69 1 2024-02-22T15:50:48.416 INFO:tasks.workunit.client.0.smithi019.stdout:3/846: stat d9/d26/d2d/d53/lff 0 2024-02-22T15:50:48.417 INFO:tasks.workunit.client.1.smithi062.stdout:7/595: unlink d0/d14/d28/l3c 0 2024-02-22T15:50:48.418 INFO:tasks.workunit.client.0.smithi019.stdout:4/900: dwrite d2/db/d92/ffc [8388608,4194304] 0 2024-02-22T15:50:48.419 INFO:tasks.workunit.client.0.smithi019.stdout:9/963: fdatasync d2/d7/dc/d27/dbc/fd8 0 2024-02-22T15:50:48.420 INFO:tasks.workunit.client.1.smithi062.stdout:2/853: creat d1/d8/df/f120 x:0 0 0 2024-02-22T15:50:48.421 INFO:tasks.workunit.client.1.smithi062.stdout:3/650: rename d5/d16/d56/f7e to d5/d16/d21/d32/d6a/d7b/fcc 0 2024-02-22T15:50:48.421 INFO:tasks.workunit.client.1.smithi062.stdout:5/676: dwrite d0/d3/da/d13/f2a [0,4194304] 0 2024-02-22T15:50:48.423 INFO:tasks.workunit.client.0.smithi019.stdout:9/964: truncate d2/d1f/d2e/d48/d36/f115 991774 0 2024-02-22T15:50:48.426 INFO:tasks.workunit.client.1.smithi062.stdout:8/867: link d8/cd d8/d42/d87/c10c 0 2024-02-22T15:50:48.429 INFO:tasks.workunit.client.0.smithi019.stdout:9/965: mknod d2/d1f/d2e/d48/d45/c14b 0 2024-02-22T15:50:48.430 INFO:tasks.workunit.client.0.smithi019.stdout:4/901: chown d2/db/d14/d23/d9e/d10a/cde 0 1 2024-02-22T15:50:48.431 INFO:tasks.workunit.client.0.smithi019.stdout:3/847: dwrite d9/d22/d27/d96/fca [0,4194304] 0 2024-02-22T15:50:48.433 INFO:tasks.workunit.client.1.smithi062.stdout:3/651: rmdir d5/d26/d2d 39 2024-02-22T15:50:48.435 INFO:tasks.workunit.client.1.smithi062.stdout:5/677: unlink d0/d3/d2f/d59/d67/d69/cd1 0 2024-02-22T15:50:48.441 INFO:tasks.workunit.client.0.smithi019.stdout:4/902: mkdir d2/db/d14/d23/d21/d42/dc4/d11f/d120 0 2024-02-22T15:50:48.441 INFO:tasks.workunit.client.0.smithi019.stdout:9/966: dwrite d2/d7/dc/d27/dbc/d110/d138/d60/fe9 [0,4194304] 0 2024-02-22T15:50:48.441 INFO:tasks.workunit.client.1.smithi062.stdout:1/676: sync 2024-02-22T15:50:48.442 INFO:tasks.workunit.client.1.smithi062.stdout:9/744: sync 2024-02-22T15:50:48.442 INFO:tasks.workunit.client.1.smithi062.stdout:4/861: sync 2024-02-22T15:50:48.442 INFO:tasks.workunit.client.1.smithi062.stdout:6/767: sync 2024-02-22T15:50:48.442 INFO:tasks.workunit.client.1.smithi062.stdout:2/854: dwrite d1/d1e/d38/f10c [0,4194304] 0 2024-02-22T15:50:48.442 INFO:tasks.workunit.client.1.smithi062.stdout:3/652: fsync d5/d1a/f1e 0 2024-02-22T15:50:48.442 INFO:tasks.workunit.client.1.smithi062.stdout:7/596: dwrite d0/d14/d19/d30/d37/fa1 [0,4194304] 0 2024-02-22T15:50:48.443 INFO:tasks.workunit.client.0.smithi019.stdout:9/967: fsync d2/d7/dc/d27/d87/fb1 0 2024-02-22T15:50:48.446 INFO:tasks.workunit.client.0.smithi019.stdout:4/903: mkdir d2/db/d14/d23/d67/d121 0 2024-02-22T15:50:48.446 INFO:tasks.workunit.client.1.smithi062.stdout:0/842: dread d1/d13/dde/d76/d8c/d5d/dc3/f24 [4194304,4194304] 0 2024-02-22T15:50:48.447 INFO:tasks.workunit.client.0.smithi019.stdout:9/968: truncate d2/d1f/d2e/d48/d89/f54 601566 0 2024-02-22T15:50:48.447 INFO:tasks.workunit.client.0.smithi019.stdout:8/907: dread d9/f32 [4194304,4194304] 0 2024-02-22T15:50:48.448 INFO:tasks.workunit.client.0.smithi019.stdout:9/969: dread - d2/d1f/d2e/d48/d45/d5a/f140 zero size 2024-02-22T15:50:48.449 INFO:tasks.workunit.client.1.smithi062.stdout:9/745: read d4/d1c/f2c [4024776,52288] 0 2024-02-22T15:50:48.452 INFO:tasks.workunit.client.0.smithi019.stdout:8/908: creat d9/d14/d17/d1c/f131 x:0 0 0 2024-02-22T15:50:48.453 INFO:tasks.workunit.client.0.smithi019.stdout:8/909: readlink d9/d14/d6f/d7e/l11f 0 2024-02-22T15:50:48.458 INFO:tasks.workunit.client.0.smithi019.stdout:8/910: unlink d9/d14/f38 0 2024-02-22T15:50:48.459 INFO:tasks.workunit.client.1.smithi062.stdout:2/855: fsync d1/d47/d39/d5d/fcf 0 2024-02-22T15:50:48.459 INFO:tasks.workunit.client.0.smithi019.stdout:9/970: dwrite d2/d7/dc/fb7 [0,4194304] 0 2024-02-22T15:50:48.459 INFO:tasks.workunit.client.0.smithi019.stdout:8/911: chown d9/d14/d1f/f92 1033459139 1 2024-02-22T15:50:48.461 INFO:tasks.workunit.client.0.smithi019.stdout:9/971: chown d2/d7/dc/d27/dbc/l18 339795350 1 2024-02-22T15:50:48.461 INFO:tasks.workunit.client.1.smithi062.stdout:3/653: mkdir d5/d16/d21/d32/d6a/dcd 0 2024-02-22T15:50:48.461 INFO:tasks.workunit.client.1.smithi062.stdout:8/868: creat d8/d9/f10d x:0 0 0 2024-02-22T15:50:48.462 INFO:tasks.workunit.client.1.smithi062.stdout:7/597: mknod d0/d14/d19/d30/d37/cb7 0 2024-02-22T15:50:48.463 INFO:tasks.workunit.client.1.smithi062.stdout:0/843: creat d1/d13/dde/d76/d8c/d2c/d87/f116 x:0 0 0 2024-02-22T15:50:48.467 INFO:tasks.workunit.client.0.smithi019.stdout:9/972: mknod d2/dec/dd4/c14c 0 2024-02-22T15:50:48.469 INFO:tasks.workunit.client.1.smithi062.stdout:6/768: rename d5/df/d32/dd3 to d5/df/dbc/dcd/de9/d100 0 2024-02-22T15:50:48.469 INFO:tasks.workunit.client.1.smithi062.stdout:5/678: dread d0/d3/da/d13/d23/d4e/d5a/f5e [0,4194304] 0 2024-02-22T15:50:48.469 INFO:tasks.workunit.client.1.smithi062.stdout:2/856: creat d1/d1e/d38/d76/dc5/f121 x:0 0 0 2024-02-22T15:50:48.469 INFO:tasks.workunit.client.1.smithi062.stdout:8/869: stat d8/c11 0 2024-02-22T15:50:48.469 INFO:tasks.workunit.client.0.smithi019.stdout:9/973: dread d2/d7/dc/d27/dbc/f105 [0,4194304] 0 2024-02-22T15:50:48.470 INFO:tasks.workunit.client.1.smithi062.stdout:3/654: write d5/d16/d21/d55/fa0 [1938893,127347] 0 2024-02-22T15:50:48.471 INFO:tasks.workunit.client.0.smithi019.stdout:9/974: write d2/d7/dc/d27/dbc/d110/d138/d60/f132 [274489,61199] 0 2024-02-22T15:50:48.472 INFO:tasks.workunit.client.1.smithi062.stdout:0/844: mkdir d1/d13/dde/d76/d8c/d2c/d87/d117 0 2024-02-22T15:50:48.472 INFO:tasks.workunit.client.1.smithi062.stdout:5/679: symlink d0/d3/d2f/ldd 0 2024-02-22T15:50:48.474 INFO:tasks.workunit.client.0.smithi019.stdout:9/975: dread d2/d1f/d2e/d48/d45/d5a/daf/fbd [0,4194304] 0 2024-02-22T15:50:48.475 INFO:tasks.workunit.client.1.smithi062.stdout:5/680: fsync d0/d3/da/d13/d23/d4e/d60/f63 0 2024-02-22T15:50:48.479 INFO:tasks.workunit.client.1.smithi062.stdout:7/598: truncate d0/d6/d2e/d35/f94 591851 0 2024-02-22T15:50:48.479 INFO:tasks.workunit.client.1.smithi062.stdout:3/655: mknod d5/d16/d21/d36/cce 0 2024-02-22T15:50:48.480 INFO:tasks.workunit.client.1.smithi062.stdout:2/857: mknod d1/d8/c122 0 2024-02-22T15:50:48.481 INFO:tasks.workunit.client.1.smithi062.stdout:5/681: chown d0/d3/d2f/d59/d67/d69/d3f/d66/da9/lbe 1 1 2024-02-22T15:50:48.481 INFO:tasks.workunit.client.0.smithi019.stdout:9/976: dwrite d2/d1f/d2e/d48/d45/d5a/f72 [0,4194304] 0 2024-02-22T15:50:48.483 INFO:tasks.workunit.client.1.smithi062.stdout:6/769: dwrite d5/df/d40/d44/d76/d56/f5b [0,4194304] 0 2024-02-22T15:50:48.483 INFO:tasks.workunit.client.1.smithi062.stdout:3/656: chown d5/d16/d21/f79 10864054 1 2024-02-22T15:50:48.484 INFO:tasks.workunit.client.1.smithi062.stdout:8/870: mkdir d8/d9/d2c/d49/d94/d10e 0 2024-02-22T15:50:48.488 INFO:tasks.workunit.client.1.smithi062.stdout:4/862: getdents d4/d2a/d2d/d2e/d6a/d94 0 2024-02-22T15:50:48.489 INFO:tasks.workunit.client.1.smithi062.stdout:7/599: creat d0/d91/d7c/fb8 x:0 0 0 2024-02-22T15:50:48.491 INFO:tasks.workunit.client.1.smithi062.stdout:5/682: symlink d0/d3/da/d13/d23/d4e/lde 0 2024-02-22T15:50:48.493 INFO:tasks.workunit.client.1.smithi062.stdout:0/845: link d1/d3/d4/f99 d1/d3/d5f/f118 0 2024-02-22T15:50:48.494 INFO:tasks.workunit.client.0.smithi019.stdout:9/977: dwrite d2/f4f [0,4194304] 0 2024-02-22T15:50:48.494 INFO:tasks.workunit.client.1.smithi062.stdout:3/657: mknod d5/d16/d21/d36/d4a/ccf 0 2024-02-22T15:50:48.500 INFO:tasks.workunit.client.1.smithi062.stdout:7/600: creat d0/d6/d2e/d55/d80/fb9 x:0 0 0 2024-02-22T15:50:48.500 INFO:tasks.workunit.client.1.smithi062.stdout:5/683: creat d0/fdf x:0 0 0 2024-02-22T15:50:48.501 INFO:tasks.workunit.client.1.smithi062.stdout:0/846: truncate d1/d13/dde/d76/d8c/d5d/dc3/f24 9227406 0 2024-02-22T15:50:48.504 INFO:tasks.workunit.client.0.smithi019.stdout:9/978: dwrite d2/d7/dc/d27/dbc/d110/d138/d60/d8c/d66/f127 [0,4194304] 0 2024-02-22T15:50:48.505 INFO:tasks.workunit.client.1.smithi062.stdout:6/770: rename d5/df/d32/fc5 to d5/df/dbc/dcd/de9/d100/f101 0 2024-02-22T15:50:48.505 INFO:tasks.workunit.client.1.smithi062.stdout:5/684: dread - d0/d3/da/d13/d23/f49 zero size 2024-02-22T15:50:48.505 INFO:tasks.workunit.client.1.smithi062.stdout:3/658: unlink d5/d16/d21/d32/f51 0 2024-02-22T15:50:48.506 INFO:tasks.workunit.client.1.smithi062.stdout:3/659: write d5/f1d [1294964,109060] 0 2024-02-22T15:50:48.508 INFO:tasks.workunit.client.1.smithi062.stdout:5/685: read - d0/d3/d2f/d59/d67/d69/d3f/d66/f97 zero size 2024-02-22T15:50:48.511 INFO:tasks.workunit.client.1.smithi062.stdout:8/871: creat d8/d9/d48/d103/f10f x:0 0 0 2024-02-22T15:50:48.512 INFO:tasks.workunit.client.1.smithi062.stdout:4/863: rename d4/d1c/d2c/d6b/d70/de8/dfc/d12f to d4/d1c/d2c/db9/d134 0 2024-02-22T15:50:48.513 INFO:tasks.workunit.client.0.smithi019.stdout:9/979: rename d2/d7/dc/d27/dbc/f3f to d2/d7/dc/d27/dbc/dcc/f14d 0 2024-02-22T15:50:48.513 INFO:tasks.workunit.client.0.smithi019.stdout:9/980: chown d2/d7/dc/dd6 968 1 2024-02-22T15:50:48.515 INFO:tasks.workunit.client.1.smithi062.stdout:4/864: fdatasync d4/d1c/d30/d37/f123 0 2024-02-22T15:50:48.515 INFO:tasks.workunit.client.1.smithi062.stdout:0/847: mkdir d1/d13/dde/d76/d8c/d2c/db5/d119 0 2024-02-22T15:50:48.515 INFO:tasks.workunit.client.1.smithi062.stdout:5/686: dread - d0/d3/da/f68 zero size 2024-02-22T15:50:48.516 INFO:tasks.workunit.client.1.smithi062.stdout:7/601: symlink d0/d6/d2e/d2f/d88/lba 0 2024-02-22T15:50:48.518 INFO:tasks.workunit.client.0.smithi019.stdout:9/981: dread d2/dc8/fa2 [0,4194304] 0 2024-02-22T15:50:48.519 INFO:tasks.workunit.client.1.smithi062.stdout:5/687: write d0/d3/fb9 [448987,29683] 0 2024-02-22T15:50:48.519 INFO:tasks.workunit.client.1.smithi062.stdout:1/677: dread d1/d6/d7/d2b/d96/f9e [0,4194304] 0 2024-02-22T15:50:48.519 INFO:tasks.workunit.client.1.smithi062.stdout:7/602: creat d0/d91/fbb x:0 0 0 2024-02-22T15:50:48.520 INFO:tasks.workunit.client.1.smithi062.stdout:0/848: write d1/d13/dde/d76/d8c/f88 [1837665,103203] 0 2024-02-22T15:50:48.522 INFO:tasks.workunit.client.0.smithi019.stdout:9/982: creat d2/d7/d52/dc0/f14e x:0 0 0 2024-02-22T15:50:48.523 INFO:tasks.workunit.client.1.smithi062.stdout:4/865: fdatasync d4/df/f14 0 2024-02-22T15:50:48.524 INFO:tasks.workunit.client.0.smithi019.stdout:9/983: mknod d2/dec/c14f 0 2024-02-22T15:50:48.524 INFO:tasks.workunit.client.1.smithi062.stdout:7/603: write d0/d14/d19/f1a [973888,6236] 0 2024-02-22T15:50:48.525 INFO:tasks.workunit.client.0.smithi019.stdout:9/984: chown d2/d1f/db5/cf6 170653 1 2024-02-22T15:50:48.525 INFO:tasks.workunit.client.1.smithi062.stdout:5/688: chown d0/d3/d2f/d59/d67/d69 480 1 2024-02-22T15:50:48.526 INFO:tasks.workunit.client.0.smithi019.stdout:9/985: chown d2/d7/dc/d27/dbc/d130 4 1 2024-02-22T15:50:48.528 INFO:tasks.workunit.client.1.smithi062.stdout:5/689: write d0/fdf [739886,87468] 0 2024-02-22T15:50:48.529 INFO:tasks.workunit.client.0.smithi019.stdout:9/986: truncate d2/d1f/d2e/d48/d45/d5a/fa0 1365200 0 2024-02-22T15:50:48.530 INFO:tasks.workunit.client.1.smithi062.stdout:4/866: write d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f124 [413234,90883] 0 2024-02-22T15:50:48.531 INFO:tasks.workunit.client.0.smithi019.stdout:9/987: chown d2/l13 6 1 2024-02-22T15:50:48.533 INFO:tasks.workunit.client.1.smithi062.stdout:0/849: creat d1/d3/de/d50/f11a x:0 0 0 2024-02-22T15:50:48.535 INFO:tasks.workunit.client.0.smithi019.stdout:9/988: creat d2/d7/d52/dc0/f150 x:0 0 0 2024-02-22T15:50:48.538 INFO:tasks.workunit.client.1.smithi062.stdout:5/690: creat d0/d3/fe0 x:0 0 0 2024-02-22T15:50:48.539 INFO:tasks.workunit.client.1.smithi062.stdout:8/872: dread d8/d9/d2c/d49/d5e/f101 [0,4194304] 0 2024-02-22T15:50:48.540 INFO:tasks.workunit.client.0.smithi019.stdout:0/921: sync 2024-02-22T15:50:48.540 INFO:tasks.workunit.client.0.smithi019.stdout:9/989: link d2/d1f/d2e/d48/d89/ca8 d2/d7/dc/c151 0 2024-02-22T15:50:48.542 INFO:tasks.workunit.client.1.smithi062.stdout:4/867: dwrite d4/d2a/d2d/d2e/d6a/d94/dfe/f12e [0,4194304] 0 2024-02-22T15:50:48.542 INFO:tasks.workunit.client.0.smithi019.stdout:3/848: write d9/d65/f7d [780077,57359] 0 2024-02-22T15:50:48.544 INFO:tasks.workunit.client.0.smithi019.stdout:0/922: unlink d4/d22/dcc/fe4 0 2024-02-22T15:50:48.548 INFO:tasks.workunit.client.0.smithi019.stdout:3/849: truncate d9/d26/d3f/f4a 3545638 0 2024-02-22T15:50:48.548 INFO:tasks.workunit.client.0.smithi019.stdout:0/923: mknod d4/d9/da7/db2/c136 0 2024-02-22T15:50:48.550 INFO:tasks.workunit.client.1.smithi062.stdout:7/604: getdents d0/d6/d2e/d58/d5d 0 2024-02-22T15:50:48.550 INFO:tasks.workunit.client.1.smithi062.stdout:8/873: dread - d8/d9/d44/df8/d105/f4c zero size 2024-02-22T15:50:48.550 INFO:tasks.workunit.client.1.smithi062.stdout:0/850: dwrite d1/d3/de/f3a [0,4194304] 0 2024-02-22T15:50:48.550 INFO:tasks.workunit.client.1.smithi062.stdout:4/868: creat d4/d1c/d30/da7/f135 x:0 0 0 2024-02-22T15:50:48.550 INFO:tasks.workunit.client.1.smithi062.stdout:7/605: readlink d0/d6/d2e/d55/d80/d8a/lae 0 2024-02-22T15:50:48.551 INFO:tasks.workunit.client.1.smithi062.stdout:8/874: chown d8/d9/l15 931 1 2024-02-22T15:50:48.555 INFO:tasks.workunit.client.0.smithi019.stdout:3/850: link d9/d26/d3f/d5c/fe6 d9/f115 0 2024-02-22T15:50:48.555 INFO:tasks.workunit.client.1.smithi062.stdout:4/869: chown d4/d1c/d2c/d6b 29345511 1 2024-02-22T15:50:48.556 INFO:tasks.workunit.client.1.smithi062.stdout:0/851: rmdir d1/d38 39 2024-02-22T15:50:48.556 INFO:tasks.workunit.client.1.smithi062.stdout:8/875: fsync d8/d9/d2c/d74/fcc 0 2024-02-22T15:50:48.556 INFO:tasks.workunit.client.0.smithi019.stdout:0/924: dwrite d4/db/d24/f100 [0,4194304] 0 2024-02-22T15:50:48.562 INFO:tasks.workunit.client.0.smithi019.stdout:0/925: chown d4/db/d76/d77/cde 5 1 2024-02-22T15:50:48.562 INFO:tasks.workunit.client.1.smithi062.stdout:0/852: mknod d1/d13/dde/d76/d8c/d5d/dc3/d85/c11b 0 2024-02-22T15:50:48.563 INFO:tasks.workunit.client.1.smithi062.stdout:8/876: getdents d8/d86 0 2024-02-22T15:50:48.563 INFO:tasks.workunit.client.1.smithi062.stdout:6/771: dread d5/dd/d33/f48 [0,4194304] 0 2024-02-22T15:50:48.565 INFO:tasks.workunit.client.0.smithi019.stdout:0/926: fdatasync d4/d22/d2c/fdd 0 2024-02-22T15:50:48.565 INFO:tasks.workunit.client.1.smithi062.stdout:8/877: mkdir d8/d9/d44/df8/d110 0 2024-02-22T15:50:48.570 INFO:tasks.workunit.client.1.smithi062.stdout:8/878: mknod d8/d9/d2c/d49/d94/dc1/de9/dd1/c111 0 2024-02-22T15:50:48.571 INFO:tasks.workunit.client.1.smithi062.stdout:8/879: fsync d8/d42/f7c 0 2024-02-22T15:50:48.571 INFO:tasks.workunit.client.1.smithi062.stdout:9/746: sync 2024-02-22T15:50:48.574 INFO:tasks.workunit.client.1.smithi062.stdout:0/853: read d1/d3/d5f/f118 [1494624,5491] 0 2024-02-22T15:50:48.574 INFO:tasks.workunit.client.1.smithi062.stdout:3/660: dread d5/d16/d21/d36/d5a/d85/fc6 [0,4194304] 0 2024-02-22T15:50:48.575 INFO:tasks.workunit.client.0.smithi019.stdout:8/912: write d9/d14/d17/d1c/f51 [1482349,16647] 0 2024-02-22T15:50:48.578 INFO:tasks.workunit.client.0.smithi019.stdout:8/913: creat d9/d14/d1f/d6d/db5/f132 x:0 0 0 2024-02-22T15:50:48.579 INFO:tasks.workunit.client.1.smithi062.stdout:9/747: dread d4/d1c/fc1 [0,4194304] 0 2024-02-22T15:50:48.579 INFO:tasks.workunit.client.1.smithi062.stdout:8/880: unlink d8/d9/d44/df8/d105/d70/lf2 0 2024-02-22T15:50:48.579 INFO:tasks.workunit.client.1.smithi062.stdout:6/772: link d5/d8a/ld7 d5/df/dbc/dcd/de9/d100/l102 0 2024-02-22T15:50:48.583 INFO:tasks.workunit.client.1.smithi062.stdout:0/854: mkdir d1/d13/dde/d76/d8c/d2c/d11c 0 2024-02-22T15:50:48.584 INFO:tasks.workunit.client.1.smithi062.stdout:8/881: unlink d8/d9/f71 0 2024-02-22T15:50:48.585 INFO:tasks.workunit.client.1.smithi062.stdout:3/661: dwrite d5/da6/fc9 [0,4194304] 0 2024-02-22T15:50:48.585 INFO:tasks.workunit.client.1.smithi062.stdout:6/773: creat d5/df/d32/d47/f103 x:0 0 0 2024-02-22T15:50:48.590 INFO:tasks.workunit.client.1.smithi062.stdout:3/662: fsync d5/d16/d21/d32/d6a/fac 0 2024-02-22T15:50:48.593 INFO:tasks.workunit.client.1.smithi062.stdout:9/748: symlink d4/d15/d5f/d75/lec 0 2024-02-22T15:50:48.594 INFO:tasks.workunit.client.1.smithi062.stdout:0/855: readlink d1/d13/dde/d76/l7a 0 2024-02-22T15:50:48.595 INFO:tasks.workunit.client.1.smithi062.stdout:8/882: dwrite d8/d26/fb4 [0,4194304] 0 2024-02-22T15:50:48.595 INFO:tasks.workunit.client.1.smithi062.stdout:6/774: chown d5/dd/d33/f97 124549 1 2024-02-22T15:50:48.596 INFO:tasks.workunit.client.0.smithi019.stdout:8/914: dread d9/d14/d17/d3d/d83/fa6 [0,4194304] 0 2024-02-22T15:50:48.597 INFO:tasks.workunit.client.1.smithi062.stdout:8/883: dread - d8/d9/f10d zero size 2024-02-22T15:50:48.597 INFO:tasks.workunit.client.0.smithi019.stdout:8/915: chown d9/d14/d6f/d7e/d94/db8/fa1 33510420 1 2024-02-22T15:50:48.601 INFO:tasks.workunit.client.0.smithi019.stdout:8/916: rmdir d9/d14/d17/d1c 39 2024-02-22T15:50:48.603 INFO:tasks.workunit.client.1.smithi062.stdout:0/856: dread - d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/df4/f114 zero size 2024-02-22T15:50:48.604 INFO:tasks.workunit.client.1.smithi062.stdout:4/870: dread d4/d2a/d2d/f85 [0,4194304] 0 2024-02-22T15:50:48.605 INFO:tasks.workunit.client.0.smithi019.stdout:8/917: mkdir d9/d14/d6f/d133 0 2024-02-22T15:50:48.610 INFO:tasks.workunit.client.1.smithi062.stdout:4/871: read d4/d1c/d2c/d6b/d9b/fc0 [5605428,110213] 0 2024-02-22T15:50:48.613 INFO:tasks.workunit.client.1.smithi062.stdout:3/663: creat d5/d26/d2d/fd0 x:0 0 0 2024-02-22T15:50:48.614 INFO:tasks.workunit.client.0.smithi019.stdout:9/990: truncate d2/d7/dc/f29 16897 0 2024-02-22T15:50:48.614 INFO:tasks.workunit.client.1.smithi062.stdout:8/884: creat d8/d9/d44/d7e/f112 x:0 0 0 2024-02-22T15:50:48.616 INFO:tasks.workunit.client.1.smithi062.stdout:3/664: read - d5/d16/d21/d84/fca zero size 2024-02-22T15:50:48.617 INFO:tasks.workunit.client.0.smithi019.stdout:3/851: write d9/d22/d49/f7c [60979,58285] 0 2024-02-22T15:50:48.617 INFO:tasks.workunit.client.1.smithi062.stdout:0/857: truncate d1/d13/dde/d76/d8c/d2c/d84/f26 2184952 0 2024-02-22T15:50:48.618 INFO:tasks.workunit.client.1.smithi062.stdout:2/858: write d1/d8/d62/d73/dbd/d7d/f93 [1165324,24456] 0 2024-02-22T15:50:48.619 INFO:tasks.workunit.client.0.smithi019.stdout:0/927: truncate d4/d22/d2c/d68/d6d/fc9 2809427 0 2024-02-22T15:50:48.619 INFO:tasks.workunit.client.0.smithi019.stdout:3/852: truncate d9/d22/d27/f7a 4948065 0 2024-02-22T15:50:48.620 INFO:tasks.workunit.client.1.smithi062.stdout:4/872: dread d4/d1c/d2c/d6b/d70/de8/dfc/fff [0,4194304] 0 2024-02-22T15:50:48.620 INFO:tasks.workunit.client.1.smithi062.stdout:8/885: write d8/d86/ffa [1755680,106332] 0 2024-02-22T15:50:48.621 INFO:tasks.workunit.client.0.smithi019.stdout:3/853: write d9/d22/f9d [1036566,75497] 0 2024-02-22T15:50:48.622 INFO:tasks.workunit.client.0.smithi019.stdout:9/991: rename d2/d1f/d2e/d48/d45/f61 to d2/d7/d52/f152 0 2024-02-22T15:50:48.623 INFO:tasks.workunit.client.1.smithi062.stdout:6/775: link d5/df/d32/d47/f50 d5/df/f104 0 2024-02-22T15:50:48.625 INFO:tasks.workunit.client.1.smithi062.stdout:5/691: mkdir d0/d3/da/d13/d23/d4e/d60/db5/de1 0 2024-02-22T15:50:48.626 INFO:tasks.workunit.client.0.smithi019.stdout:3/854: rename d9/d26/d2d/dc7 to d9/d22/d27/d96/d116 0 2024-02-22T15:50:48.628 INFO:tasks.workunit.client.1.smithi062.stdout:0/858: symlink d1/d13/dde/d76/l11d 0 2024-02-22T15:50:48.629 INFO:tasks.workunit.client.0.smithi019.stdout:3/855: mknod d9/c117 0 2024-02-22T15:50:48.630 INFO:tasks.workunit.client.0.smithi019.stdout:8/918: truncate d9/d14/d1f/d5c/fa2 3685979 0 2024-02-22T15:50:48.631 INFO:tasks.workunit.client.1.smithi062.stdout:4/873: symlink d4/d2a/d2d/d2e/d6a/d8a/ddb/l136 0 2024-02-22T15:50:48.631 INFO:tasks.workunit.client.0.smithi019.stdout:9/992: rmdir d2/d7/dee/d147 0 2024-02-22T15:50:48.632 INFO:tasks.workunit.client.1.smithi062.stdout:6/776: dread - d5/dd/d33/d66/fa8 zero size 2024-02-22T15:50:48.633 INFO:tasks.workunit.client.1.smithi062.stdout:6/777: truncate d5/df/dbc/fe5 106234 0 2024-02-22T15:50:48.635 INFO:tasks.workunit.client.1.smithi062.stdout:3/665: creat d5/d16/d56/dbc/dcb/fd1 x:0 0 0 2024-02-22T15:50:48.636 INFO:tasks.workunit.client.1.smithi062.stdout:0/859: creat d1/d13/dde/d76/d102/f11e x:0 0 0 2024-02-22T15:50:48.637 INFO:tasks.workunit.client.0.smithi019.stdout:9/993: creat d2/d7/dc/d27/d68/dca/f153 x:0 0 0 2024-02-22T15:50:48.637 INFO:tasks.workunit.client.0.smithi019.stdout:9/994: stat d2/d1f/d2e/d88 0 2024-02-22T15:50:48.638 INFO:tasks.workunit.client.1.smithi062.stdout:2/859: creat d1/d1e/dda/de8/d112/f123 x:0 0 0 2024-02-22T15:50:48.639 INFO:tasks.workunit.client.1.smithi062.stdout:4/874: symlink d4/d1c/d2c/l137 0 2024-02-22T15:50:48.639 INFO:tasks.workunit.client.1.smithi062.stdout:2/860: read - d1/d8/d62/d73/dbd/d7d/d84/fcc zero size 2024-02-22T15:50:48.640 INFO:tasks.workunit.client.0.smithi019.stdout:9/995: truncate d2/d1f/d2e/d48/fc4 1643962 0 2024-02-22T15:50:48.640 INFO:tasks.workunit.client.1.smithi062.stdout:1/678: write d1/d6/de/f17 [2274139,23509] 0 2024-02-22T15:50:48.641 INFO:tasks.workunit.client.0.smithi019.stdout:9/996: stat d2/dc8/fb0 0 2024-02-22T15:50:48.643 INFO:tasks.workunit.client.1.smithi062.stdout:6/778: rename d5/df/d40/d44/f7c to d5/df/d32/f105 0 2024-02-22T15:50:48.643 INFO:tasks.workunit.client.1.smithi062.stdout:1/679: write d1/d6/d7/d2b/d47/d7e/fc5 [1082643,66181] 0 2024-02-22T15:50:48.644 INFO:tasks.workunit.client.0.smithi019.stdout:9/997: fdatasync d2/d1f/d2e/d48/d45/d5a/d118/fbb 0 2024-02-22T15:50:48.645 INFO:tasks.workunit.client.1.smithi062.stdout:0/860: symlink d1/d3/d4/d1e/d74/d68/l11f 0 2024-02-22T15:50:48.646 INFO:tasks.workunit.client.1.smithi062.stdout:5/692: creat d0/d3/da/fe2 x:0 0 0 2024-02-22T15:50:48.646 INFO:tasks.workunit.client.0.smithi019.stdout:9/998: fsync d2/d7/dc/d27/dbc/d110/f12d 0 2024-02-22T15:50:48.649 INFO:tasks.workunit.client.1.smithi062.stdout:2/861: mkdir d1/d1e/dda/de8/d112/d124 0 2024-02-22T15:50:48.650 INFO:tasks.workunit.client.1.smithi062.stdout:3/666: symlink d5/d1a/d3b/dc1/ld2 0 2024-02-22T15:50:48.651 INFO:tasks.workunit.client.0.smithi019.stdout:9/999: getdents d2/d7/dc/d27/d87 0 2024-02-22T15:50:48.652 INFO:tasks.workunit.client.1.smithi062.stdout:6/779: truncate d5/df/d32/d47/f7a 958107 0 2024-02-22T15:50:48.653 INFO:tasks.workunit.client.0.smithi019.stdout:8/919: dread d9/d14/d1f/f41 [0,4194304] 0 2024-02-22T15:50:48.653 INFO:tasks.workunit.client.1.smithi062.stdout:3/667: chown l2 1701114 1 2024-02-22T15:50:48.653 INFO:tasks.workunit.client.0.smithi019.stdout:8/920: chown d9/d14/d17/d3d/d83/lad 66834 1 2024-02-22T15:50:48.655 INFO:tasks.workunit.client.0.smithi019.stdout:8/921: stat d9/d14/d17/d3d/d3f/dd4/ld7 0 2024-02-22T15:50:48.657 INFO:tasks.workunit.client.0.smithi019.stdout:8/922: fsync d9/d14/d17/d3d/d83/fa6 0 2024-02-22T15:50:48.658 INFO:tasks.workunit.client.0.smithi019.stdout:8/923: fdatasync d9/d14/d17/d3d/d128/f125 0 2024-02-22T15:50:48.659 INFO:tasks.workunit.client.1.smithi062.stdout:1/680: creat d1/d6/d14/fe4 x:0 0 0 2024-02-22T15:50:48.660 INFO:tasks.workunit.client.1.smithi062.stdout:0/861: creat d1/d13/dde/d76/d8c/d5d/dc3/d85/f120 x:0 0 0 2024-02-22T15:50:48.660 INFO:tasks.workunit.client.1.smithi062.stdout:5/693: dwrite d0/d3/da/d13/d23/d4e/d60/f63 [4194304,4194304] 0 2024-02-22T15:50:48.661 INFO:tasks.workunit.client.0.smithi019.stdout:8/924: creat d9/d14/d17/d3d/d128/f134 x:0 0 0 2024-02-22T15:50:48.662 INFO:tasks.workunit.client.1.smithi062.stdout:1/681: write d1/f27 [82373,87545] 0 2024-02-22T15:50:48.662 INFO:tasks.workunit.client.1.smithi062.stdout:6/780: truncate d5/df/d40/d44/d4a/f2d 4521544 0 2024-02-22T15:50:48.663 INFO:tasks.workunit.client.1.smithi062.stdout:2/862: rename d1/d8/fdd to d1/d8/d62/d73/dbd/d7d/d84/f125 0 2024-02-22T15:50:48.664 INFO:tasks.workunit.client.1.smithi062.stdout:7/606: dwrite d0/d14/d13/f46 [0,4194304] 0 2024-02-22T15:50:48.664 INFO:tasks.workunit.client.0.smithi019.stdout:8/925: getdents d9/d14/d6f/d7e/dff 0 2024-02-22T15:50:48.669 INFO:tasks.workunit.client.1.smithi062.stdout:3/668: mknod d5/cd3 0 2024-02-22T15:50:48.670 INFO:tasks.workunit.client.1.smithi062.stdout:5/694: creat d0/d3/d2f/d59/d67/d69/d3f/fe3 x:0 0 0 2024-02-22T15:50:48.682 INFO:tasks.workunit.client.0.smithi019.stdout:8/926: truncate d9/d14/d17/d1c/f51 5937404 0 2024-02-22T15:50:48.685 INFO:tasks.workunit.client.0.smithi019.stdout:8/927: rename d9/d14/d17/c115 to d9/d14/d17/d3d/d128/dd3/d12a/c135 0 2024-02-22T15:50:48.686 INFO:tasks.workunit.client.0.smithi019.stdout:8/928: rmdir d9/d14/d17/d1c/da0 39 2024-02-22T15:50:48.689 INFO:tasks.workunit.client.0.smithi019.stdout:8/929: symlink d9/d14/d17/d3d/d3f/dd4/l136 0 2024-02-22T15:50:48.692 INFO:tasks.workunit.client.0.smithi019.stdout:8/930: rename d9/d14/d6f/d7e/d94/daa/dea/f100 to d9/d14/f137 0 2024-02-22T15:50:48.693 INFO:tasks.workunit.client.0.smithi019.stdout:8/931: truncate d9/ff 498475 0 2024-02-22T15:50:48.695 INFO:tasks.workunit.client.0.smithi019.stdout:8/932: creat d9/d14/d17/d5d/de6/f138 x:0 0 0 2024-02-22T15:50:48.695 INFO:tasks.workunit.client.0.smithi019.stdout:8/933: chown d9/d14/d1f/d5c/c5e 840 1 2024-02-22T15:50:48.698 INFO:tasks.workunit.client.0.smithi019.stdout:8/934: creat d9/d14/d17/d1c/f139 x:0 0 0 2024-02-22T15:50:48.701 INFO:tasks.workunit.client.0.smithi019.stdout:8/935: getdents d9 0 2024-02-22T15:50:48.704 INFO:tasks.workunit.client.0.smithi019.stdout:8/936: unlink d9/d14/l46 0 2024-02-22T15:50:48.705 INFO:tasks.workunit.client.0.smithi019.stdout:8/937: rmdir d9/d14/d6f 39 2024-02-22T15:50:48.706 INFO:tasks.workunit.client.0.smithi019.stdout:8/938: dread - d9/d14/d1f/d6d/db5/f132 zero size 2024-02-22T15:50:48.718 INFO:tasks.workunit.client.0.smithi019.stdout:8/939: dread d9/d14/d17/d3d/fbb [0,4194304] 0 2024-02-22T15:50:48.721 INFO:tasks.workunit.client.0.smithi019.stdout:8/940: unlink d9/d14/d1f/d20/c31 0 2024-02-22T15:50:48.722 INFO:tasks.workunit.client.0.smithi019.stdout:8/941: write d9/d14/d17/f85 [790151,110153] 0 2024-02-22T15:50:48.722 INFO:tasks.workunit.client.0.smithi019.stdout:8/942: chown d9/d14/d17/f4a 1054089843 1 2024-02-22T15:50:48.725 INFO:tasks.workunit.client.0.smithi019.stdout:8/943: dread - d9/d14/d6f/d7e/d94/d9c/fcf zero size 2024-02-22T15:50:48.727 INFO:tasks.workunit.client.0.smithi019.stdout:8/944: mkdir d9/d14/d17/d1c/d13a 0 2024-02-22T15:50:48.728 INFO:tasks.workunit.client.0.smithi019.stdout:8/945: symlink d9/d14/d6f/d7e/d94/d9c/l13b 0 2024-02-22T15:50:48.730 INFO:tasks.workunit.client.0.smithi019.stdout:8/946: dread - d9/d14/d6f/d7e/dff/f106 zero size 2024-02-22T15:50:48.734 INFO:tasks.workunit.client.1.smithi062.stdout:6/781: symlink d5/df/d40/d44/d76/l106 0 2024-02-22T15:50:48.737 INFO:tasks.workunit.client.1.smithi062.stdout:2/863: mknod d1/d8/d62/d73/dbd/d7d/c126 0 2024-02-22T15:50:48.738 INFO:tasks.workunit.client.0.smithi019.stdout:8/947: dwrite d9/d14/d1f/d6d/f8f [0,4194304] 0 2024-02-22T15:50:48.739 INFO:tasks.workunit.client.0.smithi019.stdout:8/948: fsync d9/d14/d17/d3d/d128/f125 0 2024-02-22T15:50:48.742 INFO:tasks.workunit.client.1.smithi062.stdout:9/749: dwrite d4/d15/d5f/d75/d84/f8b [0,4194304] 0 2024-02-22T15:50:48.742 INFO:tasks.workunit.client.0.smithi019.stdout:8/949: creat d9/d14/d17/d1c/dd8/f13c x:0 0 0 2024-02-22T15:50:48.743 INFO:tasks.workunit.client.1.smithi062.stdout:9/750: readlink d4/d1c/d83/ld3 0 2024-02-22T15:50:48.744 INFO:tasks.workunit.client.1.smithi062.stdout:9/751: fdatasync d4/d1c/fe6 0 2024-02-22T15:50:48.745 INFO:tasks.workunit.client.0.smithi019.stdout:8/950: creat d9/d14/d6f/d7e/d94/db8/f13d x:0 0 0 2024-02-22T15:50:48.748 INFO:tasks.workunit.client.0.smithi019.stdout:8/951: rename d9/d14/d1f/d6d/c122 to d9/d14/d17/d3d/d128/dd3/d12a/c13e 0 2024-02-22T15:50:48.751 INFO:tasks.workunit.client.0.smithi019.stdout:8/952: dread f3 [0,4194304] 0 2024-02-22T15:50:48.753 INFO:tasks.workunit.client.0.smithi019.stdout:8/953: rmdir d9/d14/d1f/d5c 39 2024-02-22T15:50:48.755 INFO:tasks.workunit.client.0.smithi019.stdout:8/954: symlink d9/d14/d17/d3d/d128/dd3/d12a/l13f 0 2024-02-22T15:50:48.755 INFO:tasks.workunit.client.0.smithi019.stdout:8/955: dread - d9/d14/d6f/d7e/dff/f12c zero size 2024-02-22T15:50:48.756 INFO:tasks.workunit.client.1.smithi062.stdout:8/886: dwrite d8/d9/d48/f79 [0,4194304] 0 2024-02-22T15:50:48.759 INFO:tasks.workunit.client.1.smithi062.stdout:8/887: fsync d8/d9/d44/d4d/fee 0 2024-02-22T15:50:48.765 INFO:tasks.workunit.client.1.smithi062.stdout:4/875: write d4/d1c/d2c/d6b/d70/de8/dfc/fff [914248,109378] 0 2024-02-22T15:50:48.773 INFO:tasks.workunit.client.1.smithi062.stdout:4/876: dwrite d4/d1c/d2c/d6b/fca [0,4194304] 0 2024-02-22T15:50:48.775 INFO:tasks.workunit.client.0.smithi019.stdout:8/956: dread d9/d14/d1f/d5c/fc8 [0,4194304] 0 2024-02-22T15:50:48.778 INFO:tasks.workunit.client.0.smithi019.stdout:8/957: creat d9/d14/d1f/f140 x:0 0 0 2024-02-22T15:50:48.779 INFO:tasks.workunit.client.0.smithi019.stdout:8/958: chown d9/d14/d6f 158088130 1 2024-02-22T15:50:48.781 INFO:tasks.workunit.client.0.smithi019.stdout:8/959: creat d9/d14/d6f/d7e/d94/f141 x:0 0 0 2024-02-22T15:50:48.782 INFO:tasks.workunit.client.1.smithi062.stdout:0/862: dwrite d1/d13/dde/f34 [4194304,4194304] 0 2024-02-22T15:50:48.782 INFO:tasks.workunit.client.0.smithi019.stdout:8/960: write d9/d14/d17/d3d/d3f/dd4/f12b [762618,93662] 0 2024-02-22T15:50:48.783 INFO:tasks.workunit.client.1.smithi062.stdout:0/863: fsync d1/fc2 0 2024-02-22T15:50:48.787 INFO:tasks.workunit.client.1.smithi062.stdout:6/782: rmdir d5/df/d32/d47 39 2024-02-22T15:50:48.788 INFO:tasks.workunit.client.1.smithi062.stdout:5/695: fdatasync d0/d3/d30/f46 0 2024-02-22T15:50:48.789 INFO:tasks.workunit.client.1.smithi062.stdout:6/783: write d5/df/d40/fb2 [214419,116720] 0 2024-02-22T15:50:48.795 INFO:tasks.workunit.client.1.smithi062.stdout:8/888: unlink d8/d26/f9e 0 2024-02-22T15:50:48.801 INFO:tasks.workunit.client.1.smithi062.stdout:4/877: dread d4/d1c/d30/d3b/d49/f95 [0,4194304] 0 2024-02-22T15:50:48.803 INFO:tasks.workunit.client.1.smithi062.stdout:6/784: dread d5/df/d40/d44/d76/f52 [4194304,4194304] 0 2024-02-22T15:50:48.805 INFO:tasks.workunit.client.1.smithi062.stdout:8/889: truncate d8/d9/d44/d4d/f9f 346991 0 2024-02-22T15:50:48.809 INFO:tasks.workunit.client.1.smithi062.stdout:9/752: symlink d4/d15/d5f/d75/d97/led 0 2024-02-22T15:50:48.813 INFO:tasks.workunit.client.1.smithi062.stdout:5/696: fsync d0/d3/da/d13/d23/d4e/f5d 0 2024-02-22T15:50:48.815 INFO:tasks.workunit.client.0.smithi019.stdout:8/961: dread d9/d14/d17/d1c/f48 [0,4194304] 0 2024-02-22T15:50:48.815 INFO:tasks.workunit.client.1.smithi062.stdout:4/878: mkdir d4/d1c/d138 0 2024-02-22T15:50:48.816 INFO:tasks.workunit.client.0.smithi019.stdout:8/962: stat d9/d14/d17/d1c/l2b 0 2024-02-22T15:50:48.818 INFO:tasks.workunit.client.0.smithi019.stdout:8/963: write d9/d14/d6f/d7e/d94/daa/dea/f10a [208928,92082] 0 2024-02-22T15:50:48.821 INFO:tasks.workunit.client.1.smithi062.stdout:8/890: dwrite d8/d9/d2c/d74/fa0 [0,4194304] 0 2024-02-22T15:50:48.824 INFO:tasks.workunit.client.0.smithi019.stdout:8/964: write d9/d14/d1f/d4c/f7f [124106,67966] 0 2024-02-22T15:50:48.825 INFO:tasks.workunit.client.0.smithi019.stdout:4/904: sync 2024-02-22T15:50:48.826 INFO:tasks.workunit.client.1.smithi062.stdout:5/697: dwrite d0/d3/f10 [0,4194304] 0 2024-02-22T15:50:48.827 INFO:tasks.workunit.client.0.smithi019.stdout:8/965: creat d9/d14/d1f/d6d/d103/f142 x:0 0 0 2024-02-22T15:50:48.828 INFO:tasks.workunit.client.1.smithi062.stdout:0/864: getdents d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dd7 0 2024-02-22T15:50:48.829 INFO:tasks.workunit.client.1.smithi062.stdout:4/879: mkdir d4/d1c/d30/d3b/d49/d139 0 2024-02-22T15:50:48.830 INFO:tasks.workunit.client.0.smithi019.stdout:8/966: fdatasync d9/d14/fce 0 2024-02-22T15:50:48.832 INFO:tasks.workunit.client.0.smithi019.stdout:4/905: link d2/db/d14/d23/d21/c9d d2/d3f/d48/de9/c122 0 2024-02-22T15:50:48.832 INFO:tasks.workunit.client.1.smithi062.stdout:6/785: getdents d5/dd 0 2024-02-22T15:50:48.833 INFO:tasks.workunit.client.0.smithi019.stdout:4/906: write d2/db/d14/d23/f2f [1931168,84568] 0 2024-02-22T15:50:48.833 INFO:tasks.workunit.client.0.smithi019.stdout:8/967: mkdir d9/d14/d6f/d7e/d94/d9c/d143 0 2024-02-22T15:50:48.835 INFO:tasks.workunit.client.1.smithi062.stdout:9/753: dwrite d4/d15/d5f/d75/f65 [0,4194304] 0 2024-02-22T15:50:48.837 INFO:tasks.workunit.client.0.smithi019.stdout:8/968: fsync d9/d14/d17/d3d/d128/ff1 0 2024-02-22T15:50:48.839 INFO:tasks.workunit.client.1.smithi062.stdout:6/786: chown d5/df/d40/d44/f95 503155 1 2024-02-22T15:50:48.839 INFO:tasks.workunit.client.1.smithi062.stdout:0/865: mkdir d1/d3/dd5/d121 0 2024-02-22T15:50:48.842 INFO:tasks.workunit.client.0.smithi019.stdout:4/907: dwrite d2/db/d14/d23/f28 [0,4194304] 0 2024-02-22T15:50:48.842 INFO:tasks.workunit.client.1.smithi062.stdout:8/891: rename d8/d9/d44/df8/d105/d4e/f8e to d8/d9/d2c/d74/df5/f113 0 2024-02-22T15:50:48.843 INFO:tasks.workunit.client.0.smithi019.stdout:8/969: mknod d9/d14/d17/d3d/d128/deb/c144 0 2024-02-22T15:50:48.845 INFO:tasks.workunit.client.1.smithi062.stdout:8/892: write d8/d26/fb4 [2763154,68484] 0 2024-02-22T15:50:48.846 INFO:tasks.workunit.client.1.smithi062.stdout:4/880: creat d4/d1c/f13a x:0 0 0 2024-02-22T15:50:48.846 INFO:tasks.workunit.client.0.smithi019.stdout:4/908: creat d2/db/d14/d23/d67/d121/f123 x:0 0 0 2024-02-22T15:50:48.849 INFO:tasks.workunit.client.0.smithi019.stdout:8/970: rename d9/d14/f52 to d9/d14/d17/d3d/de5/f145 0 2024-02-22T15:50:48.850 INFO:tasks.workunit.client.0.smithi019.stdout:4/909: getdents d2/db/d14/d23/d21/d35/dc0 0 2024-02-22T15:50:48.851 INFO:tasks.workunit.client.0.smithi019.stdout:8/971: write d9/f107 [4352816,33362] 0 2024-02-22T15:50:48.853 INFO:tasks.workunit.client.1.smithi062.stdout:6/787: symlink d5/df/d40/d44/l107 0 2024-02-22T15:50:48.855 INFO:tasks.workunit.client.1.smithi062.stdout:4/881: dread d4/d1c/d2c/f101 [0,4194304] 0 2024-02-22T15:50:48.855 INFO:tasks.workunit.client.1.smithi062.stdout:6/788: write d5/df/d32/d8f/fa1 [2171218,9692] 0 2024-02-22T15:50:48.857 INFO:tasks.workunit.client.1.smithi062.stdout:9/754: rename d4/d15/d5e/d63/da0/d92/lcd to d4/lee 0 2024-02-22T15:50:48.859 INFO:tasks.workunit.client.1.smithi062.stdout:4/882: dread - d4/d1c/d2c/d6b/f100 zero size 2024-02-22T15:50:48.860 INFO:tasks.workunit.client.1.smithi062.stdout:2/864: write d1/d28/fc7 [924189,46382] 0 2024-02-22T15:50:48.861 INFO:tasks.workunit.client.1.smithi062.stdout:1/682: dwrite d1/d6/d14/f6d [0,4194304] 0 2024-02-22T15:50:48.861 INFO:tasks.workunit.client.1.smithi062.stdout:0/866: rename d1/d3/de/df0 to d1/d3/d4/d1e/d74/d122 0 2024-02-22T15:50:48.865 INFO:tasks.workunit.client.1.smithi062.stdout:2/865: chown d1/d47/d39/d5d 11240 1 2024-02-22T15:50:48.866 INFO:tasks.workunit.client.1.smithi062.stdout:1/683: readlink d1/d6/de/lcd 0 2024-02-22T15:50:48.867 INFO:tasks.workunit.client.1.smithi062.stdout:6/789: getdents d5/d8a/ddd 0 2024-02-22T15:50:48.868 INFO:tasks.workunit.client.1.smithi062.stdout:9/755: truncate d4/d35/f49 1319031 0 2024-02-22T15:50:48.868 INFO:tasks.workunit.client.1.smithi062.stdout:4/883: write d4/d1c/d2c/d6b/d70/f76 [3628426,36459] 0 2024-02-22T15:50:48.869 INFO:tasks.workunit.client.1.smithi062.stdout:1/684: dread - d1/d6/d14/dce/fd0 zero size 2024-02-22T15:50:48.871 INFO:tasks.workunit.client.0.smithi019.stdout:4/910: dwrite d2/db/f17 [0,4194304] 0 2024-02-22T15:50:48.873 INFO:tasks.workunit.client.1.smithi062.stdout:4/884: stat d4/d1c/d2c/d6b/d70/de8/c106 0 2024-02-22T15:50:48.874 INFO:tasks.workunit.client.0.smithi019.stdout:4/911: symlink d2/db/d116/l124 0 2024-02-22T15:50:48.875 INFO:tasks.workunit.client.1.smithi062.stdout:2/866: mkdir d1/d47/d32/d104/d127 0 2024-02-22T15:50:48.875 INFO:tasks.workunit.client.1.smithi062.stdout:9/756: symlink d4/d15/d5f/d75/d84/lef 0 2024-02-22T15:50:48.878 INFO:tasks.workunit.client.0.smithi019.stdout:4/912: dread d2/db/d14/d23/d67/d8f/f113 [0,4194304] 0 2024-02-22T15:50:48.880 INFO:tasks.workunit.client.1.smithi062.stdout:4/885: mkdir d4/d2a/d2d/d46/d13b 0 2024-02-22T15:50:48.882 INFO:tasks.workunit.client.0.smithi019.stdout:0/928: sync 2024-02-22T15:50:48.882 INFO:tasks.workunit.client.0.smithi019.stdout:3/856: sync 2024-02-22T15:50:48.883 INFO:tasks.workunit.client.0.smithi019.stdout:8/972: dread d9/d14/d1f/fa5 [0,4194304] 0 2024-02-22T15:50:48.884 INFO:tasks.workunit.client.0.smithi019.stdout:3/857: chown d9/d26/d2d 11895 1 2024-02-22T15:50:48.884 INFO:tasks.workunit.client.1.smithi062.stdout:9/757: chown d4/d15/d31 55483821 1 2024-02-22T15:50:48.886 INFO:tasks.workunit.client.0.smithi019.stdout:8/973: fdatasync d9/d14/d1f/d4c/f7f 0 2024-02-22T15:50:48.886 INFO:tasks.workunit.client.0.smithi019.stdout:4/913: dwrite d2/db/d14/d23/f28 [4194304,4194304] 0 2024-02-22T15:50:48.889 INFO:tasks.workunit.client.0.smithi019.stdout:0/929: fdatasync d4/db/d76/d77/faa 0 2024-02-22T15:50:48.890 INFO:tasks.workunit.client.0.smithi019.stdout:3/858: symlink d9/d16/d107/l118 0 2024-02-22T15:50:48.891 INFO:tasks.workunit.client.0.smithi019.stdout:3/859: dread - d9/d16/fd7 zero size 2024-02-22T15:50:48.891 INFO:tasks.workunit.client.0.smithi019.stdout:8/974: fsync d9/d14/d1f/f80 0 2024-02-22T15:50:48.892 INFO:tasks.workunit.client.1.smithi062.stdout:9/758: mknod d4/d28/d50/cf0 0 2024-02-22T15:50:48.892 INFO:tasks.workunit.client.1.smithi062.stdout:0/867: dwrite d1/d13/dde/d76/d8c/d92/f96 [0,4194304] 0 2024-02-22T15:50:48.892 INFO:tasks.workunit.client.0.smithi019.stdout:4/914: symlink d2/db/d54/l125 0 2024-02-22T15:50:48.894 INFO:tasks.workunit.client.0.smithi019.stdout:3/860: truncate d9/d26/d2d/d53/fe1 4544380 0 2024-02-22T15:50:48.895 INFO:tasks.workunit.client.0.smithi019.stdout:4/915: chown d2/db/d59/ff9 133075 1 2024-02-22T15:50:48.895 INFO:tasks.workunit.client.0.smithi019.stdout:0/930: write d4/f56 [1873536,8460] 0 2024-02-22T15:50:48.895 INFO:tasks.workunit.client.1.smithi062.stdout:8/893: dread d8/f3a [0,4194304] 0 2024-02-22T15:50:48.898 INFO:tasks.workunit.client.0.smithi019.stdout:0/931: chown d4/db/d1b/d63/db0/cbd 6821 1 2024-02-22T15:50:48.898 INFO:tasks.workunit.client.0.smithi019.stdout:4/916: write d2/db/d14/d23/d21/d35/dc0/f104 [570775,89404] 0 2024-02-22T15:50:48.899 INFO:tasks.workunit.client.0.smithi019.stdout:3/861: mknod d9/d26/d3f/d5c/c119 0 2024-02-22T15:50:48.901 INFO:tasks.workunit.client.0.smithi019.stdout:0/932: dread - d4/fdc zero size 2024-02-22T15:50:48.901 INFO:tasks.workunit.client.0.smithi019.stdout:4/917: chown d2/db/d14/d23/d21/d42/dc4 210 1 2024-02-22T15:50:48.902 INFO:tasks.workunit.client.0.smithi019.stdout:8/975: link d9/d14/d17/d5d/c118 d9/d14/d17/c146 0 2024-02-22T15:50:48.902 INFO:tasks.workunit.client.0.smithi019.stdout:0/933: readlink d4/db/d1b/d40/lfb 0 2024-02-22T15:50:48.903 INFO:tasks.workunit.client.0.smithi019.stdout:3/862: unlink d9/d22/ce5 0 2024-02-22T15:50:48.903 INFO:tasks.workunit.client.1.smithi062.stdout:2/867: rename d1/d47/d39/d5d/cbe to d1/d1e/d38/d76/dc5/dec/c128 0 2024-02-22T15:50:48.904 INFO:tasks.workunit.client.0.smithi019.stdout:4/918: mkdir d2/d3f/d48/de9/d126 0 2024-02-22T15:50:48.905 INFO:tasks.workunit.client.0.smithi019.stdout:4/919: fsync d2/db/d44/f114 0 2024-02-22T15:50:48.905 INFO:tasks.workunit.client.0.smithi019.stdout:8/976: getdents d9/d14/d17/d1c/d13a 0 2024-02-22T15:50:48.906 INFO:tasks.workunit.client.1.smithi062.stdout:9/759: creat d4/d1c/ff1 x:0 0 0 2024-02-22T15:50:48.907 INFO:tasks.workunit.client.0.smithi019.stdout:8/977: write d9/fa [849658,37775] 0 2024-02-22T15:50:48.908 INFO:tasks.workunit.client.1.smithi062.stdout:6/790: dwrite d5/df/d32/d47/fa2 [0,4194304] 0 2024-02-22T15:50:48.908 INFO:tasks.workunit.client.1.smithi062.stdout:8/894: rmdir d8/d9/d44/df8/d105/d70 39 2024-02-22T15:50:48.909 INFO:tasks.workunit.client.0.smithi019.stdout:4/920: write d2/db/d14/d23/d21/d35/d3d/fe0 [158849,17046] 0 2024-02-22T15:50:48.910 INFO:tasks.workunit.client.1.smithi062.stdout:1/685: dwrite d1/d6/de/f31 [4194304,4194304] 0 2024-02-22T15:50:48.913 INFO:tasks.workunit.client.0.smithi019.stdout:8/978: truncate d9/d14/d17/d3d/f91 107744 0 2024-02-22T15:50:48.913 INFO:tasks.workunit.client.1.smithi062.stdout:2/868: stat d1/d8/d62/d73/dbd/dbb/fee 0 2024-02-22T15:50:48.913 INFO:tasks.workunit.client.1.smithi062.stdout:9/760: write d4/d15/d5f/f9b [269783,42191] 0 2024-02-22T15:50:48.914 INFO:tasks.workunit.client.0.smithi019.stdout:0/934: truncate d4/db/d1b/d63/db0/da1/dbb/d52/f62 1439348 0 2024-02-22T15:50:48.914 INFO:tasks.workunit.client.1.smithi062.stdout:4/886: dwrite d4/d1c/d30/da7/f135 [0,4194304] 0 2024-02-22T15:50:48.916 INFO:tasks.workunit.client.0.smithi019.stdout:8/979: mkdir d9/d14/d6f/d7e/d94/daa/d147 0 2024-02-22T15:50:48.916 INFO:tasks.workunit.client.0.smithi019.stdout:0/935: dread - d4/d22/d3d/de8/ff8 zero size 2024-02-22T15:50:48.918 INFO:tasks.workunit.client.0.smithi019.stdout:4/921: rename d2/db/d14/d23/d33/l40 to d2/db/d14/d23/d21/dbf/l127 0 2024-02-22T15:50:48.918 INFO:tasks.workunit.client.1.smithi062.stdout:1/686: chown d1/d6/d7/f54 80938766 1 2024-02-22T15:50:48.920 INFO:tasks.workunit.client.0.smithi019.stdout:8/980: getdents d9/d14/d6f/d7e/dff 0 2024-02-22T15:50:48.924 INFO:tasks.workunit.client.1.smithi062.stdout:1/687: dread - d1/d6/d7/d4a/fde zero size 2024-02-22T15:50:48.926 INFO:tasks.workunit.client.1.smithi062.stdout:6/791: rename d5/df/d40/d44/d4a/d41/f72 to d5/df/d40/db3/f108 0 2024-02-22T15:50:48.926 INFO:tasks.workunit.client.0.smithi019.stdout:0/936: getdents d4/db/d1b/d63/db0/da1/dbb/d7 0 2024-02-22T15:50:48.927 INFO:tasks.workunit.client.1.smithi062.stdout:2/869: fdatasync d1/d47/d39/f4b 0 2024-02-22T15:50:48.927 INFO:tasks.workunit.client.0.smithi019.stdout:0/937: chown d4/db/d49 159886618 1 2024-02-22T15:50:48.928 INFO:tasks.workunit.client.0.smithi019.stdout:0/938: fsync d4/d22/d2c/fac 0 2024-02-22T15:50:48.928 INFO:tasks.workunit.client.1.smithi062.stdout:9/761: fdatasync d4/d15/d5f/d6a/fa8 0 2024-02-22T15:50:48.929 INFO:tasks.workunit.client.1.smithi062.stdout:1/688: creat d1/d6/d36/fe5 x:0 0 0 2024-02-22T15:50:48.930 INFO:tasks.workunit.client.1.smithi062.stdout:8/895: creat d8/f114 x:0 0 0 2024-02-22T15:50:48.931 INFO:tasks.workunit.client.1.smithi062.stdout:4/887: dread d4/d13/d20/fc2 [0,4194304] 0 2024-02-22T15:50:48.931 INFO:tasks.workunit.client.1.smithi062.stdout:2/870: stat d1/d1e/d38/f4f 0 2024-02-22T15:50:48.931 INFO:tasks.workunit.client.1.smithi062.stdout:6/792: rmdir d5/df/dbc 39 2024-02-22T15:50:48.935 INFO:tasks.workunit.client.0.smithi019.stdout:0/939: dwrite d4/d22/d2c/fac [0,4194304] 0 2024-02-22T15:50:48.936 INFO:tasks.workunit.client.1.smithi062.stdout:6/793: chown d5/df/d40/d44/f82 237 1 2024-02-22T15:50:48.936 INFO:tasks.workunit.client.1.smithi062.stdout:1/689: rmdir d1/d6/d7/d2b/d56 39 2024-02-22T15:50:48.937 INFO:tasks.workunit.client.1.smithi062.stdout:0/868: dwrite d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/fb7 [0,4194304] 0 2024-02-22T15:50:48.938 INFO:tasks.workunit.client.0.smithi019.stdout:3/863: dread d9/d26/d2d/df4/ffc [0,4194304] 0 2024-02-22T15:50:48.941 INFO:tasks.workunit.client.1.smithi062.stdout:9/762: creat d4/d15/d5f/d75/ff2 x:0 0 0 2024-02-22T15:50:48.941 INFO:tasks.workunit.client.0.smithi019.stdout:0/940: link d4/db/d1b/d40/f5d d4/db/d1b/d40/d44/f137 0 2024-02-22T15:50:48.944 INFO:tasks.workunit.client.0.smithi019.stdout:0/941: read - d4/db/d49/fb1 zero size 2024-02-22T15:50:48.947 INFO:tasks.workunit.client.0.smithi019.stdout:3/864: dwrite d9/d26/d3f/d10b/d66/f7f [0,4194304] 0 2024-02-22T15:50:48.950 INFO:tasks.workunit.client.1.smithi062.stdout:2/871: creat d1/d8/d62/d73/dbd/d7d/f129 x:0 0 0 2024-02-22T15:50:48.950 INFO:tasks.workunit.client.1.smithi062.stdout:1/690: mkdir d1/d6/d7/de6 0 2024-02-22T15:50:48.950 INFO:tasks.workunit.client.1.smithi062.stdout:8/896: dwrite d8/f20 [0,4194304] 0 2024-02-22T15:50:48.952 INFO:tasks.workunit.client.1.smithi062.stdout:8/897: readlink d8/d9/d44/d4d/laf 0 2024-02-22T15:50:48.953 INFO:tasks.workunit.client.1.smithi062.stdout:9/763: chown d4/d15/d5e/d63/da0/f8c 1 1 2024-02-22T15:50:48.953 INFO:tasks.workunit.client.1.smithi062.stdout:2/872: rmdir d1/d47/d32/d104 39 2024-02-22T15:50:48.955 INFO:tasks.workunit.client.0.smithi019.stdout:8/981: dwrite d9/d14/d17/d3d/d3f/f9e [0,4194304] 0 2024-02-22T15:50:48.957 INFO:tasks.workunit.client.0.smithi019.stdout:3/865: link d9/d11/c110 d9/d26/d3f/d5c/c11a 0 2024-02-22T15:50:48.957 INFO:tasks.workunit.client.1.smithi062.stdout:0/869: dwrite d1/d3/de/f3a [4194304,4194304] 0 2024-02-22T15:50:48.957 INFO:tasks.workunit.client.1.smithi062.stdout:1/691: creat d1/d6/d7/de6/fe7 x:0 0 0 2024-02-22T15:50:48.958 INFO:tasks.workunit.client.0.smithi019.stdout:8/982: dread - d9/d14/d6f/d7e/d94/f141 zero size 2024-02-22T15:50:48.963 INFO:tasks.workunit.client.1.smithi062.stdout:8/898: fdatasync d8/d86/f96 0 2024-02-22T15:50:48.963 INFO:tasks.workunit.client.1.smithi062.stdout:2/873: readlink d1/d8/d62/d73/dbd/d7d/ldb 0 2024-02-22T15:50:48.964 INFO:tasks.workunit.client.0.smithi019.stdout:3/866: creat d9/d26/d32/f11b x:0 0 0 2024-02-22T15:50:48.964 INFO:tasks.workunit.client.1.smithi062.stdout:6/794: dread d5/df/d40/d44/d76/d56/f5b [0,4194304] 0 2024-02-22T15:50:48.964 INFO:tasks.workunit.client.1.smithi062.stdout:8/899: chown d8/d9/f10d 499 1 2024-02-22T15:50:48.965 INFO:tasks.workunit.client.1.smithi062.stdout:0/870: write d1/d13/dde/d76/d8c/f88 [2725607,61435] 0 2024-02-22T15:50:48.965 INFO:tasks.workunit.client.1.smithi062.stdout:9/764: mknod d4/d28/d50/cf3 0 2024-02-22T15:50:48.966 INFO:tasks.workunit.client.1.smithi062.stdout:8/900: chown d8/l81 99 1 2024-02-22T15:50:48.967 INFO:tasks.workunit.client.0.smithi019.stdout:8/983: read d9/d14/d6f/d7e/d94/d9c/fa3 [726302,41794] 0 2024-02-22T15:50:48.972 INFO:tasks.workunit.client.1.smithi062.stdout:1/692: mknod d1/d6/d7/d2b/d49/d74/dad/ce8 0 2024-02-22T15:50:48.974 INFO:tasks.workunit.client.1.smithi062.stdout:2/874: truncate d1/fb8 2871301 0 2024-02-22T15:50:48.978 INFO:tasks.workunit.client.1.smithi062.stdout:9/765: creat d4/d28/d50/ff4 x:0 0 0 2024-02-22T15:50:48.979 INFO:tasks.workunit.client.1.smithi062.stdout:1/693: write d1/d6/d36/d76/fa1 [328281,76782] 0 2024-02-22T15:50:48.980 INFO:tasks.workunit.client.1.smithi062.stdout:6/795: read d5/dd/d33/d66/db9/fa6 [2134120,9259] 0 2024-02-22T15:50:48.981 INFO:tasks.workunit.client.1.smithi062.stdout:8/901: rename d8/d9/d44/c80 to d8/d9/d2c/dd5/c115 0 2024-02-22T15:50:48.982 INFO:tasks.workunit.client.1.smithi062.stdout:8/902: fdatasync d8/f22 0 2024-02-22T15:50:48.983 INFO:tasks.workunit.client.1.smithi062.stdout:2/875: unlink d1/d47/d32/d52/d80/l11f 0 2024-02-22T15:50:48.984 INFO:tasks.workunit.client.1.smithi062.stdout:2/876: readlink d1/d8/df/le6 0 2024-02-22T15:50:48.985 INFO:tasks.workunit.client.1.smithi062.stdout:8/903: dread - d8/d9/d44/fde zero size 2024-02-22T15:50:48.985 INFO:tasks.workunit.client.1.smithi062.stdout:6/796: mkdir d5/df/d40/d109 0 2024-02-22T15:50:48.986 INFO:tasks.workunit.client.0.smithi019.stdout:0/942: dwrite d4/db/d24/d80/d122/f12e [4194304,4194304] 0 2024-02-22T15:50:48.986 INFO:tasks.workunit.client.0.smithi019.stdout:8/984: write d9/f3b [1120557,87572] 0 2024-02-22T15:50:48.988 INFO:tasks.workunit.client.1.smithi062.stdout:9/766: rename d4/d28/c96 to d4/d28/d50/d5a/cf5 0 2024-02-22T15:50:48.989 INFO:tasks.workunit.client.1.smithi062.stdout:1/694: creat d1/d6/d7/d8a/dca/fe9 x:0 0 0 2024-02-22T15:50:48.990 INFO:tasks.workunit.client.1.smithi062.stdout:6/797: fdatasync d5/f38 0 2024-02-22T15:50:48.994 INFO:tasks.workunit.client.0.smithi019.stdout:8/985: dread d9/d14/d1f/d5c/fc8 [0,4194304] 0 2024-02-22T15:50:48.994 INFO:tasks.workunit.client.0.smithi019.stdout:0/943: mkdir d4/db/d49/dc2/dc1/d138 0 2024-02-22T15:50:48.995 INFO:tasks.workunit.client.1.smithi062.stdout:9/767: creat d4/d15/d5e/d63/da0/d92/db6/ff6 x:0 0 0 2024-02-22T15:50:48.995 INFO:tasks.workunit.client.0.smithi019.stdout:8/986: chown d9/d14/d1f/f92 10288 1 2024-02-22T15:50:48.996 INFO:tasks.workunit.client.1.smithi062.stdout:2/877: mknod d1/d1e/d38/d11a/c12a 0 2024-02-22T15:50:49.000 INFO:tasks.workunit.client.1.smithi062.stdout:8/904: rename d8/d9/f18 to d8/d9/d44/df8/d110/f116 0 2024-02-22T15:50:49.004 INFO:tasks.workunit.client.1.smithi062.stdout:6/798: mkdir d5/dd/d33/d10a 0 2024-02-22T15:50:49.005 INFO:tasks.workunit.client.0.smithi019.stdout:3/867: dread d9/d22/d27/d96/d116/ff7 [0,4194304] 0 2024-02-22T15:50:49.006 INFO:tasks.workunit.client.0.smithi019.stdout:8/987: truncate d9/d14/d1f/d6d/f8f 180415 0 2024-02-22T15:50:49.008 INFO:tasks.workunit.client.1.smithi062.stdout:6/799: rmdir d5/df/d40/d44/d76 39 2024-02-22T15:50:49.009 INFO:tasks.workunit.client.0.smithi019.stdout:3/868: mkdir d9/d26/d3f/d111/d11c 0 2024-02-22T15:50:49.010 INFO:tasks.workunit.client.0.smithi019.stdout:8/988: creat d9/d14/d17/d3d/d128/dd3/d12a/f148 x:0 0 0 2024-02-22T15:50:49.011 INFO:tasks.workunit.client.0.smithi019.stdout:0/944: dwrite d4/db/d1b/d63/db0/da1/dbb/d7/f72 [0,4194304] 0 2024-02-22T15:50:49.011 INFO:tasks.workunit.client.1.smithi062.stdout:9/768: creat d4/d15/ff7 x:0 0 0 2024-02-22T15:50:49.012 INFO:tasks.workunit.client.1.smithi062.stdout:0/871: dwrite d1/d3/d4/fe5 [0,4194304] 0 2024-02-22T15:50:49.015 INFO:tasks.workunit.client.0.smithi019.stdout:0/945: mkdir d4/db/d76/d77/d139 0 2024-02-22T15:50:49.017 INFO:tasks.workunit.client.0.smithi019.stdout:3/869: rename d9/d11/l10 to d9/d26/d3f/l11d 0 2024-02-22T15:50:49.017 INFO:tasks.workunit.client.1.smithi062.stdout:9/769: symlink d4/d28/lf8 0 2024-02-22T15:50:49.019 INFO:tasks.workunit.client.1.smithi062.stdout:6/800: truncate d5/df/f84 1236689 0 2024-02-22T15:50:49.020 INFO:tasks.workunit.client.0.smithi019.stdout:0/946: mkdir d4/db/d1b/dea/d13a 0 2024-02-22T15:50:49.021 INFO:tasks.workunit.client.0.smithi019.stdout:8/989: creat d9/d14/d17/d3d/d128/f149 x:0 0 0 2024-02-22T15:50:49.022 INFO:tasks.workunit.client.1.smithi062.stdout:8/905: dwrite d8/d9/d2c/d49/d5e/f101 [0,4194304] 0 2024-02-22T15:50:49.022 INFO:tasks.workunit.client.0.smithi019.stdout:3/870: chown d9/d22/d27/lb2 4 1 2024-02-22T15:50:49.024 INFO:tasks.workunit.client.0.smithi019.stdout:8/990: write d9/d14/f19 [4211434,45816] 0 2024-02-22T15:50:49.025 INFO:tasks.workunit.client.0.smithi019.stdout:8/991: stat d9/d14/d17/d3d/d128/f149 0 2024-02-22T15:50:49.026 INFO:tasks.workunit.client.0.smithi019.stdout:3/871: rmdir d9/d26/d2d/df4 39 2024-02-22T15:50:49.027 INFO:tasks.workunit.client.0.smithi019.stdout:3/872: chown d9/d16/dcf/dfa 0 1 2024-02-22T15:50:49.030 INFO:tasks.workunit.client.1.smithi062.stdout:2/878: write d1/d8/d62/d73/dbd/d7d/d84/fd1 [717745,35282] 0 2024-02-22T15:50:49.030 INFO:tasks.workunit.client.1.smithi062.stdout:6/801: mkdir d5/d8a/d10b 0 2024-02-22T15:50:49.031 INFO:tasks.workunit.client.0.smithi019.stdout:0/947: link d4/d22/d2c/f9a d4/db/d49/dc2/dc1/d138/f13b 0 2024-02-22T15:50:49.031 INFO:tasks.workunit.client.0.smithi019.stdout:8/992: creat d9/d14/d17/d5d/de6/f14a x:0 0 0 2024-02-22T15:50:49.031 INFO:tasks.workunit.client.1.smithi062.stdout:0/872: creat d1/d13/dde/d76/d8c/d5d/f123 x:0 0 0 2024-02-22T15:50:49.032 INFO:tasks.workunit.client.0.smithi019.stdout:3/873: getdents d9/d16/dcf/dfa 0 2024-02-22T15:50:49.034 INFO:tasks.workunit.client.1.smithi062.stdout:9/770: dwrite d4/d15/d5f/d75/d97/faf [0,4194304] 0 2024-02-22T15:50:49.037 INFO:tasks.workunit.client.0.smithi019.stdout:3/874: read d9/d65/da7/fde [2350545,104497] 0 2024-02-22T15:50:49.037 INFO:tasks.workunit.client.0.smithi019.stdout:8/993: write d9/d14/d17/d3d/d3f/dd4/f101 [1321988,118531] 0 2024-02-22T15:50:49.041 INFO:tasks.workunit.client.0.smithi019.stdout:3/875: symlink d9/d16/dcf/l11e 0 2024-02-22T15:50:49.044 INFO:tasks.workunit.client.0.smithi019.stdout:8/994: dwrite d9/d14/d17/d1c/f9d [0,4194304] 0 2024-02-22T15:50:49.045 INFO:tasks.workunit.client.1.smithi062.stdout:8/906: dwrite d8/d9/d44/df8/d105/fc6 [0,4194304] 0 2024-02-22T15:50:49.046 INFO:tasks.workunit.client.0.smithi019.stdout:3/876: unlink d9/d26/d2d/d39/l98 0 2024-02-22T15:50:49.046 INFO:tasks.workunit.client.1.smithi062.stdout:0/873: creat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dd7/f124 x:0 0 0 2024-02-22T15:50:49.047 INFO:tasks.workunit.client.0.smithi019.stdout:8/995: truncate d9/d14/d1f/d4c/f129 393834 0 2024-02-22T15:50:49.049 INFO:tasks.workunit.client.1.smithi062.stdout:0/874: chown d1/d13/dde/d76/d8c/d2c/db5/d119 12 1 2024-02-22T15:50:49.050 INFO:tasks.workunit.client.0.smithi019.stdout:8/996: stat d9/d14/d6f/cab 0 2024-02-22T15:50:49.050 INFO:tasks.workunit.client.0.smithi019.stdout:3/877: mkdir d9/d26/d3f/d11f 0 2024-02-22T15:50:49.052 INFO:tasks.workunit.client.0.smithi019.stdout:8/997: write d9/d14/d1f/f92 [2460339,44833] 0 2024-02-22T15:50:49.053 INFO:tasks.workunit.client.1.smithi062.stdout:2/879: getdents d1/d8/d62/d73/dbd/dbb 0 2024-02-22T15:50:49.053 INFO:tasks.workunit.client.1.smithi062.stdout:8/907: creat d8/f117 x:0 0 0 2024-02-22T15:50:49.054 INFO:tasks.workunit.client.1.smithi062.stdout:0/875: mknod d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/dfd/c125 0 2024-02-22T15:50:49.055 INFO:tasks.workunit.client.0.smithi019.stdout:8/998: write d9/d14/d1f/f140 [271535,92343] 0 2024-02-22T15:50:49.059 INFO:tasks.workunit.client.1.smithi062.stdout:8/908: truncate d8/d9/d44/d58/fdb 940211 0 2024-02-22T15:50:49.059 INFO:tasks.workunit.client.1.smithi062.stdout:2/880: truncate d1/d8/d62/f9c 3991068 0 2024-02-22T15:50:49.060 INFO:tasks.workunit.client.1.smithi062.stdout:8/909: mkdir d8/d9/d44/df1/d118 0 2024-02-22T15:50:49.061 INFO:tasks.workunit.client.0.smithi019.stdout:3/878: dwrite d9/d26/d3f/d5c/f6e [0,4194304] 0 2024-02-22T15:50:49.061 INFO:tasks.workunit.client.1.smithi062.stdout:0/876: truncate d1/d13/dde/d76/d8c/d2c/d84/f2a 6779771 0 2024-02-22T15:50:49.062 INFO:tasks.workunit.client.1.smithi062.stdout:2/881: mknod d1/d47/d39/d5d/c12b 0 2024-02-22T15:50:49.064 INFO:tasks.workunit.client.1.smithi062.stdout:2/882: mkdir d1/d1e/d12c 0 2024-02-22T15:50:49.065 INFO:tasks.workunit.client.0.smithi019.stdout:3/879: dread d9/d26/d3f/f6c [0,4194304] 0 2024-02-22T15:50:49.066 INFO:tasks.workunit.client.0.smithi019.stdout:8/999: dwrite d9/d14/d6f/d7e/d94/db8/fa1 [0,4194304] 0 2024-02-22T15:50:49.066 INFO:tasks.workunit.client.1.smithi062.stdout:2/883: stat d1/d47/d39/d5d/ce2 0 2024-02-22T15:50:49.067 INFO:tasks.workunit.client.0.smithi019.stdout:0/948: write d4/f92 [2642043,65905] 0 2024-02-22T15:50:49.067 INFO:tasks.workunit.client.0.smithi019.stdout:3/880: dread - d9/d26/d32/f102 zero size 2024-02-22T15:50:49.069 INFO:tasks.workunit.client.0.smithi019.stdout:4/922: sync 2024-02-22T15:50:49.070 INFO:tasks.workunit.client.1.smithi062.stdout:0/877: rename d1/d13/dde/d76/d8c/d5d/dc3/d85/f120 to d1/d13/dde/d76/d8c/d2c/db5/f126 0 2024-02-22T15:50:49.076 INFO:tasks.workunit.client.0.smithi019.stdout:3/881: getdents d9/d22/d27 0 2024-02-22T15:50:49.080 INFO:tasks.workunit.client.0.smithi019.stdout:3/882: write f6 [2700271,113928] 0 2024-02-22T15:50:49.080 INFO:tasks.workunit.client.0.smithi019.stdout:4/923: dwrite d2/db/d14/d23/d21/d35/d3d/fe8 [0,4194304] 0 2024-02-22T15:50:49.081 INFO:tasks.workunit.client.1.smithi062.stdout:2/884: dwrite d1/d8/d62/d73/dbd/d7d/d84/fd1 [0,4194304] 0 2024-02-22T15:50:49.081 INFO:tasks.workunit.client.0.smithi019.stdout:3/883: chown d9/d26/d2d/d53/c57 223863 1 2024-02-22T15:50:49.082 INFO:tasks.workunit.client.0.smithi019.stdout:4/924: chown d2/db/d14/d23/d33/l62 282355627 1 2024-02-22T15:50:49.083 INFO:tasks.workunit.client.0.smithi019.stdout:3/884: write d9/d14/f86 [1113932,14449] 0 2024-02-22T15:50:49.084 INFO:tasks.workunit.client.0.smithi019.stdout:3/885: dread - d9/d22/d27/fb6 zero size 2024-02-22T15:50:49.085 INFO:tasks.workunit.client.1.smithi062.stdout:9/771: dwrite d4/d15/f23 [0,4194304] 0 2024-02-22T15:50:49.085 INFO:tasks.workunit.client.0.smithi019.stdout:4/925: creat d2/db/d54/f128 x:0 0 0 2024-02-22T15:50:49.087 INFO:tasks.workunit.client.0.smithi019.stdout:3/886: symlink d9/d26/d2d/d53/da0/l120 0 2024-02-22T15:50:49.091 INFO:tasks.workunit.client.0.smithi019.stdout:3/887: read d9/d26/d2d/d39/f5f [317517,4287] 0 2024-02-22T15:50:49.093 INFO:tasks.workunit.client.1.smithi062.stdout:9/772: chown d4/d15/l19 1 1 2024-02-22T15:50:49.093 INFO:tasks.workunit.client.0.smithi019.stdout:3/888: write d9/d26/d3f/d5c/f105 [376696,97087] 0 2024-02-22T15:50:49.099 INFO:tasks.workunit.client.1.smithi062.stdout:2/885: dwrite d1/d8/d62/d73/dbd/d7d/f11d [0,4194304] 0 2024-02-22T15:50:49.101 INFO:tasks.workunit.client.1.smithi062.stdout:8/910: dread d8/d26/f41 [0,4194304] 0 2024-02-22T15:50:49.102 INFO:tasks.workunit.client.0.smithi019.stdout:4/926: dwrite d2/db/d14/d23/d4b/f82 [4194304,4194304] 0 2024-02-22T15:50:49.102 INFO:tasks.workunit.client.1.smithi062.stdout:8/911: write d8/d86/db6/fe3 [4914457,96820] 0 2024-02-22T15:50:49.105 INFO:tasks.workunit.client.1.smithi062.stdout:2/886: creat d1/d8/d62/d73/dbd/d65/f12d x:0 0 0 2024-02-22T15:50:49.106 INFO:tasks.workunit.client.1.smithi062.stdout:8/912: symlink d8/d9/d44/l119 0 2024-02-22T15:50:49.107 INFO:tasks.workunit.client.0.smithi019.stdout:3/889: dwrite d9/d16/fd7 [0,4194304] 0 2024-02-22T15:50:49.110 INFO:tasks.workunit.client.1.smithi062.stdout:2/887: dread d1/d8/df/f31 [0,4194304] 0 2024-02-22T15:50:49.111 INFO:tasks.workunit.client.0.smithi019.stdout:3/890: mknod d9/d22/c121 0 2024-02-22T15:50:49.115 INFO:tasks.workunit.client.0.smithi019.stdout:3/891: mknod d9/d26/d3f/d111/d11c/c122 0 2024-02-22T15:50:49.116 INFO:tasks.workunit.client.1.smithi062.stdout:2/888: unlink d1/d47/d32/d52/d80/fdc 0 2024-02-22T15:50:49.117 INFO:tasks.workunit.client.0.smithi019.stdout:3/892: write d9/d26/d3f/d10b/d66/fb3 [3632876,27520] 0 2024-02-22T15:50:49.119 INFO:tasks.workunit.client.0.smithi019.stdout:3/893: chown d9/d26/d3f/d10b/c10f 37184 1 2024-02-22T15:50:49.121 INFO:tasks.workunit.client.1.smithi062.stdout:2/889: mkdir d1/d47/d39/d12e 0 2024-02-22T15:50:49.124 INFO:tasks.workunit.client.1.smithi062.stdout:9/773: dwrite d4/d1c/f2c [0,4194304] 0 2024-02-22T15:50:49.124 INFO:tasks.workunit.client.1.smithi062.stdout:0/878: dread d1/d13/dde/d76/d8c/d2c/d87/fb9 [4194304,4194304] 0 2024-02-22T15:50:49.125 INFO:tasks.workunit.client.1.smithi062.stdout:2/890: write d1/d47/f83 [756314,40287] 0 2024-02-22T15:50:49.125 INFO:tasks.workunit.client.0.smithi019.stdout:3/894: dwrite d9/d26/d3f/d10b/d66/f7f [0,4194304] 0 2024-02-22T15:50:49.128 INFO:tasks.workunit.client.1.smithi062.stdout:0/879: fdatasync d1/d13/dde/f56 0 2024-02-22T15:50:49.128 INFO:tasks.workunit.client.1.smithi062.stdout:2/891: mknod d1/d1e/d38/d76/dc5/df3/c12f 0 2024-02-22T15:50:49.130 INFO:tasks.workunit.client.1.smithi062.stdout:9/774: creat d4/d1c/d83/ff9 x:0 0 0 2024-02-22T15:50:49.130 INFO:tasks.workunit.client.0.smithi019.stdout:3/895: symlink d9/d65/l123 0 2024-02-22T15:50:49.132 INFO:tasks.workunit.client.1.smithi062.stdout:2/892: mknod d1/d8/d62/d73/dbd/dbb/c130 0 2024-02-22T15:50:49.134 INFO:tasks.workunit.client.0.smithi019.stdout:3/896: getdents d9/d26/d3f 0 2024-02-22T15:50:49.136 INFO:tasks.workunit.client.1.smithi062.stdout:0/880: rename d1/d13/dde/f34 to d1/d3/dd5/f127 0 2024-02-22T15:50:49.136 INFO:tasks.workunit.client.1.smithi062.stdout:8/913: truncate d8/d26/db5/f104 3191897 0 2024-02-22T15:50:49.138 INFO:tasks.workunit.client.0.smithi019.stdout:3/897: creat d9/d26/d2d/d53/da0/f124 x:0 0 0 2024-02-22T15:50:49.138 INFO:tasks.workunit.client.1.smithi062.stdout:9/775: creat d4/d15/d5f/d6a/ffa x:0 0 0 2024-02-22T15:50:49.139 INFO:tasks.workunit.client.0.smithi019.stdout:0/949: dread d4/f92 [0,4194304] 0 2024-02-22T15:50:49.139 INFO:tasks.workunit.client.1.smithi062.stdout:0/881: dread - d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dd7/f124 zero size 2024-02-22T15:50:49.140 INFO:tasks.workunit.client.1.smithi062.stdout:2/893: mknod d1/d8/d62/d73/dbd/d65/c131 0 2024-02-22T15:50:49.141 INFO:tasks.workunit.client.0.smithi019.stdout:3/898: truncate d9/d22/d49/f100 901253 0 2024-02-22T15:50:49.142 INFO:tasks.workunit.client.0.smithi019.stdout:0/950: chown d4/db/d1b/d40/d44/d75/df7 216215207 1 2024-02-22T15:50:49.143 INFO:tasks.workunit.client.0.smithi019.stdout:4/927: dwrite d2/fd [0,4194304] 0 2024-02-22T15:50:49.144 INFO:tasks.workunit.client.0.smithi019.stdout:0/951: fdatasync d4/d22/d3d/de8/ff8 0 2024-02-22T15:50:49.145 INFO:tasks.workunit.client.0.smithi019.stdout:3/899: creat d9/d26/d2d/d39/f125 x:0 0 0 2024-02-22T15:50:49.147 INFO:tasks.workunit.client.1.smithi062.stdout:8/914: fdatasync d8/d9/d44/d4d/d51/f68 0 2024-02-22T15:50:49.147 INFO:tasks.workunit.client.1.smithi062.stdout:2/894: truncate d1/d8/f42 2617726 0 2024-02-22T15:50:49.147 INFO:tasks.workunit.client.0.smithi019.stdout:4/928: creat d2/db/d14/d23/d4b/f129 x:0 0 0 2024-02-22T15:50:49.148 INFO:tasks.workunit.client.0.smithi019.stdout:0/952: rmdir d4/db/d49 39 2024-02-22T15:50:49.149 INFO:tasks.workunit.client.1.smithi062.stdout:0/882: dwrite d1/d5e/ff9 [4194304,4194304] 0 2024-02-22T15:50:49.153 INFO:tasks.workunit.client.0.smithi019.stdout:0/953: creat d4/db/d1b/d63/db0/da1/f13c x:0 0 0 2024-02-22T15:50:49.153 INFO:tasks.workunit.client.1.smithi062.stdout:8/915: creat d8/d9/d44/df1/f11a x:0 0 0 2024-02-22T15:50:49.153 INFO:tasks.workunit.client.1.smithi062.stdout:2/895: creat d1/d47/d32/d52/dba/f132 x:0 0 0 2024-02-22T15:50:49.156 INFO:tasks.workunit.client.1.smithi062.stdout:8/916: chown d8/d9/d2c/d49/d5e/c73 120 1 2024-02-22T15:50:49.156 INFO:tasks.workunit.client.0.smithi019.stdout:4/929: dwrite d2/db/d14/d23/d21/dbf/fdd [4194304,4194304] 0 2024-02-22T15:50:49.156 INFO:tasks.workunit.client.1.smithi062.stdout:2/896: truncate d1/d47/d39/d5d/f63 278126 0 2024-02-22T15:50:49.159 INFO:tasks.workunit.client.1.smithi062.stdout:2/897: write d1/d8/df/f61 [1603717,104196] 0 2024-02-22T15:50:49.161 INFO:tasks.workunit.client.1.smithi062.stdout:8/917: rename d8/d26/fec to d8/d9/d44/d4d/d51/d55/f11b 0 2024-02-22T15:50:49.161 INFO:tasks.workunit.client.1.smithi062.stdout:2/898: readlink d1/d47/d39/d5d/la7 0 2024-02-22T15:50:49.162 INFO:tasks.workunit.client.0.smithi019.stdout:3/900: read d9/d65/d106/fed [3498148,96794] 0 2024-02-22T15:50:49.162 INFO:tasks.workunit.client.0.smithi019.stdout:3/901: fsync d9/d26/d2d/d39/fcd 0 2024-02-22T15:50:49.164 INFO:tasks.workunit.client.0.smithi019.stdout:3/902: fsync d9/d26/d2d/f4b 0 2024-02-22T15:50:49.167 INFO:tasks.workunit.client.0.smithi019.stdout:0/954: dwrite d4/db/d1b/d63/db0/da1/dbb/d7/f72 [0,4194304] 0 2024-02-22T15:50:49.168 INFO:tasks.workunit.client.1.smithi062.stdout:2/899: creat d1/d1e/d38/d55/f133 x:0 0 0 2024-02-22T15:50:49.169 INFO:tasks.workunit.client.1.smithi062.stdout:8/918: mknod d8/d9/d2c/d49/d94/c11c 0 2024-02-22T15:50:49.171 INFO:tasks.workunit.client.0.smithi019.stdout:4/930: getdents d2/db/d14/d23/d21/d42/dc4 0 2024-02-22T15:50:49.171 INFO:tasks.workunit.client.0.smithi019.stdout:3/903: rename d9/d11/l60 to d9/d65/l126 0 2024-02-22T15:50:49.172 INFO:tasks.workunit.client.1.smithi062.stdout:9/776: write d4/d15/d5e/d63/fb2 [946605,110374] 0 2024-02-22T15:50:49.172 INFO:tasks.workunit.client.1.smithi062.stdout:2/900: fdatasync d1/d8/d62/d73/dbd/dbb/dc4/dfe/f11b 0 2024-02-22T15:50:49.172 INFO:tasks.workunit.client.0.smithi019.stdout:3/904: write f6 [2913550,10075] 0 2024-02-22T15:50:49.173 INFO:tasks.workunit.client.1.smithi062.stdout:8/919: read d8/d9/d2c/d74/fa0 [1795444,119372] 0 2024-02-22T15:50:49.174 INFO:tasks.workunit.client.0.smithi019.stdout:0/955: creat d4/db/d24/d105/f13d x:0 0 0 2024-02-22T15:50:49.175 INFO:tasks.workunit.client.1.smithi062.stdout:8/920: chown d8/d86/db6/ffd 244 1 2024-02-22T15:50:49.175 INFO:tasks.workunit.client.0.smithi019.stdout:3/905: mkdir d9/d65/da7/d127 0 2024-02-22T15:50:49.179 INFO:tasks.workunit.client.0.smithi019.stdout:0/956: rename d4/d22/l64 to d4/db/d76/l13e 0 2024-02-22T15:50:49.180 INFO:tasks.workunit.client.1.smithi062.stdout:9/777: rename d4/d15/f5c to d4/d15/d5f/ffb 0 2024-02-22T15:50:49.180 INFO:tasks.workunit.client.0.smithi019.stdout:3/906: creat d9/d26/d2d/d39/f128 x:0 0 0 2024-02-22T15:50:49.182 INFO:tasks.workunit.client.1.smithi062.stdout:2/901: mknod d1/d1e/d38/d11a/c134 0 2024-02-22T15:50:49.182 INFO:tasks.workunit.client.0.smithi019.stdout:0/957: rmdir d4/db/d76 39 2024-02-22T15:50:49.183 INFO:tasks.workunit.client.0.smithi019.stdout:3/907: creat d9/d26/d3f/d10b/f129 x:0 0 0 2024-02-22T15:50:49.184 INFO:tasks.workunit.client.1.smithi062.stdout:2/902: readlink d1/d8/d62/d73/dbd/d65/lb2 0 2024-02-22T15:50:49.184 INFO:tasks.workunit.client.1.smithi062.stdout:8/921: dread d8/d9/d48/f79 [0,4194304] 0 2024-02-22T15:50:49.185 INFO:tasks.workunit.client.0.smithi019.stdout:0/958: unlink d4/db/d1b/d63/db0/da1/dbb/d7/l12c 0 2024-02-22T15:50:49.185 INFO:tasks.workunit.client.1.smithi062.stdout:0/883: truncate d1/fe6 2686125 0 2024-02-22T15:50:49.187 INFO:tasks.workunit.client.1.smithi062.stdout:9/778: rename d4/d15/d5e/d63/da0/d92/cd5 to d4/d15/d31/cfc 0 2024-02-22T15:50:49.188 INFO:tasks.workunit.client.1.smithi062.stdout:2/903: write d1/d8/d62/d73/dbd/d7d/d84/fd7 [86038,100455] 0 2024-02-22T15:50:49.188 INFO:tasks.workunit.client.0.smithi019.stdout:0/959: truncate d4/fdc 691577 0 2024-02-22T15:50:49.189 INFO:tasks.workunit.client.0.smithi019.stdout:3/908: getdents d9/d22/d27/d96/d116 0 2024-02-22T15:50:49.193 INFO:tasks.workunit.client.1.smithi062.stdout:2/904: creat d1/d8/d8f/f135 x:0 0 0 2024-02-22T15:50:49.194 INFO:tasks.workunit.client.1.smithi062.stdout:2/905: readlink d1/d47/d32/l9d 0 2024-02-22T15:50:49.195 INFO:tasks.workunit.client.1.smithi062.stdout:9/779: mkdir d4/d15/dfd 0 2024-02-22T15:50:49.199 INFO:tasks.workunit.client.0.smithi019.stdout:0/960: write d4/db/d1b/d63/db0/da1/dbb/fba [2299622,18001] 0 2024-02-22T15:50:49.199 INFO:tasks.workunit.client.1.smithi062.stdout:2/906: symlink d1/d1e/d38/d76/l136 0 2024-02-22T15:50:49.202 INFO:tasks.workunit.client.0.smithi019.stdout:4/931: dread d2/d3f/d48/d76/fa8 [0,4194304] 0 2024-02-22T15:50:49.202 INFO:tasks.workunit.client.1.smithi062.stdout:8/922: link d8/d9/l2b d8/d9/d44/df8/d110/l11d 0 2024-02-22T15:50:49.203 INFO:tasks.workunit.client.1.smithi062.stdout:9/780: mknod d4/d28/d50/cfe 0 2024-02-22T15:50:49.206 INFO:tasks.workunit.client.0.smithi019.stdout:3/909: dwrite d9/d26/d76/f108 [0,4194304] 0 2024-02-22T15:50:49.209 INFO:tasks.workunit.client.1.smithi062.stdout:2/907: dwrite d1/d47/d39/d5d/fcf [0,4194304] 0 2024-02-22T15:50:49.212 INFO:tasks.workunit.client.0.smithi019.stdout:0/961: dwrite d4/db/d1b/f1f [0,4194304] 0 2024-02-22T15:50:49.212 INFO:tasks.workunit.client.1.smithi062.stdout:9/781: read d4/d15/d5e/d63/da0/f39 [2413019,32533] 0 2024-02-22T15:50:49.213 INFO:tasks.workunit.client.1.smithi062.stdout:2/908: write d1/d8/d62/d73/dbd/d7d/ffb [1021086,9583] 0 2024-02-22T15:50:49.214 INFO:tasks.workunit.client.1.smithi062.stdout:8/923: dwrite d8/d26/f34 [0,4194304] 0 2024-02-22T15:50:49.215 INFO:tasks.workunit.client.0.smithi019.stdout:3/910: getdents d9/d16/dcf/dfa 0 2024-02-22T15:50:49.218 INFO:tasks.workunit.client.1.smithi062.stdout:9/782: write d4/d15/d5f/d75/f65 [816999,85816] 0 2024-02-22T15:50:49.220 INFO:tasks.workunit.client.0.smithi019.stdout:4/932: dwrite d2/db/d14/d23/d33/f74 [0,4194304] 0 2024-02-22T15:50:49.222 INFO:tasks.workunit.client.0.smithi019.stdout:4/933: chown d2/db/d14/d23/d21/d35/d3d 4050 1 2024-02-22T15:50:49.223 INFO:tasks.workunit.client.0.smithi019.stdout:3/911: getdents d9 0 2024-02-22T15:50:49.225 INFO:tasks.workunit.client.0.smithi019.stdout:3/912: dread - d9/d16/ffe zero size 2024-02-22T15:50:49.225 INFO:tasks.workunit.client.0.smithi019.stdout:4/934: chown d2/db/d14/d23/d67/l102 1677830868 1 2024-02-22T15:50:49.229 INFO:tasks.workunit.client.0.smithi019.stdout:3/913: creat d9/d14/d19/f12a x:0 0 0 2024-02-22T15:50:49.232 INFO:tasks.workunit.client.0.smithi019.stdout:3/914: mkdir d9/d22/d12b 0 2024-02-22T15:50:49.232 INFO:tasks.workunit.client.0.smithi019.stdout:4/935: link d2/db/d14/d23/d21/d35/d3d/c10b d2/db/d14/d23/d21/d42/dc4/c12a 0 2024-02-22T15:50:49.234 INFO:tasks.workunit.client.0.smithi019.stdout:3/915: mkdir d9/d14/d12c 0 2024-02-22T15:50:49.235 INFO:tasks.workunit.client.0.smithi019.stdout:3/916: write d9/d26/d32/f102 [887555,128638] 0 2024-02-22T15:50:49.239 INFO:tasks.workunit.client.0.smithi019.stdout:3/917: dread d9/d26/fb5 [0,4194304] 0 2024-02-22T15:50:49.242 INFO:tasks.workunit.client.0.smithi019.stdout:3/918: unlink d9/d26/d2d/d53/da0/f124 0 2024-02-22T15:50:49.242 INFO:tasks.workunit.client.0.smithi019.stdout:3/919: chown d9/d16/dcf 97398 1 2024-02-22T15:50:49.244 INFO:tasks.workunit.client.0.smithi019.stdout:3/920: mkdir d9/d26/d2d/d103/d12d 0 2024-02-22T15:50:49.246 INFO:tasks.workunit.client.0.smithi019.stdout:3/921: truncate d9/d22/d27/f47 773521 0 2024-02-22T15:50:49.247 INFO:tasks.workunit.client.0.smithi019.stdout:3/922: fdatasync d9/d11/f41 0 2024-02-22T15:50:49.253 INFO:tasks.workunit.client.0.smithi019.stdout:0/962: dwrite d4/db/d1b/d40/f5d [0,4194304] 0 2024-02-22T15:50:49.256 INFO:tasks.workunit.client.0.smithi019.stdout:0/963: write d4/db/d24/d80/d122/f12e [5729713,44509] 0 2024-02-22T15:50:49.257 INFO:tasks.workunit.client.0.smithi019.stdout:0/964: stat d4/db/d1b/d40 0 2024-02-22T15:50:49.259 INFO:tasks.workunit.client.0.smithi019.stdout:0/965: creat d4/d9/f13f x:0 0 0 2024-02-22T15:50:49.262 INFO:tasks.workunit.client.0.smithi019.stdout:0/966: symlink d4/db/d24/d105/l140 0 2024-02-22T15:50:49.266 INFO:tasks.workunit.client.0.smithi019.stdout:3/923: dwrite d9/d26/d32/f43 [0,4194304] 0 2024-02-22T15:50:49.268 INFO:tasks.workunit.client.0.smithi019.stdout:3/924: chown d9/d22/d27/d96/fe7 29102848 1 2024-02-22T15:50:49.294 INFO:tasks.workunit.client.1.smithi062.stdout:9/783: readlink d4/d15/d5f/d75/d84/leb 0 2024-02-22T15:50:49.299 INFO:tasks.workunit.client.1.smithi062.stdout:9/784: truncate d4/d15/d5e/d63/f86 1619801 0 2024-02-22T15:50:49.301 INFO:tasks.workunit.client.1.smithi062.stdout:0/884: dwrite d1/d13/dde/d76/d8c/d2c/d84/f26 [0,4194304] 0 2024-02-22T15:50:49.305 INFO:tasks.workunit.client.1.smithi062.stdout:9/785: chown d4/d15/d5f/ffb 3558 1 2024-02-22T15:50:49.305 INFO:tasks.workunit.client.1.smithi062.stdout:8/924: mkdir d8/d9/d2c/d74/d107/d11e 0 2024-02-22T15:50:49.307 INFO:tasks.workunit.client.1.smithi062.stdout:8/925: write d8/d26/fb4 [1994754,37527] 0 2024-02-22T15:50:49.315 INFO:tasks.workunit.client.1.smithi062.stdout:9/786: dwrite d4/f26 [0,4194304] 0 2024-02-22T15:50:49.321 INFO:tasks.workunit.client.1.smithi062.stdout:8/926: dread d8/d9/d2c/d49/d5e/f7d [0,4194304] 0 2024-02-22T15:50:49.332 INFO:tasks.workunit.client.0.smithi019.stdout:0/967: dwrite d4/d22/f36 [0,4194304] 0 2024-02-22T15:50:49.333 INFO:tasks.workunit.client.0.smithi019.stdout:3/925: dwrite d9/d14/f9b [0,4194304] 0 2024-02-22T15:50:49.335 INFO:tasks.workunit.client.0.smithi019.stdout:3/926: rmdir d9/d26/d3f/d10b/dae 39 2024-02-22T15:50:49.343 INFO:tasks.workunit.client.0.smithi019.stdout:3/927: dwrite f4 [4194304,4194304] 0 2024-02-22T15:50:49.344 INFO:tasks.workunit.client.1.smithi062.stdout:8/927: rmdir d8/d9/d2c/d49/d94/dc1 39 2024-02-22T15:50:49.344 INFO:tasks.workunit.client.0.smithi019.stdout:3/928: fsync d9/d22/d27/f7a 0 2024-02-22T15:50:49.345 INFO:tasks.workunit.client.0.smithi019.stdout:0/968: read d4/db/d24/d80/fa0 [86373,115016] 0 2024-02-22T15:50:49.346 INFO:tasks.workunit.client.0.smithi019.stdout:3/929: truncate d9/d22/d27/d96/fca 4516148 0 2024-02-22T15:50:49.365 INFO:tasks.workunit.client.0.smithi019.stdout:0/969: dread d4/db/d1b/d40/f89 [0,4194304] 0 2024-02-22T15:50:49.366 INFO:tasks.workunit.client.1.smithi062.stdout:0/885: dread d1/d3/d4/d1e/f79 [0,4194304] 0 2024-02-22T15:50:49.366 INFO:tasks.workunit.client.0.smithi019.stdout:0/970: stat d4/db/d24/d80/d122 0 2024-02-22T15:50:49.369 INFO:tasks.workunit.client.0.smithi019.stdout:0/971: read d4/f3e [670238,70323] 0 2024-02-22T15:50:49.371 INFO:tasks.workunit.client.1.smithi062.stdout:0/886: creat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/f128 x:0 0 0 2024-02-22T15:50:49.372 INFO:tasks.workunit.client.1.smithi062.stdout:8/928: rmdir d8/d9/d48 39 2024-02-22T15:50:49.374 INFO:tasks.workunit.client.1.smithi062.stdout:0/887: creat d1/d13/f129 x:0 0 0 2024-02-22T15:50:49.375 INFO:tasks.workunit.client.0.smithi019.stdout:0/972: dwrite d4/d9/f26 [4194304,4194304] 0 2024-02-22T15:50:49.375 INFO:tasks.workunit.client.1.smithi062.stdout:0/888: fdatasync d1/d13/dde/d76/d102/f11e 0 2024-02-22T15:50:49.379 INFO:tasks.workunit.client.1.smithi062.stdout:9/787: rename d4/d28/d50/cc2 to d4/d28/cff 0 2024-02-22T15:50:49.379 INFO:tasks.workunit.client.1.smithi062.stdout:9/788: write d4/d15/ff7 [635817,31470] 0 2024-02-22T15:50:49.380 INFO:tasks.workunit.client.0.smithi019.stdout:0/973: link d4/db/d1b/d63/db0/da1/dbb/d52/dee/ff4 d4/db/d24/d80/daf/f141 0 2024-02-22T15:50:49.381 INFO:tasks.workunit.client.1.smithi062.stdout:0/889: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d100/d12a 0 2024-02-22T15:50:49.382 INFO:tasks.workunit.client.1.smithi062.stdout:9/789: fdatasync d4/d15/d31/fa5 0 2024-02-22T15:50:49.382 INFO:tasks.workunit.client.1.smithi062.stdout:8/929: symlink d8/d9/d2c/d49/d94/dc1/de9/dd1/l11f 0 2024-02-22T15:50:49.384 INFO:tasks.workunit.client.0.smithi019.stdout:3/930: dread d9/d16/f42 [0,4194304] 0 2024-02-22T15:50:49.387 INFO:tasks.workunit.client.0.smithi019.stdout:3/931: mknod d9/d26/d2d/d53/c12e 0 2024-02-22T15:50:49.389 INFO:tasks.workunit.client.0.smithi019.stdout:3/932: creat d9/d16/f12f x:0 0 0 2024-02-22T15:50:49.392 INFO:tasks.workunit.client.1.smithi062.stdout:8/930: getdents d8/d9/d44/df8/d105/dc8/d106 0 2024-02-22T15:50:49.393 INFO:tasks.workunit.client.0.smithi019.stdout:0/974: dread d4/db/d1b/d63/db0/da1/fbc [0,4194304] 0 2024-02-22T15:50:49.395 INFO:tasks.workunit.client.0.smithi019.stdout:0/975: mkdir d4/d9/da7/d142 0 2024-02-22T15:50:49.396 INFO:tasks.workunit.client.1.smithi062.stdout:8/931: rmdir d8/d9/d48 39 2024-02-22T15:50:49.397 INFO:tasks.workunit.client.0.smithi019.stdout:0/976: creat d4/d22/d2c/d109/f143 x:0 0 0 2024-02-22T15:50:49.398 INFO:tasks.workunit.client.0.smithi019.stdout:3/933: dread d9/f6a [0,4194304] 0 2024-02-22T15:50:49.400 INFO:tasks.workunit.client.0.smithi019.stdout:0/977: unlink d4/d22/d2c/d68/fa6 0 2024-02-22T15:50:49.402 INFO:tasks.workunit.client.1.smithi062.stdout:8/932: dwrite d8/d26/f34 [0,4194304] 0 2024-02-22T15:50:49.402 INFO:tasks.workunit.client.1.smithi062.stdout:0/890: unlink d1/d3/d4/d1e/fc6 0 2024-02-22T15:50:49.403 INFO:tasks.workunit.client.0.smithi019.stdout:0/978: chown d4/db/d1b/d63/db0/da1/dbb/d52/f58 1698 1 2024-02-22T15:50:49.403 INFO:tasks.workunit.client.1.smithi062.stdout:9/790: creat d4/d15/d5f/d75/f100 x:0 0 0 2024-02-22T15:50:49.404 INFO:tasks.workunit.client.0.smithi019.stdout:3/934: dwrite d9/d22/d27/d96/fca [0,4194304] 0 2024-02-22T15:50:49.405 INFO:tasks.workunit.client.1.smithi062.stdout:9/791: chown d4/d15/d5f/d6a/ce3 5565872 1 2024-02-22T15:50:49.407 INFO:tasks.workunit.client.0.smithi019.stdout:3/935: write d9/d14/f86 [1724968,12911] 0 2024-02-22T15:50:49.407 INFO:tasks.workunit.client.1.smithi062.stdout:8/933: creat d8/d9/d48/d103/f120 x:0 0 0 2024-02-22T15:50:49.409 INFO:tasks.workunit.client.1.smithi062.stdout:9/792: mkdir d4/d15/d5e/d101 0 2024-02-22T15:50:49.410 INFO:tasks.workunit.client.1.smithi062.stdout:9/793: write d4/d15/f23 [2143059,30371] 0 2024-02-22T15:50:49.413 INFO:tasks.workunit.client.0.smithi019.stdout:0/979: dread d4/db/d1b/d40/d44/d75/f82 [0,4194304] 0 2024-02-22T15:50:49.414 INFO:tasks.workunit.client.1.smithi062.stdout:2/909: unlink d1/d8/f15 0 2024-02-22T15:50:49.414 INFO:tasks.workunit.client.0.smithi019.stdout:3/936: truncate d9/d26/d76/f77 3934321 0 2024-02-22T15:50:49.416 INFO:tasks.workunit.client.0.smithi019.stdout:0/980: creat d4/d22/d2c/d68/d6d/f144 x:0 0 0 2024-02-22T15:50:49.418 INFO:tasks.workunit.client.0.smithi019.stdout:4/936: sync 2024-02-22T15:50:49.419 INFO:tasks.workunit.client.1.smithi062.stdout:2/910: symlink d1/d1e/d38/d11a/l137 0 2024-02-22T15:50:49.420 INFO:tasks.workunit.client.0.smithi019.stdout:0/981: mknod d4/db/d76/c145 0 2024-02-22T15:50:49.422 INFO:tasks.workunit.client.0.smithi019.stdout:4/937: symlink d2/d3f/d48/de9/l12b 0 2024-02-22T15:50:49.423 INFO:tasks.workunit.client.0.smithi019.stdout:3/937: getdents d9/d26/d3f/d5c 0 2024-02-22T15:50:49.423 INFO:tasks.workunit.client.0.smithi019.stdout:0/982: rmdir d4/db/d1b/d63/db0/da1/dbb/da3 39 2024-02-22T15:50:49.424 INFO:tasks.workunit.client.0.smithi019.stdout:3/938: readlink d9/d22/d27/d96/dba/le3 0 2024-02-22T15:50:49.425 INFO:tasks.workunit.client.0.smithi019.stdout:4/938: mknod d2/db/d14/d23/d33/db2/c12c 0 2024-02-22T15:50:49.425 INFO:tasks.workunit.client.0.smithi019.stdout:0/983: symlink d4/db/d76/d77/d12b/l146 0 2024-02-22T15:50:49.427 INFO:tasks.workunit.client.0.smithi019.stdout:0/984: truncate d4/d67/f115 125754 0 2024-02-22T15:50:49.427 INFO:tasks.workunit.client.1.smithi062.stdout:9/794: dwrite d4/d15/d5e/d63/da0/d92/db6/ff6 [0,4194304] 0 2024-02-22T15:50:49.427 INFO:tasks.workunit.client.0.smithi019.stdout:4/939: mknod d2/db/d44/c12d 0 2024-02-22T15:50:49.428 INFO:tasks.workunit.client.0.smithi019.stdout:4/940: dread - d2/d3f/d48/d76/ff1 zero size 2024-02-22T15:50:49.429 INFO:tasks.workunit.client.1.smithi062.stdout:0/891: write d1/d13/dde/d76/d8c/f62 [3173941,93511] 0 2024-02-22T15:50:49.430 INFO:tasks.workunit.client.0.smithi019.stdout:0/985: symlink d4/db/d1b/d40/d8c/dda/l147 0 2024-02-22T15:50:49.432 INFO:tasks.workunit.client.0.smithi019.stdout:0/986: write d4/d23/d7d/fff [1894978,30778] 0 2024-02-22T15:50:49.432 INFO:tasks.workunit.client.1.smithi062.stdout:9/795: unlink d4/d1c/d83/d87/lbf 0 2024-02-22T15:50:49.434 INFO:tasks.workunit.client.1.smithi062.stdout:2/911: dwrite d1/d8/d62/d73/dbd/f9f [0,4194304] 0 2024-02-22T15:50:49.435 INFO:tasks.workunit.client.0.smithi019.stdout:0/987: symlink d4/db/d1b/d63/db0/da1/dbb/d52/dee/l148 0 2024-02-22T15:50:49.436 INFO:tasks.workunit.client.0.smithi019.stdout:4/941: link d2/c1e d2/d3f/d48/de9/c12e 0 2024-02-22T15:50:49.436 INFO:tasks.workunit.client.1.smithi062.stdout:9/796: symlink d4/d15/d31/l102 0 2024-02-22T15:50:49.436 INFO:tasks.workunit.client.1.smithi062.stdout:2/912: readlink d1/d1e/d38/d11a/l137 0 2024-02-22T15:50:49.437 INFO:tasks.workunit.client.1.smithi062.stdout:0/892: getdents d1/d3/de 0 2024-02-22T15:50:49.439 INFO:tasks.workunit.client.1.smithi062.stdout:0/893: stat d1/d3/d4/fe8 0 2024-02-22T15:50:49.441 INFO:tasks.workunit.client.1.smithi062.stdout:9/797: dread d4/d15/d5f/d75/f89 [0,4194304] 0 2024-02-22T15:50:49.441 INFO:tasks.workunit.client.1.smithi062.stdout:2/913: creat d1/d47/d32/d52/dbc/f138 x:0 0 0 2024-02-22T15:50:49.441 INFO:tasks.workunit.client.0.smithi019.stdout:0/988: rename d4/db/d1b/d63/db0/da1/dbb/c3f to d4/db/d1b/d63/db0/da1/dbb/dc/d12a/c149 0 2024-02-22T15:50:49.442 INFO:tasks.workunit.client.1.smithi062.stdout:0/894: chown d1/d13/dde/d76/d8c/d5d/dc3/d85/c11b 68568 1 2024-02-22T15:50:49.443 INFO:tasks.workunit.client.0.smithi019.stdout:0/989: chown d4/db/d49/dc2/dc1 562 1 2024-02-22T15:50:49.445 INFO:tasks.workunit.client.1.smithi062.stdout:2/914: creat d1/d8/d62/d73/dbd/d65/f139 x:0 0 0 2024-02-22T15:50:49.445 INFO:tasks.workunit.client.0.smithi019.stdout:0/990: write d4/db/d76/d77/fca [12921699,77171] 0 2024-02-22T15:50:49.446 INFO:tasks.workunit.client.1.smithi062.stdout:9/798: truncate d4/f8 4890474 0 2024-02-22T15:50:49.447 INFO:tasks.workunit.client.0.smithi019.stdout:0/991: read - d4/d22/d2c/d68/d6d/f107 zero size 2024-02-22T15:50:49.448 INFO:tasks.workunit.client.1.smithi062.stdout:0/895: mkdir d1/d13/dde/d76/d8c/d2c/db5/d119/d12b 0 2024-02-22T15:50:49.448 INFO:tasks.workunit.client.0.smithi019.stdout:4/942: link d2/d3f/d48/d76/ffe d2/db/d14/d61/f12f 0 2024-02-22T15:50:49.448 INFO:tasks.workunit.client.0.smithi019.stdout:4/943: fdatasync d2/f6 0 2024-02-22T15:50:49.449 INFO:tasks.workunit.client.1.smithi062.stdout:0/896: fdatasync d1/d13/dde/d76/fe2 0 2024-02-22T15:50:49.453 INFO:tasks.workunit.client.0.smithi019.stdout:3/939: truncate d9/d22/d27/d6d/fe2 302269 0 2024-02-22T15:50:49.454 INFO:tasks.workunit.client.1.smithi062.stdout:0/897: mkdir d1/d3/de/d12c 0 2024-02-22T15:50:49.455 INFO:tasks.workunit.client.0.smithi019.stdout:4/944: symlink d2/db/d14/l130 0 2024-02-22T15:50:49.458 INFO:tasks.workunit.client.0.smithi019.stdout:3/940: mkdir d9/d22/d58/d130 0 2024-02-22T15:50:49.459 INFO:tasks.workunit.client.1.smithi062.stdout:0/898: dread - d1/d13/dde/d76/d102/f11e zero size 2024-02-22T15:50:49.460 INFO:tasks.workunit.client.1.smithi062.stdout:9/799: getdents d4/d1c/d83/d87 0 2024-02-22T15:50:49.462 INFO:tasks.workunit.client.0.smithi019.stdout:3/941: dread d9/d22/d58/ff0 [0,4194304] 0 2024-02-22T15:50:49.463 INFO:tasks.workunit.client.1.smithi062.stdout:9/800: readlink d4/d15/d5e/d63/da0/d92/le0 0 2024-02-22T15:50:49.463 INFO:tasks.workunit.client.1.smithi062.stdout:2/915: truncate d1/d47/d39/fac 527656 0 2024-02-22T15:50:49.466 INFO:tasks.workunit.client.0.smithi019.stdout:4/945: dwrite d2/db/d14/d23/d21/f2c [0,4194304] 0 2024-02-22T15:50:49.467 INFO:tasks.workunit.client.1.smithi062.stdout:2/916: read d1/d47/d39/fcd [1515458,77533] 0 2024-02-22T15:50:49.468 INFO:tasks.workunit.client.0.smithi019.stdout:4/946: stat d2/db/d14/d23/d21/d35/dc0 0 2024-02-22T15:50:49.469 INFO:tasks.workunit.client.1.smithi062.stdout:0/899: dwrite d1/d5e/fe9 [0,4194304] 0 2024-02-22T15:50:49.472 INFO:tasks.workunit.client.1.smithi062.stdout:9/801: dread d4/d15/d5f/f9b [0,4194304] 0 2024-02-22T15:50:49.473 INFO:tasks.workunit.client.1.smithi062.stdout:0/900: fdatasync d1/d13/dde/d76/fb1 0 2024-02-22T15:50:49.475 INFO:tasks.workunit.client.0.smithi019.stdout:0/992: truncate d4/db/d1b/d63/db0/da1/dbb/d7/f72 2320946 0 2024-02-22T15:50:49.475 INFO:tasks.workunit.client.0.smithi019.stdout:4/947: write d2/d3f/d48/d76/fa8 [696820,79419] 0 2024-02-22T15:50:49.475 INFO:tasks.workunit.client.0.smithi019.stdout:0/993: dread - d4/db/d1b/d63/db0/da1/dbb/d7/fe0 zero size 2024-02-22T15:50:49.476 INFO:tasks.workunit.client.1.smithi062.stdout:0/901: chown d1/d13/dde/d76/d8c/d5d/dc3/d100/l103 4 1 2024-02-22T15:50:49.476 INFO:tasks.workunit.client.1.smithi062.stdout:9/802: creat d4/d1c/d83/d87/f103 x:0 0 0 2024-02-22T15:50:49.478 INFO:tasks.workunit.client.0.smithi019.stdout:4/948: creat d2/db/d54/d8c/dbb/f131 x:0 0 0 2024-02-22T15:50:49.479 INFO:tasks.workunit.client.0.smithi019.stdout:4/949: readlink d2/db/d14/d23/d21/l2e 0 2024-02-22T15:50:49.479 INFO:tasks.workunit.client.1.smithi062.stdout:2/917: truncate d1/d28/fc7 387035 0 2024-02-22T15:50:49.480 INFO:tasks.workunit.client.1.smithi062.stdout:9/803: rename d4/d35/dde to d4/d15/d5e/d63/d104 0 2024-02-22T15:50:49.481 INFO:tasks.workunit.client.0.smithi019.stdout:4/950: truncate d2/db/d14/f81 1261942 0 2024-02-22T15:50:49.481 INFO:tasks.workunit.client.1.smithi062.stdout:2/918: readlink d1/d8/d62/dcb/l103 0 2024-02-22T15:50:49.481 INFO:tasks.workunit.client.1.smithi062.stdout:2/919: chown d1/d47/f2b 1824638 1 2024-02-22T15:50:49.482 INFO:tasks.workunit.client.1.smithi062.stdout:0/902: creat d1/d38/f12d x:0 0 0 2024-02-22T15:50:49.483 INFO:tasks.workunit.client.0.smithi019.stdout:3/942: truncate d9/d65/f7d 8070 0 2024-02-22T15:50:49.484 INFO:tasks.workunit.client.0.smithi019.stdout:4/951: mknod d2/db/d14/d23/d33/c132 0 2024-02-22T15:50:49.484 INFO:tasks.workunit.client.1.smithi062.stdout:9/804: creat d4/d15/d5e/d63/da0/d92/f105 x:0 0 0 2024-02-22T15:50:49.485 INFO:tasks.workunit.client.1.smithi062.stdout:2/920: mknod d1/d8/d62/d73/dbd/d65/c13a 0 2024-02-22T15:50:49.486 INFO:tasks.workunit.client.0.smithi019.stdout:4/952: rmdir d2/db/d14/d23/d9e/d10a 39 2024-02-22T15:50:49.487 INFO:tasks.workunit.client.0.smithi019.stdout:3/943: truncate d9/d22/f50 829840 0 2024-02-22T15:50:49.488 INFO:tasks.workunit.client.1.smithi062.stdout:9/805: mkdir d4/d15/d31/d106 0 2024-02-22T15:50:49.489 INFO:tasks.workunit.client.0.smithi019.stdout:3/944: write d9/d14/f1b [2948962,25336] 0 2024-02-22T15:50:49.491 INFO:tasks.workunit.client.1.smithi062.stdout:2/921: creat d1/f13b x:0 0 0 2024-02-22T15:50:49.493 INFO:tasks.workunit.client.1.smithi062.stdout:9/806: dread d4/d15/ff7 [0,4194304] 0 2024-02-22T15:50:49.493 INFO:tasks.workunit.client.1.smithi062.stdout:0/903: dwrite d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/fb7 [0,4194304] 0 2024-02-22T15:50:49.493 INFO:tasks.workunit.client.1.smithi062.stdout:2/922: unlink d1/de1/c117 0 2024-02-22T15:50:49.494 INFO:tasks.workunit.client.1.smithi062.stdout:9/807: stat d4/d15/d31/fe4 0 2024-02-22T15:50:49.496 INFO:tasks.workunit.client.1.smithi062.stdout:2/923: write d1/d8/d62/d73/dbd/d7d/f111 [32772,66919] 0 2024-02-22T15:50:49.498 INFO:tasks.workunit.client.0.smithi019.stdout:4/953: getdents d2/db/dea/ded 0 2024-02-22T15:50:49.499 INFO:tasks.workunit.client.1.smithi062.stdout:9/808: mkdir d4/d28/d50/d107 0 2024-02-22T15:50:49.502 INFO:tasks.workunit.client.1.smithi062.stdout:0/904: link d1/d13/dde/d76/d8c/d2c/lac d1/d13/dde/d1d/ddb/l12e 0 2024-02-22T15:50:49.504 INFO:tasks.workunit.client.1.smithi062.stdout:9/809: rmdir d4/d28/d50/dcf 0 2024-02-22T15:50:49.505 INFO:tasks.workunit.client.1.smithi062.stdout:2/924: truncate d1/d8/d62/d73/dbd/d7d/faa 808303 0 2024-02-22T15:50:49.507 INFO:tasks.workunit.client.1.smithi062.stdout:2/925: chown d1/d8/df/le6 4861203 1 2024-02-22T15:50:49.508 INFO:tasks.workunit.client.1.smithi062.stdout:9/810: readlink d4/d28/lc7 0 2024-02-22T15:50:49.510 INFO:tasks.workunit.client.0.smithi019.stdout:0/994: dread d4/db/d1b/d63/db0/da1/dbb/d7/fcb [4194304,4194304] 0 2024-02-22T15:50:49.511 INFO:tasks.workunit.client.1.smithi062.stdout:2/926: fdatasync d1/d8/d62/f9c 0 2024-02-22T15:50:49.513 INFO:tasks.workunit.client.1.smithi062.stdout:2/927: truncate d1/f13b 245513 0 2024-02-22T15:50:49.514 INFO:tasks.workunit.client.1.smithi062.stdout:7/607: sync 2024-02-22T15:50:49.514 INFO:tasks.workunit.client.1.smithi062.stdout:3/669: sync 2024-02-22T15:50:49.516 INFO:tasks.workunit.client.1.smithi062.stdout:9/811: dwrite d4/d1c/f56 [0,4194304] 0 2024-02-22T15:50:49.517 INFO:tasks.workunit.client.1.smithi062.stdout:2/928: chown d1/d8/d62/dcb/l103 982 1 2024-02-22T15:50:49.518 INFO:tasks.workunit.client.1.smithi062.stdout:7/608: rmdir d0/d6/d2e/d4a 39 2024-02-22T15:50:49.518 INFO:tasks.workunit.client.1.smithi062.stdout:3/670: stat d5/d16/d21/d32/d6a/d7b/l9f 0 2024-02-22T15:50:49.522 INFO:tasks.workunit.client.1.smithi062.stdout:9/812: unlink d4/d15/l16 0 2024-02-22T15:50:49.524 INFO:tasks.workunit.client.1.smithi062.stdout:2/929: symlink d1/d8/d62/d73/da6/l13c 0 2024-02-22T15:50:49.527 INFO:tasks.workunit.client.1.smithi062.stdout:9/813: symlink d4/d1c/d83/d87/l108 0 2024-02-22T15:50:49.528 INFO:tasks.workunit.client.1.smithi062.stdout:7/609: mknod d0/d14/d76/cbc 0 2024-02-22T15:50:49.529 INFO:tasks.workunit.client.0.smithi019.stdout:3/945: truncate d9/d26/d32/f9c 1896636 0 2024-02-22T15:50:49.530 INFO:tasks.workunit.client.1.smithi062.stdout:3/671: dread d5/d16/d21/d32/d6a/f78 [0,4194304] 0 2024-02-22T15:50:49.530 INFO:tasks.workunit.client.0.smithi019.stdout:4/954: creat d2/d3f/d48/de9/d126/f133 x:0 0 0 2024-02-22T15:50:49.532 INFO:tasks.workunit.client.1.smithi062.stdout:2/930: creat d1/d8/d62/d73/dbd/d7d/d84/dfc/f13d x:0 0 0 2024-02-22T15:50:49.532 INFO:tasks.workunit.client.0.smithi019.stdout:3/946: unlink d9/d26/d2d/d53/cd9 0 2024-02-22T15:50:49.533 INFO:tasks.workunit.client.0.smithi019.stdout:3/947: stat d9/d22/f9d 0 2024-02-22T15:50:49.534 INFO:tasks.workunit.client.1.smithi062.stdout:0/905: truncate d1/d3/de/ff 6964216 0 2024-02-22T15:50:49.535 INFO:tasks.workunit.client.0.smithi019.stdout:4/955: creat d2/db/f134 x:0 0 0 2024-02-22T15:50:49.535 INFO:tasks.workunit.client.1.smithi062.stdout:0/906: dread - d1/d13/dde/d76/d8c/d2c/d84/f107 zero size 2024-02-22T15:50:49.536 INFO:tasks.workunit.client.0.smithi019.stdout:3/948: rename d9/d14/f1b to d9/d65/da7/dab/f131 0 2024-02-22T15:50:49.540 INFO:tasks.workunit.client.1.smithi062.stdout:2/931: creat d1/d1e/d38/d11a/f13e x:0 0 0 2024-02-22T15:50:49.541 INFO:tasks.workunit.client.1.smithi062.stdout:0/907: rename d1/d13/dde/d76/d8c/c9e to d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/c12f 0 2024-02-22T15:50:49.542 INFO:tasks.workunit.client.0.smithi019.stdout:4/956: mknod d2/db/c135 0 2024-02-22T15:50:49.543 INFO:tasks.workunit.client.1.smithi062.stdout:0/908: write d1/d13/dde/d76/d8c/d5d/dc3/f47 [5121215,91247] 0 2024-02-22T15:50:49.545 INFO:tasks.workunit.client.0.smithi019.stdout:4/957: unlink d2/db/d14/d23/d9e/dac/le7 0 2024-02-22T15:50:49.546 INFO:tasks.workunit.client.1.smithi062.stdout:0/909: chown d1/d38/dfa/l105 0 1 2024-02-22T15:50:49.549 INFO:tasks.workunit.client.0.smithi019.stdout:4/958: symlink d2/d3f/d48/de9/d117/l136 0 2024-02-22T15:50:49.550 INFO:tasks.workunit.client.0.smithi019.stdout:4/959: write d2/db/d59/ff9 [695189,63228] 0 2024-02-22T15:50:49.553 INFO:tasks.workunit.client.1.smithi062.stdout:9/814: getdents d4/d15/d5f/d6a 0 2024-02-22T15:50:49.553 INFO:tasks.workunit.client.0.smithi019.stdout:4/960: rmdir d2/db/d14/d23/d33/db2/d107 39 2024-02-22T15:50:49.554 INFO:tasks.workunit.client.0.smithi019.stdout:4/961: fsync d2/db/d92/ffc 0 2024-02-22T15:50:49.557 INFO:tasks.workunit.client.0.smithi019.stdout:4/962: creat d2/db/d44/f137 x:0 0 0 2024-02-22T15:50:49.558 INFO:tasks.workunit.client.0.smithi019.stdout:4/963: fdatasync d2/db/d14/d23/f36 0 2024-02-22T15:50:49.560 INFO:tasks.workunit.client.1.smithi062.stdout:0/910: creat d1/d3/d4/f130 x:0 0 0 2024-02-22T15:50:49.560 INFO:tasks.workunit.client.0.smithi019.stdout:4/964: chown d2/db/d14/d23/d21/d35/d85/def 66 1 2024-02-22T15:50:49.560 INFO:tasks.workunit.client.0.smithi019.stdout:3/949: dread d9/f115 [0,4194304] 0 2024-02-22T15:50:49.562 INFO:tasks.workunit.client.0.smithi019.stdout:4/965: fdatasync d2/db/d14/d23/d21/d35/f38 0 2024-02-22T15:50:49.563 INFO:tasks.workunit.client.1.smithi062.stdout:2/932: write d1/d8/d62/d73/f75 [40874,118720] 0 2024-02-22T15:50:49.563 INFO:tasks.workunit.client.1.smithi062.stdout:7/610: dwrite d0/d6/d2e/d55/f87 [0,4194304] 0 2024-02-22T15:50:49.569 INFO:tasks.workunit.client.1.smithi062.stdout:3/672: dwrite d5/d16/d21/d32/f66 [0,4194304] 0 2024-02-22T15:50:49.570 INFO:tasks.workunit.client.1.smithi062.stdout:2/933: dread - d1/d8/df/f78 zero size 2024-02-22T15:50:49.570 INFO:tasks.workunit.client.1.smithi062.stdout:9/815: truncate d4/d28/d50/d5a/f85 3173892 0 2024-02-22T15:50:49.571 INFO:tasks.workunit.client.1.smithi062.stdout:2/934: dread - d1/d1e/dda/de8/d112/f123 zero size 2024-02-22T15:50:49.572 INFO:tasks.workunit.client.1.smithi062.stdout:7/611: mkdir d0/d14/d19/d30/dbd 0 2024-02-22T15:50:49.573 INFO:tasks.workunit.client.1.smithi062.stdout:9/816: read - d4/d1c/d83/fc5 zero size 2024-02-22T15:50:49.574 INFO:tasks.workunit.client.1.smithi062.stdout:3/673: mknod d5/d16/d21/d32/d6a/d7b/cd4 0 2024-02-22T15:50:49.574 INFO:tasks.workunit.client.1.smithi062.stdout:2/935: readlink d1/d8/d62/d73/lc6 0 2024-02-22T15:50:49.575 INFO:tasks.workunit.client.0.smithi019.stdout:0/995: truncate d4/db/d1b/f1f 599853 0 2024-02-22T15:50:49.576 INFO:tasks.workunit.client.1.smithi062.stdout:3/674: chown d5/d16/d21/d32/d6a/l99 737960 1 2024-02-22T15:50:49.581 INFO:tasks.workunit.client.1.smithi062.stdout:2/936: dwrite d1/d8/d62/d73/dbd/f9f [0,4194304] 0 2024-02-22T15:50:49.583 INFO:tasks.workunit.client.0.smithi019.stdout:0/996: symlink d4/db/d1b/d63/db0/da1/dbb/da3/de1/d12f/l14a 0 2024-02-22T15:50:49.584 INFO:tasks.workunit.client.0.smithi019.stdout:0/997: write d4/db/d24/f100 [792205,4337] 0 2024-02-22T15:50:49.587 INFO:tasks.workunit.client.0.smithi019.stdout:0/998: unlink d4/db/d24/d105/l140 0 2024-02-22T15:50:49.591 INFO:tasks.workunit.client.0.smithi019.stdout:4/966: dwrite d2/db/d54/d8c/fa4 [0,4194304] 0 2024-02-22T15:50:49.594 INFO:tasks.workunit.client.0.smithi019.stdout:0/999: truncate d4/db/d1b/d63/db0/da1/dbb/f39 3775832 0 2024-02-22T15:50:49.601 INFO:tasks.workunit.client.1.smithi062.stdout:0/911: dread d1/d13/dde/d76/d8c/f64 [0,4194304] 0 2024-02-22T15:50:49.700 INFO:tasks.workunit.client.1.smithi062.stdout:4/888: sync 2024-02-22T15:50:49.700 INFO:tasks.workunit.client.1.smithi062.stdout:1/695: sync 2024-02-22T15:50:49.700 INFO:tasks.workunit.client.1.smithi062.stdout:5/698: sync 2024-02-22T15:50:49.703 INFO:tasks.workunit.client.1.smithi062.stdout:2/937: symlink d1/d1e/dda/de8/d112/l13f 0 2024-02-22T15:50:49.704 INFO:tasks.workunit.client.1.smithi062.stdout:1/696: read d1/d6/de/f17 [1462786,78300] 0 2024-02-22T15:50:49.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:49 smithi062 ceph-mon[48130]: pgmap v40: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 107 MiB/s rd, 287 MiB/s wr, 554 op/s 2024-02-22T15:50:49.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:49 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:49.721 INFO:tasks.workunit.client.1.smithi062.stdout:4/889: mknod d4/d1c/d2c/d6b/d70/dd6/c13c 0 2024-02-22T15:50:49.723 INFO:tasks.workunit.client.1.smithi062.stdout:3/675: mkdir d5/d1a/d3b/dd5 0 2024-02-22T15:50:49.726 INFO:tasks.workunit.client.1.smithi062.stdout:1/697: mknod d1/d6/d7/d4a/d65/cea 0 2024-02-22T15:50:49.727 INFO:tasks.workunit.client.1.smithi062.stdout:7/612: creat d0/d6/d2e/d4a/d5a/db3/fbe x:0 0 0 2024-02-22T15:50:49.728 INFO:tasks.workunit.client.1.smithi062.stdout:3/676: read d5/d16/d21/d32/d6a/f78 [2255241,75035] 0 2024-02-22T15:50:49.730 INFO:tasks.workunit.client.1.smithi062.stdout:1/698: readlink d1/d6/d7/d2b/d47/ld2 0 2024-02-22T15:50:49.730 INFO:tasks.workunit.client.1.smithi062.stdout:4/890: creat d4/d1c/dac/f13d x:0 0 0 2024-02-22T15:50:49.730 INFO:tasks.workunit.client.1.smithi062.stdout:9/817: dwrite d4/d15/d31/f4e [0,4194304] 0 2024-02-22T15:50:49.732 INFO:tasks.workunit.client.1.smithi062.stdout:5/699: rename d0/d3/da/d13/d23/d4e/c98 to d0/d3/da/d32/d33/ce4 0 2024-02-22T15:50:49.733 INFO:tasks.workunit.client.1.smithi062.stdout:1/699: fsync d1/d6/d7/d2b/d49/d74/f85 0 2024-02-22T15:50:49.741 INFO:tasks.workunit.client.1.smithi062.stdout:4/891: dwrite d4/d2a/d2d/ff9 [0,4194304] 0 2024-02-22T15:50:49.744 INFO:tasks.workunit.client.1.smithi062.stdout:1/700: fdatasync d1/d6/d7/f98 0 2024-02-22T15:50:49.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:49 smithi019 ceph-mon[67418]: pgmap v40: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 107 MiB/s rd, 287 MiB/s wr, 554 op/s 2024-02-22T15:50:49.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:49 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:50:49.746 INFO:tasks.workunit.client.1.smithi062.stdout:9/818: rmdir d4/d15/d5f/d75 39 2024-02-22T15:50:49.748 INFO:tasks.workunit.client.1.smithi062.stdout:9/819: chown d4/d15/d5f/d6a 7123 1 2024-02-22T15:50:49.749 INFO:tasks.workunit.client.1.smithi062.stdout:2/938: getdents d1/d47/d32/d52 0 2024-02-22T15:50:49.749 INFO:tasks.workunit.client.1.smithi062.stdout:1/701: dread d1/d6/d36/d76/fa1 [0,4194304] 0 2024-02-22T15:50:49.755 INFO:tasks.workunit.client.1.smithi062.stdout:4/892: dread d4/d1c/d2c/d6b/d9b/db1/dc6/fd8 [0,4194304] 0 2024-02-22T15:50:49.758 INFO:tasks.workunit.client.1.smithi062.stdout:7/613: getdents d0/d6/d5b/d60 0 2024-02-22T15:50:49.758 INFO:tasks.workunit.client.1.smithi062.stdout:4/893: read d4/d34/f103 [25950,33338] 0 2024-02-22T15:50:49.759 INFO:tasks.workunit.client.1.smithi062.stdout:4/894: fsync d4/d2a/d2d/d46/fad 0 2024-02-22T15:50:49.760 INFO:tasks.workunit.client.1.smithi062.stdout:7/614: chown d0/d6/d2e/d55/d80/fb4 895 1 2024-02-22T15:50:49.761 INFO:tasks.workunit.client.1.smithi062.stdout:4/895: stat d4/d1c/d30/c114 0 2024-02-22T15:50:49.762 INFO:tasks.workunit.client.1.smithi062.stdout:1/702: symlink d1/leb 0 2024-02-22T15:50:49.763 INFO:tasks.workunit.client.1.smithi062.stdout:1/703: write d1/d66/f75 [3320504,10302] 0 2024-02-22T15:50:49.765 INFO:tasks.workunit.client.1.smithi062.stdout:7/615: creat d0/d6/d2e/d58/fbf x:0 0 0 2024-02-22T15:50:49.766 INFO:tasks.workunit.client.1.smithi062.stdout:1/704: write d1/d6/d7/d2b/d49/d74/f85 [963500,121248] 0 2024-02-22T15:50:49.767 INFO:tasks.workunit.client.1.smithi062.stdout:4/896: symlink d4/d1c/d2c/db9/l13e 0 2024-02-22T15:50:49.770 INFO:tasks.workunit.client.1.smithi062.stdout:7/616: mkdir d0/d6/d2e/d58/dc0 0 2024-02-22T15:50:49.771 INFO:tasks.workunit.client.1.smithi062.stdout:2/939: dread d1/fb8 [0,4194304] 0 2024-02-22T15:50:49.771 INFO:tasks.workunit.client.1.smithi062.stdout:0/912: write d1/d3/d5f/f118 [3833211,2885] 0 2024-02-22T15:50:49.773 INFO:tasks.workunit.client.1.smithi062.stdout:0/913: fdatasync d1/d13/dde/d1d/ddb/dfe/d112/fd1 0 2024-02-22T15:50:49.774 INFO:tasks.workunit.client.1.smithi062.stdout:4/897: rmdir d4/d2a/d2d/d64 39 2024-02-22T15:50:49.777 INFO:tasks.workunit.client.0.smithi019.stdout:4/967: sync 2024-02-22T15:50:49.777 INFO:tasks.workunit.client.0.smithi019.stdout:3/950: sync 2024-02-22T15:50:49.778 INFO:tasks.workunit.client.1.smithi062.stdout:2/940: unlink d1/d1e/d38/d76/dc5/f121 0 2024-02-22T15:50:49.780 INFO:tasks.workunit.client.0.smithi019.stdout:3/951: dread d9/d26/d2d/d39/fcd [0,4194304] 0 2024-02-22T15:50:49.780 INFO:tasks.workunit.client.1.smithi062.stdout:4/898: readlink d4/d2a/d2d/d2e/d6a/l83 0 2024-02-22T15:50:49.781 INFO:tasks.workunit.client.1.smithi062.stdout:1/705: getdents d1/d6/d7/d2b/d49/dcc 0 2024-02-22T15:50:49.781 INFO:tasks.workunit.client.0.smithi019.stdout:4/968: rename d2/db/ff3 to d2/db/d14/d23/d21/d42/dc4/d11f/f138 0 2024-02-22T15:50:49.782 INFO:tasks.workunit.client.1.smithi062.stdout:7/617: dwrite d0/d14/d19/d30/d37/f95 [0,4194304] 0 2024-02-22T15:50:49.783 INFO:tasks.workunit.client.0.smithi019.stdout:4/969: write d2/db/d14/d23/f2f [1296722,129220] 0 2024-02-22T15:50:49.783 INFO:tasks.workunit.client.1.smithi062.stdout:1/706: chown d1/d6/d7 2049461557 1 2024-02-22T15:50:49.787 INFO:tasks.workunit.client.0.smithi019.stdout:3/952: getdents d9/d65/d106 0 2024-02-22T15:50:49.788 INFO:tasks.workunit.client.1.smithi062.stdout:5/700: write d0/d3/da/d13/d23/d4e/d5a/fcf [309870,14040] 0 2024-02-22T15:50:49.788 INFO:tasks.workunit.client.1.smithi062.stdout:3/677: write d5/f8 [848886,99094] 0 2024-02-22T15:50:49.788 INFO:tasks.workunit.client.0.smithi019.stdout:3/953: rmdir d9/d26/d2d 39 2024-02-22T15:50:49.791 INFO:tasks.workunit.client.1.smithi062.stdout:1/707: mkdir d1/d6/d7/d8a/dec 0 2024-02-22T15:50:49.792 INFO:tasks.workunit.client.1.smithi062.stdout:4/899: unlink d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd7/c109 0 2024-02-22T15:50:49.793 INFO:tasks.workunit.client.1.smithi062.stdout:0/914: dread d1/d3/d4/d1e/d74/f70 [0,4194304] 0 2024-02-22T15:50:49.796 INFO:tasks.workunit.client.1.smithi062.stdout:3/678: symlink d5/d16/d21/ld6 0 2024-02-22T15:50:49.799 INFO:tasks.workunit.client.1.smithi062.stdout:2/941: creat d1/d8/f140 x:0 0 0 2024-02-22T15:50:49.800 INFO:tasks.workunit.client.1.smithi062.stdout:7/618: mknod d0/d6/d2e/d58/dc0/cc1 0 2024-02-22T15:50:49.800 INFO:tasks.workunit.client.1.smithi062.stdout:1/708: creat d1/d6/fed x:0 0 0 2024-02-22T15:50:49.804 INFO:tasks.workunit.client.1.smithi062.stdout:3/679: chown d5/d16/d21/d36 640223 1 2024-02-22T15:50:49.804 INFO:tasks.workunit.client.1.smithi062.stdout:0/915: mkdir d1/d3/de/d131 0 2024-02-22T15:50:49.805 INFO:tasks.workunit.client.1.smithi062.stdout:4/900: creat d4/d1c/d2c/d6b/d70/d130/f13f x:0 0 0 2024-02-22T15:50:49.806 INFO:tasks.workunit.client.1.smithi062.stdout:5/701: dwrite d0/d3/da/d13/d23/d78/d81/fb4 [0,4194304] 0 2024-02-22T15:50:49.808 INFO:tasks.workunit.client.1.smithi062.stdout:1/709: mkdir d1/d6/d36/d37/dee 0 2024-02-22T15:50:49.809 INFO:tasks.workunit.client.1.smithi062.stdout:2/942: readlink d1/d47/d32/d52/dbc/lf0 0 2024-02-22T15:50:49.811 INFO:tasks.workunit.client.1.smithi062.stdout:9/820: dwrite d4/d15/d5f/d75/d84/f7c [0,4194304] 0 2024-02-22T15:50:49.813 INFO:tasks.workunit.client.0.smithi019.stdout:4/970: dread d2/db/d14/d23/d21/d42/dc4/d11f/f138 [0,4194304] 0 2024-02-22T15:50:49.814 INFO:tasks.workunit.client.0.smithi019.stdout:4/971: fsync d2/db/d59/ff9 0 2024-02-22T15:50:49.815 INFO:tasks.workunit.client.1.smithi062.stdout:7/619: read d0/d14/d13/f48 [2216061,56699] 0 2024-02-22T15:50:49.815 INFO:tasks.workunit.client.1.smithi062.stdout:3/680: unlink d5/d16/d21/d84/fca 0 2024-02-22T15:50:49.819 INFO:tasks.workunit.client.1.smithi062.stdout:2/943: rmdir d1/d8/d62/d73 39 2024-02-22T15:50:49.822 INFO:tasks.workunit.client.0.smithi019.stdout:4/972: dwrite d2/db/d14/d23/d67/f68 [0,4194304] 0 2024-02-22T15:50:49.823 INFO:tasks.workunit.client.1.smithi062.stdout:0/916: rename d1/d13/dde/d76/d8c/d92/f96 to d1/d3/d4/d1e/d74/d122/f132 0 2024-02-22T15:50:49.823 INFO:tasks.workunit.client.1.smithi062.stdout:0/917: fsync d1/ff3 0 2024-02-22T15:50:49.826 INFO:tasks.workunit.client.1.smithi062.stdout:0/918: write d1/d3/d4/f99 [3202122,35453] 0 2024-02-22T15:50:49.827 INFO:tasks.workunit.client.1.smithi062.stdout:9/821: mkdir d4/d15/d109 0 2024-02-22T15:50:49.827 INFO:tasks.workunit.client.0.smithi019.stdout:4/973: rename d2/db/d14/d23/d67/f11a to d2/db/d14/d23/d21/d42/f139 0 2024-02-22T15:50:49.830 INFO:tasks.workunit.client.0.smithi019.stdout:4/974: write d2/db/d14/d23/d21/d42/dc4/d11f/f138 [390816,47988] 0 2024-02-22T15:50:49.833 INFO:tasks.workunit.client.0.smithi019.stdout:4/975: unlink d2/db/d14/d23/d9e/dac/cbc 0 2024-02-22T15:50:49.835 INFO:tasks.workunit.client.1.smithi062.stdout:1/710: creat d1/d6/d7/fef x:0 0 0 2024-02-22T15:50:49.836 INFO:tasks.workunit.client.0.smithi019.stdout:4/976: getdents d2/db/d14/d23/d33/db1 0 2024-02-22T15:50:49.837 INFO:tasks.workunit.client.1.smithi062.stdout:4/901: rmdir d4/d1c/d138 0 2024-02-22T15:50:49.837 INFO:tasks.workunit.client.1.smithi062.stdout:9/822: dwrite d4/d28/d50/da3/fcb [0,4194304] 0 2024-02-22T15:50:49.838 INFO:tasks.workunit.client.0.smithi019.stdout:4/977: truncate d2/db/d14/d23/d33/f74 4571136 0 2024-02-22T15:50:49.839 INFO:tasks.workunit.client.1.smithi062.stdout:6/802: sync 2024-02-22T15:50:49.839 INFO:tasks.workunit.client.1.smithi062.stdout:8/934: sync 2024-02-22T15:50:49.840 INFO:tasks.workunit.client.0.smithi019.stdout:4/978: fdatasync d2/db/d14/fc7 0 2024-02-22T15:50:49.842 INFO:tasks.workunit.client.1.smithi062.stdout:8/935: chown d8/d9/d2c/d49/d5e/cca 10581 1 2024-02-22T15:50:49.843 INFO:tasks.workunit.client.1.smithi062.stdout:2/944: write d1/d47/d32/d52/d80/f9b [2261496,86991] 0 2024-02-22T15:50:49.843 INFO:tasks.workunit.client.1.smithi062.stdout:5/702: write d0/d3/d2f/f44 [695755,99706] 0 2024-02-22T15:50:49.845 INFO:tasks.workunit.client.0.smithi019.stdout:4/979: mknod d2/d3f/d48/d76/c13a 0 2024-02-22T15:50:49.845 INFO:tasks.workunit.client.1.smithi062.stdout:3/681: write d5/d16/d21/f34 [551593,120877] 0 2024-02-22T15:50:49.849 INFO:tasks.workunit.client.0.smithi019.stdout:4/980: write d2/db/d54/d8c/dbb/f131 [451374,126762] 0 2024-02-22T15:50:49.849 INFO:tasks.workunit.client.1.smithi062.stdout:4/902: mknod d4/d13/d20/c140 0 2024-02-22T15:50:49.849 INFO:tasks.workunit.client.1.smithi062.stdout:6/803: truncate d5/d8a/fcf 936278 0 2024-02-22T15:50:49.850 INFO:tasks.workunit.client.1.smithi062.stdout:1/711: dwrite d1/d6/d7/de6/fe7 [0,4194304] 0 2024-02-22T15:50:49.850 INFO:tasks.workunit.client.0.smithi019.stdout:4/981: fdatasync d2/db/d14/d23/f28 0 2024-02-22T15:50:49.853 INFO:tasks.workunit.client.1.smithi062.stdout:7/620: truncate d0/d6/d2e/f86 2530693 0 2024-02-22T15:50:49.854 INFO:tasks.workunit.client.1.smithi062.stdout:3/682: rename d5/d16/d56/dbc/dcb/fd1 to d5/d1a/d3b/dd5/fd7 0 2024-02-22T15:50:49.857 INFO:tasks.workunit.client.1.smithi062.stdout:5/703: dread d0/d3/da/d13/d23/d4e/d5a/f5e [0,4194304] 0 2024-02-22T15:50:49.858 INFO:tasks.workunit.client.1.smithi062.stdout:1/712: mknod d1/d6/d36/d76/cf0 0 2024-02-22T15:50:49.858 INFO:tasks.workunit.client.0.smithi019.stdout:4/982: dwrite d2/db/d14/d23/d21/f2c [0,4194304] 0 2024-02-22T15:50:49.858 INFO:tasks.workunit.client.1.smithi062.stdout:4/903: write d4/d1c/d2c/d6b/d70/de8/dfc/fff [1068723,108943] 0 2024-02-22T15:50:49.859 INFO:tasks.workunit.client.0.smithi019.stdout:4/983: chown d2/db/fdc 58909947 1 2024-02-22T15:50:49.862 INFO:tasks.workunit.client.1.smithi062.stdout:4/904: fsync d4/df/daf/d10c/f121 0 2024-02-22T15:50:49.864 INFO:tasks.workunit.client.1.smithi062.stdout:7/621: creat d0/d6/d2e/d4a/d5a/db3/fc2 x:0 0 0 2024-02-22T15:50:49.865 INFO:tasks.workunit.client.1.smithi062.stdout:3/683: creat d5/d16/d97/fd8 x:0 0 0 2024-02-22T15:50:49.866 INFO:tasks.workunit.client.1.smithi062.stdout:1/713: chown d1/d6/d7/d2b/d56/d95 1439 1 2024-02-22T15:50:49.867 INFO:tasks.workunit.client.0.smithi019.stdout:4/984: dwrite d2/db/d14/d23/f28 [0,4194304] 0 2024-02-22T15:50:49.869 INFO:tasks.workunit.client.1.smithi062.stdout:2/945: link d1/d8/d62/d73/lc6 d1/d1e/d38/d76/df5/l141 0 2024-02-22T15:50:49.870 INFO:tasks.workunit.client.0.smithi019.stdout:4/985: creat d2/d3f/d48/f13b x:0 0 0 2024-02-22T15:50:49.871 INFO:tasks.workunit.client.1.smithi062.stdout:0/919: dread d1/d13/dde/d76/d8c/d5d/f37 [0,4194304] 0 2024-02-22T15:50:49.872 INFO:tasks.workunit.client.1.smithi062.stdout:5/704: rename d0/d3/d2f/d59/d67/f95 to d0/d3/d2f/d59/d67/d69/d3f/d66/da9/fe5 0 2024-02-22T15:50:49.873 INFO:tasks.workunit.client.1.smithi062.stdout:9/823: dwrite d4/d28/f38 [0,4194304] 0 2024-02-22T15:50:49.874 INFO:tasks.workunit.client.1.smithi062.stdout:4/905: creat d4/d1c/d2c/d6b/d9b/db1/dc6/f141 x:0 0 0 2024-02-22T15:50:49.874 INFO:tasks.workunit.client.0.smithi019.stdout:4/986: creat d2/db/d14/d23/f13c x:0 0 0 2024-02-22T15:50:49.876 INFO:tasks.workunit.client.0.smithi019.stdout:3/954: sync 2024-02-22T15:50:49.878 INFO:tasks.workunit.client.0.smithi019.stdout:4/987: creat d2/db/d14/d23/d21/d42/dc4/dd5/f13d x:0 0 0 2024-02-22T15:50:49.880 INFO:tasks.workunit.client.0.smithi019.stdout:3/955: rename d9/d26/d76/l112 to d9/d14/d12c/l132 0 2024-02-22T15:50:49.881 INFO:tasks.workunit.client.1.smithi062.stdout:7/622: mkdir d0/d14/d19/d30/d45/dc3 0 2024-02-22T15:50:49.881 INFO:tasks.workunit.client.1.smithi062.stdout:2/946: symlink d1/d1e/d38/d76/dc5/df3/l142 0 2024-02-22T15:50:49.881 INFO:tasks.workunit.client.1.smithi062.stdout:4/906: chown d4/d66/c110 9308 1 2024-02-22T15:50:49.881 INFO:tasks.workunit.client.1.smithi062.stdout:0/920: fsync d1/d3/d4/d1e/d74/d68/fc7 0 2024-02-22T15:50:49.881 INFO:tasks.workunit.client.1.smithi062.stdout:7/623: chown d0/d14/d19/f7b 224171710 1 2024-02-22T15:50:49.882 INFO:tasks.workunit.client.1.smithi062.stdout:9/824: write d4/d28/f72 [8304920,126540] 0 2024-02-22T15:50:49.883 INFO:tasks.workunit.client.0.smithi019.stdout:3/956: readlink d9/d26/d2d/l70 0 2024-02-22T15:50:49.884 INFO:tasks.workunit.client.1.smithi062.stdout:9/825: chown d4/d15/d31/d106 2 1 2024-02-22T15:50:49.884 INFO:tasks.workunit.client.1.smithi062.stdout:5/705: dread d0/d3/da/d13/d23/d4e/d5a/f5e [0,4194304] 0 2024-02-22T15:50:49.885 INFO:tasks.workunit.client.0.smithi019.stdout:3/957: mknod d9/d22/d27/d96/c133 0 2024-02-22T15:50:49.887 INFO:tasks.workunit.client.0.smithi019.stdout:3/958: creat d9/d65/da7/dab/f134 x:0 0 0 2024-02-22T15:50:49.890 INFO:tasks.workunit.client.0.smithi019.stdout:3/959: rename d9/d26/d3f/d10b/c10f to d9/d65/da7/d127/c135 0 2024-02-22T15:50:49.891 INFO:tasks.workunit.client.0.smithi019.stdout:3/960: dread - d9/d26/d3f/d10b/d66/f87 zero size 2024-02-22T15:50:49.892 INFO:tasks.workunit.client.0.smithi019.stdout:3/961: chown d9/d22/d27/lb2 1370 1 2024-02-22T15:50:49.893 INFO:tasks.workunit.client.1.smithi062.stdout:5/706: creat d0/d3/da/d32/d42/fe6 x:0 0 0 2024-02-22T15:50:49.893 INFO:tasks.workunit.client.0.smithi019.stdout:3/962: fsync d9/d14/d19/f10c 0 2024-02-22T15:50:49.897 INFO:tasks.workunit.client.0.smithi019.stdout:3/963: rename d9/d22/d27/c75 to d9/d26/d3f/d11f/c136 0 2024-02-22T15:50:49.898 INFO:tasks.workunit.client.0.smithi019.stdout:4/988: truncate d2/db/d14/d61/f73 1128859 0 2024-02-22T15:50:49.899 INFO:tasks.workunit.client.0.smithi019.stdout:4/989: symlink d2/d3f/l13e 0 2024-02-22T15:50:49.900 INFO:tasks.workunit.client.0.smithi019.stdout:4/990: truncate d2/db/f26 2500003 0 2024-02-22T15:50:49.902 INFO:tasks.workunit.client.1.smithi062.stdout:9/826: dwrite d4/d28/d50/fdf [0,4194304] 0 2024-02-22T15:50:49.908 INFO:tasks.workunit.client.1.smithi062.stdout:4/907: dwrite d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f129 [0,4194304] 0 2024-02-22T15:50:49.910 INFO:tasks.workunit.client.1.smithi062.stdout:3/684: dread d5/d16/d21/d84/f9a [0,4194304] 0 2024-02-22T15:50:49.911 INFO:tasks.workunit.client.1.smithi062.stdout:4/908: chown d4/d1c/d30/d3b/d49/f95 970030 1 2024-02-22T15:50:49.913 INFO:tasks.workunit.client.1.smithi062.stdout:5/707: mknod d0/d3/da/d13/d57/ce7 0 2024-02-22T15:50:49.918 INFO:tasks.workunit.client.1.smithi062.stdout:3/685: readlink d5/l7 0 2024-02-22T15:50:49.920 INFO:tasks.workunit.client.1.smithi062.stdout:2/947: dread d1/d47/f88 [0,4194304] 0 2024-02-22T15:50:49.920 INFO:tasks.workunit.client.1.smithi062.stdout:1/714: dread d1/d66/f75 [0,4194304] 0 2024-02-22T15:50:49.922 INFO:tasks.workunit.client.1.smithi062.stdout:5/708: chown d0/d3/da/d13/d23/d31/fba 187 1 2024-02-22T15:50:49.922 INFO:tasks.workunit.client.1.smithi062.stdout:3/686: stat d5/d16/d21/d32/d75 0 2024-02-22T15:50:49.924 INFO:tasks.workunit.client.1.smithi062.stdout:5/709: fsync d0/f45 0 2024-02-22T15:50:49.925 INFO:tasks.workunit.client.0.smithi019.stdout:3/964: dread d9/d65/da7/dab/f131 [0,4194304] 0 2024-02-22T15:50:49.926 INFO:tasks.workunit.client.1.smithi062.stdout:4/909: dread d4/d2a/fd1 [0,4194304] 0 2024-02-22T15:50:49.927 INFO:tasks.workunit.client.0.smithi019.stdout:3/965: write d9/d26/d76/f108 [1746192,104331] 0 2024-02-22T15:50:49.929 INFO:tasks.workunit.client.1.smithi062.stdout:1/715: fdatasync d1/d6/d14/fe4 0 2024-02-22T15:50:49.930 INFO:tasks.workunit.client.1.smithi062.stdout:7/624: dwrite d0/d6/d2e/d55/d80/d8a/f90 [0,4194304] 0 2024-02-22T15:50:49.930 INFO:tasks.workunit.client.1.smithi062.stdout:2/948: readlink d1/d8/d62/d73/da6/la9 0 2024-02-22T15:50:49.930 INFO:tasks.workunit.client.0.smithi019.stdout:3/966: link d9/d11/fb4 d9/d65/d106/f137 0 2024-02-22T15:50:49.931 INFO:tasks.workunit.client.0.smithi019.stdout:3/967: stat d9/d65/da7/fde 0 2024-02-22T15:50:49.933 INFO:tasks.workunit.client.1.smithi062.stdout:4/910: chown d4/d1c/d2c/d6b/fca 512825 1 2024-02-22T15:50:49.934 INFO:tasks.workunit.client.1.smithi062.stdout:9/827: dwrite d4/d28/f72 [0,4194304] 0 2024-02-22T15:50:49.938 INFO:tasks.workunit.client.1.smithi062.stdout:2/949: write d1/d8/d62/d73/f75 [18670,120319] 0 2024-02-22T15:50:49.938 INFO:tasks.workunit.client.1.smithi062.stdout:5/710: fdatasync d0/d3/da/d13/d23/d4e/fa3 0 2024-02-22T15:50:49.941 INFO:tasks.workunit.client.1.smithi062.stdout:5/711: chown d0/d3/da/d13/d23/d4e/d5a/fcf 26845979 1 2024-02-22T15:50:49.943 INFO:tasks.workunit.client.1.smithi062.stdout:0/921: dwrite d1/fe6 [0,4194304] 0 2024-02-22T15:50:49.945 INFO:tasks.workunit.client.1.smithi062.stdout:2/950: dread d1/d8/d62/d73/dbd/d7d/f111 [0,4194304] 0 2024-02-22T15:50:49.945 INFO:tasks.workunit.client.1.smithi062.stdout:0/922: readlink d1/d3/d4/d1e/d74/d68/l11f 0 2024-02-22T15:50:49.947 INFO:tasks.workunit.client.1.smithi062.stdout:9/828: creat d4/d15/f10a x:0 0 0 2024-02-22T15:50:49.949 INFO:tasks.workunit.client.1.smithi062.stdout:2/951: symlink d1/d8/d62/d73/dbd/dbb/dc4/dfe/l143 0 2024-02-22T15:50:49.950 INFO:tasks.workunit.client.1.smithi062.stdout:0/923: creat d1/d3/de/d50/f133 x:0 0 0 2024-02-22T15:50:49.951 INFO:tasks.workunit.client.1.smithi062.stdout:5/712: truncate d0/d3/da/d13/d23/d4e/d6d/f80 764043 0 2024-02-22T15:50:49.952 INFO:tasks.workunit.client.0.smithi019.stdout:3/968: truncate d9/d26/d32/f33 143652 0 2024-02-22T15:50:49.952 INFO:tasks.workunit.client.0.smithi019.stdout:4/991: dread d2/db/d59/ff9 [0,4194304] 0 2024-02-22T15:50:49.953 INFO:tasks.workunit.client.1.smithi062.stdout:9/829: mkdir d4/d15/d5e/d63/d10b 0 2024-02-22T15:50:49.955 INFO:tasks.workunit.client.0.smithi019.stdout:3/969: mkdir d9/d26/d2d/d103/d138 0 2024-02-22T15:50:49.955 INFO:tasks.workunit.client.0.smithi019.stdout:4/992: dread - d2/db/d14/d23/d21/d42/dc4/ff0 zero size 2024-02-22T15:50:49.956 INFO:tasks.workunit.client.1.smithi062.stdout:3/687: write d5/d16/d21/f22 [28051,48550] 0 2024-02-22T15:50:49.956 INFO:tasks.workunit.client.0.smithi019.stdout:3/970: chown d9/d26/d2d/d53/da0/lac 3 1 2024-02-22T15:50:49.956 INFO:tasks.workunit.client.0.smithi019.stdout:3/971: stat d9/d11/cc6 0 2024-02-22T15:50:49.958 INFO:tasks.workunit.client.0.smithi019.stdout:4/993: symlink d2/db/d14/d23/d21/d35/d85/l13f 0 2024-02-22T15:50:49.960 INFO:tasks.workunit.client.1.smithi062.stdout:2/952: dwrite d1/d47/f83 [0,4194304] 0 2024-02-22T15:50:49.960 INFO:tasks.workunit.client.1.smithi062.stdout:5/713: read d0/d3/da/d13/d23/d4e/d60/f63 [2333481,9970] 0 2024-02-22T15:50:49.960 INFO:tasks.workunit.client.1.smithi062.stdout:9/830: getdents d4/d15/d5e/d101 0 2024-02-22T15:50:49.960 INFO:tasks.workunit.client.0.smithi019.stdout:3/972: mkdir d9/d22/d139 0 2024-02-22T15:50:49.961 INFO:tasks.workunit.client.1.smithi062.stdout:3/688: write d5/d16/d21/d36/d5a/d85/f93 [46868,124077] 0 2024-02-22T15:50:49.962 INFO:tasks.workunit.client.0.smithi019.stdout:4/994: rename d2/db/d14/d23/d33 to d2/d3f/d48/de9/d126/d140 0 2024-02-22T15:50:49.965 INFO:tasks.workunit.client.0.smithi019.stdout:4/995: chown d2/db/d14/d23/d21/d35/d3d/c83 15 1 2024-02-22T15:50:49.966 INFO:tasks.workunit.client.1.smithi062.stdout:2/953: chown d1/d8/d62/d73/dbd/d7d/ffb 1 1 2024-02-22T15:50:49.966 INFO:tasks.workunit.client.1.smithi062.stdout:9/831: readlink d4/d15/l19 0 2024-02-22T15:50:49.969 INFO:tasks.workunit.client.0.smithi019.stdout:3/973: link d9/d14/c36 d9/d22/d27/c13a 0 2024-02-22T15:50:49.969 INFO:tasks.workunit.client.1.smithi062.stdout:5/714: creat d0/d3/da/d13/d23/d4e/d6d/dd2/fe8 x:0 0 0 2024-02-22T15:50:49.971 INFO:tasks.workunit.client.0.smithi019.stdout:4/996: getdents d2/db/d14/d23/d67 0 2024-02-22T15:50:49.971 INFO:tasks.workunit.client.1.smithi062.stdout:3/689: rename d5/d16/d21/d84/lb3 to d5/d16/d56/ld9 0 2024-02-22T15:50:49.972 INFO:tasks.workunit.client.1.smithi062.stdout:4/911: dread d4/d2a/d2d/d46/fad [0,4194304] 0 2024-02-22T15:50:49.972 INFO:tasks.workunit.client.0.smithi019.stdout:4/997: write d2/db/d14/d23/d21/d42/dc4/dd5/f105 [40005,58096] 0 2024-02-22T15:50:49.973 INFO:tasks.workunit.client.1.smithi062.stdout:3/690: stat d5/d1a/f25 0 2024-02-22T15:50:49.973 INFO:tasks.workunit.client.1.smithi062.stdout:2/954: chown d1/d8/d62/d73/dbd/f46 2773 1 2024-02-22T15:50:49.974 INFO:tasks.workunit.client.0.smithi019.stdout:4/998: fsync d2/d3f/ffa 0 2024-02-22T15:50:49.974 INFO:tasks.workunit.client.0.smithi019.stdout:3/974: read d9/d26/d2d/d53/f56 [2508309,31679] 0 2024-02-22T15:50:49.975 INFO:tasks.workunit.client.0.smithi019.stdout:3/975: mkdir d9/d22/d49/d13b 0 2024-02-22T15:50:49.977 INFO:tasks.workunit.client.1.smithi062.stdout:3/691: chown d5/d16/d21/d32/f4c 17871769 1 2024-02-22T15:50:49.977 INFO:tasks.workunit.client.0.smithi019.stdout:4/999: rename d2/db/d14/d23/d4b/f129 to d2/db/d14/d23/d21/d42/dc4/dd5/f141 0 2024-02-22T15:50:49.978 INFO:tasks.workunit.client.1.smithi062.stdout:5/715: dread d0/d3/da/d13/d23/f3b [0,4194304] 0 2024-02-22T15:50:49.979 INFO:tasks.workunit.client.1.smithi062.stdout:9/832: creat d4/d15/d5f/d75/d97/f10c x:0 0 0 2024-02-22T15:50:49.980 INFO:tasks.workunit.client.1.smithi062.stdout:4/912: symlink d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd0/l142 0 2024-02-22T15:50:49.980 INFO:tasks.workunit.client.0.smithi019.stdout:3/976: rename d9/d26/l37 to d9/d16/d107/l13c 0 2024-02-22T15:50:49.981 INFO:tasks.workunit.client.1.smithi062.stdout:3/692: creat d5/d16/d21/d32/d75/fda x:0 0 0 2024-02-22T15:50:49.981 INFO:tasks.workunit.client.1.smithi062.stdout:2/955: read d1/d1e/f9a [5979833,127240] 0 2024-02-22T15:50:49.982 INFO:tasks.workunit.client.0.smithi019.stdout:3/977: mkdir d9/d26/d3f/d13d 0 2024-02-22T15:50:49.982 INFO:tasks.workunit.client.1.smithi062.stdout:3/693: chown d5/d1a/d3c/l9c 1 1 2024-02-22T15:50:49.984 INFO:tasks.workunit.client.0.smithi019.stdout:3/978: truncate d9/f115 1321482 0 2024-02-22T15:50:49.984 INFO:tasks.workunit.client.1.smithi062.stdout:5/716: mkdir d0/d3/da/d13/d57/d79/dd0/de9 0 2024-02-22T15:50:49.985 INFO:tasks.workunit.client.0.smithi019.stdout:3/979: fdatasync d9/d26/d3f/d10b/d66/f7f 0 2024-02-22T15:50:49.986 INFO:tasks.workunit.client.1.smithi062.stdout:3/694: write d5/d26/f61 [4730091,130082] 0 2024-02-22T15:50:49.990 INFO:tasks.workunit.client.1.smithi062.stdout:1/716: dread d1/d6/de/f17 [0,4194304] 0 2024-02-22T15:50:49.993 INFO:tasks.workunit.client.1.smithi062.stdout:3/695: chown d5/d16/d56/dbc/lbf 37 1 2024-02-22T15:50:49.995 INFO:tasks.workunit.client.1.smithi062.stdout:5/717: mknod d0/d3/da/d13/d57/dc7/dda/cea 0 2024-02-22T15:50:49.996 INFO:tasks.workunit.client.1.smithi062.stdout:4/913: dwrite d4/d13/d20/fea [0,4194304] 0 2024-02-22T15:50:49.997 INFO:tasks.workunit.client.1.smithi062.stdout:9/833: dread d4/d15/d5f/d75/d97/faf [0,4194304] 0 2024-02-22T15:50:49.998 INFO:tasks.workunit.client.1.smithi062.stdout:4/914: chown d4/d66/dfa/l10a 97817067 1 2024-02-22T15:50:49.998 INFO:tasks.workunit.client.1.smithi062.stdout:1/717: mknod d1/d6/d7/d2b/d49/dcc/cf1 0 2024-02-22T15:50:49.999 INFO:tasks.workunit.client.1.smithi062.stdout:5/718: dread - d0/d3/da/d13/d23/d4e/d60/db5/fc5 zero size 2024-02-22T15:50:50.001 INFO:tasks.workunit.client.1.smithi062.stdout:3/696: mkdir d5/d26/d2d/d3f/ddb 0 2024-02-22T15:50:50.002 INFO:tasks.workunit.client.1.smithi062.stdout:9/834: symlink d4/d28/l10d 0 2024-02-22T15:50:50.004 INFO:tasks.workunit.client.1.smithi062.stdout:9/835: chown d4/d1c/l1f 11929 1 2024-02-22T15:50:50.007 INFO:tasks.workunit.client.1.smithi062.stdout:4/915: truncate d4/d66/f6f 1260827 0 2024-02-22T15:50:50.011 INFO:tasks.workunit.client.1.smithi062.stdout:9/836: creat d4/d15/d5e/d63/da0/d92/db6/f10e x:0 0 0 2024-02-22T15:50:50.014 INFO:tasks.workunit.client.1.smithi062.stdout:3/697: dwrite d5/d16/d21/d36/d5a/d85/fb2 [4194304,4194304] 0 2024-02-22T15:50:50.016 INFO:tasks.workunit.client.1.smithi062.stdout:9/837: unlink d4/d15/d5f/d6a/ce3 0 2024-02-22T15:50:50.016 INFO:tasks.workunit.client.1.smithi062.stdout:2/956: dread d1/d47/d39/fcd [0,4194304] 0 2024-02-22T15:50:50.020 INFO:tasks.workunit.client.1.smithi062.stdout:3/698: fsync d5/d26/f35 0 2024-02-22T15:50:50.021 INFO:tasks.workunit.client.1.smithi062.stdout:1/718: truncate d1/d6/d7/d2b/d47/d63/f87 1313769 0 2024-02-22T15:50:50.027 INFO:tasks.workunit.client.1.smithi062.stdout:2/957: read d1/d28/fc7 [89474,35822] 0 2024-02-22T15:50:50.031 INFO:tasks.workunit.client.1.smithi062.stdout:3/699: symlink d5/d1a/d3b/ldc 0 2024-02-22T15:50:50.032 INFO:tasks.workunit.client.1.smithi062.stdout:5/719: dwrite d0/d3/da/f18 [0,4194304] 0 2024-02-22T15:50:50.034 INFO:tasks.workunit.client.1.smithi062.stdout:9/838: link d4/d15/d5f/d75/d84/c45 d4/d15/d5e/d63/da0/c10f 0 2024-02-22T15:50:50.037 INFO:tasks.workunit.client.1.smithi062.stdout:4/916: dwrite d4/d34/f48 [4194304,4194304] 0 2024-02-22T15:50:50.047 INFO:tasks.workunit.client.1.smithi062.stdout:1/719: dwrite d1/d6/d7/d2b/fc4 [0,4194304] 0 2024-02-22T15:50:50.052 INFO:tasks.workunit.client.1.smithi062.stdout:4/917: truncate d4/d1c/d30/d37/f123 285866 0 2024-02-22T15:50:50.056 INFO:tasks.workunit.client.1.smithi062.stdout:2/958: dwrite d1/d1e/ff8 [0,4194304] 0 2024-02-22T15:50:50.056 INFO:tasks.workunit.client.1.smithi062.stdout:4/918: creat d4/d1c/dac/f143 x:0 0 0 2024-02-22T15:50:50.058 INFO:tasks.workunit.client.1.smithi062.stdout:3/700: link d5/da6/ca8 d5/d16/d21/d36/d5a/d85/cdd 0 2024-02-22T15:50:50.058 INFO:tasks.workunit.client.1.smithi062.stdout:9/839: dwrite d4/d1c/d83/fc5 [0,4194304] 0 2024-02-22T15:50:50.066 INFO:tasks.workunit.client.1.smithi062.stdout:1/720: rename d1/d6/d36/fe5 to d1/d6/d7/d2b/d47/d24/ff2 0 2024-02-22T15:50:50.067 INFO:tasks.workunit.client.1.smithi062.stdout:9/840: mkdir d4/d15/d5f/d75/d84/d110 0 2024-02-22T15:50:50.069 INFO:tasks.workunit.client.1.smithi062.stdout:9/841: chown d4/f10 167656365 1 2024-02-22T15:50:50.070 INFO:tasks.workunit.client.1.smithi062.stdout:9/842: chown d4/d28 495 1 2024-02-22T15:50:50.072 INFO:tasks.workunit.client.1.smithi062.stdout:2/959: dwrite d1/d47/d32/d52/dbc/f138 [0,4194304] 0 2024-02-22T15:50:50.074 INFO:tasks.workunit.client.1.smithi062.stdout:1/721: rename d1/d6/d7/d2b/d47/d63/db6 to d1/d6/d7/d2b/d96/df3 0 2024-02-22T15:50:50.079 INFO:tasks.workunit.client.1.smithi062.stdout:1/722: creat d1/d6/d7/d2b/d47/d63/ff4 x:0 0 0 2024-02-22T15:50:50.081 INFO:tasks.workunit.client.1.smithi062.stdout:2/960: creat d1/d1e/d38/d76/dc5/df3/d10f/f144 x:0 0 0 2024-02-22T15:50:50.083 INFO:tasks.workunit.client.1.smithi062.stdout:2/961: fdatasync d1/f50 0 2024-02-22T15:50:50.085 INFO:tasks.workunit.client.1.smithi062.stdout:4/919: truncate d4/d2a/d2d/ff9 1766213 0 2024-02-22T15:50:50.088 INFO:tasks.workunit.client.1.smithi062.stdout:3/701: dwrite d5/d16/d21/d32/d6a/f8e [0,4194304] 0 2024-02-22T15:50:50.091 INFO:tasks.workunit.client.1.smithi062.stdout:3/702: truncate f3 3964871 0 2024-02-22T15:50:50.094 INFO:tasks.workunit.client.1.smithi062.stdout:3/703: write d5/d1a/d3b/f71 [464258,109178] 0 2024-02-22T15:50:50.094 INFO:tasks.workunit.client.1.smithi062.stdout:9/843: dwrite d4/d15/d5e/d63/da0/d92/db6/fc4 [0,4194304] 0 2024-02-22T15:50:50.096 INFO:tasks.workunit.client.1.smithi062.stdout:4/920: write d4/f51 [788091,103530] 0 2024-02-22T15:50:50.108 INFO:tasks.workunit.client.1.smithi062.stdout:9/844: dwrite d4/d15/ff7 [0,4194304] 0 2024-02-22T15:50:50.111 INFO:tasks.workunit.client.1.smithi062.stdout:9/845: stat d4/d15/f44 0 2024-02-22T15:50:50.113 INFO:tasks.workunit.client.1.smithi062.stdout:3/704: write d5/d26/d2d/d3f/f7c [567194,59423] 0 2024-02-22T15:50:50.114 INFO:tasks.workunit.client.1.smithi062.stdout:9/846: chown d4/d15/dfd 2825 1 2024-02-22T15:50:50.115 INFO:tasks.workunit.client.1.smithi062.stdout:3/705: creat d5/da6/fde x:0 0 0 2024-02-22T15:50:50.117 INFO:tasks.workunit.client.1.smithi062.stdout:9/847: mkdir d4/d28/d50/d5a/d111 0 2024-02-22T15:50:50.117 INFO:tasks.workunit.client.1.smithi062.stdout:9/848: chown d4/l41 50147343 1 2024-02-22T15:50:50.121 INFO:tasks.workunit.client.1.smithi062.stdout:9/849: creat d4/d15/d5f/d6a/f112 x:0 0 0 2024-02-22T15:50:50.128 INFO:tasks.workunit.client.1.smithi062.stdout:3/706: dwrite d5/d26/f27 [0,4194304] 0 2024-02-22T15:50:50.130 INFO:tasks.workunit.client.1.smithi062.stdout:3/707: write d5/fbd [1428160,72527] 0 2024-02-22T15:50:50.132 INFO:tasks.workunit.client.1.smithi062.stdout:9/850: dwrite d4/d15/d5e/d63/da0/d92/db6/fc4 [0,4194304] 0 2024-02-22T15:50:50.135 INFO:tasks.workunit.client.1.smithi062.stdout:3/708: symlink d5/d16/d56/dbc/dcb/ldf 0 2024-02-22T15:50:50.137 INFO:tasks.workunit.client.1.smithi062.stdout:9/851: creat d4/d28/d50/da3/f113 x:0 0 0 2024-02-22T15:50:50.138 INFO:tasks.workunit.client.0.smithi019.stdout:3/980: sync 2024-02-22T15:50:50.139 INFO:tasks.workunit.client.1.smithi062.stdout:3/709: symlink d5/d1a/d3b/le0 0 2024-02-22T15:50:50.140 INFO:tasks.workunit.client.1.smithi062.stdout:9/852: getdents d4/d15/dfd 0 2024-02-22T15:50:50.141 INFO:tasks.workunit.client.0.smithi019.stdout:3/981: mknod d9/d65/d106/c13e 0 2024-02-22T15:50:50.142 INFO:tasks.workunit.client.1.smithi062.stdout:3/710: creat d5/d16/d21/d96/fe1 x:0 0 0 2024-02-22T15:50:50.149 INFO:tasks.workunit.client.0.smithi019.stdout:3/982: read d9/d22/f9d [1098054,56836] 0 2024-02-22T15:50:50.152 INFO:tasks.workunit.client.1.smithi062.stdout:9/853: dwrite d4/f12 [4194304,4194304] 0 2024-02-22T15:50:50.154 INFO:tasks.workunit.client.1.smithi062.stdout:9/854: creat d4/d15/d5f/d75/d84/f114 x:0 0 0 2024-02-22T15:50:50.156 INFO:tasks.workunit.client.1.smithi062.stdout:3/711: dread d5/d1a/d3c/f54 [0,4194304] 0 2024-02-22T15:50:50.156 INFO:tasks.workunit.client.1.smithi062.stdout:9/855: mknod d4/d15/d5e/d63/da0/d92/c115 0 2024-02-22T15:50:50.156 INFO:tasks.workunit.client.1.smithi062.stdout:3/712: stat d5/d1a/f1e 0 2024-02-22T15:50:50.157 INFO:tasks.workunit.client.1.smithi062.stdout:9/856: fsync d4/d1c/d83/fdc 0 2024-02-22T15:50:50.159 INFO:tasks.workunit.client.1.smithi062.stdout:9/857: stat d4/d15/ld0 0 2024-02-22T15:50:50.159 INFO:tasks.workunit.client.1.smithi062.stdout:9/858: stat d4/d15/d5f/d75 0 2024-02-22T15:50:50.168 INFO:tasks.workunit.client.1.smithi062.stdout:9/859: dread d4/d1c/f6b [0,4194304] 0 2024-02-22T15:50:50.174 INFO:tasks.workunit.client.1.smithi062.stdout:3/713: dread d5/d16/d21/d32/d6a/d7b/fcc [0,4194304] 0 2024-02-22T15:50:50.176 INFO:tasks.workunit.client.1.smithi062.stdout:3/714: truncate d5/d16/f19 3798854 0 2024-02-22T15:50:50.177 INFO:tasks.workunit.client.1.smithi062.stdout:3/715: mkdir d5/d16/d21/d36/d5a/de2 0 2024-02-22T15:50:50.178 INFO:tasks.workunit.client.1.smithi062.stdout:3/716: readlink d5/d1a/d3c/l57 0 2024-02-22T15:50:50.179 INFO:tasks.workunit.client.1.smithi062.stdout:3/717: readlink d5/d16/l28 0 2024-02-22T15:50:50.179 INFO:tasks.workunit.client.0.smithi019.stdout:3/983: write d9/d22/d27/fb0 [545079,13511] 0 2024-02-22T15:50:50.180 INFO:tasks.workunit.client.1.smithi062.stdout:3/718: write d5/d16/d21/d36/d4a/f4f [4563614,14148] 0 2024-02-22T15:50:50.181 INFO:tasks.workunit.client.1.smithi062.stdout:3/719: chown d5/da6 75674120 1 2024-02-22T15:50:50.181 INFO:tasks.workunit.client.0.smithi019.stdout:3/984: write d9/d26/d2d/d53/f56 [416873,102296] 0 2024-02-22T15:50:50.183 INFO:tasks.workunit.client.1.smithi062.stdout:3/720: mkdir d5/da6/de3 0 2024-02-22T15:50:50.184 INFO:tasks.workunit.client.0.smithi019.stdout:3/985: truncate d9/d26/d3f/f6c 1066262 0 2024-02-22T15:50:50.184 INFO:tasks.workunit.client.1.smithi062.stdout:3/721: creat d5/d16/d21/d32/d75/fe4 x:0 0 0 2024-02-22T15:50:50.192 INFO:tasks.workunit.client.1.smithi062.stdout:3/722: dwrite d5/da6/fde [0,4194304] 0 2024-02-22T15:50:50.197 INFO:tasks.workunit.client.1.smithi062.stdout:3/723: dread d5/d16/d21/f34 [0,4194304] 0 2024-02-22T15:50:50.199 INFO:tasks.workunit.client.1.smithi062.stdout:3/724: truncate d5/d16/d21/d32/d6a/d7b/f5f 1809340 0 2024-02-22T15:50:50.201 INFO:tasks.workunit.client.0.smithi019.stdout:3/986: dread d9/d11/f4d [0,4194304] 0 2024-02-22T15:50:50.202 INFO:tasks.workunit.client.1.smithi062.stdout:3/725: creat d5/d26/fe5 x:0 0 0 2024-02-22T15:50:50.203 INFO:tasks.workunit.client.1.smithi062.stdout:3/726: readlink d5/d1a/l40 0 2024-02-22T15:50:50.204 INFO:tasks.workunit.client.0.smithi019.stdout:3/987: creat d9/d22/d139/f13f x:0 0 0 2024-02-22T15:50:50.204 INFO:tasks.workunit.client.0.smithi019.stdout:3/988: chown d9/d26/d32/f43 8 1 2024-02-22T15:50:50.205 INFO:tasks.workunit.client.1.smithi062.stdout:3/727: mkdir d5/d26/d2d/d3f/de6 0 2024-02-22T15:50:50.207 INFO:tasks.workunit.client.0.smithi019.stdout:3/989: rmdir d9/d22/d58/d130 0 2024-02-22T15:50:50.208 INFO:tasks.workunit.client.1.smithi062.stdout:8/936: sync 2024-02-22T15:50:50.212 INFO:tasks.workunit.client.1.smithi062.stdout:8/937: readlink d8/d42/l52 0 2024-02-22T15:50:50.213 INFO:tasks.workunit.client.1.smithi062.stdout:8/938: chown d8/d9/d44/df8/f100 14793 1 2024-02-22T15:50:50.213 INFO:tasks.workunit.client.0.smithi019.stdout:3/990: dwrite d9/d65/da7/dab/f134 [0,4194304] 0 2024-02-22T15:50:50.215 INFO:tasks.workunit.client.1.smithi062.stdout:8/939: creat d8/d9/d44/d58/f121 x:0 0 0 2024-02-22T15:50:50.215 INFO:tasks.workunit.client.0.smithi019.stdout:3/991: chown d9/d26/d3f/d10b/ld8 0 1 2024-02-22T15:50:50.218 INFO:tasks.workunit.client.1.smithi062.stdout:8/940: symlink d8/d9/d2c/d49/d94/dc1/l122 0 2024-02-22T15:50:50.219 INFO:tasks.workunit.client.0.smithi019.stdout:3/992: mkdir d9/d22/d27/dd0/d140 0 2024-02-22T15:50:50.220 INFO:tasks.workunit.client.1.smithi062.stdout:8/941: write d8/f53 [9339764,53811] 0 2024-02-22T15:50:50.229 INFO:tasks.workunit.client.1.smithi062.stdout:8/942: truncate d8/d26/f41 2232358 0 2024-02-22T15:50:50.234 INFO:tasks.workunit.client.1.smithi062.stdout:8/943: rename d8/d9/d44/df8/d105/f4c to d8/d9/d2c/d49/d94/dc1/de9/dd1/f123 0 2024-02-22T15:50:50.235 INFO:tasks.workunit.client.1.smithi062.stdout:3/728: dread d5/f6 [0,4194304] 0 2024-02-22T15:50:50.246 INFO:tasks.workunit.client.1.smithi062.stdout:8/944: link d8/d9/d2c/d49/l8b d8/d9/d2c/d49/d94/l124 0 2024-02-22T15:50:50.246 INFO:tasks.workunit.client.0.smithi019.stdout:3/993: dread d9/d65/d106/f137 [0,4194304] 0 2024-02-22T15:50:50.246 INFO:tasks.workunit.client.1.smithi062.stdout:3/729: write d5/d16/d21/d32/f70 [4237341,65734] 0 2024-02-22T15:50:50.247 INFO:tasks.workunit.client.0.smithi019.stdout:3/994: chown d9/d14/f95 5 1 2024-02-22T15:50:50.249 INFO:tasks.workunit.client.1.smithi062.stdout:3/730: truncate d5/d26/d2d/fb4 890636 0 2024-02-22T15:50:50.250 INFO:tasks.workunit.client.1.smithi062.stdout:3/731: fsync d5/d16/d21/d32/d6a/d7b/f5f 0 2024-02-22T15:50:50.254 INFO:tasks.workunit.client.0.smithi019.stdout:3/995: dwrite d9/d26/d3f/d10b/f129 [0,4194304] 0 2024-02-22T15:50:50.255 INFO:tasks.workunit.client.1.smithi062.stdout:3/732: truncate d5/fbd 1101652 0 2024-02-22T15:50:50.256 INFO:tasks.workunit.client.0.smithi019.stdout:3/996: creat d9/d65/d106/f141 x:0 0 0 2024-02-22T15:50:50.257 INFO:tasks.workunit.client.0.smithi019.stdout:3/997: chown d9/d26/d2d/d53/da0 1680612 1 2024-02-22T15:50:50.258 INFO:tasks.workunit.client.1.smithi062.stdout:3/733: mkdir d5/d1a/d3b/de7 0 2024-02-22T15:50:50.259 INFO:tasks.workunit.client.0.smithi019.stdout:3/998: symlink d9/d16/l142 0 2024-02-22T15:50:50.260 INFO:tasks.workunit.client.1.smithi062.stdout:3/734: creat d5/da6/de3/fe8 x:0 0 0 2024-02-22T15:50:50.260 INFO:tasks.workunit.client.0.smithi019.stdout:3/999: chown d9/d22/d49/f100 49 1 2024-02-22T15:50:50.261 INFO:tasks.workunit.client.1.smithi062.stdout:3/735: readlink d5/d16/d21/d36/d5a/d85/l94 0 2024-02-22T15:50:50.264 INFO:tasks.workunit.client.0.smithi019.stderr:+ rm -rf -- ./tmp.rgMABlLemQ 2024-02-22T15:50:50.276 INFO:tasks.workunit.client.1.smithi062.stdout:3/736: dread d5/da6/fc9 [0,4194304] 0 2024-02-22T15:50:50.279 INFO:tasks.workunit.client.1.smithi062.stdout:3/737: link d5/da6/fc9 d5/d16/d21/d32/d75/fe9 0 2024-02-22T15:50:50.280 INFO:tasks.workunit.client.1.smithi062.stdout:3/738: fsync d5/d16/d21/d55/fa0 0 2024-02-22T15:50:50.286 INFO:tasks.workunit.client.1.smithi062.stdout:8/945: dread d8/d9/d2c/d49/d94/fb3 [0,4194304] 0 2024-02-22T15:50:50.290 INFO:tasks.workunit.client.1.smithi062.stdout:3/739: dwrite d5/f6f [0,4194304] 0 2024-02-22T15:50:50.295 INFO:tasks.workunit.client.1.smithi062.stdout:8/946: dwrite d8/d9/d44/fb2 [4194304,4194304] 0 2024-02-22T15:50:50.296 INFO:tasks.workunit.client.1.smithi062.stdout:3/740: write d5/d26/d2d/f8f [924454,56147] 0 2024-02-22T15:50:50.301 INFO:tasks.workunit.client.1.smithi062.stdout:8/947: rename d8/d9/d2c/d49/fb8 to d8/d42/f125 0 2024-02-22T15:50:50.302 INFO:tasks.workunit.client.1.smithi062.stdout:3/741: mknod d5/d26/d2d/da2/dab/cea 0 2024-02-22T15:50:50.304 INFO:tasks.workunit.client.1.smithi062.stdout:8/948: creat d8/d9/d44/d58/de0/f126 x:0 0 0 2024-02-22T15:50:50.306 INFO:tasks.workunit.client.1.smithi062.stdout:8/949: truncate d8/d9/f10d 752616 0 2024-02-22T15:50:50.324 INFO:tasks.workunit.client.1.smithi062.stdout:3/742: dread f3 [0,4194304] 0 2024-02-22T15:50:50.328 INFO:tasks.workunit.client.1.smithi062.stdout:3/743: dread d5/d16/d21/d32/d6a/d7b/fcc [0,4194304] 0 2024-02-22T15:50:50.334 INFO:tasks.workunit.client.1.smithi062.stdout:3/744: dread d5/d1a/f1b [8388608,4194304] 0 2024-02-22T15:50:50.336 INFO:tasks.workunit.client.1.smithi062.stdout:8/950: dread d8/d9/fe [0,4194304] 0 2024-02-22T15:50:50.337 INFO:tasks.workunit.client.1.smithi062.stdout:3/745: mkdir d5/deb 0 2024-02-22T15:50:50.339 INFO:tasks.workunit.client.1.smithi062.stdout:3/746: creat d5/da6/fec x:0 0 0 2024-02-22T15:50:50.342 INFO:tasks.workunit.client.1.smithi062.stdout:3/747: chown d5/d26/d2d/da2/dab/lba 1 1 2024-02-22T15:50:50.349 INFO:tasks.workunit.client.1.smithi062.stdout:3/748: truncate d5/d16/d21/d32/f66 1639920 0 2024-02-22T15:50:50.354 INFO:tasks.workunit.client.1.smithi062.stdout:8/951: dwrite d8/d9/d44/df8/d105/d70/fc5 [0,4194304] 0 2024-02-22T15:50:50.358 INFO:tasks.workunit.client.1.smithi062.stdout:8/952: chown d8/d9/d2c/d74/df5/fff 492 1 2024-02-22T15:50:50.360 INFO:tasks.workunit.client.1.smithi062.stdout:3/749: dwrite d5/d26/d2d/d3f/fa7 [0,4194304] 0 2024-02-22T15:50:50.362 INFO:tasks.workunit.client.1.smithi062.stdout:3/750: write d5/d26/d2d/fb4 [1575498,88710] 0 2024-02-22T15:50:50.365 INFO:tasks.workunit.client.1.smithi062.stdout:8/953: stat d8/d9/d44/df8/d105/d4e/lc9 0 2024-02-22T15:50:50.368 INFO:tasks.workunit.client.1.smithi062.stdout:3/751: mkdir d5/d1a/ded 0 2024-02-22T15:50:50.370 INFO:tasks.workunit.client.1.smithi062.stdout:3/752: creat d5/d26/d2d/d3f/fee x:0 0 0 2024-02-22T15:50:50.372 INFO:tasks.workunit.client.1.smithi062.stdout:8/954: rename d8/c14 to d8/d26/db5/c127 0 2024-02-22T15:50:50.373 INFO:tasks.workunit.client.1.smithi062.stdout:8/955: truncate d8/d9/d48/fa4 19991 0 2024-02-22T15:50:50.374 INFO:tasks.workunit.client.1.smithi062.stdout:8/956: fdatasync d8/f22 0 2024-02-22T15:50:50.375 INFO:tasks.workunit.client.1.smithi062.stdout:3/753: rename d5/d16/d56/cad to d5/d16/d21/d36/d5a/de2/cef 0 2024-02-22T15:50:50.377 INFO:tasks.workunit.client.1.smithi062.stdout:3/754: write d5/d16/d21/d32/f8b [765872,95522] 0 2024-02-22T15:50:50.380 INFO:tasks.workunit.client.1.smithi062.stdout:3/755: unlink d5/d1a/f1e 0 2024-02-22T15:50:50.387 INFO:tasks.workunit.client.1.smithi062.stdout:3/756: dwrite d5/d26/d2d/da2/dab/faf [0,4194304] 0 2024-02-22T15:50:50.389 INFO:tasks.workunit.client.1.smithi062.stdout:3/757: read - d5/da6/de3/fe8 zero size 2024-02-22T15:50:50.394 INFO:tasks.workunit.client.1.smithi062.stdout:8/957: dwrite d8/d9/d44/df8/d105/f35 [0,4194304] 0 2024-02-22T15:50:50.396 INFO:tasks.workunit.client.1.smithi062.stdout:6/804: sync 2024-02-22T15:50:50.396 INFO:tasks.workunit.client.1.smithi062.stdout:3/758: fdatasync d5/d16/d21/d55/fb9 0 2024-02-22T15:50:50.399 INFO:tasks.workunit.client.1.smithi062.stdout:3/759: write d5/d1a/d3c/d62/fb7 [2935182,114868] 0 2024-02-22T15:50:50.399 INFO:tasks.workunit.client.1.smithi062.stdout:8/958: unlink d8/d9/d44/df8/d105/d4e/fcf 0 2024-02-22T15:50:50.400 INFO:tasks.workunit.client.1.smithi062.stdout:6/805: chown d5/df/d32/d8f/leb 503541 1 2024-02-22T15:50:50.401 INFO:tasks.workunit.client.1.smithi062.stdout:8/959: readlink d8/d9/d2c/d49/d94/lce 0 2024-02-22T15:50:50.402 INFO:tasks.workunit.client.1.smithi062.stdout:6/806: read d5/dd/f2c [197112,25081] 0 2024-02-22T15:50:50.403 INFO:tasks.workunit.client.1.smithi062.stdout:3/760: creat d5/d16/d56/dbc/ff0 x:0 0 0 2024-02-22T15:50:50.406 INFO:tasks.workunit.client.1.smithi062.stdout:8/960: rename d8/d86/ffa to d8/d9/d44/d58/f128 0 2024-02-22T15:50:50.408 INFO:tasks.workunit.client.1.smithi062.stdout:8/961: rmdir d8/d9/d44/d4d/d51/d55 39 2024-02-22T15:50:50.409 INFO:tasks.workunit.client.1.smithi062.stdout:3/761: unlink d5/d16/d21/d32/d75/cae 0 2024-02-22T15:50:50.409 INFO:tasks.workunit.client.1.smithi062.stdout:8/962: write d8/d9/d44/df8/d105/d4e/faa [3585676,90896] 0 2024-02-22T15:50:50.411 INFO:tasks.workunit.client.1.smithi062.stdout:3/762: write d5/d1a/fa9 [740970,44678] 0 2024-02-22T15:50:50.411 INFO:tasks.workunit.client.1.smithi062.stdout:6/807: getdents d5/df/dbc/dce 0 2024-02-22T15:50:50.415 INFO:tasks.workunit.client.1.smithi062.stdout:8/963: creat d8/d9/d44/df1/d118/f129 x:0 0 0 2024-02-22T15:50:50.418 INFO:tasks.workunit.client.1.smithi062.stdout:6/808: creat d5/dd/d33/f10c x:0 0 0 2024-02-22T15:50:50.422 INFO:tasks.workunit.client.1.smithi062.stdout:8/964: creat d8/d42/d87/d108/f12a x:0 0 0 2024-02-22T15:50:50.423 INFO:tasks.workunit.client.1.smithi062.stdout:6/809: rmdir d5/df/d40/d44/d76/d56 39 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:1/723: sync 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:0/924: sync 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:5/720: sync 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:9/860: sync 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:2/962: sync 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:7/625: sync 2024-02-22T15:50:50.582 INFO:tasks.workunit.client.1.smithi062.stdout:4/921: sync 2024-02-22T15:50:50.584 INFO:tasks.workunit.client.1.smithi062.stdout:9/861: symlink d4/d35/l116 0 2024-02-22T15:50:50.586 INFO:tasks.workunit.client.1.smithi062.stdout:0/925: fsync d1/d13/dde/d76/d8c/d5d/dc3/f47 0 2024-02-22T15:50:50.586 INFO:tasks.workunit.client.1.smithi062.stdout:2/963: dread - d1/d1e/d38/d55/fc9 zero size 2024-02-22T15:50:50.587 INFO:tasks.workunit.client.1.smithi062.stdout:1/724: mkdir d1/d6/d7/d2b/d49/d74/df5 0 2024-02-22T15:50:50.588 INFO:tasks.workunit.client.1.smithi062.stdout:4/922: dread d4/d1c/d30/d37/f123 [0,4194304] 0 2024-02-22T15:50:50.589 INFO:tasks.workunit.client.1.smithi062.stdout:2/964: fdatasync d1/d1e/dda/de8/f10b 0 2024-02-22T15:50:50.591 INFO:tasks.workunit.client.1.smithi062.stdout:2/965: write d1/d47/d39/d5d/f102 [809418,70104] 0 2024-02-22T15:50:50.592 INFO:tasks.workunit.client.1.smithi062.stdout:9/862: unlink d4/d15/d31/l61 0 2024-02-22T15:50:50.592 INFO:tasks.workunit.client.1.smithi062.stdout:2/966: fsync d1/d28/fc2 0 2024-02-22T15:50:50.594 INFO:tasks.workunit.client.1.smithi062.stdout:5/721: creat d0/d3/da/d13/d23/d4e/feb x:0 0 0 2024-02-22T15:50:50.595 INFO:tasks.workunit.client.1.smithi062.stdout:1/725: dread - d1/d6/d7/d2b/d49/fb1 zero size 2024-02-22T15:50:50.595 INFO:tasks.workunit.client.1.smithi062.stdout:2/967: chown d1/d8/df/l45 15 1 2024-02-22T15:50:50.596 INFO:tasks.workunit.client.1.smithi062.stdout:0/926: dwrite d1/d3/d4/d1e/fd8 [0,4194304] 0 2024-02-22T15:50:50.598 INFO:tasks.workunit.client.1.smithi062.stdout:2/968: write d1/d47/f107 [565345,33705] 0 2024-02-22T15:50:50.602 INFO:tasks.workunit.client.1.smithi062.stdout:0/927: creat d1/d38/f134 x:0 0 0 2024-02-22T15:50:50.603 INFO:tasks.workunit.client.1.smithi062.stdout:9/863: symlink d4/d15/d5f/d75/l117 0 2024-02-22T15:50:50.604 INFO:tasks.workunit.client.1.smithi062.stdout:9/864: write d4/d15/d5f/d6a/f112 [755165,36983] 0 2024-02-22T15:50:50.605 INFO:tasks.workunit.client.1.smithi062.stdout:0/928: chown d1/d13/dde/d76/d8c/d2c/d84/c10c 973166 1 2024-02-22T15:50:50.610 INFO:tasks.workunit.client.1.smithi062.stdout:2/969: chown d1/d1e/d38/d76/df5/l141 3 1 2024-02-22T15:50:50.612 INFO:tasks.workunit.client.1.smithi062.stdout:9/865: rmdir d4/d15/d31/d106 0 2024-02-22T15:50:50.614 INFO:tasks.workunit.client.1.smithi062.stdout:0/929: dwrite d1/d3/dd5/f127 [0,4194304] 0 2024-02-22T15:50:50.616 INFO:tasks.workunit.client.1.smithi062.stdout:9/866: mkdir d4/d1c/d118 0 2024-02-22T15:50:50.617 INFO:tasks.workunit.client.1.smithi062.stdout:9/867: chown d4/d15/d31/fa5 422902467 1 2024-02-22T15:50:50.619 INFO:tasks.workunit.client.1.smithi062.stdout:0/930: rename d1/d3/d4/d1e/fd8 to d1/d13/dde/d76/d8c/d2c/d11c/f135 0 2024-02-22T15:50:50.620 INFO:tasks.workunit.client.1.smithi062.stdout:7/626: truncate d0/d14/f29 2633395 0 2024-02-22T15:50:50.621 INFO:tasks.workunit.client.1.smithi062.stdout:0/931: read - d1/d13/dde/d76/d8c/d5d/f123 zero size 2024-02-22T15:50:50.623 INFO:tasks.workunit.client.1.smithi062.stdout:0/932: write d1/d3/d4/d1e/fb2 [4072633,91238] 0 2024-02-22T15:50:50.628 INFO:tasks.workunit.client.1.smithi062.stdout:1/726: truncate d1/d6/d7/d2b/fae 2430224 0 2024-02-22T15:50:50.629 INFO:tasks.workunit.client.1.smithi062.stdout:9/868: link d4/d1c/d83/d87/fea d4/d35/f119 0 2024-02-22T15:50:50.633 INFO:tasks.workunit.client.1.smithi062.stdout:5/722: dwrite d0/d3/da/d13/d23/f24 [0,4194304] 0 2024-02-22T15:50:50.634 INFO:tasks.workunit.client.1.smithi062.stdout:1/727: creat d1/d6/d7/d2b/d47/d7e/ff6 x:0 0 0 2024-02-22T15:50:50.635 INFO:tasks.workunit.client.1.smithi062.stdout:5/723: dread - d0/d3/da/d13/d57/fbf zero size 2024-02-22T15:50:50.638 INFO:tasks.workunit.client.1.smithi062.stdout:7/627: write d0/d14/f20 [3737486,25034] 0 2024-02-22T15:50:50.644 INFO:tasks.workunit.client.1.smithi062.stdout:1/728: link d1/d6/d7/d2b/d47/d7e/ff6 d1/ff7 0 2024-02-22T15:50:50.645 INFO:tasks.workunit.client.1.smithi062.stdout:7/628: mkdir d0/d6/d2e/d58/d5d/dc4 0 2024-02-22T15:50:50.646 INFO:tasks.workunit.client.1.smithi062.stdout:9/869: dwrite d4/d15/d5f/d75/d84/f5b [0,4194304] 0 2024-02-22T15:50:50.649 INFO:tasks.workunit.client.1.smithi062.stdout:8/965: sync 2024-02-22T15:50:50.649 INFO:tasks.workunit.client.1.smithi062.stdout:3/763: sync 2024-02-22T15:50:50.649 INFO:tasks.workunit.client.1.smithi062.stdout:6/810: sync 2024-02-22T15:50:50.650 INFO:tasks.workunit.client.1.smithi062.stdout:3/764: chown d5/d16/c8d 109191 1 2024-02-22T15:50:50.653 INFO:tasks.workunit.client.1.smithi062.stdout:7/629: read d0/d6/d2e/d35/f94 [114521,89010] 0 2024-02-22T15:50:50.654 INFO:tasks.workunit.client.1.smithi062.stdout:9/870: dread - d4/d28/d50/d5a/f9c zero size 2024-02-22T15:50:50.656 INFO:tasks.workunit.client.1.smithi062.stdout:9/871: chown d4/d15/d5e/d63/da0/d92/db6/fc3 13446421 1 2024-02-22T15:50:50.657 INFO:tasks.workunit.client.1.smithi062.stdout:5/724: link d0/d3/da/d13/d23/d78/d81/fb4 d0/d3/da/d13/d23/fec 0 2024-02-22T15:50:50.660 INFO:tasks.workunit.client.1.smithi062.stdout:7/630: mknod d0/d6/d2e/d55/d80/cc5 0 2024-02-22T15:50:50.661 INFO:tasks.workunit.client.1.smithi062.stdout:1/729: rmdir d1/d6/d7/d8a/dec 0 2024-02-22T15:50:50.663 INFO:tasks.workunit.client.1.smithi062.stdout:9/872: creat d4/d28/d50/d5a/f11a x:0 0 0 2024-02-22T15:50:50.665 INFO:tasks.workunit.client.1.smithi062.stdout:1/730: read d1/d6/f61 [1639098,7038] 0 2024-02-22T15:50:50.666 INFO:tasks.workunit.client.1.smithi062.stdout:6/811: link d5/df/d32/d47/f103 d5/dd/d33/d66/db9/f10d 0 2024-02-22T15:50:50.667 INFO:tasks.workunit.client.1.smithi062.stdout:5/725: dwrite d0/d3/da/f18 [4194304,4194304] 0 2024-02-22T15:50:50.671 INFO:tasks.workunit.client.1.smithi062.stdout:9/873: mkdir d4/d15/d5f/d75/d84/d11b 0 2024-02-22T15:50:50.671 INFO:tasks.workunit.client.1.smithi062.stdout:3/765: link d5/d1a/d3c/d62/cc8 d5/d16/d21/d32/d6a/cf1 0 2024-02-22T15:50:50.674 INFO:tasks.workunit.client.1.smithi062.stdout:6/812: fsync d5/df/d40/d44/d4a/d41/f55 0 2024-02-22T15:50:50.676 INFO:tasks.workunit.client.1.smithi062.stdout:7/631: symlink d0/d6/d2e/d55/lc6 0 2024-02-22T15:50:50.676 INFO:tasks.workunit.client.1.smithi062.stdout:5/726: rmdir d0/d3/da/d13/d57/d79 39 2024-02-22T15:50:50.682 INFO:tasks.workunit.client.1.smithi062.stdout:1/731: dwrite d1/d6/d7/d2b/d56/d95/fe3 [0,4194304] 0 2024-02-22T15:50:50.694 INFO:tasks.workunit.client.1.smithi062.stdout:3/766: rename d5/d16/d21/d32/lbe to d5/deb/lf2 0 2024-02-22T15:50:50.694 INFO:tasks.workunit.client.1.smithi062.stdout:9/874: unlink d4/d28/d50/ld7 0 2024-02-22T15:50:50.695 INFO:tasks.workunit.client.1.smithi062.stdout:5/727: fdatasync d0/d3/da/d13/d23/d4e/f5d 0 2024-02-22T15:50:50.695 INFO:tasks.workunit.client.1.smithi062.stdout:8/966: dwrite d8/d9/d44/df8/d105/d70/f97 [0,4194304] 0 2024-02-22T15:50:50.697 INFO:tasks.workunit.client.1.smithi062.stdout:8/967: stat d8/d9/f37 0 2024-02-22T15:50:50.697 INFO:tasks.workunit.client.1.smithi062.stdout:6/813: write f0 [54876,43842] 0 2024-02-22T15:50:50.706 INFO:tasks.workunit.client.1.smithi062.stdout:8/968: dwrite d8/f3a [0,4194304] 0 2024-02-22T15:50:50.725 INFO:tasks.workunit.client.1.smithi062.stdout:7/632: link d0/d14/d13/f48 d0/d91/d7c/fc7 0 2024-02-22T15:50:50.726 INFO:tasks.workunit.client.1.smithi062.stdout:1/732: fsync d1/d6/d7/f91 0 2024-02-22T15:50:50.729 INFO:tasks.workunit.client.1.smithi062.stdout:3/767: rename d5/d16/d21/d32/d6a/f78 to d5/d1a/ded/ff3 0 2024-02-22T15:50:50.729 INFO:tasks.workunit.client.1.smithi062.stdout:6/814: truncate d5/df/d40/d44/f82 1244262 0 2024-02-22T15:50:50.731 INFO:tasks.workunit.client.1.smithi062.stdout:9/875: creat d4/d15/d5f/d75/d84/d110/f11c x:0 0 0 2024-02-22T15:50:50.733 INFO:tasks.workunit.client.1.smithi062.stdout:7/633: mkdir d0/d14/d19/d30/dc8 0 2024-02-22T15:50:50.734 INFO:tasks.workunit.client.1.smithi062.stdout:5/728: rmdir d0/d3/da/d13/d57/d79 39 2024-02-22T15:50:50.735 INFO:tasks.workunit.client.1.smithi062.stdout:3/768: symlink d5/d16/d21/d96/lf4 0 2024-02-22T15:50:50.736 INFO:tasks.workunit.client.1.smithi062.stdout:8/969: symlink d8/d9/d2c/d74/l12b 0 2024-02-22T15:50:50.737 INFO:tasks.workunit.client.1.smithi062.stdout:3/769: write d5/d26/d2d/fa1 [1269533,30390] 0 2024-02-22T15:50:50.739 INFO:tasks.workunit.client.1.smithi062.stdout:7/634: dread d0/d6/d2e/d4a/d5a/d5e/f6c [0,4194304] 0 2024-02-22T15:50:50.743 INFO:tasks.workunit.client.1.smithi062.stdout:7/635: fsync d0/d14/d13/f36 0 2024-02-22T15:50:50.747 INFO:tasks.workunit.client.1.smithi062.stdout:6/815: getdents d5/dd/d33/d66 0 2024-02-22T15:50:50.749 INFO:tasks.workunit.client.1.smithi062.stdout:3/770: dread d5/fbd [0,4194304] 0 2024-02-22T15:50:50.753 INFO:tasks.workunit.client.1.smithi062.stdout:8/970: dread d8/d42/f7c [0,4194304] 0 2024-02-22T15:50:50.754 INFO:tasks.workunit.client.1.smithi062.stdout:6/816: rename d5/df/l67 to d5/df/dbc/dcd/de9/d100/l10e 0 2024-02-22T15:50:50.756 INFO:tasks.workunit.client.1.smithi062.stdout:3/771: mknod d5/d1a/d3b/dc1/cf5 0 2024-02-22T15:50:50.756 INFO:tasks.workunit.client.1.smithi062.stdout:5/729: link d0/d3/da/d13/d23/d4e/d60/l6a d0/d3/d2f/d59/d67/led 0 2024-02-22T15:50:50.758 INFO:tasks.workunit.client.1.smithi062.stdout:5/730: dread - d0/d3/da/d32/d33/dbc/fc4 zero size 2024-02-22T15:50:50.758 INFO:tasks.workunit.client.1.smithi062.stdout:8/971: mknod d8/d9/d44/d4d/d8a/c12c 0 2024-02-22T15:50:50.759 INFO:tasks.workunit.client.1.smithi062.stdout:6/817: mknod d5/dd/d33/d71/c10f 0 2024-02-22T15:50:50.764 INFO:tasks.workunit.client.1.smithi062.stdout:5/731: creat d0/d3/d2f/d59/d67/d69/fee x:0 0 0 2024-02-22T15:50:50.765 INFO:tasks.workunit.client.1.smithi062.stdout:3/772: dwrite d5/d1a/d3b/dc1/fc3 [0,4194304] 0 2024-02-22T15:50:50.769 INFO:tasks.workunit.client.1.smithi062.stdout:8/972: rmdir d8/d9/d44/d4d/d51/d55 39 2024-02-22T15:50:50.771 INFO:tasks.workunit.client.1.smithi062.stdout:6/818: symlink d5/d8a/ddd/de6/df1/l110 0 2024-02-22T15:50:50.775 INFO:tasks.workunit.client.1.smithi062.stdout:3/773: dread d5/d16/d21/d32/d75/fe9 [0,4194304] 0 2024-02-22T15:50:50.780 INFO:tasks.workunit.client.1.smithi062.stdout:3/774: mknod d5/da6/de3/cf6 0 2024-02-22T15:50:50.782 INFO:tasks.workunit.client.1.smithi062.stdout:5/732: rmdir d0/d3/da/d13/d57/d79/dd0/de9 0 2024-02-22T15:50:50.783 INFO:tasks.workunit.client.1.smithi062.stdout:3/775: truncate d5/f8 332729 0 2024-02-22T15:50:50.786 INFO:tasks.workunit.client.1.smithi062.stdout:5/733: rmdir d0/d3/d2f/d59/d67/d69/d3f/d5f 39 2024-02-22T15:50:50.787 INFO:tasks.workunit.client.1.smithi062.stdout:3/776: chown d5/d16/d21/d36/d5a/d85/fb2 5046 1 2024-02-22T15:50:50.789 INFO:tasks.workunit.client.1.smithi062.stdout:7/636: dwrite d0/d14/d19/f1a [0,4194304] 0 2024-02-22T15:50:50.792 INFO:tasks.workunit.client.1.smithi062.stdout:5/734: creat d0/d3/da/d13/d57/dc7/dda/fef x:0 0 0 2024-02-22T15:50:50.796 INFO:tasks.workunit.client.1.smithi062.stdout:5/735: chown d0/d3/da/d32/d4f/c6b 19910859 1 2024-02-22T15:50:50.797 INFO:tasks.workunit.client.1.smithi062.stdout:6/819: write d5/f38 [83199,51801] 0 2024-02-22T15:50:50.801 INFO:tasks.workunit.client.1.smithi062.stdout:5/736: chown d0/d3/da/d13/d23/d4e/d6d/f93 161598547 1 2024-02-22T15:50:50.809 INFO:tasks.workunit.client.1.smithi062.stdout:3/777: dwrite d5/d16/f19 [0,4194304] 0 2024-02-22T15:50:50.809 INFO:tasks.workunit.client.1.smithi062.stdout:7/637: dwrite d0/d6/d2e/d58/f5c [0,4194304] 0 2024-02-22T15:50:50.811 INFO:tasks.workunit.client.1.smithi062.stdout:3/778: fsync d5/d1a/d3c/d62/fb8 0 2024-02-22T15:50:50.811 INFO:tasks.workunit.client.1.smithi062.stdout:5/737: rename d0/d3/d2f/d59/d67/d69/d3f/la6 to d0/d3/da/d32/d4f/lf0 0 2024-02-22T15:50:50.815 INFO:tasks.workunit.client.1.smithi062.stdout:3/779: symlink d5/d16/d21/lf7 0 2024-02-22T15:50:50.815 INFO:tasks.workunit.client.1.smithi062.stdout:5/738: creat d0/d3/d30/d71/d7c/ff1 x:0 0 0 2024-02-22T15:50:50.818 INFO:tasks.workunit.client.1.smithi062.stdout:3/780: write d5/d16/d21/d32/d6a/fc7 [225818,39918] 0 2024-02-22T15:50:50.818 INFO:tasks.workunit.client.1.smithi062.stdout:5/739: dread d0/d3/da/f53 [0,4194304] 0 2024-02-22T15:50:50.820 INFO:tasks.workunit.client.1.smithi062.stdout:7/638: dwrite d0/d14/f67 [4194304,4194304] 0 2024-02-22T15:50:50.822 INFO:tasks.workunit.client.1.smithi062.stdout:3/781: creat d5/d16/d21/d55/ff8 x:0 0 0 2024-02-22T15:50:50.823 INFO:tasks.workunit.client.1.smithi062.stdout:4/923: sync 2024-02-22T15:50:50.825 INFO:tasks.workunit.client.1.smithi062.stdout:5/740: write d0/d3/da/d13/d23/d4e/fa3 [1724472,27032] 0 2024-02-22T15:50:50.826 INFO:tasks.workunit.client.1.smithi062.stdout:7/639: symlink d0/d6/d2e/d58/d5d/dc4/lc9 0 2024-02-22T15:50:50.831 INFO:tasks.workunit.client.1.smithi062.stdout:3/782: truncate d5/d1a/f1b 5464575 0 2024-02-22T15:50:50.831 INFO:tasks.workunit.client.1.smithi062.stdout:4/924: unlink d4/d13/d20/c140 0 2024-02-22T15:50:50.832 INFO:tasks.workunit.client.1.smithi062.stdout:5/741: creat d0/d3/d2f/d59/d67/d69/ff2 x:0 0 0 2024-02-22T15:50:50.834 INFO:tasks.workunit.client.1.smithi062.stdout:3/783: mknod d5/d16/d21/d36/d5a/cf9 0 2024-02-22T15:50:50.834 INFO:tasks.workunit.client.1.smithi062.stdout:5/742: chown d0/d3/da/d32/d4f/d72/f99 36 1 2024-02-22T15:50:50.835 INFO:tasks.workunit.client.1.smithi062.stdout:3/784: chown d5/d16/d97 411217 1 2024-02-22T15:50:50.836 INFO:tasks.workunit.client.1.smithi062.stdout:7/640: write d0/d6/f7d [987588,93789] 0 2024-02-22T15:50:50.838 INFO:tasks.workunit.client.1.smithi062.stdout:7/641: dread - d0/d6/d2e/d55/d72/fb2 zero size 2024-02-22T15:50:50.839 INFO:tasks.workunit.client.1.smithi062.stdout:5/743: unlink d0/d3/da/d13/d23/d4e/lde 0 2024-02-22T15:50:50.839 INFO:tasks.workunit.client.1.smithi062.stdout:4/925: dwrite d4/d34/f48 [0,4194304] 0 2024-02-22T15:50:50.841 INFO:tasks.workunit.client.1.smithi062.stdout:3/785: mknod d5/da6/de3/cfa 0 2024-02-22T15:50:50.842 INFO:tasks.workunit.client.1.smithi062.stdout:4/926: dread - d4/d1c/d2c/d6b/d9b/db1/dc6/f141 zero size 2024-02-22T15:50:50.843 INFO:tasks.workunit.client.1.smithi062.stdout:3/786: readlink l2 0 2024-02-22T15:50:50.848 INFO:tasks.workunit.client.1.smithi062.stdout:5/744: mkdir d0/d3/da/d32/d4f/df3 0 2024-02-22T15:50:50.849 INFO:tasks.workunit.client.1.smithi062.stdout:4/927: mkdir d4/d2a/d2d/d46/d13b/d144 0 2024-02-22T15:50:50.853 INFO:tasks.workunit.client.1.smithi062.stdout:3/787: dread d5/d16/d21/d55/fb9 [0,4194304] 0 2024-02-22T15:50:50.853 INFO:tasks.workunit.client.1.smithi062.stdout:7/642: dwrite d0/d6/d2e/d35/faf [0,4194304] 0 2024-02-22T15:50:50.858 INFO:tasks.workunit.client.1.smithi062.stdout:3/788: mkdir d5/d1a/d3b/dc1/dfb 0 2024-02-22T15:50:50.858 INFO:tasks.workunit.client.1.smithi062.stdout:7/643: mknod d0/d14/d19/d30/dc8/cca 0 2024-02-22T15:50:50.860 INFO:tasks.workunit.client.1.smithi062.stdout:7/644: fsync d0/d6/d2e/d55/d80/d8a/f90 0 2024-02-22T15:50:50.863 INFO:tasks.workunit.client.1.smithi062.stdout:7/645: chown d0/d14/d19/c3f 13107065 1 2024-02-22T15:50:50.864 INFO:tasks.workunit.client.1.smithi062.stdout:3/789: mknod d5/d16/d21/d36/d5a/cfc 0 2024-02-22T15:50:50.865 INFO:tasks.workunit.client.1.smithi062.stdout:4/928: getdents d4/d2a/d2d/d2e/d6a/d9e/da4/db6/dbc/dd7 0 2024-02-22T15:50:50.869 INFO:tasks.workunit.client.1.smithi062.stdout:5/745: rename d0/d3/d2f/d59/d67/d69 to d0/d3/da/d32/d4f/d72/df4 0 2024-02-22T15:50:50.872 INFO:tasks.workunit.client.1.smithi062.stdout:7/646: dwrite d0/d6/d2e/d35/f70 [0,4194304] 0 2024-02-22T15:50:50.873 INFO:tasks.workunit.client.1.smithi062.stdout:4/929: mknod d4/d1c/d2c/d6b/d9b/db1/dc6/c145 0 2024-02-22T15:50:50.881 INFO:tasks.workunit.client.1.smithi062.stdout:7/647: dwrite d0/d6/d2e/d2f/f82 [0,4194304] 0 2024-02-22T15:50:50.883 INFO:tasks.workunit.client.1.smithi062.stdout:7/648: fdatasync d0/d6/d5b/faa 0 2024-02-22T15:50:50.886 INFO:tasks.workunit.client.1.smithi062.stdout:7/649: truncate d0/d6/d2e/fa3 1017184 0 2024-02-22T15:50:50.888 INFO:tasks.workunit.client.1.smithi062.stdout:4/930: rmdir d4/d2a/d2d/d46/d13b/d144 0 2024-02-22T15:50:50.888 INFO:tasks.workunit.client.1.smithi062.stdout:7/650: truncate d0/d6/d2e/d2f/f82 4465542 0 2024-02-22T15:50:50.889 INFO:tasks.workunit.client.1.smithi062.stdout:5/746: getdents d0/d3/d2f/d59/d67 0 2024-02-22T15:50:50.889 INFO:tasks.workunit.client.1.smithi062.stdout:3/790: write d5/d16/d21/f29 [215074,12146] 0 2024-02-22T15:50:50.891 INFO:tasks.workunit.client.1.smithi062.stdout:4/931: mkdir d4/df/d146 0 2024-02-22T15:50:50.895 INFO:tasks.workunit.client.1.smithi062.stdout:5/747: mknod d0/d3/da/d32/d4f/d72/df4/d3f/d66/cf5 0 2024-02-22T15:50:50.895 INFO:tasks.workunit.client.1.smithi062.stdout:7/651: read d0/d14/d19/d30/d37/f63 [2598853,69199] 0 2024-02-22T15:50:50.896 INFO:tasks.workunit.client.1.smithi062.stdout:7/652: write d0/d6/f52 [3828052,20866] 0 2024-02-22T15:50:50.897 INFO:tasks.workunit.client.1.smithi062.stdout:5/748: chown d0/d3/da/d32/d4f/d72/df4/d3f/fe3 75331 1 2024-02-22T15:50:50.904 INFO:tasks.workunit.client.1.smithi062.stdout:3/791: write f3 [4274588,101898] 0 2024-02-22T15:50:50.906 INFO:tasks.workunit.client.1.smithi062.stdout:4/932: truncate d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f129 1115547 0 2024-02-22T15:50:50.910 INFO:tasks.workunit.client.1.smithi062.stdout:7/653: dwrite d0/f89 [0,4194304] 0 2024-02-22T15:50:50.913 INFO:tasks.workunit.client.1.smithi062.stdout:5/749: dwrite d0/d3/da/d13/d23/d4e/d6d/dd2/fe8 [0,4194304] 0 2024-02-22T15:50:50.921 INFO:tasks.workunit.client.1.smithi062.stdout:4/933: write d4/d1c/d2c/d6b/d9b/db1/dc6/fef [3172424,64188] 0 2024-02-22T15:50:50.921 INFO:tasks.workunit.client.1.smithi062.stdout:5/750: mknod d0/d3/d30/d71/d7c/cf6 0 2024-02-22T15:50:50.924 INFO:tasks.workunit.client.1.smithi062.stdout:5/751: creat d0/d3/d2f/d59/ff7 x:0 0 0 2024-02-22T15:50:50.925 INFO:tasks.workunit.client.1.smithi062.stdout:4/934: creat d4/d2a/d2d/d2e/d6a/d9e/da4/f147 x:0 0 0 2024-02-22T15:50:50.926 INFO:tasks.workunit.client.1.smithi062.stdout:4/935: dread - d4/d2a/f12b zero size 2024-02-22T15:50:50.927 INFO:tasks.workunit.client.1.smithi062.stdout:4/936: write d4/d2a/f11d [398519,3291] 0 2024-02-22T15:50:50.928 INFO:tasks.workunit.client.1.smithi062.stdout:3/792: dread d5/d16/d21/d32/f66 [0,4194304] 0 2024-02-22T15:50:50.929 INFO:tasks.workunit.client.1.smithi062.stdout:3/793: readlink d5/d16/d21/ld6 0 2024-02-22T15:50:50.933 INFO:tasks.workunit.client.1.smithi062.stdout:4/937: creat d4/d1c/d2c/d6b/d9b/db1/f148 x:0 0 0 2024-02-22T15:50:50.933 INFO:tasks.workunit.client.1.smithi062.stdout:3/794: symlink d5/d16/d56/lfd 0 2024-02-22T15:50:50.934 INFO:tasks.workunit.client.1.smithi062.stdout:5/752: dwrite d0/d3/da/d13/d23/d4e/d5a/fcf [0,4194304] 0 2024-02-22T15:50:50.934 INFO:tasks.workunit.client.1.smithi062.stdout:3/795: fdatasync d5/d26/f6c 0 2024-02-22T15:50:50.936 INFO:tasks.workunit.client.1.smithi062.stdout:5/753: chown d0/d3/da/l20 1071641 1 2024-02-22T15:50:50.937 INFO:tasks.workunit.client.1.smithi062.stdout:3/796: write d5/d16/d21/d32/d6a/d7b/f5f [231675,36716] 0 2024-02-22T15:50:50.939 INFO:tasks.workunit.client.1.smithi062.stdout:5/754: chown d0/d3/da/d13/d23/d78/lb2 61306860 1 2024-02-22T15:50:50.939 INFO:tasks.workunit.client.1.smithi062.stdout:4/938: read d4/d13/d20/fc2 [565002,99374] 0 2024-02-22T15:50:50.939 INFO:tasks.workunit.client.1.smithi062.stdout:3/797: truncate d5/da6/de3/fe8 413140 0 2024-02-22T15:50:50.942 INFO:tasks.workunit.client.1.smithi062.stdout:4/939: unlink d4/d1c/d2c/d6b/fe3 0 2024-02-22T15:50:50.944 INFO:tasks.workunit.client.1.smithi062.stdout:4/940: dread - d4/d1c/d30/da7/dcb/fd4 zero size 2024-02-22T15:50:50.944 INFO:tasks.workunit.client.1.smithi062.stdout:3/798: mkdir d5/d16/d21/d96/dfe 0 2024-02-22T15:50:50.945 INFO:tasks.workunit.client.1.smithi062.stdout:5/755: creat d0/d3/da/d32/d4f/d72/df4/d3f/d5f/ff8 x:0 0 0 2024-02-22T15:50:50.947 INFO:tasks.workunit.client.1.smithi062.stdout:5/756: chown d0/d3/d30/d71/fcb 1741 1 2024-02-22T15:50:50.947 INFO:tasks.workunit.client.1.smithi062.stdout:4/941: write d4/d1c/d30/d3b/d49/f95 [1591799,99284] 0 2024-02-22T15:50:50.948 INFO:tasks.workunit.client.1.smithi062.stdout:3/799: mkdir d5/d26/d2d/d3f/ddb/dff 0 2024-02-22T15:50:50.950 INFO:tasks.workunit.client.1.smithi062.stdout:3/800: truncate d5/d16/d21/f22 746484 0 2024-02-22T15:50:50.950 INFO:tasks.workunit.client.1.smithi062.stdout:5/757: mkdir d0/d3/da/df9 0 2024-02-22T15:50:50.951 INFO:tasks.workunit.client.1.smithi062.stdout:5/758: readlink d0/d3/da/l20 0 2024-02-22T15:50:50.953 INFO:tasks.workunit.client.1.smithi062.stdout:4/942: getdents d4/d2a/d7d 0 2024-02-22T15:50:50.953 INFO:tasks.workunit.client.1.smithi062.stdout:5/759: stat d0/d3/da/d13/d23/d4e/d5a/dd6 0 2024-02-22T15:50:50.955 INFO:tasks.workunit.client.1.smithi062.stdout:4/943: mkdir d4/d1c/d30/d149 0 2024-02-22T15:50:50.957 INFO:tasks.workunit.client.1.smithi062.stdout:5/760: symlink d0/d3/da/d32/d4f/d72/dca/lfa 0 2024-02-22T15:50:50.958 INFO:tasks.workunit.client.1.smithi062.stdout:3/801: dread d5/d16/d21/f29 [0,4194304] 0 2024-02-22T15:50:50.959 INFO:tasks.workunit.client.1.smithi062.stdout:5/761: truncate d0/d3/d30/f4a 1883401 0 2024-02-22T15:50:50.967 INFO:tasks.workunit.client.1.smithi062.stdout:3/802: dwrite d5/d26/d2d/fa1 [4194304,4194304] 0 2024-02-22T15:50:50.969 INFO:tasks.workunit.client.1.smithi062.stdout:5/762: dread d0/d3/da/f18 [0,4194304] 0 2024-02-22T15:50:50.969 INFO:tasks.workunit.client.1.smithi062.stdout:3/803: chown d5/d16/d56/dbc/dcb 32 1 2024-02-22T15:50:50.974 INFO:tasks.workunit.client.1.smithi062.stdout:4/944: dread d4/d1c/d30/d3b/d49/f95 [0,4194304] 0 2024-02-22T15:50:50.976 INFO:tasks.workunit.client.1.smithi062.stdout:5/763: mknod d0/d3/da/d13/d57/dc7/dda/cfb 0 2024-02-22T15:50:50.977 INFO:tasks.workunit.client.1.smithi062.stdout:3/804: rename d5/d26/d2d/d3f/fa7 to d5/da6/de3/f100 0 2024-02-22T15:50:50.978 INFO:tasks.workunit.client.1.smithi062.stdout:4/945: readlink d4/df/l29 0 2024-02-22T15:50:50.980 INFO:tasks.workunit.client.1.smithi062.stdout:5/764: chown d0/d3/da/d32/d4f/d72/df4/d3f/d66/da9/fe5 1544 1 2024-02-22T15:50:50.981 INFO:tasks.workunit.client.1.smithi062.stdout:3/805: mknod d5/d16/d21/d32/d9e/dc2/c101 0 2024-02-22T15:50:50.982 INFO:tasks.workunit.client.1.smithi062.stdout:5/765: chown d0/d3/da/d32/d4f/d72/df4/d3f/d66/fce 111 1 2024-02-22T15:50:50.982 INFO:tasks.workunit.client.1.smithi062.stdout:4/946: dread d4/d2a/fb8 [0,4194304] 0 2024-02-22T15:50:50.986 INFO:tasks.workunit.client.1.smithi062.stdout:3/806: symlink d5/d16/d21/d32/d6a/l102 0 2024-02-22T15:50:50.987 INFO:tasks.workunit.client.1.smithi062.stdout:3/807: write d5/d16/d21/d32/d75/fda [808859,10707] 0 2024-02-22T15:50:50.989 INFO:tasks.workunit.client.1.smithi062.stdout:3/808: chown d5/d16/d21 0 1 2024-02-22T15:50:50.990 INFO:tasks.workunit.client.1.smithi062.stdout:3/809: chown d5/d26/d2d/da2/dab/faf 989419 1 2024-02-22T15:50:50.991 INFO:tasks.workunit.client.1.smithi062.stdout:3/810: write d5/d26/fe5 [140994,53688] 0 2024-02-22T15:50:50.992 INFO:tasks.workunit.client.1.smithi062.stdout:4/947: truncate d4/d2a/d2d/d2e/d6a/d94/dfe/f12e 2574604 0 2024-02-22T15:50:50.993 INFO:tasks.workunit.client.1.smithi062.stdout:4/948: truncate d4/d2a/f11d 734053 0 2024-02-22T15:50:50.994 INFO:tasks.workunit.client.1.smithi062.stdout:3/811: creat d5/d16/d21/d32/d9e/dc2/f103 x:0 0 0 2024-02-22T15:50:50.996 INFO:tasks.workunit.client.1.smithi062.stdout:4/949: mkdir d4/d34/d14a 0 2024-02-22T15:50:51.002 INFO:tasks.workunit.client.1.smithi062.stdout:5/766: dread d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:51.004 INFO:tasks.workunit.client.1.smithi062.stdout:5/767: mkdir d0/d3/da/d13/d57/d79/dd0/dfc 0 2024-02-22T15:50:51.006 INFO:tasks.workunit.client.1.smithi062.stdout:5/768: mknod d0/d3/d2f/d59/d67/cfd 0 2024-02-22T15:50:51.007 INFO:tasks.workunit.client.1.smithi062.stdout:5/769: chown d0/d3/da/fe2 13343 1 2024-02-22T15:50:51.008 INFO:tasks.workunit.client.1.smithi062.stdout:5/770: creat d0/d3/d30/d71/ffe x:0 0 0 2024-02-22T15:50:51.009 INFO:tasks.workunit.client.1.smithi062.stdout:3/812: dread d5/d26/f35 [4194304,4194304] 0 2024-02-22T15:50:51.010 INFO:tasks.workunit.client.1.smithi062.stdout:5/771: mkdir d0/d3/da/d32/d33/dff 0 2024-02-22T15:50:51.011 INFO:tasks.workunit.client.1.smithi062.stdout:3/813: creat d5/da6/de3/f104 x:0 0 0 2024-02-22T15:50:51.011 INFO:tasks.workunit.client.1.smithi062.stdout:5/772: fdatasync d0/f9 0 2024-02-22T15:50:51.013 INFO:tasks.workunit.client.1.smithi062.stdout:3/814: creat d5/d16/d21/f105 x:0 0 0 2024-02-22T15:50:51.014 INFO:tasks.workunit.client.1.smithi062.stdout:3/815: mknod d5/d16/d21/d36/c106 0 2024-02-22T15:50:51.016 INFO:tasks.workunit.client.1.smithi062.stdout:3/816: mkdir d5/d26/d2d/da2/dab/d107 0 2024-02-22T15:50:51.017 INFO:tasks.workunit.client.1.smithi062.stdout:3/817: chown d5/d16/d21/d32/d6a/fac 21 1 2024-02-22T15:50:51.017 INFO:tasks.workunit.client.1.smithi062.stdout:3/818: truncate d5/d16/d21/f105 24733 0 2024-02-22T15:50:51.020 INFO:tasks.workunit.client.1.smithi062.stdout:3/819: rename d5/d16/d21/d55/fb9 to d5/d26/d2d/d3f/de6/f108 0 2024-02-22T15:50:51.024 INFO:tasks.workunit.client.1.smithi062.stdout:3/820: dread d5/d1a/fa9 [0,4194304] 0 2024-02-22T15:50:51.026 INFO:tasks.workunit.client.1.smithi062.stdout:3/821: unlink d5/d1a/d3b/f71 0 2024-02-22T15:50:51.029 INFO:tasks.workunit.client.1.smithi062.stdout:3/822: rename d5/d1a/d3c/f53 to d5/d16/d21/f109 0 2024-02-22T15:50:51.031 INFO:tasks.workunit.client.1.smithi062.stdout:3/823: mknod d5/d16/c10a 0 2024-02-22T15:50:51.032 INFO:tasks.workunit.client.1.smithi062.stdout:3/824: dread d5/d16/d21/f105 [0,4194304] 0 2024-02-22T15:50:51.033 INFO:tasks.workunit.client.1.smithi062.stdout:3/825: chown d5/da6/de3/fe8 441 1 2024-02-22T15:50:51.036 INFO:tasks.workunit.client.1.smithi062.stdout:3/826: dread d5/d26/d2d/f8f [0,4194304] 0 2024-02-22T15:50:51.038 INFO:tasks.workunit.client.1.smithi062.stdout:3/827: creat d5/d1a/f10b x:0 0 0 2024-02-22T15:50:51.038 INFO:tasks.workunit.client.1.smithi062.stdout:3/828: chown d5/d1a/ded 31553 1 2024-02-22T15:50:51.041 INFO:tasks.workunit.client.1.smithi062.stdout:3/829: rename d5/d16/d21/f79 to d5/d26/d2d/da2/dab/d107/f10c 0 2024-02-22T15:50:51.043 INFO:tasks.workunit.client.1.smithi062.stdout:3/830: unlink d5/d16/d21/d32/d9e/dc2/c101 0 2024-02-22T15:50:51.061 INFO:tasks.workunit.client.1.smithi062.stdout:3/831: dwrite d5/d1a/d3c/f54 [0,4194304] 0 2024-02-22T15:50:51.065 INFO:tasks.workunit.client.1.smithi062.stdout:3/832: mkdir d5/d16/d21/d96/dfe/d10d 0 2024-02-22T15:50:51.066 INFO:tasks.workunit.client.1.smithi062.stdout:3/833: mkdir d5/d1a/ded/d10e 0 2024-02-22T15:50:51.067 INFO:tasks.workunit.client.1.smithi062.stdout:3/834: chown d5/d16/d21/d36/d5a/cf9 40865086 1 2024-02-22T15:50:51.068 INFO:tasks.workunit.client.1.smithi062.stdout:3/835: unlink d5/d1a/c58 0 2024-02-22T15:50:51.071 INFO:tasks.workunit.client.1.smithi062.stdout:3/836: dread d5/d16/d21/d32/d6a/fc7 [0,4194304] 0 2024-02-22T15:50:51.074 INFO:tasks.workunit.client.1.smithi062.stdout:3/837: unlink d5/d16/d21/d32/d75/fda 0 2024-02-22T15:50:51.075 INFO:tasks.workunit.client.1.smithi062.stdout:3/838: stat d5/d1a/d3b/l52 0 2024-02-22T15:50:51.090 INFO:tasks.workunit.client.1.smithi062.stdout:3/839: dread d5/d16/d21/d36/d5a/d85/fc6 [0,4194304] 0 2024-02-22T15:50:51.092 INFO:tasks.workunit.client.1.smithi062.stdout:3/840: symlink d5/d1a/d3c/d62/l10f 0 2024-02-22T15:50:51.093 INFO:tasks.workunit.client.1.smithi062.stdout:3/841: chown d5/c3e 45 1 2024-02-22T15:50:51.094 INFO:tasks.workunit.client.1.smithi062.stdout:3/842: write d5/d16/d21/d55/ff8 [791910,26909] 0 2024-02-22T15:50:51.101 INFO:tasks.workunit.client.1.smithi062.stdout:3/843: dwrite d5/d26/d2d/da2/dab/faf [0,4194304] 0 2024-02-22T15:50:51.106 INFO:tasks.workunit.client.1.smithi062.stdout:3/844: rename d5/f10 to d5/d16/d56/dbc/f110 0 2024-02-22T15:50:51.108 INFO:tasks.workunit.client.1.smithi062.stdout:3/845: mknod d5/d1a/d3b/dc1/c111 0 2024-02-22T15:50:51.108 INFO:tasks.workunit.client.1.smithi062.stdout:3/846: chown d5/da6/fec 1159 1 2024-02-22T15:50:51.110 INFO:tasks.workunit.client.1.smithi062.stdout:3/847: mknod d5/d26/d2d/c112 0 2024-02-22T15:50:51.182 INFO:tasks.workunit.client.1.smithi062.stdout:2/970: sync 2024-02-22T15:50:51.182 INFO:tasks.workunit.client.1.smithi062.stdout:0/933: sync 2024-02-22T15:50:51.186 INFO:tasks.workunit.client.1.smithi062.stdout:2/971: creat d1/d1e/d38/d76/dc5/dec/d116/f145 x:0 0 0 2024-02-22T15:50:51.188 INFO:tasks.workunit.client.1.smithi062.stdout:0/934: dwrite d1/d13/f65 [0,4194304] 0 2024-02-22T15:50:51.188 INFO:tasks.workunit.client.1.smithi062.stdout:2/972: chown d1/d1e/d38/leb 3239158 1 2024-02-22T15:50:51.194 INFO:tasks.workunit.client.1.smithi062.stdout:0/935: rename d1/d13/dde/d76/d8c/d5d/dc1/ld3 to d1/d3/d4/d1e/d74/l136 0 2024-02-22T15:50:51.196 INFO:tasks.workunit.client.1.smithi062.stdout:0/936: stat d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/df4/lff 0 2024-02-22T15:50:51.197 INFO:tasks.workunit.client.1.smithi062.stdout:2/973: rename d1/d8/d62/d73/dbd/d7d/ldb to d1/d47/d32/d104/l146 0 2024-02-22T15:50:51.198 INFO:tasks.workunit.client.1.smithi062.stdout:2/974: readlink d1/d8/d62/d73/dbd/dbb/l108 0 2024-02-22T15:50:51.198 INFO:tasks.workunit.client.1.smithi062.stdout:0/937: rmdir d1/d3/d4/d1e/d74/d122 39 2024-02-22T15:50:51.216 INFO:tasks.workunit.client.1.smithi062.stdout:2/975: dread d1/d47/f2b [0,4194304] 0 2024-02-22T15:50:51.219 INFO:tasks.workunit.client.1.smithi062.stdout:2/976: mknod d1/c147 0 2024-02-22T15:50:51.220 INFO:tasks.workunit.client.1.smithi062.stdout:0/938: dread d1/d3/de/f3a [0,4194304] 0 2024-02-22T15:50:51.224 INFO:tasks.workunit.client.1.smithi062.stdout:0/939: creat d1/dd/f137 x:0 0 0 2024-02-22T15:50:51.225 INFO:tasks.workunit.client.1.smithi062.stdout:2/977: dwrite d1/d47/d32/d52/d80/fb0 [0,4194304] 0 2024-02-22T15:50:51.227 INFO:tasks.workunit.client.1.smithi062.stdout:0/940: chown d1/d13/dde/d1d/c77 729 1 2024-02-22T15:50:51.227 INFO:tasks.workunit.client.1.smithi062.stdout:2/978: chown d1/d1e/d38/d76/dc5/df3/d10f 1321134 1 2024-02-22T15:50:51.231 INFO:tasks.workunit.client.1.smithi062.stdout:2/979: creat d1/d1e/d38/d11a/f148 x:0 0 0 2024-02-22T15:50:51.231 INFO:tasks.workunit.client.1.smithi062.stdout:0/941: dread d1/dd/f6c [0,4194304] 0 2024-02-22T15:50:51.233 INFO:tasks.workunit.client.1.smithi062.stdout:2/980: write d1/d8/d62/d73/dbd/d7d/d84/fcc [699512,57437] 0 2024-02-22T15:50:51.233 INFO:tasks.workunit.client.1.smithi062.stdout:0/942: unlink d1/d3/d5f/f118 0 2024-02-22T15:50:51.239 INFO:tasks.workunit.client.1.smithi062.stdout:0/943: rename d1/d13/dde/d76/d8c/d5d/c2e to d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/c138 0 2024-02-22T15:50:51.241 INFO:tasks.workunit.client.1.smithi062.stdout:0/944: creat d1/d13/dde/d76/da5/f139 x:0 0 0 2024-02-22T15:50:51.243 INFO:tasks.workunit.client.1.smithi062.stdout:0/945: mknod d1/c13a 0 2024-02-22T15:50:51.248 INFO:tasks.workunit.client.1.smithi062.stdout:0/946: rename d1/d13/dde/d1d to d1/d13/dde/d76/d8c/d2c/db5/d13b 0 2024-02-22T15:50:51.257 INFO:tasks.workunit.client.1.smithi062.stdout:2/981: dwrite d1/d8/d62/d73/dbd/dbb/fee [0,4194304] 0 2024-02-22T15:50:51.335 INFO:tasks.workunit.client.1.smithi062.stdout:2/982: creat d1/d8/d62/d73/dbd/d7d/f149 x:0 0 0 2024-02-22T15:50:51.335 INFO:tasks.workunit.client.1.smithi062.stdout:0/947: write d1/d3/de/ff [4309222,64428] 0 2024-02-22T15:50:51.338 INFO:tasks.workunit.client.1.smithi062.stdout:0/948: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d83/d13c 0 2024-02-22T15:50:51.340 INFO:tasks.workunit.client.1.smithi062.stdout:2/983: getdents d1/d1e/d38/d76/dc5/dec/d116 0 2024-02-22T15:50:51.343 INFO:tasks.workunit.client.1.smithi062.stdout:0/949: rename d1/d13/dde/l27 to d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/d97/df4/l13d 0 2024-02-22T15:50:51.344 INFO:tasks.workunit.client.1.smithi062.stdout:2/984: mknod d1/d8/d62/dcb/c14a 0 2024-02-22T15:50:51.345 INFO:tasks.workunit.client.1.smithi062.stdout:0/950: symlink d1/d13/dde/d76/d8c/d92/l13e 0 2024-02-22T15:50:51.346 INFO:tasks.workunit.client.1.smithi062.stdout:2/985: creat d1/d47/d32/d52/d80/f14b x:0 0 0 2024-02-22T15:50:51.348 INFO:tasks.workunit.client.1.smithi062.stdout:0/951: creat d1/d3/d4/d1e/d74/d122/f13f x:0 0 0 2024-02-22T15:50:51.350 INFO:tasks.workunit.client.1.smithi062.stdout:0/952: read d1/d3/d4/f99 [1351531,45337] 0 2024-02-22T15:50:51.351 INFO:tasks.workunit.client.1.smithi062.stdout:0/953: stat d1/d13/dde/d76/d8c/f64 0 2024-02-22T15:50:51.353 INFO:tasks.workunit.client.1.smithi062.stdout:0/954: mknod d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/dfd/c140 0 2024-02-22T15:50:51.358 INFO:tasks.workunit.client.1.smithi062.stdout:0/955: dwrite d1/d3/d5f/fed [4194304,4194304] 0 2024-02-22T15:50:51.362 INFO:tasks.workunit.client.1.smithi062.stdout:0/956: creat d1/d13/dde/d76/d8c/d5d/dc3/d100/d12a/f141 x:0 0 0 2024-02-22T15:50:51.365 INFO:tasks.workunit.client.1.smithi062.stdout:0/957: creat d1/d13/dde/d76/d8c/d5d/f142 x:0 0 0 2024-02-22T15:50:51.369 INFO:tasks.workunit.client.1.smithi062.stdout:0/958: creat d1/d13/dde/d76/d8c/d5d/dc3/d100/f143 x:0 0 0 2024-02-22T15:50:51.377 INFO:tasks.workunit.client.1.smithi062.stdout:0/959: write d1/f12 [4487723,15022] 0 2024-02-22T15:50:51.385 INFO:tasks.workunit.client.1.smithi062.stdout:0/960: dread d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/fa1 [0,4194304] 0 2024-02-22T15:50:51.392 INFO:tasks.workunit.client.1.smithi062.stdout:0/961: dwrite d1/d13/dde/d76/d8c/d2c/d84/f107 [0,4194304] 0 2024-02-22T15:50:51.394 INFO:tasks.workunit.client.1.smithi062.stdout:0/962: readlink d1/d13/dde/d76/d8c/d5d/dc3/d100/l103 0 2024-02-22T15:50:51.395 INFO:tasks.workunit.client.1.smithi062.stdout:0/963: read d1/d3/d5f/fed [5111211,59848] 0 2024-02-22T15:50:51.400 INFO:tasks.workunit.client.1.smithi062.stdout:0/964: link d1/d3/d4/f23 d1/d3/dd5/f144 0 2024-02-22T15:50:51.403 INFO:tasks.workunit.client.1.smithi062.stdout:0/965: mknod d1/d13/dde/d76/d8c/d2c/db5/d13b/ddb/dfe/c145 0 2024-02-22T15:50:51.405 INFO:tasks.workunit.client.1.smithi062.stdout:0/966: creat d1/d13/dde/d76/d102/f146 x:0 0 0 2024-02-22T15:50:51.411 INFO:tasks.workunit.client.1.smithi062.stdout:0/967: write d1/d38/f98 [4369269,52587] 0 2024-02-22T15:50:51.416 INFO:tasks.workunit.client.1.smithi062.stdout:0/968: dwrite d1/f12 [0,4194304] 0 2024-02-22T15:50:51.423 INFO:tasks.workunit.client.1.smithi062.stdout:0/969: symlink d1/d3/d4/l147 0 2024-02-22T15:50:51.424 INFO:tasks.workunit.client.1.smithi062.stdout:0/970: creat d1/d13/dde/d76/d8c/d2c/db5/d119/d12b/f148 x:0 0 0 2024-02-22T15:50:51.426 INFO:tasks.workunit.client.1.smithi062.stdout:0/971: mkdir d1/d3/d5f/d149 0 2024-02-22T15:50:51.431 INFO:tasks.workunit.client.1.smithi062.stdout:0/972: creat d1/d13/dde/d76/d8c/d5d/dc1/f14a x:0 0 0 2024-02-22T15:50:51.434 INFO:tasks.workunit.client.1.smithi062.stdout:0/973: creat d1/d13/dde/d76/d8c/d5d/dc1/f14b x:0 0 0 2024-02-22T15:50:51.435 INFO:tasks.workunit.client.1.smithi062.stdout:0/974: fsync d1/d13/dde/d76/d8c/d5d/dc3/d100/f143 0 2024-02-22T15:50:51.466 INFO:tasks.workunit.client.1.smithi062.stdout:1/733: sync 2024-02-22T15:50:51.466 INFO:tasks.workunit.client.1.smithi062.stdout:6/820: sync 2024-02-22T15:50:51.466 INFO:tasks.workunit.client.1.smithi062.stdout:8/973: sync 2024-02-22T15:50:51.466 INFO:tasks.workunit.client.1.smithi062.stdout:9/876: sync 2024-02-22T15:50:51.467 INFO:tasks.workunit.client.1.smithi062.stdout:6/821: readlink d5/df/d40/d44/d76/l106 0 2024-02-22T15:50:51.469 INFO:tasks.workunit.client.1.smithi062.stdout:8/974: read d8/d9/fc4 [126546,78364] 0 2024-02-22T15:50:51.470 INFO:tasks.workunit.client.1.smithi062.stdout:6/822: creat d5/d6e/f111 x:0 0 0 2024-02-22T15:50:51.470 INFO:tasks.workunit.client.1.smithi062.stdout:9/877: mknod d4/d15/d5f/d75/d97/c11d 0 2024-02-22T15:50:51.475 INFO:tasks.workunit.client.1.smithi062.stdout:1/734: dwrite d1/d6/fed [0,4194304] 0 2024-02-22T15:50:51.475 INFO:tasks.workunit.client.1.smithi062.stdout:9/878: mkdir d4/d28/d50/d11e 0 2024-02-22T15:50:51.479 INFO:tasks.workunit.client.1.smithi062.stdout:6/823: rename d5/df/dbc/dcd/de9 to d5/df/dbc/dce/d112 0 2024-02-22T15:50:51.481 INFO:tasks.workunit.client.1.smithi062.stdout:8/975: dwrite d8/d9/d44/d4d/fee [0,4194304] 0 2024-02-22T15:50:51.482 INFO:tasks.workunit.client.1.smithi062.stdout:9/879: chown d4/d15/d5e/d63/da0/d92/c115 678203 1 2024-02-22T15:50:51.483 INFO:tasks.workunit.client.1.smithi062.stdout:6/824: symlink d5/d6e/l113 0 2024-02-22T15:50:51.484 INFO:tasks.workunit.client.1.smithi062.stdout:1/735: read d1/d6/d7/d2b/d47/d7e/fc5 [286879,123770] 0 2024-02-22T15:50:51.487 INFO:tasks.workunit.client.1.smithi062.stdout:9/880: write d4/d1c/f1d [4687192,76310] 0 2024-02-22T15:50:51.492 INFO:tasks.workunit.client.1.smithi062.stdout:9/881: mkdir d4/d1c/d83/d87/d11f 0 2024-02-22T15:50:51.492 INFO:tasks.workunit.client.1.smithi062.stdout:6/825: read d5/dd/d33/d66/f79 [1216027,40387] 0 2024-02-22T15:50:51.493 INFO:tasks.workunit.client.1.smithi062.stdout:8/976: getdents d8/d9/d44/df8/d105/d4e 0 2024-02-22T15:50:51.493 INFO:tasks.workunit.client.1.smithi062.stdout:6/826: stat d5/d8a/ddd/lda 0 2024-02-22T15:50:51.497 INFO:tasks.workunit.client.1.smithi062.stdout:6/827: mkdir d5/dd/d33/d10a/d114 0 2024-02-22T15:50:51.500 INFO:tasks.workunit.client.1.smithi062.stdout:8/977: rename d8/d9/d44/df8/d105/d70/fc5 to d8/d9/d44/d7e/f12d 0 2024-02-22T15:50:51.501 INFO:tasks.workunit.client.1.smithi062.stdout:9/882: getdents d4/d15/d5f/d75/d97 0 2024-02-22T15:50:51.502 INFO:tasks.workunit.client.1.smithi062.stdout:8/978: readlink d8/d9/d2c/d49/d94/da1/ldd 0 2024-02-22T15:50:51.505 INFO:tasks.workunit.client.1.smithi062.stdout:1/736: write d1/d6/d7/d2b/d47/fb0 [156424,68149] 0 2024-02-22T15:50:51.506 INFO:tasks.workunit.client.1.smithi062.stdout:6/828: mkdir d5/d8a/d10b/d115 0 2024-02-22T15:50:51.508 INFO:tasks.workunit.client.1.smithi062.stdout:9/883: unlink d4/cd9 0 2024-02-22T15:50:51.511 INFO:tasks.workunit.client.1.smithi062.stdout:6/829: rmdir d5/df/d40/db3 39 2024-02-22T15:50:51.516 INFO:tasks.workunit.client.1.smithi062.stdout:8/979: getdents d8/d9/d2c/d49/d94/dc1 0 2024-02-22T15:50:51.520 INFO:tasks.workunit.client.1.smithi062.stdout:6/830: rename d5/df/d40/d44/d4a/d41/f55 to d5/df/f116 0 2024-02-22T15:50:51.520 INFO:tasks.workunit.client.1.smithi062.stdout:8/980: dread d8/f3a [0,4194304] 0 2024-02-22T15:50:51.522 INFO:tasks.workunit.client.1.smithi062.stdout:1/737: dread d1/d6/de/d9/fd [0,4194304] 0 2024-02-22T15:50:51.524 INFO:tasks.workunit.client.1.smithi062.stdout:8/981: fsync d8/d26/db5/f104 0 2024-02-22T15:50:51.528 INFO:tasks.workunit.client.1.smithi062.stdout:9/884: dread d4/d15/d5f/d75/f9a [0,4194304] 0 2024-02-22T15:50:51.529 INFO:tasks.workunit.client.1.smithi062.stdout:8/982: rmdir d8/d9/d2c/dd5 39 2024-02-22T15:50:51.530 INFO:tasks.workunit.client.1.smithi062.stdout:9/885: readlink d4/d15/d5f/d75/d97/ldb 0 2024-02-22T15:50:51.535 INFO:tasks.workunit.client.1.smithi062.stdout:9/886: rmdir d4 39 2024-02-22T15:50:51.535 INFO:tasks.workunit.client.1.smithi062.stdout:1/738: dwrite d1/d6/d36/d76/fa1 [0,4194304] 0 2024-02-22T15:50:51.539 INFO:tasks.workunit.client.1.smithi062.stdout:6/831: rename d5/df/d40/db3/f108 to d5/dd/d33/f117 0 2024-02-22T15:50:51.542 INFO:tasks.workunit.client.1.smithi062.stdout:9/887: symlink d4/d15/d5f/d75/d97/l120 0 2024-02-22T15:50:51.543 INFO:tasks.workunit.client.1.smithi062.stdout:8/983: rename d8/d26/fb4 to d8/d86/f12e 0 2024-02-22T15:50:51.543 INFO:tasks.workunit.client.1.smithi062.stdout:9/888: chown d4/d15/f44 91787865 1 2024-02-22T15:50:51.545 INFO:tasks.workunit.client.1.smithi062.stdout:6/832: creat d5/df/d40/dd2/f118 x:0 0 0 2024-02-22T15:50:51.548 INFO:tasks.workunit.client.1.smithi062.stdout:9/889: mknod d4/d15/d5f/c121 0 2024-02-22T15:50:51.549 INFO:tasks.workunit.client.1.smithi062.stdout:9/890: stat d4/d1c/d83/cb4 0 2024-02-22T15:50:51.553 INFO:tasks.workunit.client.1.smithi062.stdout:9/891: creat d4/d28/d50/f122 x:0 0 0 2024-02-22T15:50:51.553 INFO:tasks.workunit.client.1.smithi062.stdout:9/892: chown d4/d28/d50/d107 31614702 1 2024-02-22T15:50:51.556 INFO:tasks.workunit.client.1.smithi062.stdout:9/893: mkdir d4/d15/d5f/d123 0 2024-02-22T15:50:51.557 INFO:tasks.workunit.client.1.smithi062.stdout:8/984: write d8/d9/d2c/d49/d5e/fe2 [368802,130250] 0 2024-02-22T15:50:51.562 INFO:tasks.workunit.client.1.smithi062.stdout:8/985: symlink d8/d9/d2c/d74/l12f 0 2024-02-22T15:50:51.565 INFO:tasks.workunit.client.1.smithi062.stdout:1/739: dread d1/d6/de/f31 [4194304,4194304] 0 2024-02-22T15:50:51.566 INFO:tasks.workunit.client.1.smithi062.stdout:6/833: dwrite d5/dd/fe3 [0,4194304] 0 2024-02-22T15:50:51.568 INFO:tasks.workunit.client.1.smithi062.stdout:6/834: dread - d5/df/dbc/dcd/ff8 zero size 2024-02-22T15:50:51.568 INFO:tasks.workunit.client.1.smithi062.stdout:8/986: fdatasync d8/d9/d2c/d49/d94/fb3 0 2024-02-22T15:50:51.573 INFO:tasks.workunit.client.1.smithi062.stdout:8/987: write d8/d9/d44/d58/fef [32955,70524] 0 2024-02-22T15:50:51.576 INFO:tasks.workunit.client.1.smithi062.stdout:9/894: dread d4/d15/d5f/d75/d84/f3e [0,4194304] 0 2024-02-22T15:50:51.578 INFO:tasks.workunit.client.1.smithi062.stdout:9/895: mkdir d4/d15/d5e/d101/d124 0 2024-02-22T15:50:51.580 INFO:tasks.workunit.client.1.smithi062.stdout:9/896: creat d4/d15/d109/f125 x:0 0 0 2024-02-22T15:50:51.603 INFO:tasks.workunit.client.1.smithi062.stdout:9/897: mkdir d4/d15/d5e/d126 0 2024-02-22T15:50:51.605 INFO:tasks.workunit.client.1.smithi062.stdout:8/988: dwrite d8/d9/d2c/d49/f6b [0,4194304] 0 2024-02-22T15:50:51.611 INFO:tasks.workunit.client.1.smithi062.stdout:9/898: dwrite d4/f10 [0,4194304] 0 2024-02-22T15:50:51.652 INFO:tasks.workunit.client.1.smithi062.stdout:5/773: sync 2024-02-22T15:50:51.652 INFO:tasks.workunit.client.1.smithi062.stdout:2/986: sync 2024-02-22T15:50:51.652 INFO:tasks.workunit.client.1.smithi062.stdout:4/950: sync 2024-02-22T15:50:51.652 INFO:tasks.workunit.client.1.smithi062.stdout:7/654: sync 2024-02-22T15:50:51.653 INFO:tasks.workunit.client.1.smithi062.stdout:3/848: sync 2024-02-22T15:50:51.656 INFO:tasks.workunit.client.1.smithi062.stdout:7/655: chown d0/d6/d2e/d35/f70 28080 1 2024-02-22T15:50:51.657 INFO:tasks.workunit.client.1.smithi062.stdout:2/987: fdatasync d1/d8/f3d 0 2024-02-22T15:50:51.658 INFO:tasks.workunit.client.1.smithi062.stdout:5/774: chown d0/d3/da/d13/d23/d78/d8c/dbd 5334 1 2024-02-22T15:50:51.665 INFO:tasks.workunit.client.1.smithi062.stdout:8/989: dwrite d8/d26/fbd [0,4194304] 0 2024-02-22T15:50:51.665 INFO:tasks.workunit.client.1.smithi062.stdout:3/849: truncate d5/f6f 2701757 0 2024-02-22T15:50:51.668 INFO:tasks.workunit.client.1.smithi062.stdout:2/988: mknod d1/d8/d62/d73/dbd/d65/dd4/c14c 0 2024-02-22T15:50:51.672 INFO:tasks.workunit.client.1.smithi062.stdout:8/990: unlink d8/f21 0 2024-02-22T15:50:51.672 INFO:tasks.workunit.client.1.smithi062.stdout:7/656: mkdir d0/d14/d19/d30/d45/dc3/dcb 0 2024-02-22T15:50:51.673 INFO:tasks.workunit.client.1.smithi062.stdout:9/899: dwrite d4/d35/fd8 [0,4194304] 0 2024-02-22T15:50:51.676 INFO:tasks.workunit.client.1.smithi062.stdout:8/991: dread - d8/f117 zero size 2024-02-22T15:50:51.677 INFO:tasks.workunit.client.1.smithi062.stdout:9/900: chown d4/d15/d5f/d75/l117 1 1 2024-02-22T15:50:51.678 INFO:tasks.workunit.client.1.smithi062.stdout:5/775: rename d0/d3/da/d13/d23/d4e/d5a/f5e to d0/d3/da/d13/d57/f100 0 2024-02-22T15:50:51.682 INFO:tasks.workunit.client.1.smithi062.stdout:7/657: write d0/d14/d19/d77/fb5 [1390963,75026] 0 2024-02-22T15:50:51.682 INFO:tasks.workunit.client.1.smithi062.stdout:3/850: dwrite d5/d26/d2d/fd0 [0,4194304] 0 2024-02-22T15:50:51.683 INFO:tasks.workunit.client.1.smithi062.stdout:8/992: write d8/d9/f2a [2487798,19552] 0 2024-02-22T15:50:51.685 INFO:tasks.workunit.client.1.smithi062.stdout:4/951: truncate d4/d1c/d2c/d6b/fca 2310602 0 2024-02-22T15:50:51.686 INFO:tasks.workunit.client.1.smithi062.stdout:7/658: readlink d0/d91/l79 0 2024-02-22T15:50:51.701 INFO:tasks.workunit.client.1.smithi062.stdout:2/989: dread d1/d8/d62/d73/dbd/d7d/ffb [0,4194304] 0 2024-02-22T15:50:51.701 INFO:tasks.workunit.client.1.smithi062.stdout:3/851: chown d5/d16/d56/c6b 5372594 1 2024-02-22T15:50:51.702 INFO:tasks.workunit.client.1.smithi062.stdout:5/776: link d0/d3/d30/f4a d0/d3/f101 0 2024-02-22T15:50:51.705 INFO:tasks.workunit.client.1.smithi062.stdout:9/901: dwrite d4/d1c/d83/d87/f103 [0,4194304] 0 2024-02-22T15:50:51.705 INFO:tasks.workunit.client.1.smithi062.stdout:5/777: stat d0/d3/da/d13/l1f 0 2024-02-22T15:50:51.707 INFO:tasks.workunit.client.1.smithi062.stdout:9/902: dread - d4/d15/d31/fe4 zero size 2024-02-22T15:50:51.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:51 smithi062 ceph-mon[48130]: pgmap v41: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 144 MiB/s rd, 384 MiB/s wr, 769 op/s 2024-02-22T15:50:51.711 INFO:tasks.workunit.client.1.smithi062.stdout:9/903: write d4/d1c/f2c [4659788,90679] 0 2024-02-22T15:50:51.712 INFO:tasks.workunit.client.1.smithi062.stdout:5/778: write d0/d3/da/d32/d4f/d72/f99 [108483,3078] 0 2024-02-22T15:50:51.713 INFO:tasks.workunit.client.1.smithi062.stdout:8/993: dwrite d8/d9/d2c/d49/d5e/f101 [0,4194304] 0 2024-02-22T15:50:51.721 INFO:tasks.workunit.client.1.smithi062.stdout:5/779: dwrite d0/d3/da/d32/d4f/d72/f99 [0,4194304] 0 2024-02-22T15:50:51.740 INFO:tasks.workunit.client.1.smithi062.stdout:3/852: rename d5/d26/d2d/da2 to d5/d16/d21/d32/d9e/dc2/d113 0 2024-02-22T15:50:51.743 INFO:tasks.workunit.client.1.smithi062.stdout:5/780: unlink d0/d3/da/d32/d4f/d72/df4/d3f/f5c 0 2024-02-22T15:50:51.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:51 smithi019 ceph-mon[67418]: pgmap v41: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 144 MiB/s rd, 384 MiB/s wr, 769 op/s 2024-02-22T15:50:51.746 INFO:tasks.workunit.client.1.smithi062.stdout:8/994: mknod d8/d9/d2c/c130 0 2024-02-22T15:50:51.751 INFO:tasks.workunit.client.1.smithi062.stdout:9/904: link d4/d15/d5e/d63/c6d d4/d1c/d118/c127 0 2024-02-22T15:50:51.753 INFO:tasks.workunit.client.1.smithi062.stdout:5/781: creat d0/d3/da/d13/d23/d4e/d5a/dd6/f102 x:0 0 0 2024-02-22T15:50:51.754 INFO:tasks.workunit.client.1.smithi062.stdout:4/952: getdents d4/d2a/d2d/d2e/d6a/df7 0 2024-02-22T15:50:51.755 INFO:tasks.workunit.client.1.smithi062.stdout:4/953: chown d4/d1c/dac/lcd 459799621 1 2024-02-22T15:50:51.757 INFO:tasks.workunit.client.1.smithi062.stdout:4/954: stat d4/d2a/d2d/d2e/c33 0 2024-02-22T15:50:51.758 INFO:tasks.workunit.client.1.smithi062.stdout:4/955: dread - d4/d1c/d2c/d6b/f100 zero size 2024-02-22T15:50:51.759 INFO:tasks.workunit.client.1.smithi062.stdout:9/905: rename d4/d15/d109/f125 to d4/d28/d50/f128 0 2024-02-22T15:50:51.760 INFO:tasks.workunit.client.1.smithi062.stdout:4/956: dread d4/d2a/fd1 [0,4194304] 0 2024-02-22T15:50:51.763 INFO:tasks.workunit.client.1.smithi062.stdout:5/782: creat d0/d3/d2f/d59/d67/f103 x:0 0 0 2024-02-22T15:50:51.764 INFO:tasks.workunit.client.1.smithi062.stdout:4/957: truncate d4/df/daf/d10c/f121 521562 0 2024-02-22T15:50:51.765 INFO:tasks.workunit.client.1.smithi062.stdout:4/958: stat d4/d34/c55 0 2024-02-22T15:50:51.767 INFO:tasks.workunit.client.1.smithi062.stdout:5/783: truncate d0/d3/da/fe2 869935 0 2024-02-22T15:50:51.767 INFO:tasks.workunit.client.1.smithi062.stdout:9/906: dwrite d4/f12 [4194304,4194304] 0 2024-02-22T15:50:51.768 INFO:tasks.workunit.client.1.smithi062.stdout:2/990: write d1/d8/d8f/f95 [85153,66466] 0 2024-02-22T15:50:51.769 INFO:tasks.workunit.client.1.smithi062.stdout:8/995: dread d8/d9/d44/d4d/f82 [0,4194304] 0 2024-02-22T15:50:51.770 INFO:tasks.workunit.client.1.smithi062.stdout:9/907: dread - d4/d28/d50/f122 zero size 2024-02-22T15:50:51.771 INFO:tasks.workunit.client.1.smithi062.stdout:4/959: unlink d4/d1c/d30/d3b/c71 0 2024-02-22T15:50:51.775 INFO:tasks.workunit.client.1.smithi062.stdout:2/991: symlink d1/d1e/d38/d76/df5/l14d 0 2024-02-22T15:50:51.776 INFO:tasks.workunit.client.1.smithi062.stdout:3/853: truncate d5/d26/d2d/fb4 398581 0 2024-02-22T15:50:51.777 INFO:tasks.workunit.client.1.smithi062.stdout:4/960: creat d4/d1c/dac/f14b x:0 0 0 2024-02-22T15:50:51.781 INFO:tasks.workunit.client.1.smithi062.stdout:8/996: mkdir d8/d42/d131 0 2024-02-22T15:50:51.781 INFO:tasks.workunit.client.1.smithi062.stdout:9/908: mknod d4/d28/d50/d107/c129 0 2024-02-22T15:50:51.786 INFO:tasks.workunit.client.1.smithi062.stdout:4/961: creat d4/d2a/d2d/d46/d13b/f14c x:0 0 0 2024-02-22T15:50:51.791 INFO:tasks.workunit.client.1.smithi062.stdout:0/975: sync 2024-02-22T15:50:51.791 INFO:tasks.workunit.client.1.smithi062.stdout:3/854: unlink d5/d16/d56/dbc/f110 0 2024-02-22T15:50:51.792 INFO:tasks.workunit.client.1.smithi062.stdout:4/962: mkdir d4/d13/d14d 0 2024-02-22T15:50:51.793 INFO:tasks.workunit.client.1.smithi062.stdout:5/784: dwrite d0/d3/da/f1d [0,4194304] 0 2024-02-22T15:50:51.794 INFO:tasks.workunit.client.1.smithi062.stdout:4/963: fsync d4/d1c/dac/f14b 0 2024-02-22T15:50:51.794 INFO:tasks.workunit.client.1.smithi062.stdout:0/976: chown d1/d13/dde/d76/d8c/d2c/db5/d13b 5009192 1 2024-02-22T15:50:51.795 INFO:tasks.workunit.client.1.smithi062.stdout:9/909: mkdir d4/d1c/d83/d87/d11f/d12a 0 2024-02-22T15:50:51.799 INFO:tasks.workunit.client.1.smithi062.stdout:0/977: mknod d1/d13/dde/d76/d8c/d5d/dc3/c14c 0 2024-02-22T15:50:51.801 INFO:tasks.workunit.client.1.smithi062.stdout:5/785: read - d0/d3/da/d32/d33/dbc/fc4 zero size 2024-02-22T15:50:51.801 INFO:tasks.workunit.client.1.smithi062.stdout:8/997: write d8/d9/d44/d4d/fb0 [245086,118853] 0 2024-02-22T15:50:51.802 INFO:tasks.workunit.client.1.smithi062.stdout:3/855: creat d5/d1a/d3b/f114 x:0 0 0 2024-02-22T15:50:51.803 INFO:tasks.workunit.client.1.smithi062.stdout:9/910: mknod d4/d1c/d118/c12b 0 2024-02-22T15:50:51.806 INFO:tasks.workunit.client.1.smithi062.stdout:4/964: symlink d4/d13/d14d/l14e 0 2024-02-22T15:50:51.806 INFO:tasks.workunit.client.1.smithi062.stdout:3/856: readlink d5/d1a/d3c/l57 0 2024-02-22T15:50:51.807 INFO:tasks.workunit.client.1.smithi062.stdout:2/992: dwrite d1/d1e/d38/d76/dc5/dec/ff9 [0,4194304] 0 2024-02-22T15:50:51.807 INFO:tasks.workunit.client.1.smithi062.stdout:0/978: symlink d1/d38/dfa/l14d 0 2024-02-22T15:50:51.810 INFO:tasks.workunit.client.1.smithi062.stdout:7/659: sync 2024-02-22T15:50:51.810 INFO:tasks.workunit.client.1.smithi062.stdout:6/835: sync 2024-02-22T15:50:51.810 INFO:tasks.workunit.client.1.smithi062.stdout:1/740: sync 2024-02-22T15:50:51.813 INFO:tasks.workunit.client.1.smithi062.stdout:9/911: write d4/d15/f24 [1248346,79516] 0 2024-02-22T15:50:51.813 INFO:tasks.workunit.client.1.smithi062.stdout:8/998: truncate d8/d26/f41 303827 0 2024-02-22T15:50:51.817 INFO:tasks.workunit.client.1.smithi062.stdout:1/741: rename d1/d6 to d1/d6/d7/d2b/d47/d7e/df8 22 2024-02-22T15:50:51.817 INFO:tasks.workunit.client.1.smithi062.stdout:9/912: stat d4/d28/d50/f7a 0 2024-02-22T15:50:51.818 INFO:tasks.workunit.client.1.smithi062.stdout:8/999: chown d8/d9/d2c/d49/d94 1683950 1 2024-02-22T15:50:51.819 INFO:tasks.workunit.client.1.smithi062.stdout:4/965: creat d4/d34/f14f x:0 0 0 2024-02-22T15:50:51.821 INFO:tasks.workunit.client.1.smithi062.stdout:3/857: unlink d5/d1a/c20 0 2024-02-22T15:50:51.821 INFO:tasks.workunit.client.1.smithi062.stdout:0/979: symlink d1/d3/dd5/l14e 0 2024-02-22T15:50:51.822 INFO:tasks.workunit.client.1.smithi062.stdout:5/786: getdents d0/d3/da/d13/d23/d4e/d5a/dd6 0 2024-02-22T15:50:51.827 INFO:tasks.workunit.client.1.smithi062.stdout:6/836: creat d5/df/d40/d44/d4a/d41/d85/dd1/f119 x:0 0 0 2024-02-22T15:50:51.827 INFO:tasks.workunit.client.1.smithi062.stdout:5/787: write d0/d3/da/d32/d4f/d72/df4/d3f/fe3 [168856,67743] 0 2024-02-22T15:50:51.832 INFO:tasks.workunit.client.1.smithi062.stdout:9/913: dread d4/d35/f119 [0,4194304] 0 2024-02-22T15:50:51.833 INFO:tasks.workunit.client.1.smithi062.stdout:2/993: creat d1/d8/d62/d73/dbd/f14e x:0 0 0 2024-02-22T15:50:51.835 INFO:tasks.workunit.client.1.smithi062.stdout:2/994: write d1/d1e/d38/d76/dc5/dec/ff9 [1313567,46627] 0 2024-02-22T15:50:51.835 INFO:tasks.workunit.client.1.smithi062.stdout:1/742: fdatasync d1/d6/d7/d2b/d47/d63/f87 0 2024-02-22T15:50:51.838 INFO:tasks.workunit.client.1.smithi062.stdout:2/995: write d1/d8/d62/d73/f75 [622366,114562] 0 2024-02-22T15:50:51.839 INFO:tasks.workunit.client.1.smithi062.stdout:5/788: mknod d0/d3/da/d13/d23/d78/d8c/c104 0 2024-02-22T15:50:51.845 INFO:tasks.workunit.client.1.smithi062.stdout:7/660: link d0/d6/d2e/d4a/d5a/d5e/c7a d0/d14/d19/d30/dbd/ccc 0 2024-02-22T15:50:51.846 INFO:tasks.workunit.client.1.smithi062.stdout:6/837: read d5/df/d40/d44/d76/d56/fca [2979275,29863] 0 2024-02-22T15:50:51.847 INFO:tasks.workunit.client.1.smithi062.stdout:7/661: fdatasync d0/d6/d2e/d55/d80/d8a/f8f 0 2024-02-22T15:50:51.848 INFO:tasks.workunit.client.1.smithi062.stdout:1/743: symlink d1/d6/d7/d2b/d47/d24/lf9 0 2024-02-22T15:50:51.849 INFO:tasks.workunit.client.1.smithi062.stdout:2/996: creat d1/d1e/d38/d11a/f14f x:0 0 0 2024-02-22T15:50:51.851 INFO:tasks.workunit.client.1.smithi062.stdout:0/980: creat d1/d13/dde/d76/d8c/d5d/dc3/d83/f14f x:0 0 0 2024-02-22T15:50:51.852 INFO:tasks.workunit.client.1.smithi062.stdout:2/997: dread d1/d28/fc2 [0,4194304] 0 2024-02-22T15:50:51.854 INFO:tasks.workunit.client.1.smithi062.stdout:5/789: creat d0/d3/da/d32/d4f/d72/f105 x:0 0 0 2024-02-22T15:50:51.855 INFO:tasks.workunit.client.1.smithi062.stdout:4/966: link d4/d2a/d2d/d2e/l3a d4/df/daf/l150 0 2024-02-22T15:50:51.856 INFO:tasks.workunit.client.1.smithi062.stdout:3/858: getdents d5/d16 0 2024-02-22T15:50:51.856 INFO:tasks.workunit.client.1.smithi062.stdout:6/838: truncate d5/fb 5293756 0 2024-02-22T15:50:51.858 INFO:tasks.workunit.client.1.smithi062.stdout:7/662: symlink d0/d14/d28/lcd 0 2024-02-22T15:50:51.861 INFO:tasks.workunit.client.1.smithi062.stdout:1/744: creat d1/d6/d7/d2b/d47/d24/ffa x:0 0 0 2024-02-22T15:50:51.861 INFO:tasks.workunit.client.1.smithi062.stdout:2/998: fsync d1/d47/d39/fcd 0 2024-02-22T15:50:51.867 INFO:tasks.workunit.client.1.smithi062.stdout:6/839: mkdir d5/df/d40/dac/d11a 0 2024-02-22T15:50:51.868 INFO:tasks.workunit.client.1.smithi062.stdout:7/663: dwrite d0/d6/f52 [0,4194304] 0 2024-02-22T15:50:51.869 INFO:tasks.workunit.client.1.smithi062.stdout:9/914: dread d4/d15/d5f/d75/d97/da7/fad [0,4194304] 0 2024-02-22T15:50:51.871 INFO:tasks.workunit.client.1.smithi062.stdout:9/915: chown d4/d1c/d83/fc5 468435 1 2024-02-22T15:50:51.873 INFO:tasks.workunit.client.1.smithi062.stdout:3/859: rename d5/d26/c72 to d5/d1a/d3c/d62/c115 0 2024-02-22T15:50:51.874 INFO:tasks.workunit.client.1.smithi062.stdout:0/981: mkdir d1/d13/dde/d76/d8c/d5d/d150 0 2024-02-22T15:50:51.876 INFO:tasks.workunit.client.1.smithi062.stdout:1/745: mknod d1/d6/d36/d37/cfb 0 2024-02-22T15:50:51.880 INFO:tasks.workunit.client.1.smithi062.stdout:4/967: symlink d4/d2a/d2d/d2e/d6a/l151 0 2024-02-22T15:50:51.881 INFO:tasks.workunit.client.1.smithi062.stdout:6/840: mknod d5/df/d40/d44/d76/dae/c11b 0 2024-02-22T15:50:51.881 INFO:tasks.workunit.client.1.smithi062.stdout:2/999: chown d1/d8/d62/d73/dbd/d7d/d84/f125 51626920 1 2024-02-22T15:50:51.882 INFO:tasks.workunit.client.1.smithi062.stdout:6/841: stat d5/c1b 0 2024-02-22T15:50:51.882 INFO:tasks.workunit.client.1.smithi062.stdout:0/982: creat d1/d3/d5f/f151 x:0 0 0 2024-02-22T15:50:51.883 INFO:tasks.workunit.client.1.smithi062.stdout:4/968: write d4/d1c/d2c/d6b/d9b/f120 [910669,15881] 0 2024-02-22T15:50:51.885 INFO:tasks.workunit.client.1.smithi062.stdout:3/860: truncate d5/d16/d21/d36/f95 1199537 0 2024-02-22T15:50:51.885 INFO:tasks.workunit.client.1.smithi062.stdout:7/664: link d0/d14/d19/d30/d37/f95 d0/d91/fce 0 2024-02-22T15:50:51.886 INFO:tasks.workunit.client.1.smithi062.stdout:3/861: read - d5/da6/fec zero size 2024-02-22T15:50:51.887 INFO:tasks.workunit.client.1.smithi062.stdout:4/969: chown d4/d2a/d2d/d2e/d6a/d9e/da4 13154 1 2024-02-22T15:50:51.887 INFO:tasks.workunit.client.1.smithi062.stdout:6/842: mknod d5/dd/d33/d66/db9/c11c 0 2024-02-22T15:50:51.890 INFO:tasks.workunit.client.1.smithi062.stdout:9/916: creat d4/d15/d5e/d63/da0/f12c x:0 0 0 2024-02-22T15:50:51.891 INFO:tasks.workunit.client.1.smithi062.stdout:7/665: dread - d0/d14/d13/f40 zero size 2024-02-22T15:50:51.892 INFO:tasks.workunit.client.1.smithi062.stdout:0/983: mknod d1/d3/de/d131/c152 0 2024-02-22T15:50:51.893 INFO:tasks.workunit.client.1.smithi062.stdout:5/790: write d0/d3/da/d13/d57/f100 [1470428,18420] 0 2024-02-22T15:50:51.896 INFO:tasks.workunit.client.1.smithi062.stdout:5/791: stat d0/d3/d2f 0 2024-02-22T15:50:51.896 INFO:tasks.workunit.client.1.smithi062.stdout:9/917: dread - d4/f4f zero size 2024-02-22T15:50:51.897 INFO:tasks.workunit.client.1.smithi062.stdout:4/970: dwrite d4/d1c/d2c/d6b/d70/f76 [0,4194304] 0 2024-02-22T15:50:51.898 INFO:tasks.workunit.client.1.smithi062.stdout:6/843: read d5/df/d40/d44/d78/f5c [678153,80464] 0 2024-02-22T15:50:51.898 INFO:tasks.workunit.client.1.smithi062.stdout:7/666: mkdir d0/d14/d19/d77/dcf 0 2024-02-22T15:50:51.899 INFO:tasks.workunit.client.1.smithi062.stdout:3/862: dread d5/d26/f38 [0,4194304] 0 2024-02-22T15:50:51.905 INFO:tasks.workunit.client.1.smithi062.stdout:1/746: getdents d1/d6/d7/d2b/d49/d74 0 2024-02-22T15:50:51.905 INFO:tasks.workunit.client.1.smithi062.stdout:0/984: write d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dd7/f124 [63774,29732] 0 2024-02-22T15:50:51.905 INFO:tasks.workunit.client.1.smithi062.stdout:9/918: chown d4/d28/d50/da3/fcb 9273435 1 2024-02-22T15:50:51.910 INFO:tasks.workunit.client.1.smithi062.stdout:1/747: write d1/d6/d7/de6/fe7 [3460445,56619] 0 2024-02-22T15:50:51.911 INFO:tasks.workunit.client.1.smithi062.stdout:6/844: read d5/df/d32/d8f/f7e [3776952,44028] 0 2024-02-22T15:50:51.914 INFO:tasks.workunit.client.1.smithi062.stdout:3/863: mknod d5/d26/d2d/d3f/ddb/dff/c116 0 2024-02-22T15:50:51.915 INFO:tasks.workunit.client.1.smithi062.stdout:6/845: readlink d5/df/d40/d44/d4a/d41/d43/l6f 0 2024-02-22T15:50:51.915 INFO:tasks.workunit.client.1.smithi062.stdout:0/985: creat d1/d3/de/d12c/f153 x:0 0 0 2024-02-22T15:50:51.916 INFO:tasks.workunit.client.1.smithi062.stdout:1/748: creat d1/d6/d7/d2b/d96/ffc x:0 0 0 2024-02-22T15:50:51.918 INFO:tasks.workunit.client.1.smithi062.stdout:5/792: rename d0/d3/d30/d71/f77 to d0/d3/da/d13/d23/d78/f106 0 2024-02-22T15:50:51.920 INFO:tasks.workunit.client.1.smithi062.stdout:9/919: dwrite d4/d28/f72 [0,4194304] 0 2024-02-22T15:50:51.922 INFO:tasks.workunit.client.1.smithi062.stdout:5/793: chown d0/d3/da/d13/d57/l90 0 1 2024-02-22T15:50:51.924 INFO:tasks.workunit.client.1.smithi062.stdout:1/749: mknod d1/d6/d7/d2b/d49/da9/cfd 0 2024-02-22T15:50:51.925 INFO:tasks.workunit.client.1.smithi062.stdout:9/920: fdatasync d4/d15/d5f/ffb 0 2024-02-22T15:50:51.927 INFO:tasks.workunit.client.1.smithi062.stdout:3/864: rmdir d5/d1a/d3b/de7 0 2024-02-22T15:50:51.928 INFO:tasks.workunit.client.1.smithi062.stdout:0/986: symlink d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/l154 0 2024-02-22T15:50:51.929 INFO:tasks.workunit.client.1.smithi062.stdout:5/794: dread d0/fdf [0,4194304] 0 2024-02-22T15:50:51.932 INFO:tasks.workunit.client.1.smithi062.stdout:4/971: write d4/f39 [8150731,43784] 0 2024-02-22T15:50:51.933 INFO:tasks.workunit.client.1.smithi062.stdout:7/667: write d0/d6/f93 [862063,116616] 0 2024-02-22T15:50:51.938 INFO:tasks.workunit.client.1.smithi062.stdout:6/846: dwrite d5/dd/d33/d66/db9/fa6 [0,4194304] 0 2024-02-22T15:50:51.943 INFO:tasks.workunit.client.1.smithi062.stdout:9/921: dread d4/d1c/f53 [0,4194304] 0 2024-02-22T15:50:51.947 INFO:tasks.workunit.client.1.smithi062.stdout:7/668: creat d0/d6/d2e/fd0 x:0 0 0 2024-02-22T15:50:51.947 INFO:tasks.workunit.client.1.smithi062.stdout:4/972: rename d4/d1c/d30/c98 to d4/d2a/d2d/d46/d13b/c152 0 2024-02-22T15:50:51.948 INFO:tasks.workunit.client.1.smithi062.stdout:6/847: write d5/dd/d33/d66/fcc [760121,545] 0 2024-02-22T15:50:51.955 INFO:tasks.workunit.client.1.smithi062.stdout:1/750: dwrite d1/d6/d7/d2b/d47/d24/f9f [0,4194304] 0 2024-02-22T15:50:51.956 INFO:tasks.workunit.client.1.smithi062.stdout:9/922: dread - d4/d15/d5f/d75/d84/f88 zero size 2024-02-22T15:50:51.958 INFO:tasks.workunit.client.1.smithi062.stdout:0/987: rename d1/d13/dde/d76/d8c/d2c/db5/c110 to d1/d38/dfa/c155 0 2024-02-22T15:50:51.961 INFO:tasks.workunit.client.1.smithi062.stdout:3/865: dwrite d5/f6f [0,4194304] 0 2024-02-22T15:50:51.963 INFO:tasks.workunit.client.1.smithi062.stdout:6/848: chown d5/df/d40/d44/d76/d56/fca 255 1 2024-02-22T15:50:51.967 INFO:tasks.workunit.client.1.smithi062.stdout:1/751: chown d1/d6/de/f72 934 1 2024-02-22T15:50:51.969 INFO:tasks.workunit.client.1.smithi062.stdout:5/795: dwrite d0/d3/d2f/d59/d67/fd8 [0,4194304] 0 2024-02-22T15:50:51.970 INFO:tasks.workunit.client.1.smithi062.stdout:1/752: write d1/f27 [68960,18688] 0 2024-02-22T15:50:51.972 INFO:tasks.workunit.client.1.smithi062.stdout:7/669: dwrite d0/d14/d13/f2a [4194304,4194304] 0 2024-02-22T15:50:51.973 INFO:tasks.workunit.client.1.smithi062.stdout:3/866: write d5/d16/d21/f29 [868288,69725] 0 2024-02-22T15:50:51.974 INFO:tasks.workunit.client.1.smithi062.stdout:9/923: link d4/f14 d4/d1c/d83/d87/f12d 0 2024-02-22T15:50:51.975 INFO:tasks.workunit.client.1.smithi062.stdout:1/753: chown d1/d6/d7/d2b/d47/f52 454744 1 2024-02-22T15:50:51.975 INFO:tasks.workunit.client.1.smithi062.stdout:6/849: chown d5/dd/d33/d66/db9/l9b 110 1 2024-02-22T15:50:51.976 INFO:tasks.workunit.client.1.smithi062.stdout:5/796: readlink d0/d3/da/l1e 0 2024-02-22T15:50:51.979 INFO:tasks.workunit.client.1.smithi062.stdout:0/988: symlink d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dce/dfd/l156 0 2024-02-22T15:50:51.980 INFO:tasks.workunit.client.1.smithi062.stdout:5/797: chown d0/d3/da/d32/d42/fe6 127 1 2024-02-22T15:50:51.981 INFO:tasks.workunit.client.1.smithi062.stdout:7/670: unlink d0/d14/f67 0 2024-02-22T15:50:51.982 INFO:tasks.workunit.client.1.smithi062.stdout:1/754: mkdir d1/d6/d36/d76/dfe 0 2024-02-22T15:50:51.985 INFO:tasks.workunit.client.1.smithi062.stdout:9/924: creat d4/d1c/d118/f12e x:0 0 0 2024-02-22T15:50:51.987 INFO:tasks.workunit.client.1.smithi062.stdout:0/989: truncate d1/d13/dde/d76/fe2 417430 0 2024-02-22T15:50:51.987 INFO:tasks.workunit.client.1.smithi062.stdout:5/798: symlink d0/l107 0 2024-02-22T15:50:51.990 INFO:tasks.workunit.client.1.smithi062.stdout:3/867: dwrite d5/d1a/d3b/f114 [0,4194304] 0 2024-02-22T15:50:51.990 INFO:tasks.workunit.client.1.smithi062.stdout:6/850: creat d5/df/d40/d44/f11d x:0 0 0 2024-02-22T15:50:51.992 INFO:tasks.workunit.client.1.smithi062.stdout:9/925: creat d4/d1c/d83/f12f x:0 0 0 2024-02-22T15:50:51.997 INFO:tasks.workunit.client.1.smithi062.stdout:1/755: mkdir d1/d6/d7/d2b/d47/dff 0 2024-02-22T15:50:51.998 INFO:tasks.workunit.client.1.smithi062.stdout:3/868: symlink d5/d16/d21/l117 0 2024-02-22T15:50:52.001 INFO:tasks.workunit.client.1.smithi062.stdout:3/869: stat d5/d1a/d3b/dc1/ld2 0 2024-02-22T15:50:52.001 INFO:tasks.workunit.client.1.smithi062.stdout:4/973: dread d4/d2a/d2d/d2e/d6a/d9e/da4/db6/f129 [0,4194304] 0 2024-02-22T15:50:52.001 INFO:tasks.workunit.client.1.smithi062.stdout:5/799: rename d0/d3/da/d13/d23/f3b to d0/d3/da/d32/d4f/df3/f108 0 2024-02-22T15:50:52.002 INFO:tasks.workunit.client.1.smithi062.stdout:6/851: symlink d5/df/d32/l11e 0 2024-02-22T15:50:52.004 INFO:tasks.workunit.client.1.smithi062.stdout:1/756: truncate d1/d6/f1a 412733 0 2024-02-22T15:50:52.006 INFO:tasks.workunit.client.1.smithi062.stdout:0/990: mkdir d1/d13/dde/d76/d8c/d5d/dc3/d85/d8d/dd7/d104/d157 0 2024-02-22T15:50:52.011 INFO:tasks.workunit.client.1.smithi062.stdout:5/800: fdatasync d0/d3/da/d13/d23/d4e/d6d/f80 0 2024-02-22T15:50:52.016 INFO:tasks.workunit.client.1.smithi062.stdout:3/870: truncate d5/f8a 1931716 0 2024-02-22T15:50:52.020 INFO:tasks.workunit.client.1.smithi062.stdout:9/926: dwrite d4/f10 [0,4194304] 0 2024-02-22T15:50:52.023 INFO:tasks.workunit.client.1.smithi062.stdout:6/852: truncate d5/df/d40/d44/d76/d56/f98 5019150 0 2024-02-22T15:50:52.023 INFO:tasks.workunit.client.1.smithi062.stdout:4/974: dwrite d4/d2a/d2d/f116 [0,4194304] 0 2024-02-22T15:50:52.026 INFO:tasks.workunit.client.1.smithi062.stdout:7/671: dread d0/d14/d13/f36 [4194304,4194304] 0 2024-02-22T15:50:52.026 INFO:tasks.workunit.client.1.smithi062.stdout:5/801: symlink d0/d3/da/d13/d23/d4e/d6d/l109 0 2024-02-22T15:50:52.029 INFO:tasks.workunit.client.1.smithi062.stdout:9/927: creat d4/d1c/d118/f130 x:0 0 0 2024-02-22T15:50:52.032 INFO:tasks.workunit.client.1.smithi062.stdout:1/757: dwrite d1/d6/d7/fda [4194304,4194304] 0 2024-02-22T15:50:52.032 INFO:tasks.workunit.client.1.smithi062.stdout:3/871: unlink d5/d16/d21/d32/d9e/dc2/d113/dab/cea 0 2024-02-22T15:50:52.034 INFO:tasks.workunit.client.1.smithi062.stdout:4/975: chown d4/d2a/d2d/d64 15536 1 2024-02-22T15:50:52.038 INFO:tasks.workunit.client.1.smithi062.stdout:0/991: link d1/d3/d4/d1e/d74/d122/f132 d1/d13/dde/d76/d8c/d5d/dc1/f158 0 2024-02-22T15:50:52.040 INFO:tasks.workunit.client.1.smithi062.stdout:9/928: symlink d4/d15/d5e/d101/l131 0 2024-02-22T15:50:52.045 INFO:tasks.workunit.client.1.smithi062.stdout:1/758: mkdir d1/d6/de/d9/d100 0 2024-02-22T15:50:52.046 INFO:tasks.workunit.client.1.smithi062.stdout:7/672: truncate d0/d14/f29 1943321 0 2024-02-22T15:50:52.049 INFO:tasks.workunit.client.1.smithi062.stdout:3/872: dread d5/d16/d21/d32/d6a/d7b/fcc [0,4194304] 0 2024-02-22T15:50:52.052 INFO:tasks.workunit.client.1.smithi062.stdout:4/976: mknod d4/d1c/d2c/d6b/d70/c153 0 2024-02-22T15:50:52.052 INFO:tasks.workunit.client.1.smithi062.stdout:6/853: creat d5/df/d40/dac/d11a/f11f x:0 0 0 2024-02-22T15:50:52.052 INFO:tasks.workunit.client.1.smithi062.stdout:0/992: rmdir d1/d3 39 2024-02-22T15:50:52.053 INFO:tasks.workunit.client.1.smithi062.stdout:5/802: symlink d0/d3/da/d13/d23/d78/d8c/dbd/dc9/l10a 0 2024-02-22T15:50:52.053 INFO:tasks.workunit.client.1.smithi062.stdout:9/929: symlink d4/d28/d50/d107/l132 0 2024-02-22T15:50:52.053 INFO:tasks.workunit.client.1.smithi062.stdout:7/673: mknod d0/d14/d13/cd1 0 2024-02-22T15:50:52.054 INFO:tasks.workunit.client.1.smithi062.stdout:3/873: mkdir d5/d16/d21/d96/d118 0 2024-02-22T15:50:52.055 INFO:tasks.workunit.client.1.smithi062.stdout:9/930: stat d4/d15/d5e/d63 0 2024-02-22T15:50:52.056 INFO:tasks.workunit.client.1.smithi062.stdout:7/674: write d0/f10 [751211,23608] 0 2024-02-22T15:50:52.060 INFO:tasks.workunit.client.1.smithi062.stdout:5/803: creat d0/d3/da/d32/d4f/df3/f10b x:0 0 0 2024-02-22T15:50:52.062 INFO:tasks.workunit.client.1.smithi062.stdout:9/931: symlink d4/d28/d50/d5a/l133 0 2024-02-22T15:50:52.065 INFO:tasks.workunit.client.1.smithi062.stdout:3/874: creat d5/d16/d21/d32/d9e/f119 x:0 0 0 2024-02-22T15:50:52.066 INFO:tasks.workunit.client.1.smithi062.stdout:9/932: dread - d4/d1c/d83/ff9 zero size 2024-02-22T15:50:52.067 INFO:tasks.workunit.client.1.smithi062.stdout:5/804: write d0/d3/da/d13/d23/d31/fba [365577,11708] 0 2024-02-22T15:50:52.067 INFO:tasks.workunit.client.1.smithi062.stdout:7/675: rmdir d0/d14/d19/d30/d45/d6a 0 2024-02-22T15:50:52.068 INFO:tasks.workunit.client.1.smithi062.stdout:6/854: dwrite d5/df/d40/d44/f11d [0,4194304] 0 2024-02-22T15:50:52.070 INFO:tasks.workunit.client.1.smithi062.stdout:5/805: chown d0/d3/d30 497122565 1 2024-02-22T15:50:52.072 INFO:tasks.workunit.client.1.smithi062.stdout:6/855: stat d5/dd/d33/d71/c10f 0 2024-02-22T15:50:52.074 INFO:tasks.workunit.client.1.smithi062.stdout:3/875: write d5/d16/d21/d36/d5a/d85/f93 [841856,69871] 0 2024-02-22T15:50:52.079 INFO:tasks.workunit.client.1.smithi062.stdout:5/806: creat d0/d3/da/d13/d23/d4e/d60/f10c x:0 0 0 2024-02-22T15:50:52.081 INFO:tasks.workunit.client.1.smithi062.stdout:1/759: truncate d1/d6/de/f31 2425937 0 2024-02-22T15:50:52.081 INFO:tasks.workunit.client.1.smithi062.stdout:3/876: readlink d5/d16/d21/l6e 0 2024-02-22T15:50:52.084 INFO:tasks.workunit.client.1.smithi062.stdout:5/807: chown d0/d3/da/d13/f3d 6 1 2024-02-22T15:50:52.087 INFO:tasks.workunit.client.1.smithi062.stdout:6/856: mknod d5/df/d40/d44/c120 0 2024-02-22T15:50:52.089 INFO:tasks.workunit.client.1.smithi062.stdout:3/877: symlink d5/d16/d21/d84/l11a 0 2024-02-22T15:50:52.093 INFO:tasks.workunit.client.1.smithi062.stdout:9/933: dwrite d4/d15/fe7 [0,4194304] 0 2024-02-22T15:50:52.102 INFO:tasks.workunit.client.1.smithi062.stdout:4/977: dwrite d4/df/fbb [0,4194304] 0 2024-02-22T15:50:52.102 INFO:tasks.workunit.client.1.smithi062.stdout:4/978: write d4/f53 [2315014,31625] 0 2024-02-22T15:50:52.108 INFO:tasks.workunit.client.1.smithi062.stdout:7/676: dread d0/d6/d2e/d58/f5c [0,4194304] 0 2024-02-22T15:50:52.113 INFO:tasks.workunit.client.1.smithi062.stdout:6/857: link d5/dd/f20 d5/df/d40/dac/db6/dc0/f121 0 2024-02-22T15:50:52.113 INFO:tasks.workunit.client.1.smithi062.stdout:3/878: read d5/d16/d21/d36/d5a/d85/fb2 [1562738,52201] 0 2024-02-22T15:50:52.115 INFO:tasks.workunit.client.1.smithi062.stdout:5/808: link d0/d3/da/d13/d57/dc7/dda/fef d0/d3/da/d13/d23/d4e/d60/db5/de1/f10d 0 2024-02-22T15:50:52.120 INFO:tasks.workunit.client.1.smithi062.stdout:1/760: dwrite d1/d6/d7/d2b/d47/d7e/fc5 [0,4194304] 0 2024-02-22T15:50:52.120 INFO:tasks.workunit.client.1.smithi062.stdout:7/677: fsync d0/d6/d2e/d55/f81 0 2024-02-22T15:50:52.121 INFO:tasks.workunit.client.1.smithi062.stdout:9/934: getdents d4/d1c/d83 0 2024-02-22T15:50:52.126 INFO:tasks.workunit.client.1.smithi062.stdout:9/935: chown d4/d15/d5e/d63/da0 971623 1 2024-02-22T15:50:52.127 INFO:tasks.workunit.client.1.smithi062.stdout:6/858: mknod d5/dd/d33/c122 0 2024-02-22T15:50:52.127 INFO:tasks.workunit.client.1.smithi062.stdout:3/879: creat d5/d16/d21/d32/d75/f11b x:0 0 0 2024-02-22T15:50:52.129 INFO:tasks.workunit.client.1.smithi062.stdout:9/936: write d4/d15/d5f/d75/d84/f8b [3005653,77758] 0 2024-02-22T15:50:52.129 INFO:tasks.workunit.client.1.smithi062.stdout:3/880: read - d5/d16/d21/d32/d6a/fac zero size 2024-02-22T15:50:52.132 INFO:tasks.workunit.client.1.smithi062.stdout:3/881: write d5/d16/d21/d96/fe1 [365174,102178] 0 2024-02-22T15:50:52.132 INFO:tasks.workunit.client.1.smithi062.stdout:7/678: chown d0/d6/d2e/d55/d72/fb2 149631 1 2024-02-22T15:50:52.132 INFO:tasks.workunit.client.1.smithi062.stdout:1/761: write d1/d6/d7/d2b/d96/df3/fc6 [203418,101761] 0 2024-02-22T15:50:52.135 INFO:tasks.workunit.client.1.smithi062.stdout:4/979: rename d4/d1c/d2c/d6b/d9b to d4/d1c/d2c/d6b/d154 0 2024-02-22T15:50:52.136 INFO:tasks.workunit.client.1.smithi062.stdout:5/809: unlink d0/d3/da/d13/d57/fbf 0 2024-02-22T15:50:52.139 INFO:tasks.workunit.client.1.smithi062.stdout:6/859: read - d5/df/d40/d44/d76/dae/fde zero size 2024-02-22T15:50:52.141 INFO:tasks.workunit.client.1.smithi062.stdout:9/937: dwrite d4/d15/fe7 [0,4194304] 0 2024-02-22T15:50:52.143 INFO:tasks.workunit.client.1.smithi062.stdout:4/980: creat d4/d2a/d75/f155 x:0 0 0 2024-02-22T15:50:52.143 INFO:tasks.workunit.client.1.smithi062.stdout:5/810: unlink d0/d3/da/d13/d23/d4e/d6d/f80 0 2024-02-22T15:50:52.144 INFO:tasks.workunit.client.1.smithi062.stdout:7/679: read d0/d6/f50 [259392,10515] 0 2024-02-22T15:50:52.144 INFO:tasks.workunit.client.1.smithi062.stdout:6/860: stat d5/dd/d33/fb7 0 2024-02-22T15:50:52.148 INFO:tasks.workunit.client.1.smithi062.stdout:4/981: chown d4/d13/d20/l31 1035513385 1 2024-02-22T15:50:52.150 INFO:tasks.workunit.client.1.smithi062.stdout:9/938: truncate d4/d15/d5e/d63/f86 1885845 0 2024-02-22T15:50:52.151 INFO:tasks.workunit.client.1.smithi062.stdout:5/811: dread - d0/d3/da/d13/d23/d78/d8c/dbd/fc6 zero size 2024-02-22T15:50:52.152 INFO:tasks.workunit.client.1.smithi062.stdout:7/680: symlink d0/d6/d2e/d4a/d5a/d5e/ld2 0 2024-02-22T15:50:52.153 INFO:tasks.workunit.client.1.smithi062.stdout:4/982: mkdir d4/d1c/d30/d149/d156 0 2024-02-22T15:50:52.157 INFO:tasks.workunit.client.1.smithi062.stdout:5/812: stat d0/d3/d2f/d59/d67/led 0 2024-02-22T15:50:52.158 INFO:tasks.workunit.client.1.smithi062.stdout:7/681: mknod d0/d6/d5b/cd3 0 2024-02-22T15:50:52.213 INFO:tasks.workunit.client.1.smithi062.stdout:1/762: dwrite d1/d6/d36/d37/f8e [0,4194304] 0 2024-02-22T15:50:52.214 INFO:tasks.workunit.client.1.smithi062.stdout:5/813: fsync d0/d3/da/d32/d33/dbc/fc4 0 2024-02-22T15:50:52.219 INFO:tasks.workunit.client.1.smithi062.stdout:4/983: dwrite d4/d1c/fe4 [0,4194304] 0 2024-02-22T15:50:52.220 INFO:tasks.workunit.client.1.smithi062.stdout:1/763: creat d1/f101 x:0 0 0 2024-02-22T15:50:52.227 INFO:tasks.workunit.client.1.smithi062.stdout:6/861: dwrite d5/df/f116 [0,4194304] 0 2024-02-22T15:50:52.227 INFO:tasks.workunit.client.1.smithi062.stdout:5/814: chown d0/d3/da/d13/d57/dc7/dda/fef 102 1 2024-02-22T15:50:52.229 INFO:tasks.workunit.client.1.smithi062.stdout:9/939: dwrite d4/d15/d5e/d63/da0/d92/db6/fc3 [0,4194304] 0 2024-02-22T15:50:52.229 INFO:tasks.workunit.client.1.smithi062.stdout:4/984: mkdir d4/d2a/d2d/d46/d13b/d157 0 2024-02-22T15:50:52.229 INFO:tasks.workunit.client.1.smithi062.stdout:1/764: chown d1/d6/d7/d2b/d47/d7e/cac 158168 1 2024-02-22T15:50:52.238 INFO:tasks.workunit.client.1.smithi062.stdout:7/682: dwrite d0/d91/d7c/fc7 [0,4194304] 0 2024-02-22T15:50:52.238 INFO:tasks.workunit.client.1.smithi062.stdout:1/765: chown d1/f82 0 1 2024-02-22T15:50:52.238 INFO:tasks.workunit.client.1.smithi062.stdout:9/940: chown d4/d15/d5e/d63/da0/fae 14973 1 2024-02-22T15:50:52.243 INFO:tasks.workunit.client.1.smithi062.stdout:9/941: chown d4/d15/d5e/d63/da0/d92/le0 67 1 2024-02-22T15:50:52.243 INFO:tasks.workunit.client.1.smithi062.stdout:7/683: rename d0/d14/d13/cd1 to d0/d6/d2e/d35/cd4 0 2024-02-22T15:50:52.243 INFO:tasks.workunit.client.1.smithi062.stdout:9/942: stat d4/d28/d50/d11e 0 2024-02-22T15:50:52.243 INFO:tasks.workunit.client.1.smithi062.stdout:6/862: creat d5/df/d40/d44/f123 x:0 0 0 2024-02-22T15:50:52.249 INFO:tasks.workunit.client.1.smithi062.stdout:4/985: link d4/d2a/d2d/d46/d13b/c152 d4/d1c/d2c/d6b/d70/dd6/c158 0 2024-02-22T15:50:52.249 INFO:tasks.workunit.client.1.smithi062.stdout:5/815: dread d0/d3/da/d13/d57/f100 [0,4194304] 0 2024-02-22T15:50:52.250 INFO:tasks.workunit.client.1.smithi062.stdout:6/863: fdatasync d5/df/d32/f34 0 2024-02-22T15:50:52.250 INFO:tasks.workunit.client.1.smithi062.stdout:9/943: symlink d4/d28/d50/d107/l134 0 2024-02-22T15:50:52.253 INFO:tasks.workunit.client.1.smithi062.stdout:1/766: dread d1/d6/d7/d4a/d65/d83/f8b [0,4194304] 0 2024-02-22T15:50:52.256 INFO:tasks.workunit.client.1.smithi062.stdout:5/816: creat d0/d3/da/d13/d57/dcc/f10e x:0 0 0 2024-02-22T15:50:52.256 INFO:tasks.workunit.client.1.smithi062.stdout:1/767: chown d1/d6/d7/f7a 59294760 1 2024-02-22T15:50:52.257 INFO:tasks.workunit.client.1.smithi062.stdout:9/944: read d4/f7d [531748,75104] 0 2024-02-22T15:50:52.259 INFO:tasks.workunit.client.1.smithi062.stdout:7/684: dwrite d0/d6/d2e/d55/d80/d8a/f90 [0,4194304] 0 2024-02-22T15:50:52.260 INFO:tasks.workunit.client.1.smithi062.stdout:6/864: rename d5/df/d40/d44/d76/d56/f77 to d5/df/d40/f124 0 2024-02-22T15:50:52.263 INFO:tasks.workunit.client.1.smithi062.stdout:1/768: creat d1/d6/d7/d2b/d49/da9/f102 x:0 0 0 2024-02-22T15:50:52.264 INFO:tasks.workunit.client.1.smithi062.stdout:5/817: creat d0/d3/da/d13/d23/d78/d8c/f10f x:0 0 0 2024-02-22T15:50:52.269 INFO:tasks.workunit.client.1.smithi062.stdout:6/865: rename d5/d8a/ddd/de6/df1/l110 to d5/d6e/l125 0 2024-02-22T15:50:52.271 INFO:tasks.workunit.client.1.smithi062.stdout:1/769: write d1/d6/f61 [546732,113887] 0 2024-02-22T15:50:52.273 INFO:tasks.workunit.client.1.smithi062.stdout:7/685: dwrite d0/d14/d13/f2a [4194304,4194304] 0 2024-02-22T15:50:52.273 INFO:tasks.workunit.client.1.smithi062.stdout:5/818: mknod d0/d3/da/d13/d23/d78/d8c/dbd/c110 0 2024-02-22T15:50:52.276 INFO:tasks.workunit.client.1.smithi062.stdout:9/945: dwrite d4/d15/d5e/d63/da0/fae [0,4194304] 0 2024-02-22T15:50:52.278 INFO:tasks.workunit.client.1.smithi062.stdout:5/819: write d0/d3/da/d32/d4f/d72/df4/d3f/d5f/fdb [352034,101367] 0 2024-02-22T15:50:52.286 INFO:tasks.workunit.client.1.smithi062.stdout:5/820: chown d0/d3/da/d32/d4f/d72/df4/d3f/d66/da9/lbe 26 1 2024-02-22T15:50:52.286 INFO:tasks.workunit.client.1.smithi062.stdout:1/770: rmdir d1/d6/d7/d2b/d49/d74/dad 39 2024-02-22T15:50:52.289 INFO:tasks.workunit.client.1.smithi062.stdout:7/686: rmdir d0/d14/d19/d30/dbd 39 2024-02-22T15:50:52.289 INFO:tasks.workunit.client.1.smithi062.stdout:9/946: unlink d4/d15/d5e/d101/l131 0 2024-02-22T15:50:52.289 INFO:tasks.workunit.client.1.smithi062.stdout:7/687: chown d0/d91/d7c/fb8 0 1 2024-02-22T15:50:52.290 INFO:tasks.workunit.client.1.smithi062.stdout:5/821: rmdir d0/d3/da/d13/d57/dc7 39 2024-02-22T15:50:52.291 INFO:tasks.workunit.client.1.smithi062.stdout:4/986: truncate d4/d1c/d2c/d6b/d70/dd6/fa2 2908942 0 2024-02-22T15:50:52.292 INFO:tasks.workunit.client.1.smithi062.stdout:1/771: truncate d1/d6/d10/f1c 5075048 0 2024-02-22T15:50:52.294 INFO:tasks.workunit.client.1.smithi062.stdout:7/688: fsync d0/d14/f20 0 2024-02-22T15:50:52.295 INFO:tasks.workunit.client.1.smithi062.stdout:9/947: creat d4/d1c/d83/d87/d11f/f135 x:0 0 0 2024-02-22T15:50:52.296 INFO:tasks.workunit.client.1.smithi062.stdout:9/948: write d4/d1c/f56 [9407623,107293] 0 2024-02-22T15:50:52.303 INFO:tasks.workunit.client.1.smithi062.stdout:1/772: unlink d1/d6/d36/d37/f8e 0 2024-02-22T15:50:52.306 INFO:tasks.workunit.client.1.smithi062.stdout:1/773: dread - d1/d6/d7/fef zero size 2024-02-22T15:50:52.308 INFO:tasks.workunit.client.1.smithi062.stdout:9/949: dwrite d4/f26 [0,4194304] 0 2024-02-22T15:50:52.312 INFO:tasks.workunit.client.1.smithi062.stdout:4/987: write d4/df/fda [229982,27991] 0 2024-02-22T15:50:52.316 INFO:tasks.workunit.client.1.smithi062.stdout:6/866: dwrite d5/df/d40/d44/d4a/d41/f5f [0,4194304] 0 2024-02-22T15:50:52.322 INFO:tasks.workunit.client.1.smithi062.stdout:1/774: getdents d1/d6/de/d9/d100 0 2024-02-22T15:50:52.324 INFO:tasks.workunit.client.1.smithi062.stdout:9/950: symlink d4/d15/d5f/d123/l136 0 2024-02-22T15:50:52.324 INFO:tasks.workunit.client.1.smithi062.stdout:4/988: read d4/d1c/d2c/d6b/fca [108374,83425] 0 2024-02-22T15:50:52.333 INFO:tasks.workunit.client.1.smithi062.stdout:5/822: dwrite d0/d3/da/d32/d4f/f50 [0,4194304] 0 2024-02-22T15:50:52.333 INFO:tasks.workunit.client.1.smithi062.stdout:7/689: dwrite d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:52.338 INFO:tasks.workunit.client.1.smithi062.stdout:1/775: mkdir d1/d6/d7/d4a/d65/d103 0 2024-02-22T15:50:52.340 INFO:tasks.workunit.client.1.smithi062.stdout:4/989: dread d4/f51 [0,4194304] 0 2024-02-22T15:50:52.341 INFO:tasks.workunit.client.1.smithi062.stdout:6/867: dread d5/df/d32/d47/f50 [4194304,4194304] 0 2024-02-22T15:50:52.341 INFO:tasks.workunit.client.1.smithi062.stdout:5/823: rename d0/d3/d2f/f3a to d0/d3/da/d13/d23/d31/f111 0 2024-02-22T15:50:52.342 INFO:tasks.workunit.client.1.smithi062.stdout:7/690: mknod d0/d6/d2e/d4a/d5a/cd5 0 2024-02-22T15:50:52.345 INFO:tasks.workunit.client.1.smithi062.stdout:7/691: chown d0/d6/d2e/d55/d80/fb9 6883762 1 2024-02-22T15:50:52.346 INFO:tasks.workunit.client.1.smithi062.stdout:5/824: fdatasync d0/d3/da/d13/d23/d78/d8c/dbd/fc6 0 2024-02-22T15:50:52.347 INFO:tasks.workunit.client.1.smithi062.stdout:4/990: symlink d4/d1c/dac/l159 0 2024-02-22T15:50:52.347 INFO:tasks.workunit.client.1.smithi062.stdout:9/951: fdatasync d4/f14 0 2024-02-22T15:50:52.349 INFO:tasks.workunit.client.1.smithi062.stdout:5/825: read d0/fa8 [434362,1393] 0 2024-02-22T15:50:52.352 INFO:tasks.workunit.client.1.smithi062.stdout:4/991: stat d4/d1c/d2c/d6b/d70/dd6 0 2024-02-22T15:50:52.353 INFO:tasks.workunit.client.1.smithi062.stdout:6/868: link d5/dd/d33/l75 d5/d8a/ddd/de6/df1/l126 0 2024-02-22T15:50:52.355 INFO:tasks.workunit.client.1.smithi062.stdout:7/692: creat d0/d14/d19/d30/d45/dc3/dcb/fd6 x:0 0 0 2024-02-22T15:50:52.360 INFO:tasks.workunit.client.1.smithi062.stdout:1/776: dwrite d1/d6/d10/f4b [0,4194304] 0 2024-02-22T15:50:52.360 INFO:tasks.workunit.client.1.smithi062.stdout:1/777: readlink d1/d6/de/d9/l41 0 2024-02-22T15:50:52.362 INFO:tasks.workunit.client.1.smithi062.stdout:4/992: read d4/d1c/faa [4617135,86310] 0 2024-02-22T15:50:52.367 INFO:tasks.workunit.client.1.smithi062.stdout:4/993: creat d4/d1c/d30/da7/f15a x:0 0 0 2024-02-22T15:50:52.372 INFO:tasks.workunit.client.1.smithi062.stdout:1/778: link d1/d6/d7/d2b/d49/dcc/cf1 d1/d6/d36/d76/dfe/c104 0 2024-02-22T15:50:52.374 INFO:tasks.workunit.client.1.smithi062.stdout:4/994: getdents d4/d1c/d2c/d6b/d154 0 2024-02-22T15:50:52.382 INFO:tasks.workunit.client.1.smithi062.stdout:9/952: dwrite d4/d15/d5f/d75/f89 [0,4194304] 0 2024-02-22T15:50:52.383 INFO:tasks.workunit.client.1.smithi062.stdout:6/869: dwrite d5/df/d40/d44/d4a/d41/f4d [0,4194304] 0 2024-02-22T15:50:52.385 INFO:tasks.workunit.client.1.smithi062.stdout:4/995: symlink d4/d2a/d2d/d2e/d6a/d8a/ddb/l15b 0 2024-02-22T15:50:52.387 INFO:tasks.workunit.client.1.smithi062.stdout:5/826: dread d0/d3/da/d13/d23/d4e/d5a/fcf [0,4194304] 0 2024-02-22T15:50:52.389 INFO:tasks.workunit.client.1.smithi062.stdout:5/827: chown d0/d3/da/d13/d23/d78/d8c/dbd/dc9 24 1 2024-02-22T15:50:52.389 INFO:tasks.workunit.client.1.smithi062.stdout:9/953: write d4/d15/d5f/fa2 [1667736,74189] 0 2024-02-22T15:50:52.391 INFO:tasks.workunit.client.1.smithi062.stdout:6/870: chown d5/df/d32/c4c 439 1 2024-02-22T15:50:52.391 INFO:tasks.workunit.client.1.smithi062.stdout:4/996: symlink d4/d2a/d2d/d2e/d6a/d94/dfe/l15c 0 2024-02-22T15:50:52.394 INFO:tasks.workunit.client.1.smithi062.stdout:9/954: unlink d4/d28/lf8 0 2024-02-22T15:50:52.396 INFO:tasks.workunit.client.1.smithi062.stdout:6/871: symlink d5/df/dbc/dce/l127 0 2024-02-22T15:50:52.397 INFO:tasks.workunit.client.1.smithi062.stdout:9/955: read d4/d28/d50/fbd [142066,99970] 0 2024-02-22T15:50:52.400 INFO:tasks.workunit.client.1.smithi062.stdout:1/779: dread d1/f44 [0,4194304] 0 2024-02-22T15:50:52.402 INFO:tasks.workunit.client.1.smithi062.stdout:5/828: dread d0/d3/da/d13/d23/d31/fba [0,4194304] 0 2024-02-22T15:50:52.404 INFO:tasks.workunit.client.1.smithi062.stdout:1/780: fdatasync d1/d66/f75 0 2024-02-22T15:50:52.404 INFO:tasks.workunit.client.1.smithi062.stdout:6/872: fsync d5/dd/f20 0 2024-02-22T15:50:52.405 INFO:tasks.workunit.client.1.smithi062.stdout:5/829: mknod d0/d3/da/d32/d4f/c112 0 2024-02-22T15:50:52.406 INFO:tasks.workunit.client.1.smithi062.stdout:6/873: unlink d5/f73 0 2024-02-22T15:50:52.407 INFO:tasks.workunit.client.1.smithi062.stdout:5/830: dread d0/fdf [0,4194304] 0 2024-02-22T15:50:52.410 INFO:tasks.workunit.client.1.smithi062.stdout:1/781: mkdir d1/d6/d7/d2b/d49/d105 0 2024-02-22T15:50:52.412 INFO:tasks.workunit.client.1.smithi062.stdout:5/831: mknod d0/d3/da/d32/d33/dbc/c113 0 2024-02-22T15:50:52.413 INFO:tasks.workunit.client.1.smithi062.stdout:4/997: truncate d4/f3f 1778321 0 2024-02-22T15:50:52.414 INFO:tasks.workunit.client.1.smithi062.stdout:9/956: write d4/f14 [4085816,20824] 0 2024-02-22T15:50:52.415 INFO:tasks.workunit.client.1.smithi062.stdout:1/782: read d1/f32 [240536,47940] 0 2024-02-22T15:50:52.415 INFO:tasks.workunit.client.1.smithi062.stdout:5/832: write d0/d3/d30/d71/d7c/ff1 [253559,108603] 0 2024-02-22T15:50:52.420 INFO:tasks.workunit.client.1.smithi062.stdout:5/833: write d0/d3/da/d13/d57/d79/fa5 [1578291,70573] 0 2024-02-22T15:50:52.420 INFO:tasks.workunit.client.1.smithi062.stdout:6/874: write d5/dd/d33/d66/db9/f10d [816535,16263] 0 2024-02-22T15:50:52.421 INFO:tasks.workunit.client.1.smithi062.stdout:9/957: creat d4/d28/d50/d5a/f137 x:0 0 0 2024-02-22T15:50:52.422 INFO:tasks.workunit.client.1.smithi062.stdout:4/998: symlink d4/d1c/d30/d149/d156/l15d 0 2024-02-22T15:50:52.423 INFO:tasks.workunit.client.1.smithi062.stdout:4/999: readlink d4/d34/l73 0 2024-02-22T15:50:52.424 INFO:tasks.workunit.client.1.smithi062.stdout:6/875: write d5/df/d40/d44/d4a/d41/f4d [4974092,105759] 0 2024-02-22T15:50:52.424 INFO:tasks.workunit.client.1.smithi062.stdout:1/783: rename d1/d6/d7/d2b/d96/fc9 to d1/d6/f106 0 2024-02-22T15:50:52.428 INFO:tasks.workunit.client.1.smithi062.stdout:9/958: mknod d4/d15/d31/c138 0 2024-02-22T15:50:52.438 INFO:tasks.workunit.client.1.smithi062.stdout:9/959: symlink d4/d15/d5f/d75/d97/l139 0 2024-02-22T15:50:52.442 INFO:tasks.workunit.client.1.smithi062.stdout:5/834: rename d0/d3/da/d32/d4f/lf0 to d0/d3/da/d13/l114 0 2024-02-22T15:50:52.444 INFO:tasks.workunit.client.1.smithi062.stdout:9/960: creat d4/d15/d5e/d101/f13a x:0 0 0 2024-02-22T15:50:52.448 INFO:tasks.workunit.client.1.smithi062.stdout:9/961: unlink d4/d15/d5f/d75/d84/leb 0 2024-02-22T15:50:52.449 INFO:tasks.workunit.client.1.smithi062.stdout:5/835: rmdir d0/d3/da/d32/d33/dff 0 2024-02-22T15:50:52.450 INFO:tasks.workunit.client.1.smithi062.stdout:6/876: write d5/df/d40/d44/fbf [1292414,69593] 0 2024-02-22T15:50:52.453 INFO:tasks.workunit.client.1.smithi062.stdout:5/836: rmdir d0/d3/da/d32/d4f/df3 39 2024-02-22T15:50:52.454 INFO:tasks.workunit.client.1.smithi062.stdout:5/837: dread - d0/d3/d30/d71/ffe zero size 2024-02-22T15:50:52.455 INFO:tasks.workunit.client.1.smithi062.stdout:6/877: rename d5/f6 to d5/dd/d33/d71/f128 0 2024-02-22T15:50:52.463 INFO:tasks.workunit.client.1.smithi062.stdout:5/838: creat d0/d3/d2f/d59/d67/f115 x:0 0 0 2024-02-22T15:50:52.468 INFO:tasks.workunit.client.1.smithi062.stdout:5/839: mknod d0/d3/da/d32/d4f/d72/df4/c116 0 2024-02-22T15:50:52.473 INFO:tasks.workunit.client.1.smithi062.stdout:9/962: dwrite d4/d28/d50/d5a/f9c [0,4194304] 0 2024-02-22T15:50:52.477 INFO:tasks.workunit.client.1.smithi062.stdout:9/963: fsync d4/d1c/d83/d87/f103 0 2024-02-22T15:50:52.481 INFO:tasks.workunit.client.1.smithi062.stdout:9/964: rmdir d4/d15 39 2024-02-22T15:50:52.482 INFO:tasks.workunit.client.1.smithi062.stdout:9/965: chown d4/d15/d5e/d63 15952 1 2024-02-22T15:50:52.485 INFO:tasks.workunit.client.1.smithi062.stdout:9/966: mkdir d4/d28/d50/d11e/d13b 0 2024-02-22T15:50:52.487 INFO:tasks.workunit.client.1.smithi062.stdout:0/993: sync 2024-02-22T15:50:52.487 INFO:tasks.workunit.client.1.smithi062.stdout:3/882: sync 2024-02-22T15:50:52.492 INFO:tasks.workunit.client.1.smithi062.stdout:3/883: rename d5/d16/d21/d36/l83 to d5/d16/d21/d32/d9e/dc2/d113/l11c 0 2024-02-22T15:50:52.494 INFO:tasks.workunit.client.1.smithi062.stdout:3/884: truncate d5/d26/f6c 5317578 0 2024-02-22T15:50:52.495 INFO:tasks.workunit.client.1.smithi062.stdout:5/840: dread d0/d3/da/d32/d4f/d72/f9a [0,4194304] 0 2024-02-22T15:50:52.501 INFO:tasks.workunit.client.1.smithi062.stdout:5/841: dread - d0/d3/da/d13/d57/dc7/dda/fef zero size 2024-02-22T15:50:52.502 INFO:tasks.workunit.client.1.smithi062.stdout:0/994: creat d1/d13/d43/f159 x:0 0 0 2024-02-22T15:50:52.513 INFO:tasks.workunit.client.1.smithi062.stdout:9/967: dwrite d4/d15/d5f/d75/fab [0,4194304] 0 2024-02-22T15:50:52.515 INFO:tasks.workunit.client.1.smithi062.stdout:3/885: dwrite d5/d16/d21/d32/f70 [0,4194304] 0 2024-02-22T15:50:52.515 INFO:tasks.workunit.client.1.smithi062.stdout:5/842: write d0/d3/da/d13/d23/d31/f111 [3635572,80105] 0 2024-02-22T15:50:52.528 INFO:tasks.workunit.client.1.smithi062.stdout:3/886: rename d5/d1a/d3c/f54 to d5/d16/d21/d55/f11d 0 2024-02-22T15:50:52.534 INFO:tasks.workunit.client.1.smithi062.stdout:5/843: creat d0/d3/da/d32/d4f/d72/df4/d3f/f117 x:0 0 0 2024-02-22T15:50:52.539 INFO:tasks.workunit.client.1.smithi062.stdout:9/968: truncate d4/d15/d5f/d75/d84/f3e 415348 0 2024-02-22T15:50:52.540 INFO:tasks.workunit.client.1.smithi062.stdout:5/844: fdatasync d0/d3/da/d32/d4f/d72/df4/d3f/d66/f97 0 2024-02-22T15:50:52.546 INFO:tasks.workunit.client.1.smithi062.stdout:3/887: dwrite d5/d16/d21/d36/d5a/d85/fb2 [0,4194304] 0 2024-02-22T15:50:52.546 INFO:tasks.workunit.client.1.smithi062.stdout:5/845: chown d0/d3/da/d13/d23/d4e/d5a/dd6/f102 6206 1 2024-02-22T15:50:52.547 INFO:tasks.workunit.client.1.smithi062.stdout:5/846: chown d0/d3/da/f96 7 1 2024-02-22T15:50:52.548 INFO:tasks.workunit.client.1.smithi062.stdout:9/969: mkdir d4/d15/d5e/d63/da0/d92/db6/d13c 0 2024-02-22T15:50:52.549 INFO:tasks.workunit.client.1.smithi062.stdout:0/995: dwrite d1/d13/dde/d76/d8c/d2c/d84/f2a [0,4194304] 0 2024-02-22T15:50:52.559 INFO:tasks.workunit.client.1.smithi062.stdout:3/888: truncate d5/da6/de3/f104 42186 0 2024-02-22T15:50:52.559 INFO:tasks.workunit.client.1.smithi062.stdout:5/847: creat d0/d3/da/d32/f118 x:0 0 0 2024-02-22T15:50:52.563 INFO:tasks.workunit.client.1.smithi062.stdout:9/970: dwrite d4/d28/d50/f122 [0,4194304] 0 2024-02-22T15:50:52.568 INFO:tasks.workunit.client.1.smithi062.stdout:5/848: creat d0/d3/da/d13/d57/dc7/f119 x:0 0 0 2024-02-22T15:50:52.568 INFO:tasks.workunit.client.1.smithi062.stdout:3/889: symlink d5/d16/d88/l11e 0 2024-02-22T15:50:52.571 INFO:tasks.workunit.client.1.smithi062.stdout:5/849: chown d0/d3/da/d13/d23/d4e/d60/f63 138636 1 2024-02-22T15:50:52.579 INFO:tasks.workunit.client.1.smithi062.stdout:5/850: fsync d0/d3/da/d13/d23/f24 0 2024-02-22T15:50:52.579 INFO:tasks.workunit.client.1.smithi062.stdout:9/971: rename d4/d15/d5f/d75/d97/led to d4/d15/d5e/d63/l13d 0 2024-02-22T15:50:52.579 INFO:tasks.workunit.client.1.smithi062.stdout:9/972: readlink d4/l18 0 2024-02-22T15:50:52.579 INFO:tasks.workunit.client.1.smithi062.stdout:9/973: fdatasync d4/d28/d50/fdf 0 2024-02-22T15:50:52.584 INFO:tasks.workunit.client.1.smithi062.stdout:0/996: rename d1/d13/dde/d76/d8c/d5d/f123 to d1/d3/d4/d1e/d74/d68/f15a 0 2024-02-22T15:50:52.584 INFO:tasks.workunit.client.1.smithi062.stdout:9/974: dread d4/d15/d5f/d75/fb8 [0,4194304] 0 2024-02-22T15:50:52.592 INFO:tasks.workunit.client.1.smithi062.stdout:9/975: dwrite d4/d1c/d83/fca [0,4194304] 0 2024-02-22T15:50:52.594 INFO:tasks.workunit.client.1.smithi062.stdout:5/851: creat d0/d3/da/df9/f11a x:0 0 0 2024-02-22T15:50:52.603 INFO:tasks.workunit.client.1.smithi062.stdout:9/976: rename d4/c79 to d4/d28/d50/da3/c13e 0 2024-02-22T15:50:52.606 INFO:tasks.workunit.client.1.smithi062.stdout:5/852: rmdir d0/d3/da/d13/d23/d4e/d5a 39 2024-02-22T15:50:52.606 INFO:tasks.workunit.client.1.smithi062.stdout:3/890: dwrite d5/d26/d2d/fb4 [0,4194304] 0 2024-02-22T15:50:52.608 INFO:tasks.workunit.client.1.smithi062.stdout:0/997: dwrite d1/d13/dde/d76/d8c/d5d/dc3/f24 [8388608,4194304] 0 2024-02-22T15:50:52.612 INFO:tasks.workunit.client.1.smithi062.stdout:5/853: stat d0/d3/da/d13/d23/d78/d8c/dbd 0 2024-02-22T15:50:52.614 INFO:tasks.workunit.client.1.smithi062.stdout:3/891: fdatasync d5/d26/f38 0 2024-02-22T15:50:52.625 INFO:tasks.workunit.client.1.smithi062.stdout:5/854: dwrite d0/fdf [0,4194304] 0 2024-02-22T15:50:52.627 INFO:tasks.workunit.client.1.smithi062.stdout:5/855: rename d0 to d0/d3/da/d32/d33/dbc/d11b 22 2024-02-22T15:50:52.629 INFO:tasks.workunit.client.1.smithi062.stdout:9/977: dread d4/d15/d31/f60 [0,4194304] 0 2024-02-22T15:50:52.630 INFO:tasks.workunit.client.1.smithi062.stdout:5/856: fsync d0/d3/da/d13/d23/d31/fba 0 2024-02-22T15:50:52.632 INFO:tasks.workunit.client.1.smithi062.stdout:9/978: mknod d4/d28/d50/c13f 0 2024-02-22T15:50:52.633 INFO:tasks.workunit.client.1.smithi062.stdout:9/979: write d4/d35/fd8 [1625149,9000] 0 2024-02-22T15:50:52.641 INFO:tasks.workunit.client.1.smithi062.stdout:9/980: creat d4/d15/f140 x:0 0 0 2024-02-22T15:50:52.641 INFO:tasks.workunit.client.1.smithi062.stdout:9/981: read d4/d1c/fd1 [376961,14217] 0 2024-02-22T15:50:52.642 INFO:tasks.workunit.client.1.smithi062.stdout:9/982: chown d4/d15/d5f/d6a/c7f 709347 1 2024-02-22T15:50:52.644 INFO:tasks.workunit.client.1.smithi062.stdout:9/983: mknod d4/d15/dfd/c141 0 2024-02-22T15:50:52.644 INFO:tasks.workunit.client.1.smithi062.stdout:9/984: fdatasync d4/f1b 0 2024-02-22T15:50:52.647 INFO:tasks.workunit.client.1.smithi062.stdout:9/985: symlink d4/d15/d5e/d63/da0/d92/db6/l142 0 2024-02-22T15:50:52.650 INFO:tasks.workunit.client.1.smithi062.stdout:9/986: creat d4/d15/d5f/d75/d84/d11b/f143 x:0 0 0 2024-02-22T15:50:52.651 INFO:tasks.workunit.client.1.smithi062.stdout:3/892: dread d5/d16/f19 [0,4194304] 0 2024-02-22T15:50:52.657 INFO:tasks.workunit.client.1.smithi062.stdout:3/893: chown d5/d16/d21/d32/d9e/dc2/f103 0 1 2024-02-22T15:50:52.657 INFO:tasks.workunit.client.1.smithi062.stdout:9/987: rename d4/l11 to d4/d28/d50/d107/l144 0 2024-02-22T15:50:52.657 INFO:tasks.workunit.client.1.smithi062.stdout:0/998: dread d1/d38/f7b [0,4194304] 0 2024-02-22T15:50:52.657 INFO:tasks.workunit.client.1.smithi062.stdout:3/894: dread - d5/d26/d2d/d3f/fee zero size 2024-02-22T15:50:52.657 INFO:tasks.workunit.client.1.smithi062.stdout:5/857: fdatasync d0/fdf 0 2024-02-22T15:50:52.657 INFO:tasks.workunit.client.1.smithi062.stdout:5/858: chown d0/l28 315742 1 2024-02-22T15:50:52.658 INFO:tasks.workunit.client.1.smithi062.stdout:3/895: creat d5/da6/de3/f11f x:0 0 0 2024-02-22T15:50:52.658 INFO:tasks.workunit.client.1.smithi062.stdout:5/859: chown d0/d3/da/l92 2194 1 2024-02-22T15:50:52.661 INFO:tasks.workunit.client.1.smithi062.stdout:9/988: dread d4/d1c/d83/fdc [0,4194304] 0 2024-02-22T15:50:52.665 INFO:tasks.workunit.client.1.smithi062.stdout:9/989: chown d4/d28/d50/ff4 3925290 1 2024-02-22T15:50:52.666 INFO:tasks.workunit.client.1.smithi062.stdout:0/999: dwrite d1/d13/dde/d76/d8c/d5d/dc3/d85/d8e/fd2 [0,4194304] 0 2024-02-22T15:50:52.668 INFO:tasks.workunit.client.1.smithi062.stdout:9/990: dread - d4/d15/f10a zero size 2024-02-22T15:50:52.672 INFO:tasks.workunit.client.1.smithi062.stdout:9/991: mkdir d4/d28/d145 0 2024-02-22T15:50:52.673 INFO:tasks.workunit.client.1.smithi062.stdout:5/860: dwrite d0/d3/f14 [4194304,4194304] 0 2024-02-22T15:50:52.678 INFO:tasks.workunit.client.1.smithi062.stdout:3/896: dwrite d5/f8 [0,4194304] 0 2024-02-22T15:50:52.684 INFO:tasks.workunit.client.1.smithi062.stdout:9/992: write d4/d15/d5f/d75/d97/da7/fad [352274,79485] 0 2024-02-22T15:50:52.684 INFO:tasks.workunit.client.1.smithi062.stdout:3/897: write d5/d26/f35 [3113406,108228] 0 2024-02-22T15:50:52.686 INFO:tasks.workunit.client.1.smithi062.stdout:3/898: write d5/d26/d2d/fa1 [1591383,68315] 0 2024-02-22T15:50:52.693 INFO:tasks.workunit.client.1.smithi062.stdout:9/993: dwrite d4/d15/d31/f4e [0,4194304] 0 2024-02-22T15:50:52.698 INFO:tasks.workunit.client.1.smithi062.stdout:9/994: unlink d4/d15/d5f/d75/d97/cd4 0 2024-02-22T15:50:52.708 INFO:tasks.workunit.client.1.smithi062.stdout:9/995: mkdir d4/d28/d50/d107/d146 0 2024-02-22T15:50:52.714 INFO:tasks.workunit.client.1.smithi062.stdout:7/693: sync 2024-02-22T15:50:52.716 INFO:tasks.workunit.client.1.smithi062.stdout:3/899: dread d5/f8a [0,4194304] 0 2024-02-22T15:50:52.716 INFO:tasks.workunit.client.1.smithi062.stdout:7/694: read d0/d91/fce [2123007,93616] 0 2024-02-22T15:50:52.719 INFO:tasks.workunit.client.1.smithi062.stdout:3/900: fsync d5/f8 0 2024-02-22T15:50:52.719 INFO:tasks.workunit.client.1.smithi062.stdout:3/901: chown d5/c14 5118948 1 2024-02-22T15:50:52.722 INFO:tasks.workunit.client.1.smithi062.stdout:5/861: dread d0/d3/da/d13/d23/f24 [0,4194304] 0 2024-02-22T15:50:52.724 INFO:tasks.workunit.client.1.smithi062.stdout:3/902: dwrite d5/da6/de3/f11f [0,4194304] 0 2024-02-22T15:50:52.724 INFO:tasks.workunit.client.1.smithi062.stdout:5/862: write d0/d3/da/fe2 [1544645,26036] 0 2024-02-22T15:50:52.727 INFO:tasks.workunit.client.1.smithi062.stdout:5/863: readlink d0/d3/da/d32/d4f/d72/df4/d3f/d66/l6e 0 2024-02-22T15:50:52.740 INFO:tasks.workunit.client.1.smithi062.stdout:7/695: mknod d0/d6/d2e/d4a/cd7 0 2024-02-22T15:50:52.745 INFO:tasks.workunit.client.1.smithi062.stdout:7/696: symlink d0/d14/d19/d30/d45/dc3/ld8 0 2024-02-22T15:50:52.747 INFO:tasks.workunit.client.1.smithi062.stdout:3/903: dwrite d5/d16/d21/d32/d6a/d7b/fcc [4194304,4194304] 0 2024-02-22T15:50:52.749 INFO:tasks.workunit.client.1.smithi062.stdout:3/904: write d5/d26/f27 [809962,21760] 0 2024-02-22T15:50:52.751 INFO:tasks.workunit.client.1.smithi062.stdout:5/864: link d0/d3/da/d32/d4f/d72/df4/d3f/d66/da9/lbe d0/d3/da/d13/d23/d31/dac/l11c 0 2024-02-22T15:50:52.752 INFO:tasks.workunit.client.1.smithi062.stdout:3/905: write d5/d1a/f25 [2244032,122326] 0 2024-02-22T15:50:52.753 INFO:tasks.workunit.client.1.smithi062.stdout:5/865: write d0/d3/fb9 [587767,79581] 0 2024-02-22T15:50:52.754 INFO:tasks.workunit.client.1.smithi062.stdout:9/996: getdents d4/d1c/d118 0 2024-02-22T15:50:52.755 INFO:tasks.workunit.client.1.smithi062.stdout:7/697: mkdir d0/d14/d19/d77/dcf/dd9 0 2024-02-22T15:50:52.758 INFO:tasks.workunit.client.1.smithi062.stdout:5/866: write d0/d3/da/d13/d23/d4e/feb [504093,127481] 0 2024-02-22T15:50:52.758 INFO:tasks.workunit.client.1.smithi062.stdout:3/906: dread d5/d26/d2d/f90 [0,4194304] 0 2024-02-22T15:50:52.760 INFO:tasks.workunit.client.1.smithi062.stdout:9/997: creat d4/d15/d5e/d63/d104/f147 x:0 0 0 2024-02-22T15:50:52.767 INFO:tasks.workunit.client.1.smithi062.stdout:7/698: getdents d0/d14/d19/d30/d45 0 2024-02-22T15:50:52.768 INFO:tasks.workunit.client.1.smithi062.stdout:9/998: creat d4/f148 x:0 0 0 2024-02-22T15:50:52.769 INFO:tasks.workunit.client.1.smithi062.stdout:9/999: write d4/d1c/ff1 [1000054,111825] 0 2024-02-22T15:50:52.771 INFO:tasks.workunit.client.1.smithi062.stdout:5/867: rename d0/d3/da/d13/d57/dc7 to d0/d3/da/d13/d23/d78/d11d 0 2024-02-22T15:50:52.773 INFO:tasks.workunit.client.1.smithi062.stdout:7/699: creat d0/d14/d19/d30/d45/dc3/fda x:0 0 0 2024-02-22T15:50:52.774 INFO:tasks.workunit.client.1.smithi062.stdout:5/868: symlink d0/d3/d2f/d59/d91/l11e 0 2024-02-22T15:50:52.775 INFO:tasks.workunit.client.1.smithi062.stdout:5/869: chown d0/d3/da/f1d 739 1 2024-02-22T15:50:52.776 INFO:tasks.workunit.client.1.smithi062.stdout:7/700: creat d0/d6/d2e/d4a/d5a/fdb x:0 0 0 2024-02-22T15:50:52.779 INFO:tasks.workunit.client.1.smithi062.stdout:7/701: mkdir d0/d6/d2e/d35/ddc 0 2024-02-22T15:50:52.781 INFO:tasks.workunit.client.1.smithi062.stdout:7/702: symlink d0/d14/d13/ldd 0 2024-02-22T15:50:52.782 INFO:tasks.workunit.client.1.smithi062.stdout:5/870: getdents d0/d3/da/d32/d4f/d72/df4 0 2024-02-22T15:50:52.784 INFO:tasks.workunit.client.1.smithi062.stdout:7/703: mknod d0/d6/d2e/d55/d80/d8a/cde 0 2024-02-22T15:50:52.785 INFO:tasks.workunit.client.1.smithi062.stdout:5/871: creat d0/d3/da/d32/d4f/d72/df4/d3f/d5f/f11f x:0 0 0 2024-02-22T15:50:52.864 INFO:tasks.workunit.client.1.smithi062.stdout:7/704: rmdir d0 39 2024-02-22T15:50:52.864 INFO:tasks.workunit.client.1.smithi062.stdout:3/907: dwrite d5/d16/d88/f91 [0,4194304] 0 2024-02-22T15:50:52.867 INFO:tasks.workunit.client.1.smithi062.stdout:3/908: fdatasync d5/d16/d21/d55/f11d 0 2024-02-22T15:50:52.872 INFO:tasks.workunit.client.1.smithi062.stdout:7/705: link d0/d6/d2e/d4a/d5a/d5e/f6c d0/d6/d5b/d60/fdf 0 2024-02-22T15:50:52.875 INFO:tasks.workunit.client.1.smithi062.stdout:7/706: mkdir d0/d14/d19/d30/dc8/de0 0 2024-02-22T15:50:52.876 INFO:tasks.workunit.client.1.smithi062.stdout:7/707: stat d0/d6/d2e/d2f/ca5 0 2024-02-22T15:50:52.879 INFO:tasks.workunit.client.1.smithi062.stdout:3/909: dwrite d5/d26/f38 [0,4194304] 0 2024-02-22T15:50:52.879 INFO:tasks.workunit.client.1.smithi062.stdout:7/708: mknod d0/d6/d2e/d2f/d88/ce1 0 2024-02-22T15:50:52.880 INFO:tasks.workunit.client.1.smithi062.stdout:3/910: dread - d5/d1a/f10b zero size 2024-02-22T15:50:52.883 INFO:tasks.workunit.client.1.smithi062.stdout:7/709: creat d0/d6/d2e/d4a/d5a/db3/fe2 x:0 0 0 2024-02-22T15:50:52.884 INFO:tasks.workunit.client.1.smithi062.stdout:3/911: truncate d5/d16/d21/d32/f66 441908 0 2024-02-22T15:50:52.885 INFO:tasks.workunit.client.1.smithi062.stdout:3/912: chown d5/da6/de3/fe8 38364387 1 2024-02-22T15:50:52.886 INFO:tasks.workunit.client.1.smithi062.stdout:3/913: stat d5/d16/d21/d36/d5a/d85 0 2024-02-22T15:50:52.886 INFO:tasks.workunit.client.1.smithi062.stdout:7/710: symlink d0/d14/d19/d30/d45/dc3/dcb/le3 0 2024-02-22T15:50:52.888 INFO:tasks.workunit.client.1.smithi062.stdout:3/914: write d5/d16/d21/d55/f11d [1891849,110209] 0 2024-02-22T15:50:52.888 INFO:tasks.workunit.client.1.smithi062.stdout:7/711: write d0/d14/d13/fc [1329044,4216] 0 2024-02-22T15:50:52.891 INFO:tasks.workunit.client.1.smithi062.stdout:7/712: fdatasync d0/d6/d2e/d4a/f4c 0 2024-02-22T15:50:52.899 INFO:tasks.workunit.client.1.smithi062.stdout:3/915: dwrite d5/d16/d56/dbc/ff0 [0,4194304] 0 2024-02-22T15:50:52.901 INFO:tasks.workunit.client.1.smithi062.stdout:7/713: write d0/f24 [14770,109833] 0 2024-02-22T15:50:52.905 INFO:tasks.workunit.client.1.smithi062.stdout:7/714: creat d0/d6/d2e/d58/dc0/fe4 x:0 0 0 2024-02-22T15:50:52.910 INFO:tasks.workunit.client.1.smithi062.stdout:3/916: dwrite d5/d1a/d3b/dd5/fd7 [0,4194304] 0 2024-02-22T15:50:52.912 INFO:tasks.workunit.client.1.smithi062.stdout:3/917: write d5/d26/d2d/d3f/f7c [1342965,124965] 0 2024-02-22T15:50:52.914 INFO:tasks.workunit.client.1.smithi062.stdout:7/715: dwrite d0/d6/f52 [0,4194304] 0 2024-02-22T15:50:52.917 INFO:tasks.workunit.client.1.smithi062.stdout:7/716: write d0/d14/d19/d30/d37/fa1 [3766612,37908] 0 2024-02-22T15:50:52.918 INFO:tasks.workunit.client.1.smithi062.stdout:7/717: dread - d0/d6/d2e/fd0 zero size 2024-02-22T15:50:52.919 INFO:tasks.workunit.client.1.smithi062.stdout:7/718: read d0/d6/f50 [122772,32264] 0 2024-02-22T15:50:52.921 INFO:tasks.workunit.client.1.smithi062.stdout:7/719: unlink d0/d6/d2e/d35/c8c 0 2024-02-22T15:50:52.923 INFO:tasks.workunit.client.1.smithi062.stdout:7/720: creat d0/d6/d2e/d35/ddc/fe5 x:0 0 0 2024-02-22T15:50:52.925 INFO:tasks.workunit.client.1.smithi062.stdout:7/721: truncate d0/d6/d2e/fa3 1300845 0 2024-02-22T15:50:52.928 INFO:tasks.workunit.client.1.smithi062.stdout:1/784: sync 2024-02-22T15:50:52.936 INFO:tasks.workunit.client.1.smithi062.stdout:1/785: write d1/d6/d36/d76/fa1 [392545,19519] 0 2024-02-22T15:50:52.936 INFO:tasks.workunit.client.1.smithi062.stdout:7/722: dwrite d0/d6/d2e/d55/d80/fb4 [0,4194304] 0 2024-02-22T15:50:52.940 INFO:tasks.workunit.client.1.smithi062.stdout:1/786: mknod d1/d6/d7/d2b/d56/c107 0 2024-02-22T15:50:52.940 INFO:tasks.workunit.client.1.smithi062.stdout:7/723: mkdir d0/d14/d19/d30/d45/dc3/de6 0 2024-02-22T15:50:52.943 INFO:tasks.workunit.client.1.smithi062.stdout:1/787: write d1/d6/d7/f50 [1126118,95481] 0 2024-02-22T15:50:52.943 INFO:tasks.workunit.client.1.smithi062.stdout:7/724: chown d0/d6/c5f 0 1 2024-02-22T15:50:52.946 INFO:tasks.workunit.client.1.smithi062.stdout:7/725: symlink d0/d14/d19/d77/le7 0 2024-02-22T15:50:52.948 INFO:tasks.workunit.client.1.smithi062.stdout:7/726: creat d0/d6/d2e/d4a/d5a/db3/fe8 x:0 0 0 2024-02-22T15:50:52.949 INFO:tasks.workunit.client.1.smithi062.stdout:3/918: write d5/d16/d21/f60 [317617,87474] 0 2024-02-22T15:50:52.950 INFO:tasks.workunit.client.1.smithi062.stdout:7/727: symlink d0/d14/d19/d30/dc8/le9 0 2024-02-22T15:50:52.953 INFO:tasks.workunit.client.1.smithi062.stdout:3/919: mkdir d5/d16/d21/d96/d120 0 2024-02-22T15:50:52.955 INFO:tasks.workunit.client.1.smithi062.stdout:3/920: write d5/d16/d21/d32/d9e/f119 [288927,112573] 0 2024-02-22T15:50:52.957 INFO:tasks.workunit.client.1.smithi062.stdout:1/788: dread d1/f27 [0,4194304] 0 2024-02-22T15:50:52.962 INFO:tasks.workunit.client.1.smithi062.stdout:1/789: write d1/d6/d7/d4a/d65/d83/faa [2005877,3434] 0 2024-02-22T15:50:52.962 INFO:tasks.workunit.client.1.smithi062.stdout:3/921: link d5/c14 d5/d26/d2d/d3f/de6/c121 0 2024-02-22T15:50:52.969 INFO:tasks.workunit.client.1.smithi062.stdout:3/922: dwrite d5/d26/f6c [0,4194304] 0 2024-02-22T15:50:52.972 INFO:tasks.workunit.client.1.smithi062.stdout:1/790: truncate d1/d6/de/f72 3372227 0 2024-02-22T15:50:52.973 INFO:tasks.workunit.client.1.smithi062.stdout:7/728: dread d0/d14/f8b [0,4194304] 0 2024-02-22T15:50:52.973 INFO:tasks.workunit.client.1.smithi062.stdout:7/729: chown d0/d91/l9f 75264 1 2024-02-22T15:50:52.974 INFO:tasks.workunit.client.1.smithi062.stdout:3/923: rename d5/d1a/d3b/dd5/fd7 to d5/d16/d21/d96/d120/f122 0 2024-02-22T15:50:52.980 INFO:tasks.workunit.client.1.smithi062.stdout:1/791: fdatasync d1/d6/d7/d2b/d49/f51 0 2024-02-22T15:50:52.982 INFO:tasks.workunit.client.1.smithi062.stdout:3/924: dwrite d5/d26/d2d/d3f/f7c [0,4194304] 0 2024-02-22T15:50:52.983 INFO:tasks.workunit.client.1.smithi062.stdout:1/792: write d1/f27 [542569,31763] 0 2024-02-22T15:50:52.988 INFO:tasks.workunit.client.1.smithi062.stdout:7/730: write d0/d6/d2e/d2f/f57 [324015,105961] 0 2024-02-22T15:50:52.990 INFO:tasks.workunit.client.1.smithi062.stdout:1/793: mknod d1/d6/d36/d76/c108 0 2024-02-22T15:50:52.993 INFO:tasks.workunit.client.1.smithi062.stdout:1/794: symlink d1/d6/d7/d2b/d47/d24/l109 0 2024-02-22T15:50:52.999 INFO:tasks.workunit.client.1.smithi062.stdout:1/795: link d1/d6/d7/d2b/lcf d1/d6/d7/d2b/d47/d7e/l10a 0 2024-02-22T15:50:53.003 INFO:tasks.workunit.client.1.smithi062.stdout:3/925: dread d5/d16/d21/d36/f95 [0,4194304] 0 2024-02-22T15:50:53.004 INFO:tasks.workunit.client.1.smithi062.stdout:3/926: chown d5/d1a/d3b/dc1 4 1 2024-02-22T15:50:53.006 INFO:tasks.workunit.client.1.smithi062.stdout:3/927: mknod d5/d16/c123 0 2024-02-22T15:50:53.014 INFO:tasks.workunit.client.1.smithi062.stdout:3/928: symlink d5/d16/d21/d32/d9e/l124 0 2024-02-22T15:50:53.015 INFO:tasks.workunit.client.1.smithi062.stdout:7/731: dread d0/d6/d2e/d55/d80/d8a/fa0 [4194304,4194304] 0 2024-02-22T15:50:53.018 INFO:tasks.workunit.client.1.smithi062.stdout:3/929: rename d5/d26/d2d/d3f/f7c to d5/d16/d21/d32/d9e/dc2/d113/dab/f125 0 2024-02-22T15:50:53.021 INFO:tasks.workunit.client.1.smithi062.stdout:7/732: rename d0/d6/d2e/d55/d80/d8a/fa8 to d0/d14/d19/d30/dc8/de0/fea 0 2024-02-22T15:50:53.023 INFO:tasks.workunit.client.1.smithi062.stdout:1/796: dread d1/d6/de/f72 [0,4194304] 0 2024-02-22T15:50:53.027 INFO:tasks.workunit.client.1.smithi062.stdout:3/930: dread d5/d1a/ded/ff3 [0,4194304] 0 2024-02-22T15:50:53.028 INFO:tasks.workunit.client.1.smithi062.stdout:1/797: fdatasync d1/d6/d7/d4a/d65/d83/fc1 0 2024-02-22T15:50:53.028 INFO:tasks.workunit.client.1.smithi062.stdout:7/733: link d0/d6/d2e/d4a/f4c d0/d6/d2e/d55/d72/feb 0 2024-02-22T15:50:53.031 INFO:tasks.workunit.client.1.smithi062.stdout:3/931: chown d5/d16/d88/f91 115359620 1 2024-02-22T15:50:53.034 INFO:tasks.workunit.client.1.smithi062.stdout:1/798: mkdir d1/d6/d7/d2b/d96/df3/d10b 0 2024-02-22T15:50:53.035 INFO:tasks.workunit.client.1.smithi062.stdout:7/734: write d0/d14/f38 [4726644,30043] 0 2024-02-22T15:50:53.036 INFO:tasks.workunit.client.1.smithi062.stdout:1/799: mknod d1/d6/d7/d2b/d49/d74/df5/c10c 0 2024-02-22T15:50:53.037 INFO:tasks.workunit.client.1.smithi062.stdout:7/735: symlink d0/d14/lec 0 2024-02-22T15:50:53.037 INFO:tasks.workunit.client.1.smithi062.stdout:1/800: chown d1/d6/d7/d2b/d96/df3 527999444 1 2024-02-22T15:50:53.038 INFO:tasks.workunit.client.1.smithi062.stdout:7/736: readlink d0/d6/d2e/d2f/d88/lba 0 2024-02-22T15:50:53.038 INFO:tasks.workunit.client.1.smithi062.stdout:1/801: write d1/d6/d7/d2b/d47/d63/ff4 [24002,6025] 0 2024-02-22T15:50:53.040 INFO:tasks.workunit.client.1.smithi062.stdout:7/737: chown d0/d6/d2e/d55/d80/c97 5328 1 2024-02-22T15:50:53.071 INFO:tasks.workunit.client.1.smithi062.stdout:6/878: sync 2024-02-22T15:50:53.075 INFO:tasks.workunit.client.1.smithi062.stdout:1/802: rename d1/d6/d7/d4a/d65/d83/f8b to d1/d6/d7/d2b/d56/f10d 0 2024-02-22T15:50:53.076 INFO:tasks.workunit.client.1.smithi062.stdout:1/803: write d1/d6/d14/fe4 [412592,84150] 0 2024-02-22T15:50:53.080 INFO:tasks.workunit.client.1.smithi062.stdout:1/804: mknod d1/d6/d7/d2b/d49/d74/df5/c10e 0 2024-02-22T15:50:53.088 INFO:tasks.workunit.client.1.smithi062.stdout:6/879: dwrite d5/df/d32/d47/f7a [0,4194304] 0 2024-02-22T15:50:53.088 INFO:tasks.workunit.client.1.smithi062.stdout:1/805: dwrite d1/d6/d7/d8a/dca/fe9 [0,4194304] 0 2024-02-22T15:50:53.093 INFO:tasks.workunit.client.1.smithi062.stdout:1/806: creat d1/d6/d7/d2b/d56/d95/f10f x:0 0 0 2024-02-22T15:50:53.098 INFO:tasks.workunit.client.1.smithi062.stdout:6/880: dwrite d5/dd/d33/ff6 [0,4194304] 0 2024-02-22T15:50:53.098 INFO:tasks.workunit.client.1.smithi062.stdout:1/807: chown d1/d6/d7/d2b/d96/cc8 16617398 1 2024-02-22T15:50:53.100 INFO:tasks.workunit.client.1.smithi062.stdout:7/738: dread d0/d6/d2e/d55/f87 [0,4194304] 0 2024-02-22T15:50:53.101 INFO:tasks.workunit.client.1.smithi062.stdout:6/881: stat d5/df/d40/d44/d4a/d41/d43/dc6/fd4 0 2024-02-22T15:50:53.102 INFO:tasks.workunit.client.1.smithi062.stdout:1/808: creat d1/d6/d36/f110 x:0 0 0 2024-02-22T15:50:53.105 INFO:tasks.workunit.client.1.smithi062.stdout:7/739: chown d0/d6/d2e/d58/d5d/c73 5 1 2024-02-22T15:50:53.106 INFO:tasks.workunit.client.1.smithi062.stdout:7/740: write d0/d14/d19/d30/d37/fa1 [1074693,92837] 0 2024-02-22T15:50:53.110 INFO:tasks.workunit.client.1.smithi062.stdout:1/809: rmdir d1/d6/d7/d2b/d49/d105 0 2024-02-22T15:50:53.110 INFO:tasks.workunit.client.1.smithi062.stdout:7/741: mknod d0/d14/d19/d30/dc8/ced 0 2024-02-22T15:50:53.112 INFO:tasks.workunit.client.1.smithi062.stdout:1/810: chown d1/d6/d7/d2b/d47/l55 90845256 1 2024-02-22T15:50:53.114 INFO:tasks.workunit.client.1.smithi062.stdout:7/742: symlink d0/d14/d19/d30/dc8/de0/lee 0 2024-02-22T15:50:53.116 INFO:tasks.workunit.client.1.smithi062.stdout:1/811: creat d1/d6/d7/d4a/d65/f111 x:0 0 0 2024-02-22T15:50:53.116 INFO:tasks.workunit.client.1.smithi062.stdout:7/743: mknod d0/d6/d2e/d2f/cef 0 2024-02-22T15:50:53.120 INFO:tasks.workunit.client.1.smithi062.stdout:6/882: dwrite d5/df/d32/d8f/f7e [0,4194304] 0 2024-02-22T15:50:53.125 INFO:tasks.workunit.client.1.smithi062.stdout:7/744: link d0/d91/d7c/c85 d0/d6/d2e/d4a/d5a/cf0 0 2024-02-22T15:50:53.127 INFO:tasks.workunit.client.1.smithi062.stdout:7/745: dread - d0/d6/d2e/d4a/d5a/db3/fe8 zero size 2024-02-22T15:50:53.129 INFO:tasks.workunit.client.1.smithi062.stdout:7/746: write d0/d14/d13/f46 [3615847,7851] 0 2024-02-22T15:50:53.132 INFO:tasks.workunit.client.1.smithi062.stdout:7/747: mknod d0/d14/d19/d30/cf1 0 2024-02-22T15:50:53.135 INFO:tasks.workunit.client.1.smithi062.stdout:7/748: creat d0/d6/d2e/d4a/d5a/db3/ff2 x:0 0 0 2024-02-22T15:50:53.136 INFO:tasks.workunit.client.1.smithi062.stdout:7/749: chown d0/d14/d19/d30/f7f 1112420444 1 2024-02-22T15:50:53.139 INFO:tasks.workunit.client.1.smithi062.stdout:7/750: symlink d0/d14/d19/d30/dc8/de0/lf3 0 2024-02-22T15:50:53.139 INFO:tasks.workunit.client.1.smithi062.stdout:1/812: dread d1/d6/de/f42 [0,4194304] 0 2024-02-22T15:50:53.142 INFO:tasks.workunit.client.1.smithi062.stdout:1/813: write d1/d6/de/d9/fd [2041988,104426] 0 2024-02-22T15:50:53.142 INFO:tasks.workunit.client.1.smithi062.stdout:7/751: mknod d0/d14/d19/d30/cf4 0 2024-02-22T15:50:53.143 INFO:tasks.workunit.client.1.smithi062.stdout:6/883: dread d5/df/d40/d44/f11d [0,4194304] 0 2024-02-22T15:50:53.145 INFO:tasks.workunit.client.1.smithi062.stdout:6/884: mkdir d5/dd/d129 0 2024-02-22T15:50:53.148 INFO:tasks.workunit.client.1.smithi062.stdout:1/814: getdents d1/d6/d7/d2b/d56/d95 0 2024-02-22T15:50:53.150 INFO:tasks.workunit.client.1.smithi062.stdout:6/885: creat d5/df/d40/d44/f12a x:0 0 0 2024-02-22T15:50:53.151 INFO:tasks.workunit.client.1.smithi062.stdout:6/886: chown d5/df/d40/d44/d4a/d41/d43 22646 1 2024-02-22T15:50:53.152 INFO:tasks.workunit.client.1.smithi062.stdout:1/815: mkdir d1/d6/d7/d2b/d47/d24/d112 0 2024-02-22T15:50:53.153 INFO:tasks.workunit.client.1.smithi062.stdout:1/816: readlink d1/d6/d36/d37/ld9 0 2024-02-22T15:50:53.154 INFO:tasks.workunit.client.1.smithi062.stdout:1/817: readlink d1/d6/d7/d2b/d47/l55 0 2024-02-22T15:50:53.159 INFO:tasks.workunit.client.1.smithi062.stdout:7/752: dread d0/d14/d19/d30/d37/f63 [0,4194304] 0 2024-02-22T15:50:53.161 INFO:tasks.workunit.client.1.smithi062.stdout:1/818: rename d1/d6/d7/d4a/ld8 to d1/d6/d7/de6/l113 0 2024-02-22T15:50:53.161 INFO:tasks.workunit.client.1.smithi062.stdout:7/753: chown d0/d14/d19/d30/dc8/ced 3768009 1 2024-02-22T15:50:53.163 INFO:tasks.workunit.client.1.smithi062.stdout:6/887: dwrite d5/dd/fc8 [0,4194304] 0 2024-02-22T15:50:53.166 INFO:tasks.workunit.client.1.smithi062.stdout:7/754: read d0/d14/d13/f3e [4078136,45399] 0 2024-02-22T15:50:53.168 INFO:tasks.workunit.client.1.smithi062.stdout:1/819: link d1/d6/d7/d2b/d49/da9/f102 d1/d6/d7/d4a/f114 0 2024-02-22T15:50:53.171 INFO:tasks.workunit.client.1.smithi062.stdout:1/820: chown d1/d6/d36/d76/dfe/c104 640925792 1 2024-02-22T15:50:53.180 INFO:tasks.workunit.client.1.smithi062.stdout:7/755: dwrite d0/d6/d2e/d4a/d5a/d5e/f6c [0,4194304] 0 2024-02-22T15:50:53.188 INFO:tasks.workunit.client.1.smithi062.stdout:1/821: dwrite d1/d6/d7/d2b/d49/fb1 [0,4194304] 0 2024-02-22T15:50:53.189 INFO:tasks.workunit.client.1.smithi062.stdout:1/822: readlink d1/d6/d7/d4a/d65/l97 0 2024-02-22T15:50:53.196 INFO:tasks.workunit.client.1.smithi062.stdout:1/823: dwrite d1/d6/d7/d4a/d65/d83/fbd [0,4194304] 0 2024-02-22T15:50:53.198 INFO:tasks.workunit.client.1.smithi062.stdout:1/824: readlink d1/d6/d7/d2b/d56/l7d 0 2024-02-22T15:50:53.204 INFO:tasks.workunit.client.1.smithi062.stdout:5/872: sync 2024-02-22T15:50:53.207 INFO:tasks.workunit.client.1.smithi062.stdout:5/873: chown d0/d3/da/d32/d33/f88 9330 1 2024-02-22T15:50:53.207 INFO:tasks.workunit.client.1.smithi062.stdout:5/874: readlink d0/d3/d2f/d59/d91/l11e 0 2024-02-22T15:50:53.212 INFO:tasks.workunit.client.1.smithi062.stdout:5/875: rename d0/d3/da/d13/d23/d4e/d60/c6f to d0/c120 0 2024-02-22T15:50:53.214 INFO:tasks.workunit.client.1.smithi062.stdout:5/876: creat d0/d3/d30/d71/f121 x:0 0 0 2024-02-22T15:50:53.217 INFO:tasks.workunit.client.1.smithi062.stdout:5/877: dread d0/d3/da/d32/d4f/d72/df4/d3f/d5f/fdb [0,4194304] 0 2024-02-22T15:50:53.223 INFO:tasks.workunit.client.1.smithi062.stdout:5/878: truncate d0/fa8 807192 0 2024-02-22T15:50:53.228 INFO:tasks.workunit.client.1.smithi062.stdout:5/879: truncate d0/d3/da/d32/d4f/d72/f9a 2959216 0 2024-02-22T15:50:53.231 INFO:tasks.workunit.client.1.smithi062.stdout:5/880: creat d0/d3/da/d13/d23/d4e/d6d/dd2/f122 x:0 0 0 2024-02-22T15:50:53.233 INFO:tasks.workunit.client.1.smithi062.stdout:5/881: truncate d0/d3/da/d13/d23/d78/d11d/f119 209709 0 2024-02-22T15:50:53.236 INFO:tasks.workunit.client.1.smithi062.stdout:5/882: link d0/f45 d0/d3/da/d32/d4f/f123 0 2024-02-22T15:50:53.248 INFO:tasks.workunit.client.1.smithi062.stdout:5/883: dread d0/d3/da/d13/d57/d79/fa5 [0,4194304] 0 2024-02-22T15:50:53.249 INFO:tasks.workunit.client.1.smithi062.stdout:5/884: chown d0/d3/da/d13/d23/d31/fba 24429648 1 2024-02-22T15:50:53.251 INFO:tasks.workunit.client.1.smithi062.stdout:5/885: mknod d0/d3/da/d13/d57/dcc/c124 0 2024-02-22T15:50:53.259 INFO:tasks.workunit.client.1.smithi062.stdout:5/886: rename d0/d3/da/d13/d23/d4e/d6d/dd2 to d0/d3/da/d13/d57/dcc/d125 0 2024-02-22T15:50:53.263 INFO:tasks.workunit.client.1.smithi062.stdout:5/887: link d0/d3/da/f68 d0/d3/da/d32/d4f/d72/df4/d3f/d5f/f126 0 2024-02-22T15:50:53.265 INFO:tasks.workunit.client.1.smithi062.stdout:5/888: mknod d0/d3/da/d32/d4f/df3/c127 0 2024-02-22T15:50:53.268 INFO:tasks.workunit.client.1.smithi062.stdout:5/889: getdents d0/d3/da/d13 0 2024-02-22T15:50:53.270 INFO:tasks.workunit.client.1.smithi062.stdout:5/890: dread - d0/d3/d30/fbb zero size 2024-02-22T15:50:53.275 INFO:tasks.workunit.client.1.smithi062.stdout:5/891: dread d0/d3/d30/f4a [0,4194304] 0 2024-02-22T15:50:53.277 INFO:tasks.workunit.client.1.smithi062.stdout:5/892: chown d0/d3/da/d32/d4f/d72/df4/d3f/d66/cc3 246653 1 2024-02-22T15:50:53.384 INFO:tasks.workunit.client.1.smithi062.stdout:3/932: sync 2024-02-22T15:50:53.387 INFO:tasks.workunit.client.1.smithi062.stdout:3/933: creat d5/d26/d2d/d3f/ddb/dff/f126 x:0 0 0 2024-02-22T15:50:53.388 INFO:tasks.workunit.client.1.smithi062.stdout:3/934: chown d5/d1a/d3c/d62/fb8 1385627768 1 2024-02-22T15:50:53.390 INFO:tasks.workunit.client.1.smithi062.stdout:3/935: write d5/d1a/d3b/dc1/fc3 [2849750,41503] 0 2024-02-22T15:50:53.394 INFO:tasks.workunit.client.1.smithi062.stdout:3/936: mknod d5/d16/d21/d36/d5a/de2/c127 0 2024-02-22T15:50:53.400 INFO:tasks.workunit.client.1.smithi062.stdout:3/937: write d5/d16/d21/f4e [8746815,8018] 0 2024-02-22T15:50:53.408 INFO:tasks.workunit.client.1.smithi062.stdout:3/938: dwrite d5/d16/d21/d96/fe1 [0,4194304] 0 2024-02-22T15:50:53.411 INFO:tasks.workunit.client.1.smithi062.stdout:3/939: symlink d5/d26/l128 0 2024-02-22T15:50:53.436 INFO:tasks.workunit.client.1.smithi062.stdout:3/940: dread d5/f6f [0,4194304] 0 2024-02-22T15:50:53.440 INFO:tasks.workunit.client.1.smithi062.stdout:3/941: link d5/d16/d21/d32/f70 d5/d1a/f129 0 2024-02-22T15:50:53.442 INFO:tasks.workunit.client.1.smithi062.stdout:3/942: mkdir d5/d1a/d3b/d12a 0 2024-02-22T15:50:53.443 INFO:tasks.workunit.client.1.smithi062.stdout:3/943: chown d5/d16/d21/d32/d6a/f8e 103578 1 2024-02-22T15:50:53.444 INFO:tasks.workunit.client.1.smithi062.stdout:6/888: sync 2024-02-22T15:50:53.444 INFO:tasks.workunit.client.1.smithi062.stdout:1/825: sync 2024-02-22T15:50:53.445 INFO:tasks.workunit.client.1.smithi062.stdout:5/893: sync 2024-02-22T15:50:53.445 INFO:tasks.workunit.client.1.smithi062.stdout:7/756: sync 2024-02-22T15:50:53.447 INFO:tasks.workunit.client.1.smithi062.stdout:7/757: fsync d0/d14/d13/f46 0 2024-02-22T15:50:53.447 INFO:tasks.workunit.client.1.smithi062.stdout:6/889: stat d5/dd/d33/d66/fcc 0 2024-02-22T15:50:53.449 INFO:tasks.workunit.client.1.smithi062.stdout:5/894: dread - d0/d3/da/d32/d4f/d72/df4/fd7 zero size 2024-02-22T15:50:53.452 INFO:tasks.workunit.client.1.smithi062.stdout:1/826: mkdir d1/d6/d7/d2b/d47/d63/db2/de0/d115 0 2024-02-22T15:50:53.453 INFO:tasks.workunit.client.1.smithi062.stdout:3/944: truncate d5/d1a/d3b/dc1/fc3 2723034 0 2024-02-22T15:50:53.453 INFO:tasks.workunit.client.1.smithi062.stdout:7/758: symlink d0/d6/d2e/d2f/d88/lf5 0 2024-02-22T15:50:53.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: pgmap v42: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 119 MiB/s rd, 302 MiB/s wr, 670 op/s 2024-02-22T15:50:53.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:53.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:53.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:50:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:50:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:50:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:50:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:50:53.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:50:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:50:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: Upgrade: Need to upgrade myself (mgr.smithi062.ecubmv) 2024-02-22T15:50:53.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:53 smithi062 ceph-mon[48130]: Upgrade: Need to upgrade myself (mgr.smithi062.ecubmv) 2024-02-22T15:50:53.460 INFO:tasks.workunit.client.1.smithi062.stdout:5/895: creat d0/d3/da/d13/d23/d4e/d5a/dd6/f128 x:0 0 0 2024-02-22T15:50:53.460 INFO:tasks.workunit.client.1.smithi062.stdout:6/890: getdents d5/df/dbc/dce 0 2024-02-22T15:50:53.460 INFO:tasks.workunit.client.1.smithi062.stdout:1/827: creat d1/d6/d7/d2b/d49/f116 x:0 0 0 2024-02-22T15:50:53.463 INFO:tasks.workunit.client.1.smithi062.stdout:7/759: dread d0/d14/f2c [0,4194304] 0 2024-02-22T15:50:53.463 INFO:tasks.workunit.client.1.smithi062.stdout:3/945: dwrite d5/d1a/d3c/d62/fb7 [4194304,4194304] 0 2024-02-22T15:50:53.465 INFO:tasks.workunit.client.1.smithi062.stdout:6/891: creat d5/dd/d33/d66/f12b x:0 0 0 2024-02-22T15:50:53.466 INFO:tasks.workunit.client.1.smithi062.stdout:5/896: fsync d0/d3/da/d32/d4f/df3/f108 0 2024-02-22T15:50:53.466 INFO:tasks.workunit.client.1.smithi062.stdout:3/946: chown d5/d16/d21/d36/d5a/d85/l94 126 1 2024-02-22T15:50:53.469 INFO:tasks.workunit.client.1.smithi062.stdout:7/760: mknod d0/d6/d2e/d35/ddc/cf6 0 2024-02-22T15:50:53.469 INFO:tasks.workunit.client.1.smithi062.stdout:1/828: symlink d1/d6/d7/d2b/l117 0 2024-02-22T15:50:53.469 INFO:tasks.workunit.client.1.smithi062.stdout:3/947: fsync d5/d16/d21/d55/ff8 0 2024-02-22T15:50:53.471 INFO:tasks.workunit.client.1.smithi062.stdout:6/892: creat d5/df/d32/d8f/f12c x:0 0 0 2024-02-22T15:50:53.473 INFO:tasks.workunit.client.1.smithi062.stdout:1/829: rmdir d1/d6/d7/d2b/d49/da9 39 2024-02-22T15:50:53.476 INFO:tasks.workunit.client.1.smithi062.stdout:7/761: creat d0/d6/d2e/d55/d72/ff7 x:0 0 0 2024-02-22T15:50:53.479 INFO:tasks.workunit.client.1.smithi062.stdout:6/893: mkdir d5/df/dbc/dcd/d12d 0 2024-02-22T15:50:53.482 INFO:tasks.workunit.client.1.smithi062.stdout:1/830: rename d1/d6/d7/d4a/f114 to d1/d6/d7/d8a/dca/f118 0 2024-02-22T15:50:53.483 INFO:tasks.workunit.client.1.smithi062.stdout:3/948: dwrite d5/d16/d21/d32/d9e/dc2/d113/dab/f125 [0,4194304] 0 2024-02-22T15:50:53.487 INFO:tasks.workunit.client.1.smithi062.stdout:5/897: getdents d0/d3/da/d13/d23/d31/dac 0 2024-02-22T15:50:53.487 INFO:tasks.workunit.client.1.smithi062.stdout:1/831: dread - d1/d6/d7/d2b/d96/ffc zero size 2024-02-22T15:50:53.488 INFO:tasks.workunit.client.1.smithi062.stdout:3/949: write d5/f1d [3408107,81495] 0 2024-02-22T15:50:53.490 INFO:tasks.workunit.client.1.smithi062.stdout:7/762: getdents d0/d14/d19/d77 0 2024-02-22T15:50:53.491 INFO:tasks.workunit.client.1.smithi062.stdout:3/950: write d5/d1a/d3c/d62/fb7 [3058214,74277] 0 2024-02-22T15:50:53.492 INFO:tasks.workunit.client.1.smithi062.stdout:7/763: dread - d0/d6/d2e/d58/dc0/fe4 zero size 2024-02-22T15:50:53.493 INFO:tasks.workunit.client.1.smithi062.stdout:3/951: fdatasync d5/da6/de3/f104 0 2024-02-22T15:50:53.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: pgmap v42: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 119 MiB/s rd, 302 MiB/s wr, 670 op/s 2024-02-22T15:50:53.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:50:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:50:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:50:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:50:53.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:50:53.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:50:53.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:50:53.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: Upgrade: Need to upgrade myself (mgr.smithi062.ecubmv) 2024-02-22T15:50:53.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:53 smithi019 ceph-mon[67418]: Upgrade: Need to upgrade myself (mgr.smithi062.ecubmv) 2024-02-22T15:50:53.498 INFO:tasks.workunit.client.1.smithi062.stdout:6/894: rename d5/df/d32/c6d to d5/df/d40/dac/db6/c12e 0 2024-02-22T15:50:53.498 INFO:tasks.workunit.client.1.smithi062.stdout:3/952: readlink d5/l17 0 2024-02-22T15:50:53.502 INFO:tasks.workunit.client.1.smithi062.stdout:3/953: rmdir d5/d1a/d3b/dc1 39 2024-02-22T15:50:53.504 INFO:tasks.workunit.client.1.smithi062.stdout:1/832: write d1/d6/d7/d2b/d47/f52 [1119224,12301] 0 2024-02-22T15:50:53.510 INFO:tasks.workunit.client.1.smithi062.stdout:1/833: getdents d1/d6/d7/d2b/d47/d7e 0 2024-02-22T15:50:53.511 INFO:tasks.workunit.client.1.smithi062.stdout:3/954: dwrite d5/d16/d21/d32/d9e/dc2/d113/dab/f125 [0,4194304] 0 2024-02-22T15:50:53.514 INFO:tasks.workunit.client.1.smithi062.stdout:3/955: rmdir d5/d16/d88 39 2024-02-22T15:50:53.514 INFO:tasks.workunit.client.1.smithi062.stdout:5/898: dread d0/f7 [4194304,4194304] 0 2024-02-22T15:50:53.514 INFO:tasks.workunit.client.1.smithi062.stdout:3/956: read f3 [622168,54279] 0 2024-02-22T15:50:53.520 INFO:tasks.workunit.client.1.smithi062.stdout:3/957: rmdir d5/d26/d2d/d3f/ddb 39 2024-02-22T15:50:53.524 INFO:tasks.workunit.client.1.smithi062.stdout:3/958: rmdir d5/da6 39 2024-02-22T15:50:53.526 INFO:tasks.workunit.client.1.smithi062.stdout:6/895: dwrite d5/df/d40/dac/fee [0,4194304] 0 2024-02-22T15:50:53.526 INFO:tasks.workunit.client.1.smithi062.stdout:7/764: dwrite d0/d6/d2e/d4a/f1f [4194304,4194304] 0 2024-02-22T15:50:53.529 INFO:tasks.workunit.client.1.smithi062.stdout:5/899: rmdir d0/d3/da/d13/d57/d79/dd0/dfc 0 2024-02-22T15:50:53.530 INFO:tasks.workunit.client.1.smithi062.stdout:6/896: mknod d5/d6e/dc1/c12f 0 2024-02-22T15:50:53.533 INFO:tasks.workunit.client.1.smithi062.stdout:6/897: fsync d5/dd/d33/d66/fcc 0 2024-02-22T15:50:53.534 INFO:tasks.workunit.client.1.smithi062.stdout:7/765: creat d0/d14/d19/d77/ff8 x:0 0 0 2024-02-22T15:50:53.534 INFO:tasks.workunit.client.1.smithi062.stdout:1/834: write d1/d6/d10/f5c [3990408,102993] 0 2024-02-22T15:50:53.542 INFO:tasks.workunit.client.1.smithi062.stdout:3/959: rename d5/d16/d21/d36/cce to d5/d26/d2d/d3f/ddb/c12b 0 2024-02-22T15:50:53.542 INFO:tasks.workunit.client.1.smithi062.stdout:6/898: mkdir d5/dd/d33/d10a/d130 0 2024-02-22T15:50:53.543 INFO:tasks.workunit.client.1.smithi062.stdout:5/900: dwrite d0/d3/da/d13/d23/d31/f111 [4194304,4194304] 0 2024-02-22T15:50:53.544 INFO:tasks.workunit.client.1.smithi062.stdout:7/766: creat d0/d14/d19/d30/d45/dc3/de6/ff9 x:0 0 0 2024-02-22T15:50:53.554 INFO:tasks.workunit.client.1.smithi062.stdout:6/899: fsync d5/df/d40/fb2 0 2024-02-22T15:50:53.554 INFO:tasks.workunit.client.1.smithi062.stdout:1/835: link d1/d66/l67 d1/d6/d36/d76/dfe/l119 0 2024-02-22T15:50:53.554 INFO:tasks.workunit.client.1.smithi062.stdout:7/767: fsync d0/d6/d2e/d58/f5c 0 2024-02-22T15:50:53.554 INFO:tasks.workunit.client.1.smithi062.stdout:5/901: creat d0/d3/da/d13/d57/dcc/f129 x:0 0 0 2024-02-22T15:50:53.557 INFO:tasks.workunit.client.1.smithi062.stdout:3/960: dwrite d5/d26/d2d/fa1 [0,4194304] 0 2024-02-22T15:50:53.558 INFO:tasks.workunit.client.1.smithi062.stdout:7/768: creat d0/d14/d19/d30/d45/dc3/de6/ffa x:0 0 0 2024-02-22T15:50:53.561 INFO:tasks.workunit.client.1.smithi062.stdout:5/902: rmdir d0/d3/da/d13/d23/d31 39 2024-02-22T15:50:53.566 INFO:tasks.workunit.client.1.smithi062.stdout:3/961: unlink d5/d1a/d3c/d62/fb8 0 2024-02-22T15:50:53.566 INFO:tasks.workunit.client.1.smithi062.stdout:7/769: mkdir d0/d6/d2e/d35/dfb 0 2024-02-22T15:50:53.567 INFO:tasks.workunit.client.1.smithi062.stdout:5/903: mknod d0/d3/da/d13/d23/d78/d8c/c12a 0 2024-02-22T15:50:53.569 INFO:tasks.workunit.client.1.smithi062.stdout:7/770: read - d0/d14/d19/d30/fb6 zero size 2024-02-22T15:50:53.570 INFO:tasks.workunit.client.1.smithi062.stdout:7/771: stat d0/d14/f17 0 2024-02-22T15:50:53.576 INFO:tasks.workunit.client.1.smithi062.stdout:1/836: write d1/d6/d7/d2b/d47/d63/f68 [472977,14831] 0 2024-02-22T15:50:53.577 INFO:tasks.workunit.client.1.smithi062.stdout:7/772: mknod d0/d14/d19/d30/cfc 0 2024-02-22T15:50:53.580 INFO:tasks.workunit.client.1.smithi062.stdout:1/837: symlink d1/d6/d7/de6/l11a 0 2024-02-22T15:50:53.581 INFO:tasks.workunit.client.1.smithi062.stdout:7/773: rmdir d0/d6/d2e/d58/d5d/dc4 39 2024-02-22T15:50:53.582 INFO:tasks.workunit.client.1.smithi062.stdout:6/900: dwrite d5/df/f30 [0,4194304] 0 2024-02-22T15:50:53.589 INFO:tasks.workunit.client.1.smithi062.stdout:6/901: creat d5/df/d40/d109/f131 x:0 0 0 2024-02-22T15:50:53.595 INFO:tasks.workunit.client.1.smithi062.stdout:5/904: dwrite d0/d3/da/d13/d23/d78/d11d/dda/fef [0,4194304] 0 2024-02-22T15:50:53.597 INFO:tasks.workunit.client.1.smithi062.stdout:3/962: dread d5/d26/d2d/fd0 [0,4194304] 0 2024-02-22T15:50:53.600 INFO:tasks.workunit.client.1.smithi062.stdout:6/902: mknod d5/df/d40/dac/c132 0 2024-02-22T15:50:53.601 INFO:tasks.workunit.client.1.smithi062.stdout:1/838: dwrite d1/d6/d7/d2b/d49/d74/f85 [0,4194304] 0 2024-02-22T15:50:53.603 INFO:tasks.workunit.client.1.smithi062.stdout:7/774: dwrite d0/d6/d2e/d4a/d5a/db3/fe2 [0,4194304] 0 2024-02-22T15:50:53.608 INFO:tasks.workunit.client.1.smithi062.stdout:6/903: creat d5/df/d40/d109/f133 x:0 0 0 2024-02-22T15:50:53.609 INFO:tasks.workunit.client.1.smithi062.stdout:7/775: chown d0/d14/d19/d30/dc8/de0/lf3 44 1 2024-02-22T15:50:53.610 INFO:tasks.workunit.client.1.smithi062.stdout:6/904: chown d5/df/d40/d44/d4a/faa 8898777 1 2024-02-22T15:50:53.614 INFO:tasks.workunit.client.1.smithi062.stdout:1/839: dread d1/f82 [0,4194304] 0 2024-02-22T15:50:53.615 INFO:tasks.workunit.client.1.smithi062.stdout:7/776: creat d0/d6/d2e/d58/d5d/dc4/ffd x:0 0 0 2024-02-22T15:50:53.618 INFO:tasks.workunit.client.1.smithi062.stdout:7/777: mknod d0/d6/d2e/d55/d80/cfe 0 2024-02-22T15:50:53.626 INFO:tasks.workunit.client.1.smithi062.stdout:7/778: dwrite d0/d6/d2e/d4a/d5a/db3/fc2 [0,4194304] 0 2024-02-22T15:50:53.630 INFO:tasks.workunit.client.1.smithi062.stdout:7/779: stat d0/d6/d2e/d55/d72/fb2 0 2024-02-22T15:50:53.631 INFO:tasks.workunit.client.1.smithi062.stdout:7/780: readlink d0/d91/l9f 0 2024-02-22T15:50:53.635 INFO:tasks.workunit.client.1.smithi062.stdout:5/905: dwrite d0/d3/d30/d71/d7c/fae [0,4194304] 0 2024-02-22T15:50:53.638 INFO:tasks.workunit.client.1.smithi062.stdout:7/781: mkdir d0/d6/d2e/d55/d80/dff 0 2024-02-22T15:50:53.638 INFO:tasks.workunit.client.1.smithi062.stdout:6/905: dread d5/dd/f13 [0,4194304] 0 2024-02-22T15:50:53.640 INFO:tasks.workunit.client.1.smithi062.stdout:1/840: dread d1/d6/d7/d2b/fc4 [0,4194304] 0 2024-02-22T15:50:53.640 INFO:tasks.workunit.client.1.smithi062.stdout:5/906: creat d0/f12b x:0 0 0 2024-02-22T15:50:53.644 INFO:tasks.workunit.client.1.smithi062.stdout:6/906: unlink d5/d8a/ddd/dd5/ffe 0 2024-02-22T15:50:53.648 INFO:tasks.workunit.client.1.smithi062.stdout:5/907: dread d0/d3/da/d13/d23/f3e [0,4194304] 0 2024-02-22T15:50:53.648 INFO:tasks.workunit.client.1.smithi062.stdout:6/907: mkdir d5/df/d40/db3/d134 0 2024-02-22T15:50:53.651 INFO:tasks.workunit.client.1.smithi062.stdout:7/782: write d0/d14/f8b [3187910,104641] 0 2024-02-22T15:50:53.653 INFO:tasks.workunit.client.1.smithi062.stdout:5/908: rename d0/d3/da/d13/d23/d4e/fa3 to d0/d3/da/d13/d57/d79/dd0/f12c 0 2024-02-22T15:50:53.657 INFO:tasks.workunit.client.1.smithi062.stdout:7/783: mkdir d0/d14/d19/d30/dc8/d100 0 2024-02-22T15:50:53.661 INFO:tasks.workunit.client.1.smithi062.stdout:7/784: symlink d0/d14/d19/d30/dc8/de0/l101 0 2024-02-22T15:50:53.661 INFO:tasks.workunit.client.1.smithi062.stdout:1/841: dwrite d1/f32 [0,4194304] 0 2024-02-22T15:50:53.666 INFO:tasks.workunit.client.1.smithi062.stdout:5/909: write d0/d3/da/d13/d23/d78/f106 [1066099,5001] 0 2024-02-22T15:50:53.668 INFO:tasks.workunit.client.1.smithi062.stdout:5/910: chown d0/f7 5800321 1 2024-02-22T15:50:53.672 INFO:tasks.workunit.client.1.smithi062.stdout:1/842: dwrite d1/d6/d7/f7a [0,4194304] 0 2024-02-22T15:50:53.672 INFO:tasks.workunit.client.1.smithi062.stdout:5/911: mknod d0/d3/da/d32/d4f/c12d 0 2024-02-22T15:50:53.674 INFO:tasks.workunit.client.1.smithi062.stdout:5/912: read - d0/d3/da/d13/d23/d78/d8c/f10f zero size 2024-02-22T15:50:53.687 INFO:tasks.workunit.client.1.smithi062.stdout:5/913: dwrite d0/d3/d30/f47 [0,4194304] 0 2024-02-22T15:50:53.689 INFO:tasks.workunit.client.1.smithi062.stdout:5/914: write d0/d3/da/d32/d4f/d72/df4/d3f/fe3 [1012731,46306] 0 2024-02-22T15:50:53.694 INFO:tasks.workunit.client.1.smithi062.stdout:5/915: rename d0/d3/da/d32/d4f/d72/df4/d3f/d66 to d0/d3/d30/d71/d7c/d12e 0 2024-02-22T15:50:53.695 INFO:tasks.workunit.client.1.smithi062.stdout:5/916: fsync d0/f12b 0 2024-02-22T15:50:53.698 INFO:tasks.workunit.client.1.smithi062.stdout:5/917: symlink d0/d3/da/d13/d23/d78/d8c/dbd/dc9/l12f 0 2024-02-22T15:50:53.699 INFO:tasks.workunit.client.1.smithi062.stdout:5/918: fdatasync d0/d3/d2f/f44 0 2024-02-22T15:50:53.699 INFO:tasks.workunit.client.1.smithi062.stdout:5/919: dread - d0/d3/d30/d71/f121 zero size 2024-02-22T15:50:53.702 INFO:tasks.workunit.client.1.smithi062.stdout:5/920: creat d0/d3/d2f/f130 x:0 0 0 2024-02-22T15:50:53.703 INFO:tasks.workunit.client.1.smithi062.stdout:1/843: dread d1/d6/d7/d2b/d47/f92 [0,4194304] 0 2024-02-22T15:50:53.705 INFO:tasks.workunit.client.1.smithi062.stdout:5/921: mkdir d0/d3/da/d32/d4f/d72/dca/d131 0 2024-02-22T15:50:53.708 INFO:tasks.workunit.client.1.smithi062.stdout:1/844: creat d1/d6/d7/d2b/d49/d74/f11b x:0 0 0 2024-02-22T15:50:53.714 INFO:tasks.workunit.client.1.smithi062.stdout:1/845: fsync d1/d6/d7/fda 0 2024-02-22T15:50:53.715 INFO:tasks.workunit.client.1.smithi062.stdout:1/846: fdatasync d1/d6/d7/d2b/d56/d95/fe3 0 2024-02-22T15:50:53.756 INFO:tasks.workunit.client.1.smithi062.stdout:5/922: mkdir d0/d3/da/d13/d23/d31/d132 0 2024-02-22T15:50:53.756 INFO:tasks.workunit.client.1.smithi062.stdout:5/923: chown d0/d3/d30/fbb 496902 1 2024-02-22T15:50:53.758 INFO:tasks.workunit.client.1.smithi062.stdout:1/847: mkdir d1/d6/d7/d4a/d65/d83/d11c 0 2024-02-22T15:50:53.760 INFO:tasks.workunit.client.1.smithi062.stdout:1/848: creat d1/d6/d7/d2b/d96/f11d x:0 0 0 2024-02-22T15:50:53.764 INFO:tasks.workunit.client.1.smithi062.stdout:5/924: dwrite d0/d3/da/d32/d4f/f50 [0,4194304] 0 2024-02-22T15:50:53.770 INFO:tasks.workunit.client.1.smithi062.stdout:5/925: symlink d0/d3/d2f/d59/d91/l133 0 2024-02-22T15:50:53.772 INFO:tasks.workunit.client.1.smithi062.stdout:1/849: dread d1/d6/d7/d2b/d56/f10d [0,4194304] 0 2024-02-22T15:50:53.773 INFO:tasks.workunit.client.1.smithi062.stdout:5/926: truncate d0/d3/d30/f46 3122722 0 2024-02-22T15:50:53.776 INFO:tasks.workunit.client.1.smithi062.stdout:1/850: dread d1/d6/de/f42 [0,4194304] 0 2024-02-22T15:50:53.778 INFO:tasks.workunit.client.1.smithi062.stdout:1/851: readlink d1/d6/d36/l5a 0 2024-02-22T15:50:53.778 INFO:tasks.workunit.client.1.smithi062.stdout:5/927: getdents d0/d3/da/d13 0 2024-02-22T15:50:53.778 INFO:tasks.workunit.client.1.smithi062.stdout:5/928: fdatasync d0/fdf 0 2024-02-22T15:50:53.779 INFO:tasks.workunit.client.1.smithi062.stdout:1/852: mkdir d1/d66/d11e 0 2024-02-22T15:50:53.780 INFO:tasks.workunit.client.1.smithi062.stdout:5/929: dread - d0/d3/da/d32/d4f/d72/df4/fee zero size 2024-02-22T15:50:53.786 INFO:tasks.workunit.client.1.smithi062.stdout:1/853: dwrite d1/d6/d7/fda [0,4194304] 0 2024-02-22T15:50:53.789 INFO:tasks.workunit.client.1.smithi062.stdout:1/854: mknod d1/d6/d7/d2b/d47/d63/db2/c11f 0 2024-02-22T15:50:53.795 INFO:tasks.workunit.client.1.smithi062.stdout:1/855: write d1/d6/d7/d2b/d96/f9e [5016442,69247] 0 2024-02-22T15:50:53.800 INFO:tasks.workunit.client.1.smithi062.stdout:5/930: dwrite d0/f45 [4194304,4194304] 0 2024-02-22T15:50:53.803 INFO:tasks.workunit.client.1.smithi062.stdout:1/856: getdents d1/d6/d7/d2b/d47/d63 0 2024-02-22T15:50:53.810 INFO:tasks.workunit.client.1.smithi062.stdout:1/857: dwrite d1/d6/d7/d2b/d47/fb0 [0,4194304] 0 2024-02-22T15:50:53.811 INFO:tasks.workunit.client.1.smithi062.stdout:1/858: readlink d1/l1e 0 2024-02-22T15:50:53.814 INFO:tasks.workunit.client.1.smithi062.stdout:1/859: creat d1/d6/d36/d76/f120 x:0 0 0 2024-02-22T15:50:53.833 INFO:tasks.workunit.client.1.smithi062.stdout:5/931: dread d0/d3/da/d13/d23/d4e/d60/f63 [4194304,4194304] 0 2024-02-22T15:50:53.835 INFO:tasks.workunit.client.1.smithi062.stdout:1/860: dread d1/d66/f75 [0,4194304] 0 2024-02-22T15:50:53.838 INFO:tasks.workunit.client.1.smithi062.stdout:5/932: dread d0/d3/da/d13/d57/d79/fa5 [0,4194304] 0 2024-02-22T15:50:53.843 INFO:tasks.workunit.client.1.smithi062.stdout:5/933: rename d0/d3/f101 to d0/d3/da/d13/d23/d78/f134 0 2024-02-22T15:50:53.843 INFO:tasks.workunit.client.1.smithi062.stdout:1/861: truncate d1/d6/de/d9/fd 333708 0 2024-02-22T15:50:53.845 INFO:tasks.workunit.client.1.smithi062.stdout:5/934: creat d0/d3/da/d13/d23/d78/d8c/f135 x:0 0 0 2024-02-22T15:50:53.845 INFO:tasks.workunit.client.1.smithi062.stdout:1/862: dread - d1/d6/d7/d8a/dca/f118 zero size 2024-02-22T15:50:53.846 INFO:tasks.workunit.client.1.smithi062.stdout:5/935: stat d0/d3/da/d32/d4f/d72/l8f 0 2024-02-22T15:50:53.848 INFO:tasks.workunit.client.1.smithi062.stdout:5/936: write d0/d3/da/d32/d4f/d72/df4/d3f/f117 [945476,14642] 0 2024-02-22T15:50:53.848 INFO:tasks.workunit.client.1.smithi062.stdout:1/863: read d1/d6/d7/d2b/fae [2207074,93380] 0 2024-02-22T15:50:53.854 INFO:tasks.workunit.client.1.smithi062.stdout:5/937: write d0/d3/da/d13/d23/f49 [141163,98668] 0 2024-02-22T15:50:53.854 INFO:tasks.workunit.client.1.smithi062.stdout:1/864: dread d1/d6/f61 [0,4194304] 0 2024-02-22T15:50:53.857 INFO:tasks.workunit.client.1.smithi062.stdout:1/865: symlink d1/d6/d7/d2b/d47/d24/l121 0 2024-02-22T15:50:53.858 INFO:tasks.workunit.client.1.smithi062.stdout:1/866: readlink d1/d6/d36/d37/ld9 0 2024-02-22T15:50:53.858 INFO:tasks.workunit.client.1.smithi062.stdout:5/938: dread d0/d3/da/d32/d4f/df3/f108 [0,4194304] 0 2024-02-22T15:50:53.859 INFO:tasks.workunit.client.1.smithi062.stdout:1/867: chown d1/d6/d7/d2b/d56/c9d 14637 1 2024-02-22T15:50:53.865 INFO:tasks.workunit.client.1.smithi062.stdout:5/939: dwrite d0/d3/d2f/d59/d67/fd8 [0,4194304] 0 2024-02-22T15:50:53.868 INFO:tasks.workunit.client.1.smithi062.stdout:1/868: link d1/d6/d36/c88 d1/d6/d7/d2b/d49/da9/c122 0 2024-02-22T15:50:53.868 INFO:tasks.workunit.client.1.smithi062.stdout:1/869: chown d1/d6/d7/d8a/l9c 161110792 1 2024-02-22T15:50:53.868 INFO:tasks.workunit.client.1.smithi062.stdout:1/870: stat d1/f30 0 2024-02-22T15:50:53.870 INFO:tasks.workunit.client.1.smithi062.stdout:1/871: write d1/d6/d7/d4a/d65/d83/faa [3493059,54728] 0 2024-02-22T15:50:53.870 INFO:tasks.workunit.client.1.smithi062.stdout:1/872: read d1/f27 [609361,4017] 0 2024-02-22T15:50:53.871 INFO:tasks.workunit.client.1.smithi062.stdout:5/940: rename d0/d3/da/fe2 to d0/d3/da/d32/d4f/d72/dca/f136 0 2024-02-22T15:50:53.875 INFO:tasks.workunit.client.1.smithi062.stdout:5/941: mkdir d0/d3/da/d13/d23/d31/d132/d137 0 2024-02-22T15:50:53.877 INFO:tasks.workunit.client.1.smithi062.stdout:5/942: symlink d0/d3/d30/l138 0 2024-02-22T15:50:53.879 INFO:tasks.workunit.client.1.smithi062.stdout:1/873: read d1/d6/d7/d2b/d47/d63/f68 [354633,125614] 0 2024-02-22T15:50:53.880 INFO:tasks.workunit.client.1.smithi062.stdout:5/943: rename d0/d3/da/f53 to d0/d3/da/d13/d23/d4e/d60/db5/de1/f139 0 2024-02-22T15:50:53.881 INFO:tasks.workunit.client.1.smithi062.stdout:1/874: chown d1/d6/d7/d2b/d47/d63/f68 2711949 1 2024-02-22T15:50:53.888 INFO:tasks.workunit.client.1.smithi062.stdout:1/875: creat d1/d6/d7/d2b/d49/dcc/f123 x:0 0 0 2024-02-22T15:50:53.890 INFO:tasks.workunit.client.1.smithi062.stdout:1/876: getdents d1/d6/d7/d2b/d49/d74 0 2024-02-22T15:50:53.892 INFO:tasks.workunit.client.1.smithi062.stdout:1/877: write d1/d6/d7/d4a/d65/d83/fbd [3255638,9095] 0 2024-02-22T15:50:53.896 INFO:tasks.workunit.client.1.smithi062.stdout:1/878: creat d1/d6/d7/d2b/d47/d63/db2/de0/f124 x:0 0 0 2024-02-22T15:50:53.898 INFO:tasks.workunit.client.1.smithi062.stdout:5/944: dwrite d0/d3/da/d13/d23/d78/d81/f9d [0,4194304] 0 2024-02-22T15:50:53.898 INFO:tasks.workunit.client.1.smithi062.stdout:1/879: readlink d1/d6/d7/d4a/d65/d83/laf 0 2024-02-22T15:50:53.900 INFO:tasks.workunit.client.1.smithi062.stdout:5/945: stat d0/d3/da/d13/d23/d4e/d5a 0 2024-02-22T15:50:53.902 INFO:tasks.workunit.client.1.smithi062.stdout:5/946: chown d0/d3/da/d32/d4f/c112 8553 1 2024-02-22T15:50:53.902 INFO:tasks.workunit.client.1.smithi062.stdout:1/880: creat d1/d6/d7/d2b/d47/d63/f125 x:0 0 0 2024-02-22T15:50:53.903 INFO:tasks.workunit.client.1.smithi062.stdout:5/947: read d0/d3/da/d13/d23/d78/d81/fb4 [1444704,116111] 0 2024-02-22T15:50:53.909 INFO:tasks.workunit.client.1.smithi062.stdout:5/948: getdents d0/d3/da/d13/d23/d78/d8c 0 2024-02-22T15:50:53.910 INFO:tasks.workunit.client.1.smithi062.stdout:5/949: fdatasync d0/d3/da/d13/d23/d4e/d60/f10c 0 2024-02-22T15:50:53.916 INFO:tasks.workunit.client.1.smithi062.stdout:5/950: dread d0/d3/da/d32/d4f/f50 [0,4194304] 0 2024-02-22T15:50:53.917 INFO:tasks.workunit.client.1.smithi062.stdout:5/951: write d0/d3/da/d13/d23/d4e/d60/db5/de1/f10d [966923,128932] 0 2024-02-22T15:50:53.918 INFO:tasks.workunit.client.1.smithi062.stdout:5/952: read d0/d3/d30/f4a [331469,14864] 0 2024-02-22T15:50:53.920 INFO:tasks.workunit.client.1.smithi062.stdout:5/953: mknod d0/d3/da/d13/d23/d78/d11d/dda/c13a 0 2024-02-22T15:50:53.923 INFO:tasks.workunit.client.1.smithi062.stdout:5/954: write d0/d3/fe0 [719735,125228] 0 2024-02-22T15:50:53.962 INFO:tasks.workunit.client.1.smithi062.stdout:3/963: sync 2024-02-22T15:50:53.967 INFO:tasks.workunit.client.1.smithi062.stdout:5/955: dwrite d0/d3/da/d13/fb1 [0,4194304] 0 2024-02-22T15:50:53.968 INFO:tasks.workunit.client.1.smithi062.stdout:3/964: getdents d5/d16/d21/d32/d6a 0 2024-02-22T15:50:53.971 INFO:tasks.workunit.client.1.smithi062.stdout:5/956: write d0/d3/da/d13/d23/d78/d8c/f135 [337236,64548] 0 2024-02-22T15:50:53.971 INFO:tasks.workunit.client.1.smithi062.stdout:3/965: creat d5/d16/d21/d32/d75/f12c x:0 0 0 2024-02-22T15:50:53.976 INFO:tasks.workunit.client.1.smithi062.stdout:5/957: read d0/d3/da/d13/d57/dcc/d125/fe8 [1622519,73837] 0 2024-02-22T15:50:53.976 INFO:tasks.workunit.client.1.smithi062.stdout:3/966: mknod d5/d16/d56/c12d 0 2024-02-22T15:50:53.978 INFO:tasks.workunit.client.1.smithi062.stdout:5/958: mkdir d0/d3/da/d13/d23/d4e/d6d/d13b 0 2024-02-22T15:50:53.979 INFO:tasks.workunit.client.1.smithi062.stdout:3/967: fsync d5/d1a/fa9 0 2024-02-22T15:50:53.981 INFO:tasks.workunit.client.1.smithi062.stdout:5/959: unlink d0/d3/da/d32/l75 0 2024-02-22T15:50:53.989 INFO:tasks.workunit.client.1.smithi062.stdout:5/960: dwrite d0/f12b [0,4194304] 0 2024-02-22T15:50:53.993 INFO:tasks.workunit.client.1.smithi062.stdout:3/968: dwrite d5/d1a/f1b [0,4194304] 0 2024-02-22T15:50:53.995 INFO:tasks.workunit.client.1.smithi062.stdout:3/969: write d5/d1a/d3b/f114 [4712312,22531] 0 2024-02-22T15:50:54.000 INFO:tasks.workunit.client.1.smithi062.stdout:3/970: mknod d5/d16/d21/d36/d5a/de2/c12e 0 2024-02-22T15:50:54.000 INFO:tasks.workunit.client.1.smithi062.stdout:3/971: fsync d5/d16/d21/d96/fe1 0 2024-02-22T15:50:54.002 INFO:tasks.workunit.client.1.smithi062.stdout:3/972: chown d5/d1a/d3b/f114 2072503 1 2024-02-22T15:50:54.002 INFO:tasks.workunit.client.1.smithi062.stdout:3/973: stat d5/d1a/d3c/c73 0 2024-02-22T15:50:54.006 INFO:tasks.workunit.client.1.smithi062.stdout:3/974: symlink d5/d1a/d3b/dc1/dfb/l12f 0 2024-02-22T15:50:54.007 INFO:tasks.workunit.client.1.smithi062.stdout:3/975: write d5/d16/d21/f4e [1610964,94895] 0 2024-02-22T15:50:54.008 INFO:tasks.workunit.client.1.smithi062.stdout:3/976: stat d5/d16/d97 0 2024-02-22T15:50:54.019 INFO:tasks.workunit.client.1.smithi062.stdout:5/961: dwrite d0/d3/da/d13/d23/f3e [0,4194304] 0 2024-02-22T15:50:54.021 INFO:tasks.workunit.client.1.smithi062.stdout:3/977: dwrite d5/da6/fc9 [0,4194304] 0 2024-02-22T15:50:54.027 INFO:tasks.workunit.client.1.smithi062.stdout:3/978: getdents d5/d16/d21/d36/d5a/d85 0 2024-02-22T15:50:54.030 INFO:tasks.workunit.client.1.smithi062.stdout:3/979: write d5/d1a/f10b [865334,30621] 0 2024-02-22T15:50:54.033 INFO:tasks.workunit.client.1.smithi062.stdout:5/962: dwrite d0/f39 [0,4194304] 0 2024-02-22T15:50:54.035 INFO:tasks.workunit.client.1.smithi062.stdout:5/963: mknod d0/d3/da/d13/d23/d4e/d60/c13c 0 2024-02-22T15:50:54.040 INFO:tasks.workunit.client.1.smithi062.stdout:5/964: dread d0/d3/da/d13/d23/d78/d81/f9d [0,4194304] 0 2024-02-22T15:50:54.045 INFO:tasks.workunit.client.1.smithi062.stdout:5/965: truncate d0/d3/da/d13/d23/d78/d81/fb4 3645085 0 2024-02-22T15:50:54.046 INFO:tasks.workunit.client.1.smithi062.stdout:5/966: write d0/d3/da/f1d [5017251,16921] 0 2024-02-22T15:50:54.048 INFO:tasks.workunit.client.1.smithi062.stdout:5/967: creat d0/d3/d2f/d59/f13d x:0 0 0 2024-02-22T15:50:54.053 INFO:tasks.workunit.client.1.smithi062.stdout:5/968: rename d0/d3/d2f/f44 to d0/d3/da/d32/d4f/d72/df4/d3f/f13e 0 2024-02-22T15:50:54.054 INFO:tasks.workunit.client.1.smithi062.stdout:5/969: write d0/d3/d30/d71/d7c/d12e/f97 [406180,48324] 0 2024-02-22T15:50:54.056 INFO:tasks.workunit.client.1.smithi062.stdout:5/970: truncate d0/d3/da/d13/d23/d4e/feb 742519 0 2024-02-22T15:50:54.062 INFO:tasks.workunit.client.1.smithi062.stdout:3/980: dread d5/d26/d2d/fa1 [4194304,4194304] 0 2024-02-22T15:50:54.064 INFO:tasks.workunit.client.1.smithi062.stdout:5/971: rename d0/d3/da/d13/d23/d78 to d0/d3/da/d32/d42/d13f 0 2024-02-22T15:50:54.066 INFO:tasks.workunit.client.1.smithi062.stdout:3/981: creat d5/d16/d21/d32/d9e/dc2/d113/dab/f130 x:0 0 0 2024-02-22T15:50:54.069 INFO:tasks.workunit.client.1.smithi062.stdout:5/972: unlink d0/d3/d30/f4a 0 2024-02-22T15:50:54.069 INFO:tasks.workunit.client.1.smithi062.stdout:3/982: dread d5/da6/de3/fe8 [0,4194304] 0 2024-02-22T15:50:54.070 INFO:tasks.workunit.client.1.smithi062.stdout:5/973: dread - d0/d3/da/d13/d57/dcc/f10e zero size 2024-02-22T15:50:54.071 INFO:tasks.workunit.client.1.smithi062.stdout:3/983: unlink d5/d16/d21/d55/f11d 0 2024-02-22T15:50:54.073 INFO:tasks.workunit.client.1.smithi062.stdout:5/974: mknod d0/d3/d2f/d59/d91/c140 0 2024-02-22T15:50:54.075 INFO:tasks.workunit.client.1.smithi062.stdout:3/984: truncate d5/d26/d2d/d3f/de6/f108 1503447 0 2024-02-22T15:50:54.077 INFO:tasks.workunit.client.1.smithi062.stdout:3/985: write d5/d16/d21/d96/fe1 [3130548,107526] 0 2024-02-22T15:50:54.079 INFO:tasks.workunit.client.1.smithi062.stdout:6/908: sync 2024-02-22T15:50:54.085 INFO:tasks.workunit.client.1.smithi062.stdout:3/986: dwrite d5/d1a/f10b [0,4194304] 0 2024-02-22T15:50:54.091 INFO:tasks.workunit.client.1.smithi062.stdout:5/975: write d0/d3/da/fb [2256337,55502] 0 2024-02-22T15:50:54.091 INFO:tasks.workunit.client.1.smithi062.stdout:3/987: creat d5/d16/d56/f131 x:0 0 0 2024-02-22T15:50:54.092 INFO:tasks.workunit.client.1.smithi062.stdout:5/976: fdatasync d0/d3/d2f/d59/ff7 0 2024-02-22T15:50:54.095 INFO:tasks.workunit.client.1.smithi062.stdout:3/988: write d5/d16/d21/d32/d6a/d7b/f5f [2420466,11925] 0 2024-02-22T15:50:54.095 INFO:tasks.workunit.client.1.smithi062.stdout:1/881: sync 2024-02-22T15:50:54.095 INFO:tasks.workunit.client.1.smithi062.stdout:7/785: sync 2024-02-22T15:50:54.096 INFO:tasks.workunit.client.1.smithi062.stdout:6/909: dwrite d5/df/d40/d44/d76/d56/f5b [4194304,4194304] 0 2024-02-22T15:50:54.105 INFO:tasks.workunit.client.1.smithi062.stdout:3/989: creat d5/da6/f132 x:0 0 0 2024-02-22T15:50:54.107 INFO:tasks.workunit.client.1.smithi062.stdout:7/786: creat d0/d14/d19/d30/d45/dc3/dcb/f102 x:0 0 0 2024-02-22T15:50:54.109 INFO:tasks.workunit.client.1.smithi062.stdout:5/977: getdents d0/d3/da/d13/d57/dcc/d125 0 2024-02-22T15:50:54.110 INFO:tasks.workunit.client.1.smithi062.stdout:1/882: dwrite d1/d6/d7/d8a/dca/fe9 [0,4194304] 0 2024-02-22T15:50:54.111 INFO:tasks.workunit.client.1.smithi062.stdout:7/787: chown d0/d6/d5b/cd3 9380589 1 2024-02-22T15:50:54.112 INFO:tasks.workunit.client.1.smithi062.stdout:5/978: write d0/d3/d2f/f130 [1035505,19816] 0 2024-02-22T15:50:54.112 INFO:tasks.workunit.client.1.smithi062.stdout:1/883: stat d1/d6/d7/d2b/d56/l7d 0 2024-02-22T15:50:54.117 INFO:tasks.workunit.client.1.smithi062.stdout:7/788: truncate d0/d14/d13/f3e 2615922 0 2024-02-22T15:50:54.124 INFO:tasks.workunit.client.1.smithi062.stdout:1/884: dwrite d1/d6/d7/d2b/d47/f52 [0,4194304] 0 2024-02-22T15:50:54.128 INFO:tasks.workunit.client.1.smithi062.stdout:6/910: dwrite d5/df/d40/d44/d4a/f28 [4194304,4194304] 0 2024-02-22T15:50:54.129 INFO:tasks.workunit.client.1.smithi062.stdout:7/789: truncate d0/d6/d5b/faa 3718777 0 2024-02-22T15:50:54.130 INFO:tasks.workunit.client.1.smithi062.stdout:5/979: write d0/d3/d30/f46 [3082401,110804] 0 2024-02-22T15:50:54.133 INFO:tasks.workunit.client.1.smithi062.stdout:7/790: write d0/d6/d2e/d4a/f1f [3426400,26726] 0 2024-02-22T15:50:54.138 INFO:tasks.workunit.client.1.smithi062.stdout:6/911: link d5/df/d40/d44/d4a/d41/d85/dd1/ff0 d5/df/dbc/dce/f135 0 2024-02-22T15:50:54.141 INFO:tasks.workunit.client.1.smithi062.stdout:7/791: link d0/d6/d2e/f86 d0/d14/d19/d30/dc8/f103 0 2024-02-22T15:50:54.145 INFO:tasks.workunit.client.1.smithi062.stdout:7/792: symlink d0/d6/d2e/d58/l104 0 2024-02-22T15:50:54.145 INFO:tasks.workunit.client.1.smithi062.stdout:6/912: creat d5/df/d40/d44/d76/f136 x:0 0 0 2024-02-22T15:50:54.146 INFO:tasks.workunit.client.1.smithi062.stdout:7/793: chown d0/d14/d19/d77 41 1 2024-02-22T15:50:54.147 INFO:tasks.workunit.client.1.smithi062.stdout:6/913: readlink d5/df/d32/lec 0 2024-02-22T15:50:54.147 INFO:tasks.workunit.client.1.smithi062.stdout:1/885: dread d1/d6/d7/d2b/d47/d7e/fc5 [0,4194304] 0 2024-02-22T15:50:54.149 INFO:tasks.workunit.client.1.smithi062.stdout:6/914: chown d5/df/d40/d44/l107 0 1 2024-02-22T15:50:54.150 INFO:tasks.workunit.client.1.smithi062.stdout:6/915: chown d5/df/d32/c4c 33624 1 2024-02-22T15:50:54.151 INFO:tasks.workunit.client.1.smithi062.stdout:6/916: chown d5/dd/d33/fb7 847790 1 2024-02-22T15:50:54.154 INFO:tasks.workunit.client.1.smithi062.stdout:1/886: dread d1/d6/d7/d4a/d65/d83/fbd [0,4194304] 0 2024-02-22T15:50:54.160 INFO:tasks.workunit.client.1.smithi062.stdout:3/990: dread d5/d16/d21/d36/d4a/f4f [0,4194304] 0 2024-02-22T15:50:54.165 INFO:tasks.workunit.client.1.smithi062.stdout:6/917: dwrite d5/df/d32/d47/f7a [0,4194304] 0 2024-02-22T15:50:54.165 INFO:tasks.workunit.client.1.smithi062.stdout:3/991: chown d5/d16/d21/d36/f95 23475687 1 2024-02-22T15:50:54.167 INFO:tasks.workunit.client.1.smithi062.stdout:7/794: dread d0/d91/d7c/fc7 [0,4194304] 0 2024-02-22T15:50:54.167 INFO:tasks.workunit.client.1.smithi062.stdout:1/887: write d1/d6/d7/d8a/dca/f118 [995865,100025] 0 2024-02-22T15:50:54.168 INFO:tasks.workunit.client.1.smithi062.stdout:5/980: dwrite d0/d3/d2f/d59/f9f [0,4194304] 0 2024-02-22T15:50:54.171 INFO:tasks.workunit.client.1.smithi062.stdout:6/918: unlink d5/d8a/caf 0 2024-02-22T15:50:54.171 INFO:tasks.workunit.client.1.smithi062.stdout:3/992: creat d5/d26/d2d/d3f/ddb/dff/f133 x:0 0 0 2024-02-22T15:50:54.175 INFO:tasks.workunit.client.1.smithi062.stdout:1/888: rename d1/d6/d7/d2b/d47/d63/la0 to d1/d6/d7/de6/l126 0 2024-02-22T15:50:54.178 INFO:tasks.workunit.client.1.smithi062.stdout:7/795: mkdir d0/d6/d2e/d4a/d5a/d105 0 2024-02-22T15:50:54.182 INFO:tasks.workunit.client.1.smithi062.stdout:6/919: rename d5/df/d32/d47/f7a to d5/d8a/d10b/d115/f137 0 2024-02-22T15:50:54.184 INFO:tasks.workunit.client.1.smithi062.stdout:7/796: write d0/d14/f2c [1147399,39116] 0 2024-02-22T15:50:54.186 INFO:tasks.workunit.client.1.smithi062.stdout:3/993: truncate d5/d16/d97/fb1 736429 0 2024-02-22T15:50:54.188 INFO:tasks.workunit.client.1.smithi062.stdout:5/981: dwrite d0/d3/da/d32/d42/d13f/d11d/dda/fef [4194304,4194304] 0 2024-02-22T15:50:54.188 INFO:tasks.workunit.client.1.smithi062.stdout:1/889: link d1/d6/d14/fe4 d1/d6/d7/d8a/dca/f127 0 2024-02-22T15:50:54.190 INFO:tasks.workunit.client.1.smithi062.stdout:7/797: creat d0/d6/d2e/d35/ddc/f106 x:0 0 0 2024-02-22T15:50:54.190 INFO:tasks.workunit.client.1.smithi062.stdout:6/920: symlink d5/df/d40/dd2/de4/l138 0 2024-02-22T15:50:54.191 INFO:tasks.workunit.client.1.smithi062.stdout:3/994: write d5/d1a/f25 [5534859,3888] 0 2024-02-22T15:50:54.192 INFO:tasks.workunit.client.1.smithi062.stdout:1/890: read - d1/d6/d36/d37/f86 zero size 2024-02-22T15:50:54.196 INFO:tasks.workunit.client.1.smithi062.stdout:3/995: write d5/d16/d21/d32/d6a/f8e [3731275,89848] 0 2024-02-22T15:50:54.199 INFO:tasks.workunit.client.1.smithi062.stdout:6/921: creat d5/dd/d33/d10a/d114/f139 x:0 0 0 2024-02-22T15:50:54.200 INFO:tasks.workunit.client.1.smithi062.stdout:7/798: rename d0/d14/d19/d30/dc8/f103 to d0/d91/d7c/f107 0 2024-02-22T15:50:54.205 INFO:tasks.workunit.client.1.smithi062.stdout:1/891: mknod d1/d6/d7/d4a/d65/d83/d11c/c128 0 2024-02-22T15:50:54.205 INFO:tasks.workunit.client.1.smithi062.stdout:7/799: stat d0/d6/d2e/d55/d80/fb9 0 2024-02-22T15:50:54.206 INFO:tasks.workunit.client.1.smithi062.stdout:5/982: dwrite d0/d3/d2f/d59/ff7 [0,4194304] 0 2024-02-22T15:50:54.214 INFO:tasks.workunit.client.1.smithi062.stdout:6/922: dread d5/fb [4194304,4194304] 0 2024-02-22T15:50:54.216 INFO:tasks.workunit.client.1.smithi062.stdout:3/996: dwrite d5/f8 [0,4194304] 0 2024-02-22T15:50:54.281 INFO:tasks.workunit.client.1.smithi062.stdout:5/983: unlink d0/d3/da/d13/d57/dcc/d125/fe8 0 2024-02-22T15:50:54.288 INFO:tasks.workunit.client.1.smithi062.stdout:3/997: unlink d5/c3e 0 2024-02-22T15:50:54.293 INFO:tasks.workunit.client.1.smithi062.stdout:7/800: truncate d0/d14/f20 1804119 0 2024-02-22T15:50:54.295 INFO:tasks.workunit.client.1.smithi062.stdout:1/892: link d1/d6/d7/d2b/d47/d63/fa6 d1/d6/d36/d37/dee/f129 0 2024-02-22T15:50:54.299 INFO:tasks.workunit.client.1.smithi062.stdout:5/984: creat d0/d3/da/d13/d57/f141 x:0 0 0 2024-02-22T15:50:54.301 INFO:tasks.workunit.client.1.smithi062.stdout:7/801: creat d0/d14/d13/f108 x:0 0 0 2024-02-22T15:50:54.303 INFO:tasks.workunit.client.1.smithi062.stdout:7/802: readlink d0/d14/d19/d30/la9 0 2024-02-22T15:50:54.309 INFO:tasks.workunit.client.1.smithi062.stdout:7/803: creat d0/d14/d19/d30/dc8/d100/f109 x:0 0 0 2024-02-22T15:50:54.314 INFO:tasks.workunit.client.1.smithi062.stdout:7/804: mknod d0/d14/c10a 0 2024-02-22T15:50:54.328 INFO:tasks.workunit.client.1.smithi062.stdout:5/985: creat d0/d3/da/d32/d42/d13f/d8c/f142 x:0 0 0 2024-02-22T15:50:54.364 INFO:tasks.workunit.client.1.smithi062.stdout:3/998: rmdir d5/d1a/d3b/dd5 0 2024-02-22T15:50:54.372 INFO:tasks.workunit.client.1.smithi062.stdout:3/999: chown d5/d26/d2d/d3f/de6/c121 1 1 2024-02-22T15:50:54.377 INFO:tasks.workunit.client.1.smithi062.stdout:7/805: write d0/d14/d13/f3e [3263092,50975] 0 2024-02-22T15:50:54.386 INFO:tasks.workunit.client.1.smithi062.stdout:7/806: getdents d0/d6/d5b/d60 0 2024-02-22T15:50:54.391 INFO:tasks.workunit.client.1.smithi062.stdout:7/807: mkdir d0/d14/d19/d30/d45/dc3/dcb/d10b 0 2024-02-22T15:50:54.396 INFO:tasks.workunit.client.1.smithi062.stdout:7/808: creat d0/d6/d2e/d58/d5d/f10c x:0 0 0 2024-02-22T15:50:54.399 INFO:tasks.workunit.client.1.smithi062.stdout:7/809: creat d0/d14/d19/d30/d45/dc3/f10d x:0 0 0 2024-02-22T15:50:54.401 INFO:tasks.workunit.client.1.smithi062.stdout:7/810: chown d0/d6/d2e/d58/cab 50154452 1 2024-02-22T15:50:54.412 INFO:tasks.workunit.client.1.smithi062.stdout:7/811: dwrite d0/d14/d13/f46 [4194304,4194304] 0 2024-02-22T15:50:54.425 INFO:tasks.workunit.client.1.smithi062.stdout:7/812: rename d0/d14/d19/d30/d45/dc3/dcb/le3 to d0/d6/d2e/d4a/d5a/d5e/l10e 0 2024-02-22T15:50:54.435 INFO:tasks.workunit.client.1.smithi062.stdout:7/813: dwrite d0/f10 [4194304,4194304] 0 2024-02-22T15:50:54.437 INFO:tasks.workunit.client.1.smithi062.stdout:7/814: truncate d0/d14/d19/d30/d45/dc3/f10d 798332 0 2024-02-22T15:50:54.464 INFO:tasks.workunit.client.1.smithi062.stdout:7/815: dread d0/d6/d2e/d4a/f1f [0,4194304] 0 2024-02-22T15:50:54.466 INFO:tasks.workunit.client.1.smithi062.stdout:7/816: rmdir d0/d6/d2e/d4a/d5a/db3 39 2024-02-22T15:50:54.468 INFO:tasks.workunit.client.1.smithi062.stdout:7/817: truncate d0/d14/d19/d30/d45/dc3/de6/ffa 915687 0 2024-02-22T15:50:54.469 INFO:tasks.workunit.client.1.smithi062.stdout:7/818: fdatasync d0/d6/d2e/d2f/f57 0 2024-02-22T15:50:54.471 INFO:tasks.workunit.client.1.smithi062.stdout:7/819: write d0/d6/d2e/d55/d72/ff7 [763700,16686] 0 2024-02-22T15:50:54.472 INFO:tasks.workunit.client.1.smithi062.stdout:7/820: chown d0/d14/d19/d30/f7f 211069877 1 2024-02-22T15:50:54.473 INFO:tasks.workunit.client.1.smithi062.stdout:7/821: readlink d0/d6/d2e/d58/d5d/dc4/lc9 0 2024-02-22T15:50:54.474 INFO:tasks.workunit.client.1.smithi062.stdout:7/822: write d0/d14/d19/d30/f42 [2475127,107570] 0 2024-02-22T15:50:54.475 INFO:tasks.workunit.client.1.smithi062.stdout:7/823: write d0/d14/d19/d30/d37/fa1 [3230092,19619] 0 2024-02-22T15:50:54.477 INFO:tasks.workunit.client.1.smithi062.stdout:7/824: readlink d0/d6/d2e/d55/l6f 0 2024-02-22T15:50:54.489 INFO:tasks.workunit.client.1.smithi062.stdout:6/923: sync 2024-02-22T15:50:54.492 INFO:tasks.workunit.client.1.smithi062.stdout:7/825: dread d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:54.494 INFO:tasks.workunit.client.1.smithi062.stdout:7/826: write d0/d14/d19/f98 [4672893,40558] 0 2024-02-22T15:50:54.498 INFO:tasks.workunit.client.1.smithi062.stdout:7/827: creat d0/d14/d19/d77/dcf/dd9/f10f x:0 0 0 2024-02-22T15:50:54.500 INFO:tasks.workunit.client.1.smithi062.stdout:6/924: dwrite d5/dd/f94 [0,4194304] 0 2024-02-22T15:50:54.502 INFO:tasks.workunit.client.1.smithi062.stdout:7/828: creat d0/d14/d19/d30/d45/dc3/f110 x:0 0 0 2024-02-22T15:50:54.504 INFO:tasks.workunit.client.1.smithi062.stdout:6/925: rename d5/df/d32/d47 to d5/d6e/d13a 0 2024-02-22T15:50:54.506 INFO:tasks.workunit.client.1.smithi062.stdout:6/926: read - d5/df/dbc/dce/d112/d100/f101 zero size 2024-02-22T15:50:54.508 INFO:tasks.workunit.client.1.smithi062.stdout:6/927: stat d5/df/d40/d44/d4a/d41/d85/dd1/ff0 0 2024-02-22T15:50:54.512 INFO:tasks.workunit.client.1.smithi062.stdout:6/928: rename d5/df/d40/dac/db6/dc0/cc3 to d5/d6e/dc1/c13b 0 2024-02-22T15:50:54.523 INFO:tasks.workunit.client.1.smithi062.stdout:7/829: dread d0/d14/f8b [0,4194304] 0 2024-02-22T15:50:54.524 INFO:tasks.workunit.client.1.smithi062.stdout:6/929: dread d5/f1e [0,4194304] 0 2024-02-22T15:50:54.525 INFO:tasks.workunit.client.1.smithi062.stdout:7/830: chown d0/d6/d2e/d4a/d5a/d5e 3491512 1 2024-02-22T15:50:54.526 INFO:tasks.workunit.client.1.smithi062.stdout:7/831: write d0/d14/d19/d30/d45/dc3/fda [659690,33739] 0 2024-02-22T15:50:54.530 INFO:tasks.workunit.client.1.smithi062.stdout:7/832: rename d0/d6/d2e/d4a/d5a/d5e to d0/d14/d19/d30/dbd/d111 0 2024-02-22T15:50:54.531 INFO:tasks.workunit.client.1.smithi062.stdout:6/930: link d5/dd/d33/d71/ffd d5/df/d40/d44/d78/f13c 0 2024-02-22T15:50:54.531 INFO:tasks.workunit.client.1.smithi062.stdout:7/833: readlink d0/d14/d19/d30/dbd/d111/ld2 0 2024-02-22T15:50:54.538 INFO:tasks.workunit.client.1.smithi062.stdout:6/931: dwrite d5/d6e/d13a/fa2 [0,4194304] 0 2024-02-22T15:50:54.540 INFO:tasks.workunit.client.1.smithi062.stdout:7/834: dwrite d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:54.543 INFO:tasks.workunit.client.1.smithi062.stdout:7/835: chown d0/d6/d2e/d55/f81 7763 1 2024-02-22T15:50:54.545 INFO:tasks.workunit.client.1.smithi062.stdout:5/986: sync 2024-02-22T15:50:54.545 INFO:tasks.workunit.client.1.smithi062.stdout:1/893: sync 2024-02-22T15:50:54.545 INFO:tasks.workunit.client.1.smithi062.stdout:6/932: creat d5/df/d40/d44/d4a/d41/f13d x:0 0 0 2024-02-22T15:50:54.546 INFO:tasks.workunit.client.1.smithi062.stdout:7/836: symlink d0/d6/d5b/l112 0 2024-02-22T15:50:54.548 INFO:tasks.workunit.client.1.smithi062.stdout:5/987: fsync d0/d3/da/d13/d57/dcc/f10e 0 2024-02-22T15:50:54.549 INFO:tasks.workunit.client.1.smithi062.stdout:7/837: chown d0/d6/d2e/d55/lc6 4831273 1 2024-02-22T15:50:54.552 INFO:tasks.workunit.client.1.smithi062.stdout:1/894: getdents d1/d6/d36/d37/dee 0 2024-02-22T15:50:54.553 INFO:tasks.workunit.client.1.smithi062.stdout:5/988: fdatasync d0/d3/da/d13/d23/d4e/f5d 0 2024-02-22T15:50:54.556 INFO:tasks.workunit.client.1.smithi062.stdout:7/838: fsync d0/d6/d2e/d55/d80/d8a/f90 0 2024-02-22T15:50:54.556 INFO:tasks.workunit.client.1.smithi062.stdout:6/933: dwrite d5/df/d40/dac/fee [4194304,4194304] 0 2024-02-22T15:50:54.558 INFO:tasks.workunit.client.1.smithi062.stdout:1/895: fsync d1/d6/d7/d2b/d96/df3/fc6 0 2024-02-22T15:50:54.560 INFO:tasks.workunit.client.1.smithi062.stdout:6/934: write d5/df/d40/d44/d4a/d41/f13d [261112,112227] 0 2024-02-22T15:50:54.562 INFO:tasks.workunit.client.1.smithi062.stdout:1/896: write d1/d6/d7/d2b/d47/d24/f9f [5119250,32433] 0 2024-02-22T15:50:54.563 INFO:tasks.workunit.client.1.smithi062.stdout:5/989: rename d0/fdf to d0/d3/d30/d71/f143 0 2024-02-22T15:50:54.571 INFO:tasks.workunit.client.1.smithi062.stdout:1/897: dwrite d1/d6/d7/fef [0,4194304] 0 2024-02-22T15:50:54.575 INFO:tasks.workunit.client.1.smithi062.stdout:7/839: write d0/d14/d13/f40 [260011,117841] 0 2024-02-22T15:50:54.576 INFO:tasks.workunit.client.1.smithi062.stdout:6/935: write d5/df/d32/f34 [1313700,95750] 0 2024-02-22T15:50:54.577 INFO:tasks.workunit.client.1.smithi062.stdout:1/898: getdents d1/d6/d7/d4a/d65/d83/d11c 0 2024-02-22T15:50:54.584 INFO:tasks.workunit.client.1.smithi062.stdout:7/840: truncate d0/d91/d7c/f107 1049119 0 2024-02-22T15:50:54.585 INFO:tasks.workunit.client.1.smithi062.stdout:5/990: dwrite d0/d3/f94 [0,4194304] 0 2024-02-22T15:50:54.585 INFO:tasks.workunit.client.1.smithi062.stdout:1/899: dread d1/d6/de/f72 [0,4194304] 0 2024-02-22T15:50:54.588 INFO:tasks.workunit.client.1.smithi062.stdout:7/841: fsync d0/d6/d5b/d60/fdf 0 2024-02-22T15:50:54.596 INFO:tasks.workunit.client.1.smithi062.stdout:7/842: read d0/d91/f59 [2593889,35619] 0 2024-02-22T15:50:54.597 INFO:tasks.workunit.client.1.smithi062.stdout:5/991: rename d0/d3/da/d13/d23 to d0/d3/d30/d71/d7c/d12e/d144 0 2024-02-22T15:50:54.610 INFO:tasks.workunit.client.1.smithi062.stdout:1/900: write d1/d6/d14/fe4 [962379,97484] 0 2024-02-22T15:50:54.611 INFO:tasks.workunit.client.1.smithi062.stdout:6/936: dwrite d5/df/d40/d44/d4a/f2d [0,4194304] 0 2024-02-22T15:50:54.619 INFO:tasks.workunit.client.1.smithi062.stdout:1/901: fdatasync d1/d6/d7/d2b/d47/d24/ff2 0 2024-02-22T15:50:54.619 INFO:tasks.workunit.client.1.smithi062.stdout:5/992: dwrite d0/d3/d30/d71/d7c/d12e/d144/d4e/d6d/f93 [0,4194304] 0 2024-02-22T15:50:54.622 INFO:tasks.workunit.client.1.smithi062.stdout:1/902: write d1/d6/d7/d2b/d47/fb0 [295435,120548] 0 2024-02-22T15:50:54.625 INFO:tasks.workunit.client.1.smithi062.stdout:5/993: write d0/d3/d30/d71/d7c/d12e/d144/d31/fba [909505,88915] 0 2024-02-22T15:50:54.629 INFO:tasks.workunit.client.1.smithi062.stdout:7/843: dread d0/d6/d5b/d60/fdf [0,4194304] 0 2024-02-22T15:50:54.632 INFO:tasks.workunit.client.1.smithi062.stdout:5/994: link d0/d3/da/d32/d4f/f50 d0/d3/da/d32/d42/d13f/d8c/f145 0 2024-02-22T15:50:54.632 INFO:tasks.workunit.client.1.smithi062.stdout:7/844: symlink d0/d6/d2e/d35/l113 0 2024-02-22T15:50:54.635 INFO:tasks.workunit.client.1.smithi062.stdout:7/845: write d0/d6/d2e/d58/fbf [268423,120684] 0 2024-02-22T15:50:54.648 INFO:tasks.workunit.client.1.smithi062.stdout:5/995: dwrite d0/d3/da/d13/d57/dcc/f10e [0,4194304] 0 2024-02-22T15:50:54.650 INFO:tasks.workunit.client.1.smithi062.stdout:1/903: dwrite d1/d6/d36/d76/fdc [0,4194304] 0 2024-02-22T15:50:54.653 INFO:tasks.workunit.client.1.smithi062.stdout:7/846: dwrite d0/d6/d2e/d58/d5d/dc4/ffd [0,4194304] 0 2024-02-22T15:50:54.655 INFO:tasks.workunit.client.1.smithi062.stdout:6/937: dread d5/dd/f13 [0,4194304] 0 2024-02-22T15:50:54.662 INFO:tasks.workunit.client.1.smithi062.stdout:6/938: mknod d5/df/d40/d109/c13e 0 2024-02-22T15:50:54.667 INFO:tasks.workunit.client.1.smithi062.stdout:5/996: creat d0/d3/da/d32/d42/d13f/d81/f146 x:0 0 0 2024-02-22T15:50:54.667 INFO:tasks.workunit.client.1.smithi062.stdout:1/904: creat d1/d6/d7/d2b/d49/f12a x:0 0 0 2024-02-22T15:50:54.669 INFO:tasks.workunit.client.1.smithi062.stdout:1/905: fdatasync d1/d6/d36/d76/f120 0 2024-02-22T15:50:54.669 INFO:tasks.workunit.client.1.smithi062.stdout:7/847: rename d0/d14/d19/d30/dc8/de0/fea to d0/d14/d19/d30/d45/dc3/dcb/f114 0 2024-02-22T15:50:54.670 INFO:tasks.workunit.client.1.smithi062.stdout:6/939: symlink d5/df/d40/d109/l13f 0 2024-02-22T15:50:54.671 INFO:tasks.workunit.client.1.smithi062.stdout:5/997: readlink d0/d3/d30/d71/d7c/d12e/da9/lbe 0 2024-02-22T15:50:54.671 INFO:tasks.workunit.client.1.smithi062.stdout:6/940: stat d5/df/d32/d8f/f7e 0 2024-02-22T15:50:54.673 INFO:tasks.workunit.client.1.smithi062.stdout:7/848: unlink d0/d14/d28/lcd 0 2024-02-22T15:50:54.674 INFO:tasks.workunit.client.1.smithi062.stdout:6/941: dread d5/f38 [0,4194304] 0 2024-02-22T15:50:54.675 INFO:tasks.workunit.client.1.smithi062.stdout:5/998: fdatasync d0/d3/d30/d71/d7c/d12e/d144/faa 0 2024-02-22T15:50:54.676 INFO:tasks.workunit.client.1.smithi062.stdout:6/942: read - d5/dd/d33/d66/fa8 zero size 2024-02-22T15:50:54.677 INFO:tasks.workunit.client.1.smithi062.stdout:7/849: creat d0/d14/d19/d30/dc8/de0/f115 x:0 0 0 2024-02-22T15:50:54.680 INFO:tasks.workunit.client.1.smithi062.stdout:5/999: symlink d0/d3/d30/d71/d7c/d12e/d144/d31/d132/d137/l147 0 2024-02-22T15:50:54.682 INFO:tasks.workunit.client.1.smithi062.stdout:7/850: rename d0/d6/d2e/d4a/c83 to d0/d14/d19/d77/c116 0 2024-02-22T15:50:54.684 INFO:tasks.workunit.client.1.smithi062.stdout:6/943: link d5/df/d32/d8f/f12c d5/df/d40/db3/d134/f140 0 2024-02-22T15:50:54.685 INFO:tasks.workunit.client.1.smithi062.stdout:7/851: stat d0/d14/c1d 0 2024-02-22T15:50:54.688 INFO:tasks.workunit.client.1.smithi062.stdout:1/906: write d1/d6/d7/f91 [138351,110821] 0 2024-02-22T15:50:54.688 INFO:tasks.workunit.client.1.smithi062.stdout:6/944: dread - d5/dd/d33/d66/db9/f9c zero size 2024-02-22T15:50:54.690 INFO:tasks.workunit.client.1.smithi062.stdout:6/945: fsync d5/dd/d33/d66/f12b 0 2024-02-22T15:50:54.690 INFO:tasks.workunit.client.1.smithi062.stdout:7/852: mknod d0/d6/d5b/c117 0 2024-02-22T15:50:54.692 INFO:tasks.workunit.client.1.smithi062.stdout:1/907: symlink d1/d6/d7/d2b/d49/l12b 0 2024-02-22T15:50:54.698 INFO:tasks.workunit.client.1.smithi062.stdout:6/946: dwrite d5/df/f84 [0,4194304] 0 2024-02-22T15:50:54.700 INFO:tasks.workunit.client.1.smithi062.stdout:7/853: rmdir d0/d14/d19/d77 39 2024-02-22T15:50:54.704 INFO:tasks.workunit.client.1.smithi062.stdout:7/854: mkdir d0/d14/d19/d77/dcf/d118 0 2024-02-22T15:50:54.705 INFO:tasks.workunit.client.1.smithi062.stdout:1/908: write d1/d6/d7/d4a/d65/d83/fc1 [18163,96205] 0 2024-02-22T15:50:54.708 INFO:tasks.workunit.client.1.smithi062.stdout:7/855: unlink d0/d6/d2e/d55/d72/ff7 0 2024-02-22T15:50:54.722 INFO:tasks.workunit.client.1.smithi062.stdout:7/856: rename d0/d14/d19/d30/dc8/cca to d0/d14/d19/d30/d45/dc3/c119 0 2024-02-22T15:50:54.722 INFO:tasks.workunit.client.1.smithi062.stdout:1/909: write d1/d6/d36/f5d [3821783,3762] 0 2024-02-22T15:50:54.722 INFO:tasks.workunit.client.1.smithi062.stdout:6/947: fdatasync d5/df/d40/d44/fbf 0 2024-02-22T15:50:54.723 INFO:tasks.workunit.client.1.smithi062.stdout:7/857: readlink d0/d6/d2e/d2f/d88/lf5 0 2024-02-22T15:50:54.757 INFO:tasks.workunit.client.1.smithi062.stdout:7/858: creat d0/d14/d19/d30/dbd/d111/f11a x:0 0 0 2024-02-22T15:50:54.757 INFO:tasks.workunit.client.1.smithi062.stdout:1/910: read d1/d6/d7/d8a/fa5 [2196453,106453] 0 2024-02-22T15:50:54.757 INFO:tasks.workunit.client.1.smithi062.stdout:6/948: mkdir d5/df/d141 0 2024-02-22T15:50:54.758 INFO:tasks.workunit.client.1.smithi062.stdout:1/911: fsync d1/d6/fed 0 2024-02-22T15:50:54.761 INFO:tasks.workunit.client.1.smithi062.stdout:1/912: write d1/d6/d7/d2b/d47/d63/db2/de0/f124 [63741,11376] 0 2024-02-22T15:50:54.763 INFO:tasks.workunit.client.1.smithi062.stdout:7/859: link d0/d14/d13/fc d0/d14/d19/d77/dcf/d118/f11b 0 2024-02-22T15:50:54.766 INFO:tasks.workunit.client.1.smithi062.stdout:7/860: symlink d0/d14/d19/d77/dcf/dd9/l11c 0 2024-02-22T15:50:54.769 INFO:tasks.workunit.client.1.smithi062.stdout:7/861: read d0/d14/d13/f2a [5446242,39751] 0 2024-02-22T15:50:54.769 INFO:tasks.workunit.client.1.smithi062.stdout:1/913: rename d1/d6/d7/d2b/d47/c35 to d1/d6/d7/c12c 0 2024-02-22T15:50:54.770 INFO:tasks.workunit.client.1.smithi062.stdout:7/862: chown d0/d6/d2e/da2 1560 1 2024-02-22T15:50:54.771 INFO:tasks.workunit.client.1.smithi062.stdout:1/914: fsync d1/d6/d7/d2b/d49/f116 0 2024-02-22T15:50:54.773 INFO:tasks.workunit.client.1.smithi062.stdout:6/949: dread d5/df/d40/d44/d4a/f5e [0,4194304] 0 2024-02-22T15:50:54.775 INFO:tasks.workunit.client.1.smithi062.stdout:1/915: write d1/d6/d7/d2b/d47/d7e/fc5 [4950005,93427] 0 2024-02-22T15:50:54.777 INFO:tasks.workunit.client.1.smithi062.stdout:1/916: stat d1/d6/d7/d2b/d47/l55 0 2024-02-22T15:50:54.780 INFO:tasks.workunit.client.1.smithi062.stdout:7/863: dwrite d0/d6/d2e/d55/d80/d8a/fa0 [4194304,4194304] 0 2024-02-22T15:50:54.783 INFO:tasks.workunit.client.1.smithi062.stdout:6/950: dread d5/f1a [0,4194304] 0 2024-02-22T15:50:54.784 INFO:tasks.workunit.client.1.smithi062.stdout:1/917: getdents d1/d66 0 2024-02-22T15:50:54.786 INFO:tasks.workunit.client.1.smithi062.stdout:7/864: dread - d0/d6/d2e/d4a/d5a/db3/fbe zero size 2024-02-22T15:50:54.793 INFO:tasks.workunit.client.1.smithi062.stdout:6/951: dwrite d5/dd/d33/ff6 [4194304,4194304] 0 2024-02-22T15:50:54.794 INFO:tasks.workunit.client.1.smithi062.stdout:7/865: unlink d0/d6/d2e/d55/f87 0 2024-02-22T15:50:54.796 INFO:tasks.workunit.client.1.smithi062.stdout:6/952: chown d5/df/d40/d44/d76/d56/f61 38 1 2024-02-22T15:50:54.796 INFO:tasks.workunit.client.1.smithi062.stdout:6/953: fsync d5/d6e/f111 0 2024-02-22T15:50:54.798 INFO:tasks.workunit.client.1.smithi062.stdout:7/866: creat d0/d91/f11d x:0 0 0 2024-02-22T15:50:54.803 INFO:tasks.workunit.client.1.smithi062.stdout:7/867: link d0/d6/d5b/cd3 d0/d14/d13/c11e 0 2024-02-22T15:50:54.811 INFO:tasks.workunit.client.1.smithi062.stdout:1/918: write d1/d6/de/f31 [1851017,99927] 0 2024-02-22T15:50:54.811 INFO:tasks.workunit.client.1.smithi062.stdout:7/868: dwrite d0/d14/d19/d30/dc8/d100/f109 [0,4194304] 0 2024-02-22T15:50:54.814 INFO:tasks.workunit.client.1.smithi062.stdout:6/954: truncate d5/df/d40/d44/d4a/f2d 1637208 0 2024-02-22T15:50:54.816 INFO:tasks.workunit.client.1.smithi062.stdout:7/869: write d0/d6/d2e/d58/fbf [225501,120147] 0 2024-02-22T15:50:54.818 INFO:tasks.workunit.client.1.smithi062.stdout:7/870: write d0/d14/d13/f36 [6731550,74153] 0 2024-02-22T15:50:54.820 INFO:tasks.workunit.client.1.smithi062.stdout:7/871: chown d0/d6/d2e/d55/d80/d8a/f8f 359179913 1 2024-02-22T15:50:54.820 INFO:tasks.workunit.client.1.smithi062.stdout:6/955: mknod d5/dd/d129/c142 0 2024-02-22T15:50:54.823 INFO:tasks.workunit.client.1.smithi062.stdout:6/956: rmdir d5/df/d40/d109 39 2024-02-22T15:50:54.826 INFO:tasks.workunit.client.1.smithi062.stdout:7/872: creat d0/d14/d19/d77/dcf/f11f x:0 0 0 2024-02-22T15:50:54.827 INFO:tasks.workunit.client.1.smithi062.stdout:1/919: write d1/d6/d10/f8f [2313294,68134] 0 2024-02-22T15:50:54.828 INFO:tasks.workunit.client.1.smithi062.stdout:6/957: chown d5/d8a/ddd/de6/df1/l126 16251 1 2024-02-22T15:50:54.834 INFO:tasks.workunit.client.1.smithi062.stdout:1/920: link d1/d6/d7/d4a/d65/d83/fc1 d1/f12d 0 2024-02-22T15:50:54.836 INFO:tasks.workunit.client.1.smithi062.stdout:6/958: rename d5/dd/d33/d66/fcc to d5/df/d40/d44/d76/f143 0 2024-02-22T15:50:54.836 INFO:tasks.workunit.client.1.smithi062.stdout:7/873: write d0/d14/d19/d30/d37/f95 [1662799,119972] 0 2024-02-22T15:50:54.838 INFO:tasks.workunit.client.1.smithi062.stdout:6/959: read d5/df/d40/d44/d4a/f1c [913682,117096] 0 2024-02-22T15:50:54.840 INFO:tasks.workunit.client.1.smithi062.stdout:6/960: creat d5/d6e/f144 x:0 0 0 2024-02-22T15:50:54.844 INFO:tasks.workunit.client.1.smithi062.stdout:1/921: write d1/f27 [1318988,4249] 0 2024-02-22T15:50:54.848 INFO:tasks.workunit.client.1.smithi062.stdout:1/922: creat d1/d6/d7/d2b/d49/dcc/f12e x:0 0 0 2024-02-22T15:50:54.851 INFO:tasks.workunit.client.1.smithi062.stdout:6/961: dwrite d5/df/dbc/fe5 [0,4194304] 0 2024-02-22T15:50:54.856 INFO:tasks.workunit.client.1.smithi062.stdout:6/962: rename d5/df/d40/d44/d76/l25 to d5/df/d40/d44/d4a/d41/l145 0 2024-02-22T15:50:54.858 INFO:tasks.workunit.client.1.smithi062.stdout:6/963: creat d5/d8a/d10b/f146 x:0 0 0 2024-02-22T15:50:54.860 INFO:tasks.workunit.client.1.smithi062.stdout:6/964: mknod d5/df/d40/db3/d134/c147 0 2024-02-22T15:50:54.863 INFO:tasks.workunit.client.1.smithi062.stdout:1/923: read d1/d6/de/d9/fd [34266,108452] 0 2024-02-22T15:50:54.865 INFO:tasks.workunit.client.1.smithi062.stdout:1/924: fdatasync d1/f44 0 2024-02-22T15:50:54.867 INFO:tasks.workunit.client.1.smithi062.stdout:1/925: write d1/d6/d7/d2b/d47/d24/ffa [537453,114438] 0 2024-02-22T15:50:54.869 INFO:tasks.workunit.client.1.smithi062.stdout:1/926: read d1/d6/d10/f5c [3411894,5951] 0 2024-02-22T15:50:54.874 INFO:tasks.workunit.client.1.smithi062.stdout:6/965: dwrite d5/df/d40/d44/d4a/f5e [4194304,4194304] 0 2024-02-22T15:50:54.881 INFO:tasks.workunit.client.1.smithi062.stdout:6/966: creat d5/df/d40/d109/f148 x:0 0 0 2024-02-22T15:50:54.882 INFO:tasks.workunit.client.1.smithi062.stdout:6/967: dread - d5/df/d40/d109/f133 zero size 2024-02-22T15:50:54.883 INFO:tasks.workunit.client.1.smithi062.stdout:6/968: read - d5/df/d40/d109/f133 zero size 2024-02-22T15:50:54.885 INFO:tasks.workunit.client.1.smithi062.stdout:6/969: creat d5/df/dbc/dce/d112/d100/f149 x:0 0 0 2024-02-22T15:50:54.886 INFO:tasks.workunit.client.1.smithi062.stdout:6/970: fsync f2 0 2024-02-22T15:50:54.891 INFO:tasks.workunit.client.1.smithi062.stdout:6/971: write d5/d6e/d13a/ffc [507004,74927] 0 2024-02-22T15:50:54.893 INFO:tasks.workunit.client.1.smithi062.stdout:6/972: symlink d5/df/d40/d44/d4a/d41/d85/l14a 0 2024-02-22T15:50:54.895 INFO:tasks.workunit.client.1.smithi062.stdout:6/973: mkdir d5/d8a/d14b 0 2024-02-22T15:50:54.904 INFO:tasks.workunit.client.1.smithi062.stdout:6/974: dwrite d5/df/d40/d44/d4a/d41/d85/dd1/ff0 [0,4194304] 0 2024-02-22T15:50:54.906 INFO:tasks.workunit.client.1.smithi062.stdout:6/975: fdatasync d5/df/d40/d44/f11d 0 2024-02-22T15:50:54.907 INFO:tasks.workunit.client.1.smithi062.stdout:6/976: chown d5/df/d40/d44/d4a/d41/d43/cab 21750 1 2024-02-22T15:50:54.911 INFO:tasks.workunit.client.1.smithi062.stdout:6/977: fsync d5/df/dbc/fe5 0 2024-02-22T15:50:54.917 INFO:tasks.workunit.client.1.smithi062.stdout:6/978: write d5/df/d40/d44/d76/fdb [1491515,52825] 0 2024-02-22T15:50:54.919 INFO:tasks.workunit.client.1.smithi062.stdout:6/979: mknod d5/df/d40/dac/db6/dc0/c14c 0 2024-02-22T15:50:54.922 INFO:tasks.workunit.client.1.smithi062.stdout:6/980: getdents d5/df/d40/db3/d134 0 2024-02-22T15:50:54.923 INFO:tasks.workunit.client.1.smithi062.stdout:6/981: getdents d5/df/d141 0 2024-02-22T15:50:54.924 INFO:tasks.workunit.client.1.smithi062.stdout:6/982: truncate d5/d8a/d10b/f146 618475 0 2024-02-22T15:50:54.924 INFO:tasks.workunit.client.1.smithi062.stdout:6/983: readlink d5/df/d40/d109/l13f 0 2024-02-22T15:50:54.929 INFO:tasks.workunit.client.1.smithi062.stdout:6/984: getdents d5/df/d40/dac/db6/dc0 0 2024-02-22T15:50:54.930 INFO:tasks.workunit.client.1.smithi062.stdout:6/985: truncate d5/df/dbc/dce/d112/d100/f101 354913 0 2024-02-22T15:50:54.932 INFO:tasks.workunit.client.1.smithi062.stdout:6/986: read d5/df/d40/d44/d76/d56/fca [2405631,14164] 0 2024-02-22T15:50:54.932 INFO:tasks.workunit.client.1.smithi062.stdout:6/987: read - d5/df/d40/d44/f12a zero size 2024-02-22T15:50:54.934 INFO:tasks.workunit.client.1.smithi062.stdout:6/988: truncate d5/df/d40/db3/d134/f140 791771 0 2024-02-22T15:50:54.936 INFO:tasks.workunit.client.1.smithi062.stdout:6/989: rename d5/dd/d33/d10a to d5/df/dbc/dcd/d12d/d14d 0 2024-02-22T15:50:54.939 INFO:tasks.workunit.client.1.smithi062.stdout:6/990: getdents d5/df/dbc/dcd 0 2024-02-22T15:50:54.940 INFO:tasks.workunit.client.1.smithi062.stdout:6/991: rmdir d5/df/d40/d44/d78 39 2024-02-22T15:50:54.943 INFO:tasks.workunit.client.1.smithi062.stdout:6/992: dread d5/df/d40/d44/d76/d56/f5b [4194304,4194304] 0 2024-02-22T15:50:54.944 INFO:tasks.workunit.client.1.smithi062.stdout:6/993: fdatasync d5/df/d40/f8e 0 2024-02-22T15:50:54.947 INFO:tasks.workunit.client.1.smithi062.stdout:6/994: mknod d5/df/d40/d44/d76/d56/c14e 0 2024-02-22T15:50:54.948 INFO:tasks.workunit.client.1.smithi062.stdout:6/995: read d5/d6e/d13a/f50 [1142268,53844] 0 2024-02-22T15:50:54.950 INFO:tasks.workunit.client.1.smithi062.stdout:6/996: mkdir d5/df/d40/d44/d4a/d41/d85/d88/d14f 0 2024-02-22T15:50:54.952 INFO:tasks.workunit.client.1.smithi062.stdout:6/997: read d5/dd/d33/d66/f79 [2371883,124596] 0 2024-02-22T15:50:54.954 INFO:tasks.workunit.client.1.smithi062.stdout:6/998: mknod d5/df/d40/dac/db6/c150 0 2024-02-22T15:50:54.960 INFO:tasks.workunit.client.1.smithi062.stdout:6/999: dwrite d5/dd/f94 [0,4194304] 0 2024-02-22T15:50:55.155 INFO:tasks.workunit.client.1.smithi062.stdout:1/927: sync 2024-02-22T15:50:55.155 INFO:tasks.workunit.client.1.smithi062.stdout:7/874: sync 2024-02-22T15:50:55.156 INFO:tasks.workunit.client.1.smithi062.stdout:7/875: readlink d0/d14/d19/d30/dc8/de0/lf3 0 2024-02-22T15:50:55.156 INFO:tasks.workunit.client.1.smithi062.stdout:1/928: write d1/d6/d7/d2b/d49/dcc/f123 [15041,123065] 0 2024-02-22T15:50:55.157 INFO:tasks.workunit.client.1.smithi062.stdout:1/929: read - d1/d6/d7/d2b/d49/f116 zero size 2024-02-22T15:50:55.159 INFO:tasks.workunit.client.1.smithi062.stdout:1/930: fdatasync d1/d6/d7/d2b/d49/fb1 0 2024-02-22T15:50:55.160 INFO:tasks.workunit.client.1.smithi062.stdout:1/931: dread - d1/f101 zero size 2024-02-22T15:50:55.160 INFO:tasks.workunit.client.1.smithi062.stdout:7/876: write d0/d91/f59 [2232872,19122] 0 2024-02-22T15:50:55.163 INFO:tasks.workunit.client.1.smithi062.stdout:7/877: fsync d0/d91/f59 0 2024-02-22T15:50:55.164 INFO:tasks.workunit.client.1.smithi062.stdout:1/932: write d1/d6/d7/f28 [1262384,129701] 0 2024-02-22T15:50:55.164 INFO:tasks.workunit.client.1.smithi062.stdout:7/878: chown d0/d6/d5b/cb0 11766 1 2024-02-22T15:50:55.167 INFO:tasks.workunit.client.1.smithi062.stdout:7/879: rmdir d0/d6/d2e/d4a 39 2024-02-22T15:50:55.175 INFO:tasks.workunit.client.1.smithi062.stdout:7/880: dwrite d0/d6/f23 [0,4194304] 0 2024-02-22T15:50:55.175 INFO:tasks.workunit.client.1.smithi062.stdout:1/933: dwrite d1/d6/d7/d2b/fae [0,4194304] 0 2024-02-22T15:50:55.176 INFO:tasks.workunit.client.1.smithi062.stdout:7/881: chown d0/d14/d19/d30/d37 107717125 1 2024-02-22T15:50:55.177 INFO:tasks.workunit.client.1.smithi062.stdout:7/882: readlink d0/d6/d2e/d58/d5d/dc4/lc9 0 2024-02-22T15:50:55.184 INFO:tasks.workunit.client.1.smithi062.stdout:1/934: dwrite d1/d6/d7/d2b/d47/d24/f9f [0,4194304] 0 2024-02-22T15:50:55.184 INFO:tasks.workunit.client.1.smithi062.stdout:7/883: rename d0/d14/d13/c64 to d0/d14/d19/d30/dc8/de0/c120 0 2024-02-22T15:50:55.192 INFO:tasks.workunit.client.1.smithi062.stdout:7/884: creat d0/d14/d19/d77/f121 x:0 0 0 2024-02-22T15:50:55.192 INFO:tasks.workunit.client.1.smithi062.stdout:1/935: truncate d1/d6/d7/d2b/d56/d95/f9b 1412717 0 2024-02-22T15:50:55.198 INFO:tasks.workunit.client.1.smithi062.stdout:7/885: dwrite d0/d6/d2e/d4a/d5a/db3/fbe [0,4194304] 0 2024-02-22T15:50:55.200 INFO:tasks.workunit.client.1.smithi062.stdout:7/886: readlink d0/d14/l6d 0 2024-02-22T15:50:55.202 INFO:tasks.workunit.client.1.smithi062.stdout:7/887: creat d0/d14/d19/d30/dbd/f122 x:0 0 0 2024-02-22T15:50:55.203 INFO:tasks.workunit.client.1.smithi062.stdout:7/888: write d0/d6/d2e/fa3 [533946,90666] 0 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: pgmap v43: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 106 MiB/s rd, 268 MiB/s wr, 546 op/s 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: Upgrade: Updating mgr.smithi019.yvjrkp 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:50:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:54 smithi062 ceph-mon[48130]: Deploying daemon mgr.smithi019.yvjrkp on smithi019 2024-02-22T15:50:55.213 INFO:tasks.workunit.client.1.smithi062.stdout:7/889: dwrite d0/d6/d2e/d4a/d5a/db3/fbe [0,4194304] 0 2024-02-22T15:50:55.216 INFO:tasks.workunit.client.1.smithi062.stdout:1/936: dwrite d1/d6/d14/dce/fd0 [0,4194304] 0 2024-02-22T15:50:55.216 INFO:tasks.workunit.client.1.smithi062.stdout:7/890: fdatasync d0/d91/fbb 0 2024-02-22T15:50:55.218 INFO:tasks.workunit.client.1.smithi062.stdout:7/891: write d0/d6/d2e/d2f/f57 [100771,117342] 0 2024-02-22T15:50:55.220 INFO:tasks.workunit.client.1.smithi062.stdout:7/892: read d0/d6/f52 [1758946,76979] 0 2024-02-22T15:50:55.222 INFO:tasks.workunit.client.1.smithi062.stdout:7/893: chown d0/d14/d19/c3f 706 1 2024-02-22T15:50:55.228 INFO:tasks.workunit.client.1.smithi062.stdout:7/894: read d0/d6/d5b/faa [2757143,126804] 0 2024-02-22T15:50:55.230 INFO:tasks.workunit.client.1.smithi062.stdout:7/895: write d0/d14/d19/d30/d45/dc3/fda [209958,61803] 0 2024-02-22T15:50:55.232 INFO:tasks.workunit.client.1.smithi062.stdout:1/937: dread d1/d6/d7/f28 [0,4194304] 0 2024-02-22T15:50:55.236 INFO:tasks.workunit.client.1.smithi062.stdout:1/938: link d1/d6/d7/d2b/d47/d7e/cac d1/d6/de/d9/c12f 0 2024-02-22T15:50:55.241 INFO:tasks.workunit.client.1.smithi062.stdout:7/896: write d0/d91/d7c/fb8 [206447,8263] 0 2024-02-22T15:50:55.242 INFO:tasks.workunit.client.1.smithi062.stdout:1/939: dread d1/d6/d7/f98 [0,4194304] 0 2024-02-22T15:50:55.243 INFO:tasks.workunit.client.1.smithi062.stdout:7/897: stat d0/d14/d19/d30/dc8/de0/c120 0 2024-02-22T15:50:55.245 INFO:tasks.workunit.client.1.smithi062.stdout:1/940: symlink d1/d6/d7/de6/l130 0 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: pgmap v43: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 106 MiB/s rd, 268 MiB/s wr, 546 op/s 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: Upgrade: Updating mgr.smithi019.yvjrkp 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:50:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:50:55.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:54 smithi019 ceph-mon[67418]: Deploying daemon mgr.smithi019.yvjrkp on smithi019 2024-02-22T15:50:55.246 INFO:tasks.workunit.client.1.smithi062.stdout:7/898: mkdir d0/d6/d2e/d123 0 2024-02-22T15:50:55.248 INFO:tasks.workunit.client.1.smithi062.stdout:7/899: symlink d0/d14/d19/d77/dcf/l124 0 2024-02-22T15:50:55.250 INFO:tasks.workunit.client.1.smithi062.stdout:1/941: link d1/ca d1/d6/de/d9/d100/c131 0 2024-02-22T15:50:55.250 INFO:tasks.workunit.client.1.smithi062.stdout:1/942: chown d1/d6/d14/fe4 253834 1 2024-02-22T15:50:55.251 INFO:tasks.workunit.client.1.smithi062.stdout:7/900: creat d0/d14/d19/d77/dcf/d118/f125 x:0 0 0 2024-02-22T15:50:55.251 INFO:tasks.workunit.client.1.smithi062.stdout:1/943: write d1/d6/d7/f91 [508052,56536] 0 2024-02-22T15:50:55.256 INFO:tasks.workunit.client.1.smithi062.stdout:1/944: getdents d1/d6/d10 0 2024-02-22T15:50:55.259 INFO:tasks.workunit.client.1.smithi062.stdout:7/901: dwrite d0/d6/d2e/d58/d5d/f10c [0,4194304] 0 2024-02-22T15:50:55.263 INFO:tasks.workunit.client.1.smithi062.stdout:7/902: getdents d0/d6/d2e/d123 0 2024-02-22T15:50:55.267 INFO:tasks.workunit.client.1.smithi062.stdout:1/945: dread d1/d6/d7/f29 [0,4194304] 0 2024-02-22T15:50:55.272 INFO:tasks.workunit.client.1.smithi062.stdout:1/946: link d1/d6/d7/d2b/d49/dcc/cf1 d1/d6/d7/d2b/d47/c132 0 2024-02-22T15:50:55.287 INFO:tasks.workunit.client.1.smithi062.stdout:1/947: dread d1/f30 [0,4194304] 0 2024-02-22T15:50:55.289 INFO:tasks.workunit.client.1.smithi062.stdout:1/948: mknod d1/d6/d7/de6/c133 0 2024-02-22T15:50:55.295 INFO:tasks.workunit.client.1.smithi062.stdout:1/949: dwrite d1/d6/d7/d4a/d65/f111 [0,4194304] 0 2024-02-22T15:50:55.297 INFO:tasks.workunit.client.1.smithi062.stdout:1/950: fdatasync d1/d6/de/f42 0 2024-02-22T15:50:55.299 INFO:tasks.workunit.client.1.smithi062.stdout:1/951: stat d1/d6/d36/d76/fdc 0 2024-02-22T15:50:55.301 INFO:tasks.workunit.client.1.smithi062.stdout:1/952: symlink d1/l134 0 2024-02-22T15:50:55.303 INFO:tasks.workunit.client.1.smithi062.stdout:1/953: stat d1/d6/d7/d2b/d47/ca8 0 2024-02-22T15:50:55.306 INFO:tasks.workunit.client.1.smithi062.stdout:1/954: creat d1/d6/d7/d2b/d49/f135 x:0 0 0 2024-02-22T15:50:55.312 INFO:tasks.workunit.client.1.smithi062.stdout:1/955: dwrite d1/d6/d7/d2b/d49/d74/f85 [4194304,4194304] 0 2024-02-22T15:50:55.316 INFO:tasks.workunit.client.1.smithi062.stdout:1/956: unlink d1/d6/d7/d2b/d56/d95/fe3 0 2024-02-22T15:50:55.330 INFO:tasks.workunit.client.1.smithi062.stdout:1/957: rename d1/d6/d7/d2b to d1/d136 0 2024-02-22T15:50:55.332 INFO:tasks.workunit.client.1.smithi062.stdout:1/958: mknod d1/d6/d36/c137 0 2024-02-22T15:50:55.433 INFO:tasks.workunit.client.1.smithi062.stdout:7/903: sync 2024-02-22T15:50:55.439 INFO:tasks.workunit.client.1.smithi062.stdout:7/904: creat d0/d6/d2e/d4a/f126 x:0 0 0 2024-02-22T15:50:55.441 INFO:tasks.workunit.client.1.smithi062.stdout:7/905: creat d0/d14/d19/d30/dbd/f127 x:0 0 0 2024-02-22T15:50:55.443 INFO:tasks.workunit.client.1.smithi062.stdout:7/906: write d0/d6/d2e/d4a/d5a/db3/fc2 [1497336,63488] 0 2024-02-22T15:50:55.445 INFO:tasks.workunit.client.1.smithi062.stdout:7/907: write d0/d14/d13/f46 [6443734,95517] 0 2024-02-22T15:50:55.449 INFO:tasks.workunit.client.1.smithi062.stdout:7/908: symlink d0/d6/d2e/l128 0 2024-02-22T15:50:55.449 INFO:tasks.workunit.client.1.smithi062.stdout:7/909: dread - d0/d6/d2e/d58/dc0/fe4 zero size 2024-02-22T15:50:55.453 INFO:tasks.workunit.client.1.smithi062.stdout:7/910: rename d0/d6/d2e/d4a/d5a/cb1 to d0/d14/d19/d30/dc8/d100/c129 0 2024-02-22T15:50:55.454 INFO:tasks.workunit.client.1.smithi062.stdout:7/911: symlink d0/d6/d2e/d55/d80/l12a 0 2024-02-22T15:50:55.457 INFO:tasks.workunit.client.1.smithi062.stdout:7/912: mknod d0/d14/d19/d30/d45/dc3/dcb/d10b/c12b 0 2024-02-22T15:50:55.458 INFO:tasks.workunit.client.1.smithi062.stdout:7/913: mknod d0/d14/d19/d30/d45/dc3/c12c 0 2024-02-22T15:50:55.459 INFO:tasks.workunit.client.1.smithi062.stdout:7/914: write d0/d14/d19/d30/dc8/d100/f109 [4665272,101483] 0 2024-02-22T15:50:55.461 INFO:tasks.workunit.client.1.smithi062.stdout:7/915: write d0/d14/d13/f36 [644534,127038] 0 2024-02-22T15:50:55.468 INFO:tasks.workunit.client.1.smithi062.stdout:7/916: dwrite d0/d6/d2e/d35/ddc/fe5 [0,4194304] 0 2024-02-22T15:50:55.472 INFO:tasks.workunit.client.1.smithi062.stdout:7/917: unlink d0/d14/d19/d30/cf1 0 2024-02-22T15:50:55.472 INFO:tasks.workunit.client.1.smithi062.stdout:1/959: sync 2024-02-22T15:50:55.474 INFO:tasks.workunit.client.1.smithi062.stdout:7/918: mkdir d0/d6/d2e/d123/d12d 0 2024-02-22T15:50:55.475 INFO:tasks.workunit.client.1.smithi062.stdout:1/960: rename d1/d136/d56/d95 to d1/d136/d47/d7e/d138 0 2024-02-22T15:50:55.478 INFO:tasks.workunit.client.1.smithi062.stdout:7/919: creat d0/d14/d19/d30/f12e x:0 0 0 2024-02-22T15:50:55.485 INFO:tasks.workunit.client.1.smithi062.stdout:7/920: dwrite d0/d14/d19/d30/f42 [0,4194304] 0 2024-02-22T15:50:55.487 INFO:tasks.workunit.client.1.smithi062.stdout:1/961: write d1/d6/d10/f1c [1830379,73430] 0 2024-02-22T15:50:55.489 INFO:tasks.workunit.client.1.smithi062.stdout:1/962: chown d1/d66/d11e 666941 1 2024-02-22T15:50:55.489 INFO:tasks.workunit.client.1.smithi062.stdout:1/963: chown d1/d136/d47/dff 307 1 2024-02-22T15:50:55.491 INFO:tasks.workunit.client.1.smithi062.stdout:7/921: creat d0/d6/d2e/d55/f12f x:0 0 0 2024-02-22T15:50:55.492 INFO:tasks.workunit.client.1.smithi062.stdout:1/964: mknod d1/d136/d49/d74/dad/c139 0 2024-02-22T15:50:55.495 INFO:tasks.workunit.client.1.smithi062.stdout:7/922: unlink d0/d6/d2e/d4a/d5a/cad 0 2024-02-22T15:50:55.496 INFO:tasks.workunit.client.1.smithi062.stdout:7/923: stat d0/d14/d19 0 2024-02-22T15:50:55.504 INFO:tasks.workunit.client.1.smithi062.stdout:1/965: write d1/f82 [983964,86944] 0 2024-02-22T15:50:55.507 INFO:tasks.workunit.client.1.smithi062.stdout:1/966: write d1/d6/d7/f28 [6926331,25989] 0 2024-02-22T15:50:55.510 INFO:tasks.workunit.client.1.smithi062.stdout:7/924: dwrite d0/d6/f52 [0,4194304] 0 2024-02-22T15:50:55.510 INFO:tasks.workunit.client.1.smithi062.stdout:1/967: mknod d1/d6/de/d9/d100/c13a 0 2024-02-22T15:50:55.511 INFO:tasks.workunit.client.1.smithi062.stdout:1/968: dread - d1/d6/d36/f110 zero size 2024-02-22T15:50:55.513 INFO:tasks.workunit.client.1.smithi062.stdout:7/925: mknod d0/d14/d19/d30/dbd/c130 0 2024-02-22T15:50:55.515 INFO:tasks.workunit.client.1.smithi062.stdout:1/969: unlink d1/d136/d49/f51 0 2024-02-22T15:50:55.516 INFO:tasks.workunit.client.1.smithi062.stdout:1/970: write d1/d6/d7/de6/fe7 [1820053,10278] 0 2024-02-22T15:50:55.518 INFO:tasks.workunit.client.1.smithi062.stdout:7/926: symlink d0/d6/d2e/d4a/d5a/d105/l131 0 2024-02-22T15:50:55.518 INFO:tasks.workunit.client.1.smithi062.stdout:1/971: chown d1/d6/d36/d37/dee 0 1 2024-02-22T15:50:55.519 INFO:tasks.workunit.client.1.smithi062.stdout:7/927: read - d0/d6/d2e/d4a/d5a/fdb zero size 2024-02-22T15:50:55.521 INFO:tasks.workunit.client.1.smithi062.stdout:1/972: truncate d1/d6/f61 3359060 0 2024-02-22T15:50:55.521 INFO:tasks.workunit.client.1.smithi062.stdout:7/928: symlink d0/d14/d13/l132 0 2024-02-22T15:50:55.523 INFO:tasks.workunit.client.1.smithi062.stdout:1/973: symlink d1/d6/d10/l13b 0 2024-02-22T15:50:55.526 INFO:tasks.workunit.client.1.smithi062.stdout:1/974: getdents d1/d6/d14/dce 0 2024-02-22T15:50:55.530 INFO:tasks.workunit.client.1.smithi062.stdout:7/929: dwrite d0/d14/d19/d77/dcf/f11f [0,4194304] 0 2024-02-22T15:50:55.532 INFO:tasks.workunit.client.1.smithi062.stdout:1/975: mkdir d1/d6/d7/d4a/d65/d83/d11c/d13c 0 2024-02-22T15:50:55.535 INFO:tasks.workunit.client.1.smithi062.stdout:7/930: rmdir d0/d6/d2e/d4a/d5a 39 2024-02-22T15:50:55.536 INFO:tasks.workunit.client.1.smithi062.stdout:7/931: chown d0/d14/l18 52 1 2024-02-22T15:50:55.539 INFO:tasks.workunit.client.1.smithi062.stdout:7/932: symlink d0/d6/d2e/d123/l133 0 2024-02-22T15:50:55.540 INFO:tasks.workunit.client.1.smithi062.stdout:7/933: dread - d0/d14/d19/d30/d45/dc3/dcb/f102 zero size 2024-02-22T15:50:55.542 INFO:tasks.workunit.client.1.smithi062.stdout:7/934: mkdir d0/d14/d19/d134 0 2024-02-22T15:50:55.544 INFO:tasks.workunit.client.1.smithi062.stdout:7/935: truncate d0/d14/d19/d30/dbd/d111/f11a 127626 0 2024-02-22T15:50:55.550 INFO:tasks.workunit.client.1.smithi062.stdout:7/936: dwrite d0/d14/d19/d77/dcf/d118/f125 [0,4194304] 0 2024-02-22T15:50:55.555 INFO:tasks.workunit.client.1.smithi062.stdout:7/937: mknod d0/d6/d2e/d58/dc0/c135 0 2024-02-22T15:50:55.572 INFO:tasks.workunit.client.1.smithi062.stdout:7/938: dread d0/d14/f17 [0,4194304] 0 2024-02-22T15:50:55.574 INFO:tasks.workunit.client.1.smithi062.stdout:7/939: unlink d0/d14/d19/d30/d37/la6 0 2024-02-22T15:50:55.577 INFO:tasks.workunit.client.1.smithi062.stdout:7/940: truncate d0/d14/d19/f1a 1971070 0 2024-02-22T15:50:55.587 INFO:tasks.workunit.client.1.smithi062.stdout:7/941: dread d0/d6/d2e/d55/d80/d8a/fa0 [0,4194304] 0 2024-02-22T15:50:55.589 INFO:tasks.workunit.client.1.smithi062.stdout:7/942: chown d0/d6/d5b/cd3 37187915 1 2024-02-22T15:50:55.592 INFO:tasks.workunit.client.1.smithi062.stdout:7/943: link d0/d14/c10a d0/d14/d19/d30/d45/c136 0 2024-02-22T15:50:55.595 INFO:tasks.workunit.client.1.smithi062.stdout:7/944: truncate d0/d6/d2e/d55/d80/d8a/f8f 474123 0 2024-02-22T15:50:55.596 INFO:tasks.workunit.client.1.smithi062.stdout:7/945: dread - d0/d91/f11d zero size 2024-02-22T15:50:55.599 INFO:tasks.workunit.client.1.smithi062.stdout:7/946: truncate d0/d6/d2e/d55/d80/fb9 992594 0 2024-02-22T15:50:55.604 INFO:tasks.workunit.client.1.smithi062.stdout:7/947: write d0/d14/d19/f7b [769162,88385] 0 2024-02-22T15:50:55.606 INFO:tasks.workunit.client.1.smithi062.stdout:7/948: read d0/d14/d13/f3e [2852372,102505] 0 2024-02-22T15:50:55.608 INFO:tasks.workunit.client.1.smithi062.stdout:7/949: fsync d0/d14/d13/f48 0 2024-02-22T15:50:55.610 INFO:tasks.workunit.client.1.smithi062.stdout:7/950: creat d0/d6/d2e/d55/d72/f137 x:0 0 0 2024-02-22T15:50:55.613 INFO:tasks.workunit.client.1.smithi062.stdout:7/951: dread d0/d14/d13/f36 [4194304,4194304] 0 2024-02-22T15:50:55.615 INFO:tasks.workunit.client.1.smithi062.stdout:7/952: creat d0/d14/d19/d30/d45/dc3/de6/f138 x:0 0 0 2024-02-22T15:50:55.616 INFO:tasks.workunit.client.1.smithi062.stdout:7/953: stat d0/d6/d2e/d4a/f1f 0 2024-02-22T15:50:55.618 INFO:tasks.workunit.client.1.smithi062.stdout:7/954: creat d0/d14/d13/f139 x:0 0 0 2024-02-22T15:50:55.620 INFO:tasks.workunit.client.1.smithi062.stdout:7/955: mknod d0/d14/c13a 0 2024-02-22T15:50:55.623 INFO:tasks.workunit.client.1.smithi062.stdout:7/956: truncate d0/d6/d2e/f86 2218036 0 2024-02-22T15:50:55.626 INFO:tasks.workunit.client.1.smithi062.stdout:7/957: read d0/d14/d13/f2a [6052255,108249] 0 2024-02-22T15:50:55.628 INFO:tasks.workunit.client.1.smithi062.stdout:7/958: symlink d0/d14/d19/d30/l13b 0 2024-02-22T15:50:55.629 INFO:tasks.workunit.client.1.smithi062.stdout:7/959: fsync d0/d6/d5b/d60/fdf 0 2024-02-22T15:50:55.632 INFO:tasks.workunit.client.1.smithi062.stdout:7/960: mkdir d0/d6/d2e/d4a/d5a/d105/d13c 0 2024-02-22T15:50:55.634 INFO:tasks.workunit.client.1.smithi062.stdout:7/961: creat d0/d6/d2e/d4a/d5a/d105/f13d x:0 0 0 2024-02-22T15:50:55.635 INFO:tasks.workunit.client.1.smithi062.stdout:7/962: symlink d0/d6/d2e/d4a/d5a/d105/d13c/l13e 0 2024-02-22T15:50:55.637 INFO:tasks.workunit.client.1.smithi062.stdout:7/963: mknod d0/d6/d2e/d4a/c13f 0 2024-02-22T15:50:55.638 INFO:tasks.workunit.client.1.smithi062.stdout:7/964: rmdir d0/d14/d76 39 2024-02-22T15:50:55.640 INFO:tasks.workunit.client.1.smithi062.stdout:7/965: mkdir d0/d6/d2e/d58/dc0/d140 0 2024-02-22T15:50:55.642 INFO:tasks.workunit.client.1.smithi062.stdout:7/966: symlink d0/d14/d19/d30/d45/dc3/dcb/d10b/l141 0 2024-02-22T15:50:55.644 INFO:tasks.workunit.client.1.smithi062.stdout:7/967: symlink d0/d6/d2e/d2f/d88/l142 0 2024-02-22T15:50:55.646 INFO:tasks.workunit.client.1.smithi062.stdout:7/968: creat d0/d6/d2e/d58/d5d/f143 x:0 0 0 2024-02-22T15:50:55.647 INFO:tasks.workunit.client.1.smithi062.stdout:7/969: write d0/f89 [3675409,3881] 0 2024-02-22T15:50:55.649 INFO:tasks.workunit.client.1.smithi062.stdout:7/970: truncate d0/f9c 998539 0 2024-02-22T15:50:55.652 INFO:tasks.workunit.client.1.smithi062.stdout:7/971: mknod d0/d6/d2e/d58/d5d/c144 0 2024-02-22T15:50:55.655 INFO:tasks.workunit.client.1.smithi062.stdout:7/972: mknod d0/d6/d2e/d4a/d5a/db3/c145 0 2024-02-22T15:50:55.656 INFO:tasks.workunit.client.1.smithi062.stdout:7/973: stat d0/d14/d19/d30/dc8/de0/f115 0 2024-02-22T15:50:55.657 INFO:tasks.workunit.client.1.smithi062.stdout:7/974: chown d0/d6/d2e/d55/d80 8 1 2024-02-22T15:50:55.658 INFO:tasks.workunit.client.1.smithi062.stdout:7/975: chown d0/d14/d19/d30/d45/dc3/c12c 11041465 1 2024-02-22T15:50:55.661 INFO:tasks.workunit.client.1.smithi062.stdout:7/976: symlink d0/d14/d19/d30/d45/dc3/dcb/l146 0 2024-02-22T15:50:55.663 INFO:tasks.workunit.client.1.smithi062.stdout:7/977: mkdir d0/d6/d2e/d4a/d147 0 2024-02-22T15:50:55.663 INFO:tasks.workunit.client.1.smithi062.stdout:1/976: sync 2024-02-22T15:50:55.663 INFO:tasks.workunit.client.1.smithi062.stdout:7/978: fsync d0/d6/d2e/d55/d72/f137 0 2024-02-22T15:50:55.667 INFO:tasks.workunit.client.1.smithi062.stdout:7/979: mkdir d0/d6/d2e/d35/ddc/d148 0 2024-02-22T15:50:55.668 INFO:tasks.workunit.client.1.smithi062.stdout:1/977: truncate d1/d6/f106 384820 0 2024-02-22T15:50:55.673 INFO:tasks.workunit.client.1.smithi062.stdout:7/980: truncate d0/d14/d19/d30/dbd/d111/f6c 3000564 0 2024-02-22T15:50:55.674 INFO:tasks.workunit.client.1.smithi062.stdout:1/978: rmdir d1/d136/d47/d63/db2/de0/d115 0 2024-02-22T15:50:55.674 INFO:tasks.workunit.client.1.smithi062.stdout:7/981: fsync d0/d6/d2e/d55/f12f 0 2024-02-22T15:50:55.684 INFO:tasks.workunit.client.1.smithi062.stdout:7/982: dwrite d0/d6/d2e/fa3 [0,4194304] 0 2024-02-22T15:50:55.694 INFO:tasks.workunit.client.1.smithi062.stdout:1/979: dwrite d1/f44 [0,4194304] 0 2024-02-22T15:50:55.698 INFO:tasks.workunit.client.1.smithi062.stdout:1/980: rmdir d1/d6/d7/d4a/d65 39 2024-02-22T15:50:55.699 INFO:tasks.workunit.client.1.smithi062.stdout:7/983: write d0/d6/d2e/d55/f69 [255354,52494] 0 2024-02-22T15:50:55.700 INFO:tasks.workunit.client.1.smithi062.stdout:1/981: mkdir d1/d6/d7/d8a/d13d 0 2024-02-22T15:50:55.703 INFO:tasks.workunit.client.1.smithi062.stdout:1/982: mknod d1/d136/d49/d74/c13e 0 2024-02-22T15:50:55.706 INFO:tasks.workunit.client.1.smithi062.stdout:7/984: rename d0/d14/d19/d30/d45/dc3 to d0/d6/d2e/d55/d149 0 2024-02-22T15:50:55.710 INFO:tasks.workunit.client.1.smithi062.stdout:7/985: rename d0/d14/f29 to d0/d6/d2e/da2/f14a 0 2024-02-22T15:50:55.712 INFO:tasks.workunit.client.1.smithi062.stdout:7/986: creat d0/f14b x:0 0 0 2024-02-22T15:50:55.714 INFO:tasks.workunit.client.1.smithi062.stdout:7/987: mknod d0/d6/d2e/d4a/d5a/d105/c14c 0 2024-02-22T15:50:55.717 INFO:tasks.workunit.client.1.smithi062.stdout:7/988: dread d0/d6/d2e/d55/d80/d8a/fa0 [0,4194304] 0 2024-02-22T15:50:55.718 INFO:tasks.workunit.client.1.smithi062.stdout:7/989: readlink d0/l96 0 2024-02-22T15:50:55.718 INFO:tasks.workunit.client.1.smithi062.stdout:7/990: fsync d0/d14/d13/f139 0 2024-02-22T15:50:55.726 INFO:tasks.workunit.client.1.smithi062.stdout:1/983: rename d1/d136/d49/d74/df5/c10c to d1/d6/d36/d37/dee/c13f 0 2024-02-22T15:50:55.758 INFO:tasks.workunit.client.1.smithi062.stdout:1/984: truncate d1/d6/d7/d8a/dca/fdf 509295 0 2024-02-22T15:50:55.759 INFO:tasks.workunit.client.1.smithi062.stdout:7/991: getdents d0/d6/d2e/d55/d80/d8a 0 2024-02-22T15:50:55.762 INFO:tasks.workunit.client.1.smithi062.stdout:7/992: creat d0/d6/d2e/d4a/d5a/f14d x:0 0 0 2024-02-22T15:50:55.763 INFO:tasks.workunit.client.1.smithi062.stdout:7/993: mkdir d0/d6/d2e/d2f/d14e 0 2024-02-22T15:50:55.765 INFO:tasks.workunit.client.1.smithi062.stdout:7/994: write d0/d14/d19/d30/dc8/d100/f109 [4085907,102526] 0 2024-02-22T15:50:55.788 INFO:tasks.workunit.client.1.smithi062.stdout:7/995: write d0/d6/d2e/d2f/f3a [4110487,111495] 0 2024-02-22T15:50:55.799 INFO:tasks.workunit.client.1.smithi062.stdout:7/996: dread d0/d6/d2e/da2/f14a [0,4194304] 0 2024-02-22T15:50:55.804 INFO:tasks.workunit.client.1.smithi062.stdout:7/997: dwrite d0/d14/f38 [0,4194304] 0 2024-02-22T15:50:55.807 INFO:tasks.workunit.client.1.smithi062.stdout:7/998: rename d0/d6/d2e/d58/d5d/c144 to d0/d6/d2e/d2f/d88/c14f 0 2024-02-22T15:50:55.808 INFO:tasks.workunit.client.1.smithi062.stdout:7/999: chown d0/d14/d19/d30/dbd/d111/f11a 257986044 1 2024-02-22T15:50:55.947 INFO:tasks.workunit.client.1.smithi062.stdout:1/985: sync 2024-02-22T15:50:55.956 INFO:tasks.workunit.client.1.smithi062.stdout:1/986: dwrite d1/d6/f61 [0,4194304] 0 2024-02-22T15:50:55.959 INFO:tasks.workunit.client.1.smithi062.stdout:1/987: mkdir d1/d6/de/d9/d100/d140 0 2024-02-22T15:50:55.960 INFO:tasks.workunit.client.1.smithi062.stdout:1/988: fsync d1/f30 0 2024-02-22T15:50:55.963 INFO:tasks.workunit.client.1.smithi062.stdout:1/989: mkdir d1/d6/d7/d8a/dca/d141 0 2024-02-22T15:50:55.964 INFO:tasks.workunit.client.1.smithi062.stdout:1/990: write d1/d6/d14/dce/fd0 [4930075,12196] 0 2024-02-22T15:50:55.968 INFO:tasks.workunit.client.1.smithi062.stdout:1/991: dread d1/f30 [0,4194304] 0 2024-02-22T15:50:55.970 INFO:tasks.workunit.client.1.smithi062.stdout:1/992: mknod d1/d136/d96/df3/c142 0 2024-02-22T15:50:55.970 INFO:tasks.workunit.client.1.smithi062.stdout:1/993: chown d1/d136/d96/ffc 10281078 1 2024-02-22T15:50:55.971 INFO:tasks.workunit.client.1.smithi062.stdout:1/994: fsync d1/d136/d47/d63/ff4 0 2024-02-22T15:50:55.976 INFO:tasks.workunit.client.1.smithi062.stdout:1/995: dwrite d1/d6/d7/d8a/dca/fe9 [0,4194304] 0 2024-02-22T15:50:55.977 INFO:tasks.workunit.client.1.smithi062.stdout:1/996: chown d1/d6/de/c4d 753 1 2024-02-22T15:50:56.031 INFO:tasks.workunit.client.1.smithi062.stdout:1/997: sync 2024-02-22T15:50:56.033 INFO:tasks.workunit.client.1.smithi062.stdout:1/998: chown d1/d6/d7/d4a/d65/d83/faa 93251035 1 2024-02-22T15:50:56.034 INFO:tasks.workunit.client.1.smithi062.stdout:1/999: chown d1/d6/d36/d76/dfe/c104 214119007 1 2024-02-22T15:50:56.038 INFO:tasks.workunit.client.1.smithi062.stderr:+ rm -rf -- ./tmp.WKoMFo77vo 2024-02-22T15:50:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:56 smithi062 ceph-mon[48130]: pgmap v44: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 133 MiB/s rd, 335 MiB/s wr, 736 op/s 2024-02-22T15:50:57.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:56 smithi019 ceph-mon[67418]: pgmap v44: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 133 MiB/s rd, 335 MiB/s wr, 736 op/s 2024-02-22T15:50:59.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:59 smithi019 ceph-mon[67418]: pgmap v45: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 78 MiB/s rd, 195 MiB/s wr, 458 op/s 2024-02-22T15:50:59.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:59 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:59 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:59 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:50:59.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:59 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:50:59 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:59 smithi062 ceph-mon[48130]: pgmap v45: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 78 MiB/s rd, 195 MiB/s wr, 458 op/s 2024-02-22T15:50:59.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:59 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:59 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:59 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:50:59.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:59 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:50:59.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:50:59 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:01.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:01 smithi062 ceph-mon[48130]: pgmap v46: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 81 MiB/s rd, 204 MiB/s wr, 561 op/s 2024-02-22T15:51:01.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:01 smithi019 ceph-mon[67418]: pgmap v46: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 81 MiB/s rd, 204 MiB/s wr, 561 op/s 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.850+0000 7fe803c64700 1 -- 172.21.15.19:0/453412509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fcfb0 msgr2=0x7fe7fc0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.850+0000 7fe803c64700 1 --2- 172.21.15.19:0/453412509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fcfb0 0x7fe7fc0fd420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec004940 tx=0x7fe7ec009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.850+0000 7fe803c64700 1 -- 172.21.15.19:0/453412509 shutdown_connections 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.850+0000 7fe803c64700 1 --2- 172.21.15.19:0/453412509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fcfb0 0x7fe7fc0fd420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.850+0000 7fe803c64700 1 --2- 172.21.15.19:0/453412509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 -- 172.21.15.19:0/453412509 >> 172.21.15.19:0/453412509 conn(0x7fe7fc0f74e0 msgr2=0x7fe7fc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 -- 172.21.15.19:0/453412509 shutdown_connections 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 -- 172.21.15.19:0/453412509 wait complete. 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 Processor -- start 2024-02-22T15:51:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 -- start start 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc0fcfb0 0x7fe7fc0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0fe980 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.851+0000 7fe803c64700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0feaf0 con 0x7fe7fc0fcfb0 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52212/0 (socket says 172.21.15.19:52212) 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 -- 172.21.15.19:0/2469081151 learned_addr learned my addr 172.21.15.19:0/2469081151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe8011ff700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc0fcfb0 0x7fe7fc0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 -- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc0fcfb0 msgr2=0x7fe7fc0fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc0fcfb0 0x7fe7fc0fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 -- 172.21.15.19:0/2469081151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec005040 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe801a00700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fff00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800eb70 tx=0x7fe7f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f8007cc0 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7f800f040 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.852+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc0fed70 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.853+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7f800c9f0 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.853+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc115840 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.854+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc04bed0 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.855+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7fe7f800cb50 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.855+0000 7fe7f27fc700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fe7e806aec0 0x7fe7e806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.855+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe7f8089330 con 0x7fe7fc0fbe40 2024-02-22T15:51:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.856+0000 7fe8011ff700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fe7e806aec0 0x7fe7e806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.856+0000 7fe8011ff700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fe7e806aec0 0x7fe7e806d370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000c00 tx=0x7fe7ec009e30 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:01.861+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fe7f8057100 con 0x7fe7fc0fbe40 2024-02-22T15:51:02.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.164+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7fc065dd0 con 0x7fe7e806aec0 2024-02-22T15:51:02.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.165+0000 7fe7f27fc700 1 -- 172.21.15.19:0/2469081151 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fe7fc065dd0 con 0x7fe7e806aec0 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fe7e806aec0 msgr2=0x7fe7e806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fe7e806aec0 0x7fe7e806d370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000c00 tx=0x7fe7ec009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 msgr2=0x7fe7fc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fff00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800eb70 tx=0x7fe7f800b040 comp rx=0 tx=0).stop 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 shutdown_connections 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fe7e806aec0 0x7fe7e806d370 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0fbe40 0x7fe7fc0fff00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 --2- 172.21.15.19:0/2469081151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7fc0fcfb0 0x7fe7fc0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.167+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 >> 172.21.15.19:0/2469081151 conn(0x7fe7fc0f74e0 msgr2=0x7fe7fc0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:02.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.168+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 shutdown_connections 2024-02-22T15:51:02.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.168+0000 7fe803c64700 1 -- 172.21.15.19:0/2469081151 wait complete. 2024-02-22T15:51:02.164 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/2588620690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fd440 msgr2=0x7f3a780fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/2588620690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fd440 0x7f3a780fd850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c004940 tx=0x7f3a6c009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/2588620690 shutdown_connections 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/2588620690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fc1a0 0x7f3a780fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/2588620690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fd440 0x7f3a780fd850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/2588620690 >> 172.21.15.19:0/2588620690 conn(0x7f3a780f6ca0 msgr2=0x7f3a780f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/2588620690 shutdown_connections 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/2588620690 wait complete. 2024-02-22T15:51:02.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.310+0000 7f3a7ebb5700 1 Processor -- start 2024-02-22T15:51:02.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7ebb5700 1 -- start start 2024-02-22T15:51:02.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7ebb5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 0x7f3a7810d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7ebb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fd440 0x7f3a7810de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7ebb5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7810e440 con 0x7f3a780fc1a0 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7ebb5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7810ece0 con 0x7f3a780fd440 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7c951700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 0x7f3a7810d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7c951700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 0x7f3a7810d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52242/0 (socket says 172.21.15.19:52242) 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7c951700 1 -- 172.21.15.19:0/3991836443 learned_addr learned my addr 172.21.15.19:0/3991836443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a77fff700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fd440 0x7f3a7810de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7c951700 1 -- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fd440 msgr2=0x7f3a7810de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.311+0000 7f3a7c951700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fd440 0x7f3a7810de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.312+0000 7f3a7c951700 1 -- 172.21.15.19:0/3991836443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c005040 con 0x7f3a780fc1a0 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.312+0000 7f3a77fff700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fd440 0x7f3a7810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:51:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.312+0000 7f3a7c951700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 0x7f3a7810d900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.313+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6c021070 con 0x7f3a780fc1a0 2024-02-22T15:51:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.313+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a6c01c400 con 0x7f3a780fc1a0 2024-02-22T15:51:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.313+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6c00e070 con 0x7f3a780fc1a0 2024-02-22T15:51:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.314+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7810ef60 con 0x7f3a780fc1a0 2024-02-22T15:51:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.314+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7810f410 con 0x7f3a780fc1a0 2024-02-22T15:51:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.315+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a780fa940 con 0x7f3a780fc1a0 2024-02-22T15:51:02.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.316+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f3a6c01c9c0 con 0x7f3a780fc1a0 2024-02-22T15:51:02.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.316+0000 7f3a757fa700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a6806af90 0x7f3a6806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:02.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.316+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3a6c08a3e0 con 0x7f3a780fc1a0 2024-02-22T15:51:02.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.316+0000 7f3a77fff700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a6806af90 0x7f3a6806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:02.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.317+0000 7f3a77fff700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a6806af90 0x7f3a6806d440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004fd0 tx=0x7f3a6400b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.323+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f3a6c058360 con 0x7f3a780fc1a0 2024-02-22T15:51:02.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.705+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a7810f870 con 0x7f3a6806af90 2024-02-22T15:51:02.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.707+0000 7f3a757fa700 1 -- 172.21.15.19:0/3991836443 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f3a7810f870 con 0x7f3a6806af90 2024-02-22T15:51:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a6806af90 msgr2=0x7f3a6806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a6806af90 0x7f3a6806d440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004fd0 tx=0x7f3a6400b040 comp rx=0 tx=0).stop 2024-02-22T15:51:02.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 msgr2=0x7f3a7810d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 0x7f3a7810d900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 shutdown_connections 2024-02-22T15:51:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f3a6806af90 0x7f3a6806d440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a780fc1a0 0x7f3a7810d900 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 --2- 172.21.15.19:0/3991836443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a780fd440 0x7f3a7810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.715+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 >> 172.21.15.19:0/3991836443 conn(0x7f3a780f6ca0 msgr2=0x7f3a780fb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.716+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 shutdown_connections 2024-02-22T15:51:02.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.716+0000 7f3a7ebb5700 1 -- 172.21.15.19:0/3991836443 wait complete. 2024-02-22T15:51:02.702 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:02 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:02.702 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:02 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:02.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:02 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:02.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:02 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 -- 172.21.15.19:0/4255031265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef400fda80 msgr2=0x7fef400ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 --2- 172.21.15.19:0/4255031265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef400fda80 0x7fef400ffe60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef38009ce0 tx=0x7fef38004d10 comp rx=0 tx=0).stop 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 -- 172.21.15.19:0/4255031265 shutdown_connections 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 --2- 172.21.15.19:0/4255031265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef401003a0 0x7fef40102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 --2- 172.21.15.19:0/4255031265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef400fda80 0x7fef400ffe60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 -- 172.21.15.19:0/4255031265 >> 172.21.15.19:0/4255031265 conn(0x7fef400f74e0 msgr2=0x7fef400f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 -- 172.21.15.19:0/4255031265 shutdown_connections 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 -- 172.21.15.19:0/4255031265 wait complete. 2024-02-22T15:51:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.861+0000 7fef4634a700 1 Processor -- start 2024-02-22T15:51:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef4634a700 1 -- start start 2024-02-22T15:51:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef4634a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef400fda80 0x7fef40118390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef4634a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 0x7fef401188d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef4634a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40118ed0 con 0x7fef401003a0 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef4634a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40119770 con 0x7fef400fda80 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 0x7fef401188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 0x7fef401188d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52256/0 (socket says 172.21.15.19:52256) 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3f7fe700 1 -- 172.21.15.19:0/1301772338 learned_addr learned my addr 172.21.15.19:0/1301772338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3ffff700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef400fda80 0x7fef40118390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3f7fe700 1 -- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef400fda80 msgr2=0x7fef40118390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3f7fe700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef400fda80 0x7fef40118390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.862+0000 7fef3f7fe700 1 -- 172.21.15.19:0/1301772338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30005040 con 0x7fef401003a0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.863+0000 7fef3f7fe700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 0x7fef401188d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef300096d0 tx=0x7fef30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.863+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef3000e840 con 0x7fef401003a0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.863+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef38009940 con 0x7fef401003a0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.863+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef3000f040 con 0x7fef401003a0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.863+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef30013d30 con 0x7fef401003a0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.864+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef40119da0 con 0x7fef401003a0 2024-02-22T15:51:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.865+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4004bed0 con 0x7fef401003a0 2024-02-22T15:51:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.866+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7fef3000e9a0 con 0x7fef401003a0 2024-02-22T15:51:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.866+0000 7fef3cff9700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fef2806aec0 0x7fef2806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.867+0000 7fef3ffff700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fef2806aec0 0x7fef2806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.867+0000 7fef3ffff700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fef2806aec0 0x7fef2806d370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef38015040 tx=0x7fef38004d10 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:02.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.870+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fef3000ec50 con 0x7fef401003a0 2024-02-22T15:51:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:02.873+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fef3008e060 con 0x7fef401003a0 2024-02-22T15:51:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.171+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef400f8c90 con 0x7fef2806aec0 2024-02-22T15:51:03.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.183+0000 7fef3cff9700 1 -- 172.21.15.19:0/1301772338 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fef400f8c90 con 0x7fef2806aec0 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 starting - - - - 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (9m) 75s ago 9m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (8m) 1s ago 8m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 starting - - - - 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (3m) 75s ago 3m 62.7M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (3m) 75s ago 3m 36.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (3m) 1s ago 3m 80.0M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (3m) 1s ago 3m 131M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:51:03.165 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 starting - - - - 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 running (100s) 1s ago 8m 493M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 60b7b18ac805 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (10m) 75s ago 11m 49.5M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (8m) 1s ago 8m 42.3M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 starting - - - - 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (14s) 1s ago 8m 7960k - 1.5.0 0da6a335fe13 4f5838683ba0 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 75s ago 7m 69.5M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (6m) 75s ago 6m 71.3M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (6m) 75s ago 6m 62.8M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (6m) 1s ago 6m 240M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (5m) 1s ago 5m 201M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:51:03.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (5m) 1s ago 5m 225M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:51:03.167 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 starting - - - - 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fef2806aec0 msgr2=0x7fef2806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fef2806aec0 0x7fef2806d370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef38015040 tx=0x7fef38004d10 comp rx=0 tx=0).stop 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 msgr2=0x7fef401188d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 0x7fef401188d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef300096d0 tx=0x7fef30009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 shutdown_connections 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef400fda80 0x7fef40118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7fef2806aec0 0x7fef2806d370 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 --2- 172.21.15.19:0/1301772338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef401003a0 0x7fef401188d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.186+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 >> 172.21.15.19:0/1301772338 conn(0x7fef400f74e0 msgr2=0x7fef401010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:03.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.187+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 shutdown_connections 2024-02-22T15:51:03.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.187+0000 7fef4634a700 1 -- 172.21.15.19:0/1301772338 wait complete. 2024-02-22T15:51:03.203 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-22T15:51:03.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-22T15:51:03.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 -- 172.21.15.19:0/3510430989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fd430 msgr2=0x7f63400fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 --2- 172.21.15.19:0/3510430989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fd430 0x7f63400fd840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6330004940 tx=0x7f6330009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:03.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 -- 172.21.15.19:0/3510430989 shutdown_connections 2024-02-22T15:51:03.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 --2- 172.21.15.19:0/3510430989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fc0a0 0x7f63400fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 --2- 172.21.15.19:0/3510430989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fd430 0x7f63400fd840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 -- 172.21.15.19:0/3510430989 >> 172.21.15.19:0/3510430989 conn(0x7f63400f6ca0 msgr2=0x7f63400f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:03.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 -- 172.21.15.19:0/3510430989 shutdown_connections 2024-02-22T15:51:03.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.317+0000 7f6344ab6700 1 -- 172.21.15.19:0/3510430989 wait complete. 2024-02-22T15:51:03.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.318+0000 7f6344ab6700 1 Processor -- start 2024-02-22T15:51:03.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.318+0000 7f6344ab6700 1 -- start start 2024-02-22T15:51:03.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.318+0000 7f6344ab6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 0x7f6340103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.318+0000 7f6344ab6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fd430 0x7f6340102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.318+0000 7f6344ab6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6340102a10 con 0x7f63400fc0a0 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.318+0000 7f6344ab6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6340102b80 con 0x7f63400fd430 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 0x7f6340103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 0x7f6340103f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52286/0 (socket says 172.21.15.19:52286) 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 -- 172.21.15.19:0/424207237 learned_addr learned my addr 172.21.15.19:0/424207237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633dd9b700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fd430 0x7f6340102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 -- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fd430 msgr2=0x7f6340102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fd430 0x7f6340102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 -- 172.21.15.19:0/424207237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334005040 con 0x7f63400fc0a0 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633dd9b700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fd430 0x7f6340102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f633e59c700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 0x7f6340103f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6330009fd0 tx=0x7f6330009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6330021070 con 0x7f63400fc0a0 2024-02-22T15:51:03.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f633001c400 con 0x7f63400fc0a0 2024-02-22T15:51:03.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633000e070 con 0x7f63400fc0a0 2024-02-22T15:51:03.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6330005040 con 0x7f63400fc0a0 2024-02-22T15:51:03.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.319+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6340103160 con 0x7f63400fc0a0 2024-02-22T15:51:03.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.320+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634004bed0 con 0x7f63400fc0a0 2024-02-22T15:51:03.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.322+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f633001ca00 con 0x7f63400fc0a0 2024-02-22T15:51:03.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.322+0000 7f632effd700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f632806aec0 0x7f632806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:03.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.323+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f633008b530 con 0x7f63400fc0a0 2024-02-22T15:51:03.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.323+0000 7f633dd9b700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f632806aec0 0x7f632806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:03.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.323+0000 7f633dd9b700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f632806aec0 0x7f632806d370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f63400fd170 tx=0x7f633400b040 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:03.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.327+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f63300594d0 con 0x7f63400fc0a0 2024-02-22T15:51:03.376 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:03 smithi019 ceph-mon[67418]: pgmap v47: 65 pgs: 65 active+clean; 2.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 44 MiB/s rd, 110 MiB/s wr, 357 op/s 2024-02-22T15:51:03.376 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:03 smithi019 ceph-mon[67418]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:03.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:03 smithi062 ceph-mon[48130]: pgmap v47: 65 pgs: 65 active+clean; 2.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 44 MiB/s rd, 110 MiB/s wr, 357 op/s 2024-02-22T15:51:03.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:03 smithi062 ceph-mon[48130]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.837+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6340105930 con 0x7f63400fc0a0 2024-02-22T15:51:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.837+0000 7f632effd700 1 -- 172.21.15.19:0/424207237 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+848 (secure 0 0 0) 0x7f6330059060 con 0x7f63400fc0a0 2024-02-22T15:51:03.819 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 1, 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:51:03.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:03.821 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:51:03.821 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 13, 2024-02-22T15:51:03.821 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:51:03.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:51:03.821 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:51:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f632806aec0 msgr2=0x7f632806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f632806aec0 0x7f632806d370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f63400fd170 tx=0x7f633400b040 comp rx=0 tx=0).stop 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 msgr2=0x7f6340103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 0x7f6340103f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6330009fd0 tx=0x7f6330009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 shutdown_connections 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f632806aec0 0x7f632806d370 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63400fc0a0 0x7f6340103f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 --2- 172.21.15.19:0/424207237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63400fd430 0x7f6340102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 >> 172.21.15.19:0/424207237 conn(0x7f63400f6ca0 msgr2=0x7f63401002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 shutdown_connections 2024-02-22T15:51:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.841+0000 7f6344ab6700 1 -- 172.21.15.19:0/424207237 wait complete. 2024-02-22T15:51:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.978+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3577909702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 msgr2=0x7f2a38101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.978+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3577909702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a38101ef0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a28004940 tx=0x7f2a28009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.979+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3577909702 shutdown_connections 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.979+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3577909702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 0x7f2a38104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.979+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3577909702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a38101ef0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.979+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3577909702 >> 172.21.15.19:0/3577909702 conn(0x7f2a380f7500 msgr2=0x7f2a380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.979+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3577909702 shutdown_connections 2024-02-22T15:51:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.979+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3577909702 wait complete. 2024-02-22T15:51:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3cfa5700 1 Processor -- start 2024-02-22T15:51:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3cfa5700 1 -- start start 2024-02-22T15:51:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3cfa5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a3810fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3cfa5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 0x7f2a3810ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3cfa5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38110570 con 0x7f2a38066c70 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3cfa5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38110e10 con 0x7f2a38102430 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a35d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 0x7f2a3810ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a3810fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a3810fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52298/0 (socket says 172.21.15.19:52298) 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.980+0000 7f2a3659c700 1 -- 172.21.15.19:0/3831750822 learned_addr learned my addr 172.21.15.19:0/3831750822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a35d9b700 1 -- 172.21.15.19:0/3831750822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 msgr2=0x7f2a3810fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a35d9b700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a3810fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a35d9b700 1 -- 172.21.15.19:0/3831750822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a28005040 con 0x7f2a38102430 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a35d9b700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 0x7f2a3810ff70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a380669b0 tx=0x7f2a2c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2c004e90 con 0x7f2a38102430 2024-02-22T15:51:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a2c00f040 con 0x7f2a38102430 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2c00c980 con 0x7f2a38102430 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a38111090 con 0x7f2a38102430 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.981+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381115d0 con 0x7f2a38102430 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.982+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3804bed0 con 0x7f2a38102430 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.983+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) v1 ==== 88768+0+0 (secure 0 0 0) 0x7f2a2c007cc0 con 0x7f2a38102430 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.984+0000 7f2a26ffd700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2a2006aec0 0x7f2a2006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.984+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2a2c089290 con 0x7f2a38102430 2024-02-22T15:51:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.984+0000 7f2a3659c700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2a2006aec0 0x7f2a2006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.987+0000 7f2a3659c700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2a2006aec0 0x7f2a2006d370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2a28013600 tx=0x7f2a28009e30 comp rx=0 tx=0).ready entity=mgr.24347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:03.991+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f2a2c04f4d0 con 0x7f2a38102430 2024-02-22T15:51:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.290+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 --> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a38111a60 con 0x7f2a2006aec0 2024-02-22T15:51:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.292+0000 7f2a26ffd700 1 -- 172.21.15.19:0/3831750822 <== mgr.24347 v2:172.21.15.62:6828/2665490935 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f2a38111a60 con 0x7f2a2006aec0 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "", 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mgr daemons", 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:51:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:51:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.294+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2a2006aec0 msgr2=0x7f2a2006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.294+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2a2006aec0 0x7f2a2006d370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2a28013600 tx=0x7f2a28009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.294+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 msgr2=0x7f2a3810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.294+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 0x7f2a3810ff70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a380669b0 tx=0x7f2a2c00b040 comp rx=0 tx=0).stop 2024-02-22T15:51:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 shutdown_connections 2024-02-22T15:51:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:6828/2665490935,v1:172.21.15.62:6829/2665490935] conn(0x7f2a2006aec0 0x7f2a2006d370 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a38066c70 0x7f2a3810fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 --2- 172.21.15.19:0/3831750822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a38102430 0x7f2a3810ff70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 >> 172.21.15.19:0/3831750822 conn(0x7f2a380f7500 msgr2=0x7f2a380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 shutdown_connections 2024-02-22T15:51:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:04.295+0000 7f2a3cfa5700 1 -- 172.21.15.19:0/3831750822 wait complete. 2024-02-22T15:51:04.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:04 smithi062 ceph-mon[48130]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:04.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:04 smithi062 ceph-mon[48130]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:04.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:04 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:04.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:04 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/424207237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:04.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:04 smithi019 ceph-mon[67418]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:04.748 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:04 smithi019 ceph-mon[67418]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:04.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:04 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:04.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:04 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/424207237' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:05.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:05 smithi062 ceph-mon[48130]: pgmap v48: 65 pgs: 65 active+clean; 2.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 31 MiB/s rd, 78 MiB/s wr, 304 op/s 2024-02-22T15:51:05.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:05 smithi062 ceph-mon[48130]: from='client.24453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:05.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:05 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:05.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:05 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:05.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:05 smithi019 ceph-mon[67418]: pgmap v48: 65 pgs: 65 active+clean; 2.9 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 31 MiB/s rd, 78 MiB/s wr, 304 op/s 2024-02-22T15:51:05.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:05 smithi019 ceph-mon[67418]: from='client.24453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:05.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:05 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:05.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:05 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:07.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:06 smithi019 ceph-mon[67418]: pgmap v49: 65 pgs: 65 active+clean; 1.4 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 79 MiB/s wr, 406 op/s 2024-02-22T15:51:07.097 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:06 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:07.097 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:06 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:07.097 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:06 smithi019 ceph-mon[67418]: mgrmap e25: smithi062.ecubmv(active, since 92s), standbys: smithi019.yvjrkp 2024-02-22T15:51:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:06 smithi062 ceph-mon[48130]: pgmap v49: 65 pgs: 65 active+clean; 1.4 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 79 MiB/s wr, 406 op/s 2024-02-22T15:51:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:06 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:06 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:06 smithi062 ceph-mon[48130]: mgrmap e25: smithi062.ecubmv(active, since 92s), standbys: smithi019.yvjrkp 2024-02-22T15:51:08.397 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-22T15:51:08.398 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-22T15:51:08.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: pgmap v50: 65 pgs: 65 active+clean; 1.4 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 216 op/s 2024-02-22T15:51:08.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: Standby manager daemon smithi019.yvjrkp restarted 2024-02-22T15:51:08.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: Standby manager daemon smithi019.yvjrkp started 2024-02-22T15:51:08.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/crt"}]: dispatch 2024-02-22T15:51:08.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:51:08.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/key"}]: dispatch 2024-02-22T15:51:08.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:08 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: pgmap v50: 65 pgs: 65 active+clean; 1.4 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 216 op/s 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: Standby manager daemon smithi019.yvjrkp restarted 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: Standby manager daemon smithi019.yvjrkp started 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/crt"}]: dispatch 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.yvjrkp/key"}]: dispatch 2024-02-22T15:51:09.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:08 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:51:09.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:09 smithi019 ceph-mon[67418]: mgrmap e26: smithi062.ecubmv(active, since 95s), standbys: smithi019.yvjrkp 2024-02-22T15:51:10.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:09 smithi062 ceph-mon[48130]: mgrmap e26: smithi062.ecubmv(active, since 95s), standbys: smithi019.yvjrkp 2024-02-22T15:51:11.145 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: pgmap v51: 65 pgs: 65 active+clean; 696 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 336 op/s 2024-02-22T15:51:11.145 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr fail", "who": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr fail", "who": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: osdmap e49: 6 total, 6 up, 6 in 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd='[{"prefix": "mgr fail", "who": "smithi062.ecubmv"}]': finished 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: mgrmap e27: smithi019.yvjrkp(active, starting, since 0.00836792s) 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:51:11.146 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:11.147 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:11.148 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:11.148 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:51:11.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: pgmap v51: 65 pgs: 65 active+clean; 696 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 336 op/s 2024-02-22T15:51:11.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:11.155 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 172.21.15.62:0/2872792321' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr fail", "who": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "mgr fail", "who": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: osdmap e49: 6 total, 6 up, 6 in 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.24347 ' entity='mgr.smithi062.ecubmv' cmd='[{"prefix": "mgr fail", "who": "smithi062.ecubmv"}]': finished 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: mgrmap e27: smithi019.yvjrkp(active, starting, since 0.00836792s) 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:51:11.156 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:11.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: [22/Feb/2024:15:51:11] ENGINE Bus STARTING 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: [22/Feb/2024:15:51:11] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: [22/Feb/2024:15:51:11] ENGINE Serving on http://172.21.15.19:8765 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: [22/Feb/2024:15:51:11] ENGINE Bus STARTED 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: mgrmap e28: smithi019.yvjrkp(active, since 1.00749s) 2024-02-22T15:51:12.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:12 smithi062 ceph-mon[48130]: pgmap v3: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:51:12.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:51:12.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:51:11] ENGINE Bus STARTING 2024-02-22T15:51:12.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:51:11] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:51:12.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:51:11] ENGINE Serving on http://172.21.15.19:8765 2024-02-22T15:51:12.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: [22/Feb/2024:15:51:11] ENGINE Bus STARTED 2024-02-22T15:51:12.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: mgrmap e28: smithi019.yvjrkp(active, since 1.00749s) 2024-02-22T15:51:12.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:12 smithi019 ceph-mon[67418]: pgmap v3: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:51:13.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:13 smithi019 ceph-mon[67418]: pgmap v4: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:51:14.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:13 smithi062 ceph-mon[48130]: pgmap v4: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:51:14.897 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:14 smithi062 ceph-mon[48130]: mgrmap e29: smithi019.yvjrkp(active, since 3s) 2024-02-22T15:51:15.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:14 smithi019 ceph-mon[67418]: mgrmap e29: smithi019.yvjrkp(active, since 3s) 2024-02-22T15:51:15.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:15 smithi019 ceph-mon[67418]: pgmap v5: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:51:15.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:15.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:15.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:15.996 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:16.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:15 smithi062 ceph-mon[48130]: pgmap v5: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:51:16.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:16.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:16.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:16.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:18.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:17 smithi062 ceph-mon[48130]: pgmap v6: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 58 op/s 2024-02-22T15:51:18.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:17 smithi019 ceph-mon[67418]: pgmap v6: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 58 op/s 2024-02-22T15:51:20.068 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:19 smithi062 ceph-mon[48130]: pgmap v7: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 401 op/s 2024-02-22T15:51:20.068 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:20.068 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:20.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:19 smithi019 ceph-mon[67418]: pgmap v7: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 401 op/s 2024-02-22T15:51:20.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:20.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:21.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:20 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/2527887762' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:21.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:20 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:22.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:21 smithi062 ceph-mon[48130]: pgmap v8: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 312 op/s 2024-02-22T15:51:22.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:21 smithi062 ceph-mon[48130]: mgrmap e30: smithi019.yvjrkp(active, since 10s), standbys: smithi062.ecubmv 2024-02-22T15:51:22.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:21 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:22.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:21 smithi019 ceph-mon[67418]: pgmap v8: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 312 op/s 2024-02-22T15:51:22.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:21 smithi019 ceph-mon[67418]: mgrmap e30: smithi019.yvjrkp(active, since 10s), standbys: smithi062.ecubmv 2024-02-22T15:51:22.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:21 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:24.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:23 smithi062 ceph-mon[48130]: pgmap v9: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 519 op/s 2024-02-22T15:51:24.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:23 smithi019 ceph-mon[67418]: pgmap v9: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 519 op/s 2024-02-22T15:51:25.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:25 smithi019 ceph-mon[67418]: pgmap v10: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 476 op/s 2024-02-22T15:51:25.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:25 smithi062 ceph-mon[48130]: pgmap v10: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 476 op/s 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:26.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:26 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:51:26.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:51:26.746 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:51:26.746 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:26.746 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:26 smithi019 ceph-mon[67418]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: Reconfiguring prometheus.smithi019 (dependencies changed)... 2024-02-22T15:51:27.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:27 smithi019 ceph-mon[67418]: pgmap v11: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 503 op/s 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: Reconfiguring prometheus.smithi019 (dependencies changed)... 2024-02-22T15:51:27.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:27 smithi062 ceph-mon[48130]: pgmap v11: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 503 op/s 2024-02-22T15:51:28.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:28 smithi062 ceph-mon[48130]: Reconfiguring daemon prometheus.smithi019 on smithi019 2024-02-22T15:51:28.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:28 smithi019 ceph-mon[67418]: Reconfiguring daemon prometheus.smithi019 on smithi019 2024-02-22T15:51:29.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:29 smithi062 ceph-mon[48130]: pgmap v12: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 683 op/s 2024-02-22T15:51:29.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:29 smithi019 ceph-mon[67418]: pgmap v12: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 683 op/s 2024-02-22T15:51:32.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:31 smithi019 ceph-mon[67418]: pgmap v13: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 473 op/s 2024-02-22T15:51:32.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:31 smithi062 ceph-mon[48130]: pgmap v13: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 473 op/s 2024-02-22T15:51:34.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:33 smithi062 ceph-mon[48130]: pgmap v14: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 666 op/s 2024-02-22T15:51:34.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:34.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:34.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:51:34.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:34.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:33 smithi019 ceph-mon[67418]: pgmap v14: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 666 op/s 2024-02-22T15:51:34.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:34.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:34.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:51:34.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/516217441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc0fda80 msgr2=0x7f1acc0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/516217441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc0fda80 0x7f1acc0ffe60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1abc004940 tx=0x7f1abc009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/516217441 shutdown_connections 2024-02-22T15:51:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/516217441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc1003a0 0x7f1acc102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/516217441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc0fda80 0x7f1acc0ffe60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/516217441 >> 172.21.15.19:0/516217441 conn(0x7f1acc0f74e0 msgr2=0x7f1acc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/516217441 shutdown_connections 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/516217441 wait complete. 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 Processor -- start 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.422+0000 7f1ad0b2b700 1 -- start start 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ad0b2b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc0fda80 0x7f1acc10f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ad0b2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 0x7f1acc10ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ad0b2b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc110510 con 0x7f1acc1003a0 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ad0b2b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc110db0 con 0x7f1acc0fda80 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 0x7f1acc10ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 0x7f1acc10ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60200/0 (socket says 172.21.15.19:60200) 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 -- 172.21.15.19:0/3755833012 learned_addr learned my addr 172.21.15.19:0/3755833012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1aca59c700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc0fda80 0x7f1acc10f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 -- 172.21.15.19:0/3755833012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc0fda80 msgr2=0x7f1acc10f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc0fda80 0x7f1acc10f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 -- 172.21.15.19:0/3755833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc005040 con 0x7f1acc1003a0 2024-02-22T15:51:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1aca59c700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc0fda80 0x7f1acc10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:51:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1ac9d9b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 0x7f1acc10ff10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000eb40 tx=0x7f1ac000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.423+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac0007cc0 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.424+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc111030 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.424+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ac000f040 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.424+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac000c990 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.424+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc111570 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.424+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc04bed0 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.425+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 30) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f1ac000cc00 con 0x7f1acc1003a0 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.426+0000 7f1abaffd700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f1ab406c9c0 0x7f1ab406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.426+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1ac008b930 con 0x7f1acc1003a0 2024-02-22T15:51:34.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.426+0000 7f1aca59c700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f1ab406c9c0 0x7f1ab406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:34.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.426+0000 7f1aca59c700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f1ab406c9c0 0x7f1ab406ee70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1abc000c00 tx=0x7f1abc009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:34.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.431+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f1ac0059730 con 0x7f1acc1003a0 2024-02-22T15:51:34.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.704+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1acc0fbf80 con 0x7f1ab406c9c0 2024-02-22T15:51:34.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.706+0000 7f1abaffd700 1 -- 172.21.15.19:0/3755833012 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f1acc0fbf80 con 0x7f1ab406c9c0 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f1ab406c9c0 msgr2=0x7f1ab406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f1ab406c9c0 0x7f1ab406ee70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1abc000c00 tx=0x7f1abc009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 msgr2=0x7f1acc10ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 0x7f1acc10ff10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000eb40 tx=0x7f1ac000b040 comp rx=0 tx=0).stop 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 shutdown_connections 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f1ab406c9c0 0x7f1ab406ee70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1acc0fda80 0x7f1acc10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 --2- 172.21.15.19:0/3755833012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1acc1003a0 0x7f1acc10ff10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 >> 172.21.15.19:0/3755833012 conn(0x7f1acc0f74e0 msgr2=0x7f1acc0fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 shutdown_connections 2024-02-22T15:51:34.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.708+0000 7f1ad0b2b700 1 -- 172.21.15.19:0/3755833012 wait complete. 2024-02-22T15:51:34.702 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:51:34.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.851+0000 7ff3240a4700 1 -- 172.21.15.19:0/707115017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fe1d0 msgr2=0x7ff31c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:34.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.851+0000 7ff3240a4700 1 --2- 172.21.15.19:0/707115017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fe1d0 0x7ff31c100650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff30c004940 tx=0x7ff30c009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:34.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.851+0000 7ff3240a4700 1 -- 172.21.15.19:0/707115017 shutdown_connections 2024-02-22T15:51:34.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.851+0000 7ff3240a4700 1 --2- 172.21.15.19:0/707115017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fe1d0 0x7ff31c100650 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff30c004940 tx=0x7ff30c009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:34.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.851+0000 7ff3240a4700 1 --2- 172.21.15.19:0/707115017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c0fb870 0x7ff31c0fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.851+0000 7ff3240a4700 1 -- 172.21.15.19:0/707115017 >> 172.21.15.19:0/707115017 conn(0x7ff31c0f7500 msgr2=0x7ff31c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:34.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.852+0000 7ff3240a4700 1 -- 172.21.15.19:0/707115017 shutdown_connections 2024-02-22T15:51:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.852+0000 7ff3240a4700 1 -- 172.21.15.19:0/707115017 wait complete. 2024-02-22T15:51:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.853+0000 7ff3240a4700 1 Processor -- start 2024-02-22T15:51:34.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.853+0000 7ff3240a4700 1 -- start start 2024-02-22T15:51:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.853+0000 7ff3240a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fb870 0x7ff31c114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.853+0000 7ff3240a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 0x7ff31c1153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.853+0000 7ff3240a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c114aa0 con 0x7ff31c0fb870 2024-02-22T15:51:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.853+0000 7ff3240a4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c114be0 con 0x7ff31c114550 2024-02-22T15:51:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 0x7ff31c1153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 0x7ff31c1153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:34126/0 (socket says 172.21.15.19:34126) 2024-02-22T15:51:34.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 -- 172.21.15.19:0/4257204727 learned_addr learned my addr 172.21.15.19:0/4257204727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:34.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 -- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fb870 msgr2=0x7ff31c114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:34.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fb870 0x7ff31c114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:34.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 -- 172.21.15.19:0/4257204727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c005040 con 0x7ff31c114550 2024-02-22T15:51:34.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.854+0000 7ff32163f700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 0x7ff31c1153f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff30c004940 tx=0x7ff30c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:34.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.856+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff30c00e070 con 0x7ff31c114550 2024-02-22T15:51:34.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.856+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff30c013420 con 0x7ff31c114550 2024-02-22T15:51:34.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.856+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff30c01c640 con 0x7ff31c114550 2024-02-22T15:51:34.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.856+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c115a70 con 0x7ff31c114550 2024-02-22T15:51:34.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.856+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c115f50 con 0x7ff31c114550 2024-02-22T15:51:34.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.857+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c10df90 con 0x7ff31c114550 2024-02-22T15:51:34.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.859+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 30) v1 ==== 90813+0+0 (secure 0 0 0) 0x7ff30c013970 con 0x7ff31c114550 2024-02-22T15:51:34.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.860+0000 7ff3127fc700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff30806cb50 0x7ff30806f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:34.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.860+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff30c08d220 con 0x7ff31c114550 2024-02-22T15:51:34.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.861+0000 7ff321e40700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff30806cb50 0x7ff30806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:34.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.862+0000 7ff321e40700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff30806cb50 0x7ff30806f000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3180098f0 tx=0x7ff318006da0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:34.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:34.866+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7ff30c05afd0 con 0x7ff31c114550 2024-02-22T15:51:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.204+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff31c05e3d0 con 0x7ff30806cb50 2024-02-22T15:51:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.206+0000 7ff3127fc700 1 -- 172.21.15.19:0/4257204727 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff31c05e3d0 con 0x7ff30806cb50 2024-02-22T15:51:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff30806cb50 msgr2=0x7ff30806f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff30806cb50 0x7ff30806f000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3180098f0 tx=0x7ff318006da0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 msgr2=0x7ff31c1153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 0x7ff31c1153f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff30c004940 tx=0x7ff30c009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 shutdown_connections 2024-02-22T15:51:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff30806cb50 0x7ff30806f000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31c0fb870 0x7ff31c114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 --2- 172.21.15.19:0/4257204727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff31c114550 0x7ff31c1153f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 >> 172.21.15.19:0/4257204727 conn(0x7ff31c0f7500 msgr2=0x7ff31c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 shutdown_connections 2024-02-22T15:51:35.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.208+0000 7ff3240a4700 1 -- 172.21.15.19:0/4257204727 wait complete. 2024-02-22T15:51:35.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:34 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 -- 172.21.15.19:0/3296106847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 msgr2=0x7f4ffc102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 --2- 172.21.15.19:0/3296106847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc102780 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0004940 tx=0x7f4ff0009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 -- 172.21.15.19:0/3296106847 shutdown_connections 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 --2- 172.21.15.19:0/3296106847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc102780 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 --2- 172.21.15.19:0/3296106847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 -- 172.21.15.19:0/3296106847 >> 172.21.15.19:0/3296106847 conn(0x7f4ffc0f74e0 msgr2=0x7f4ffc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 -- 172.21.15.19:0/3296106847 shutdown_connections 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 -- 172.21.15.19:0/3296106847 wait complete. 2024-02-22T15:51:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.351+0000 7f5002831700 1 Processor -- start 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f5002831700 1 -- start start 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f5002831700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc10d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f5002831700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc10dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f5002831700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc10e330 con 0x7f4ffc1003a0 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f5002831700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc10ebd0 con 0x7f4ffc0fda80 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f4ffb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc10dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc10d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc10d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:34144/0 (socket says 172.21.15.19:34144) 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f4ffb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc10dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60240/0 (socket says 172.21.15.19:60240) 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.352+0000 7f4ffbfff700 1 -- 172.21.15.19:0/3972574440 learned_addr learned my addr 172.21.15.19:0/3972574440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ffbfff700 1 -- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 msgr2=0x7f4ffc10dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ffbfff700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc10dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ffbfff700 1 -- 172.21.15.19:0/3972574440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0005040 con 0x7f4ffc0fda80 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ffbfff700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc10d7f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00eb40 tx=0x7f4fec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fec007cc0 con 0x7f4ffc0fda80 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc10ee50 con 0x7f4ffc0fda80 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4fec00f040 con 0x7f4ffc0fda80 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fec00c990 con 0x7f4ffc0fda80 2024-02-22T15:51:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.353+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc10f390 con 0x7f4ffc0fda80 2024-02-22T15:51:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.354+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc107910 con 0x7f4ffc0fda80 2024-02-22T15:51:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.355+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 30) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f4fec00cc00 con 0x7f4ffc0fda80 2024-02-22T15:51:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.355+0000 7f4ff8ff9700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4fe406c9c0 0x7f4fe406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.355+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4fec08b970 con 0x7f4ffc0fda80 2024-02-22T15:51:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.355+0000 7f4ffb7fe700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4fe406c9c0 0x7f4fe406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.356+0000 7f4ffb7fe700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4fe406c9c0 0x7f4fe406ee70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00135e0 tx=0x7f4ff0009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:35.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.361+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f4fec059770 con 0x7f4ffc0fda80 2024-02-22T15:51:35.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:34 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:51:35.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.624+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ffc02b310 con 0x7f4fe406c9c0 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.635+0000 7f4ff8ff9700 1 -- 172.21.15.19:0/3972574440 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4ffc02b310 con 0x7f4fe406c9c0 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (94s) 15s ago 9m 17.1M - 0.25.0 c8568f914cd2 36f5804ea0aa 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (10m) 15s ago 10m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (8m) 20s ago 8m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (86s) 15s ago 9m 82.1M - 9.4.7 954c08fa6188 00e17776d4e6 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (4m) 15s ago 4m 156M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (4m) 15s ago 4m 92.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (4m) 20s ago 4m 84.5M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (4m) 20s ago 4m 139M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:51:35.617 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (38s) 15s ago 11m 470M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283 running (2m) 20s ago 8m 63.7M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 60b7b18ac805 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (11m) 15s ago 11m 57.0M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (8m) 20s ago 8m 46.5M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (102s) 15s ago 10m 9256k - 1.5.0 0da6a335fe13 3023e3c39750 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (47s) 20s ago 8m 8539k - 1.5.0 0da6a335fe13 4f5838683ba0 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 15s ago 7m 192M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (7m) 15s ago 7m 208M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (7m) 15s ago 7m 171M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (6m) 20s ago 6m 225M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (6m) 20s ago 6m 201M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (5m) 20s ago 5m 202M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:51:35.618 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 starting - - - - 2024-02-22T15:51:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4fe406c9c0 msgr2=0x7f4fe406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4fe406c9c0 0x7f4fe406ee70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00135e0 tx=0x7f4ff0009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 msgr2=0x7f4ffc10d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc10d7f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00eb40 tx=0x7f4fec00b040 comp rx=0 tx=0).stop 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 shutdown_connections 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4fe406c9c0 0x7f4fe406ee70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ffc0fda80 0x7f4ffc10d7f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 --2- 172.21.15.19:0/3972574440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ffc1003a0 0x7f4ffc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 >> 172.21.15.19:0/3972574440 conn(0x7f4ffc0f74e0 msgr2=0x7f4ffc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 shutdown_connections 2024-02-22T15:51:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.639+0000 7f5002831700 1 -- 172.21.15.19:0/3972574440 wait complete. 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 -- 172.21.15.19:0/3935703586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 msgr2=0x7f8d000fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 --2- 172.21.15.19:0/3935703586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d000fd320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0004940 tx=0x7f8cf0009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 -- 172.21.15.19:0/3935703586 shutdown_connections 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 --2- 172.21.15.19:0/3935703586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d000fe0e0 0x7f8d000fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 --2- 172.21.15.19:0/3935703586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d000fd320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 -- 172.21.15.19:0/3935703586 >> 172.21.15.19:0/3935703586 conn(0x7f8d000f8620 msgr2=0x7f8d000faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 -- 172.21.15.19:0/3935703586 shutdown_connections 2024-02-22T15:51:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.766+0000 7f8d052ee700 1 -- 172.21.15.19:0/3935703586 wait complete. 2024-02-22T15:51:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8d052ee700 1 Processor -- start 2024-02-22T15:51:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8d052ee700 1 -- start start 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8d052ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d00071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8d052ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d000fe0e0 0x7f8d000702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8d052ee700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d001115b0 con 0x7f8d000fcef0 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8d052ee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d00111720 con 0x7f8d000fe0e0 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8cfeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d00071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8cfeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d00071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60262/0 (socket says 172.21.15.19:60262) 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8cfeffd700 1 -- 172.21.15.19:0/1829697128 learned_addr learned my addr 172.21.15.19:0/1829697128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.767+0000 7f8cfe7fc700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d000fe0e0 0x7f8d000702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.768+0000 7f8cfeffd700 1 -- 172.21.15.19:0/1829697128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d000fe0e0 msgr2=0x7f8d000702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.768+0000 7f8cfeffd700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d000fe0e0 0x7f8d000702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.768+0000 7f8cfeffd700 1 -- 172.21.15.19:0/1829697128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf0005040 con 0x7f8d000fcef0 2024-02-22T15:51:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.768+0000 7f8cfeffd700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d00071e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0000c00 tx=0x7f8cf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.768+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf000e040 con 0x7f8d000fcef0 2024-02-22T15:51:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.769+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8cf0009af0 con 0x7f8d000fcef0 2024-02-22T15:51:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.769+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf001c630 con 0x7f8d000fcef0 2024-02-22T15:51:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.769+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d000708e0 con 0x7f8d000fcef0 2024-02-22T15:51:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.769+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d00070e20 con 0x7f8d000fcef0 2024-02-22T15:51:35.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.770+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d000fc2a0 con 0x7f8d000fcef0 2024-02-22T15:51:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.772+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 30) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f8cf000bc50 con 0x7f8d000fcef0 2024-02-22T15:51:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.773+0000 7f8ce77fe700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8ce806cb60 0x7f8ce806f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.773+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8cf008d360 con 0x7f8d000fcef0 2024-02-22T15:51:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.773+0000 7f8cfe7fc700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8ce806cb60 0x7f8ce806f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:35.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.775+0000 7f8cfe7fc700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8ce806cb60 0x7f8ce806f010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d000723a0 tx=0x7f8cf4009ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:35.778+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f8cf0057680 con 0x7f8d000fcef0 2024-02-22T15:51:36.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:35 smithi062 ceph-mon[48130]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:36.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:35 smithi062 ceph-mon[48130]: pgmap v15: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 424 op/s 2024-02-22T15:51:36.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:35 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:51:36.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:35 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:51:36.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:35 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:36.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.259+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d0004bed0 con 0x7f8d000fcef0 2024-02-22T15:51:36.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.260+0000 7f8ce77fe700 1 -- 172.21.15.19:0/1829697128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f8cf0013e60 con 0x7f8d000fcef0 2024-02-22T15:51:36.241 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:51:36.241 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 12, 2024-02-22T15:51:36.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:51:36.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:51:36.243 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:51:36.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8ce806cb60 msgr2=0x7f8ce806f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8ce806cb60 0x7f8ce806f010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d000723a0 tx=0x7f8cf4009ee0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 msgr2=0x7f8d00071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d00071e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0000c00 tx=0x7f8cf0009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 shutdown_connections 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8ce806cb60 0x7f8ce806f010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d000fcef0 0x7f8d00071e20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 --2- 172.21.15.19:0/1829697128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d000fe0e0 0x7f8d000702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 >> 172.21.15.19:0/1829697128 conn(0x7f8d000f8620 msgr2=0x7f8d001013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 shutdown_connections 2024-02-22T15:51:36.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.262+0000 7f8d052ee700 1 -- 172.21.15.19:0/1829697128 wait complete. 2024-02-22T15:51:36.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:35 smithi019 ceph-mon[67418]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:36.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:35 smithi019 ceph-mon[67418]: pgmap v15: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 424 op/s 2024-02-22T15:51:36.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:35 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:51:36.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:35 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:51:36.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:35 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 -- 172.21.15.19:0/2195280457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe040fda80 msgr2=0x7fbe040ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/2195280457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe040fda80 0x7fbe040ffe60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbe00004940 tx=0x7fbe00009e30 comp rx=0 tx=0).stop 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 -- 172.21.15.19:0/2195280457 shutdown_connections 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/2195280457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe041003a0 0x7fbe04102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/2195280457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe040fda80 0x7fbe040ffe60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 -- 172.21.15.19:0/2195280457 >> 172.21.15.19:0/2195280457 conn(0x7fbe040f74e0 msgr2=0x7fbe040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 -- 172.21.15.19:0/2195280457 shutdown_connections 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.389+0000 7fbe0b94f700 1 -- 172.21.15.19:0/2195280457 wait complete. 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe0b94f700 1 Processor -- start 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe0b94f700 1 -- start start 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe0b94f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe040fda80 0x7fbe0406efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe0b94f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 0x7fbe0406f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe0b94f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0406fae0 con 0x7fbe041003a0 2024-02-22T15:51:36.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe0b94f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041173e0 con 0x7fbe040fda80 2024-02-22T15:51:36.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe08eea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 0x7fbe0406f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:36.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe08eea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 0x7fbe0406f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60282/0 (socket says 172.21.15.19:60282) 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe096eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe040fda80 0x7fbe0406efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.390+0000 7fbe08eea700 1 -- 172.21.15.19:0/3143683201 learned_addr learned my addr 172.21.15.19:0/3143683201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbe08eea700 1 -- 172.21.15.19:0/3143683201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe040fda80 msgr2=0x7fbe0406efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbe08eea700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe040fda80 0x7fbe0406efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbe08eea700 1 -- 172.21.15.19:0/3143683201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00005040 con 0x7fbe041003a0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbe08eea700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 0x7fbe0406f4e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400eb40 tx=0x7fbdf400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdf4007cc0 con 0x7fbe041003a0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe04117660 con 0x7fbe041003a0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbdf400f040 con 0x7fbe041003a0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdf400c990 con 0x7fbe041003a0 2024-02-22T15:51:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.391+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe04117ba0 con 0x7fbe041003a0 2024-02-22T15:51:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.393+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0404bed0 con 0x7fbe041003a0 2024-02-22T15:51:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.393+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 30) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fbdf400caf0 con 0x7fbe041003a0 2024-02-22T15:51:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.393+0000 7fbdf9ffb700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbdf006c9c0 0x7fbdf006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:51:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.393+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbdf408b860 con 0x7fbe041003a0 2024-02-22T15:51:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.394+0000 7fbe096eb700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbdf006c9c0 0x7fbdf006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:51:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.394+0000 7fbe096eb700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbdf006c9c0 0x7fbdf006ee70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b0d0 tx=0x7fbe0001c040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:51:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.399+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fbdf40596e0 con 0x7fbe041003a0 2024-02-22T15:51:36.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.682+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe041180f0 con 0x7fbdf006c9c0 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.683+0000 7fbdf9ffb700 1 -- 172.21.15.19:0/3143683201 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fbe041180f0 con 0x7fbdf006c9c0 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "2/2 daemons upgraded", 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mgr daemons", 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:51:36.665 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:51:36.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbdf006c9c0 msgr2=0x7fbdf006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:36.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbdf006c9c0 0x7fbdf006ee70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b0d0 tx=0x7fbe0001c040 comp rx=0 tx=0).stop 2024-02-22T15:51:36.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 msgr2=0x7fbe0406f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:51:36.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 0x7fbe0406f4e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400eb40 tx=0x7fbdf400b040 comp rx=0 tx=0).stop 2024-02-22T15:51:36.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 shutdown_connections 2024-02-22T15:51:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbdf006c9c0 0x7fbdf006ee70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe040fda80 0x7fbe0406efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 --2- 172.21.15.19:0/3143683201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041003a0 0x7fbe0406f4e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:51:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 >> 172.21.15.19:0/3143683201 conn(0x7fbe040f74e0 msgr2=0x7fbe040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:51:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 shutdown_connections 2024-02-22T15:51:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:51:36.686+0000 7fbe0b94f700 1 -- 172.21.15.19:0/3143683201 wait complete. 2024-02-22T15:51:37.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:36 smithi019 ceph-mon[67418]: Upgrade: Updating mgr.smithi062.ecubmv 2024-02-22T15:51:37.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:36 smithi019 ceph-mon[67418]: Deploying daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:51:37.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:36 smithi019 ceph-mon[67418]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:37.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:36 smithi019 ceph-mon[67418]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:37.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:36 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1829697128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:37.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:36 smithi062 ceph-mon[48130]: Upgrade: Updating mgr.smithi062.ecubmv 2024-02-22T15:51:37.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:36 smithi062 ceph-mon[48130]: Deploying daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:51:37.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:36 smithi062 ceph-mon[48130]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:36 smithi062 ceph-mon[48130]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:37.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:36 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1829697128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:38.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:37 smithi062 ceph-mon[48130]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:38.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:37 smithi062 ceph-mon[48130]: pgmap v16: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 452 op/s 2024-02-22T15:51:38.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:37 smithi019 ceph-mon[67418]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:51:38.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:37 smithi019 ceph-mon[67418]: pgmap v16: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 452 op/s 2024-02-22T15:51:40.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:40 smithi062 ceph-mon[48130]: pgmap v17: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 629 op/s 2024-02-22T15:51:40.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:40 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:40.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:40 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:40.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:40 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:40 smithi019 ceph-mon[67418]: pgmap v17: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 629 op/s 2024-02-22T15:51:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:40 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:40 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:40 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:41.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:41 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:41.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:41 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:42.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:42 smithi019 ceph-mon[67418]: pgmap v18: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 424 op/s 2024-02-22T15:51:42.379 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:42 smithi062 ceph-mon[48130]: pgmap v18: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 424 op/s 2024-02-22T15:51:43.655 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:43 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.656 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:43 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.656 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:43 smithi062 ceph-mon[48130]: pgmap v19: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 601 op/s 2024-02-22T15:51:43.656 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:43 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.656 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:43 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:43 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:43 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:43 smithi019 ceph-mon[67418]: pgmap v19: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 601 op/s 2024-02-22T15:51:43.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:43 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:43.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:43 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:45.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:45 smithi062 ceph-mon[48130]: pgmap v20: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 408 op/s 2024-02-22T15:51:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:45 smithi019 ceph-mon[67418]: pgmap v20: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 408 op/s 2024-02-22T15:51:48.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: pgmap v21: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-02-22T15:51:48.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.yvjrkp"}]: dispatch 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.yvjrkp"}]': finished 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi062.ecubmv"}]': finished 2024-02-22T15:51:48.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:47 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: pgmap v21: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-02-22T15:51:48.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.yvjrkp"}]: dispatch 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.yvjrkp"}]': finished 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi062.ecubmv"}]: dispatch 2024-02-22T15:51:48.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi062.ecubmv"}]': finished 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:48.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:51:48.460 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:47 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all mgr 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all rgw 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all rbd-mirror 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all ceph-exporter 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all iscsi 2024-02-22T15:51:49.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all nfs 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all nvmeof 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Standby manager daemon smithi062.ecubmv restarted 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:51:49.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:48 smithi019 ceph-mon[67418]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all mgr 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all rgw 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all rbd-mirror 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all ceph-exporter 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all iscsi 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all nfs 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all nvmeof 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv restarted 2024-02-22T15:51:49.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:51:49.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:51:49.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:51:49.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:51:49.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:48 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/4162887458' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:51:49.975 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:49 smithi019 ceph-mon[67418]: pgmap v22: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 546 op/s 2024-02-22T15:51:49.975 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:49 smithi019 ceph-mon[67418]: mgrmap e31: smithi019.yvjrkp(active, since 38s), standbys: smithi062.ecubmv 2024-02-22T15:51:50.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:49 smithi062 ceph-mon[48130]: pgmap v22: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 546 op/s 2024-02-22T15:51:50.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:49 smithi062 ceph-mon[48130]: mgrmap e31: smithi019.yvjrkp(active, since 38s), standbys: smithi062.ecubmv 2024-02-22T15:51:51.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:51 smithi019 ceph-mon[67418]: Upgrade: Updating node-exporter.smithi019 (1/2) 2024-02-22T15:51:51.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:51 smithi019 ceph-mon[67418]: Deploying daemon node-exporter.smithi019 on smithi019 2024-02-22T15:51:51.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:51 smithi062 ceph-mon[48130]: Upgrade: Updating node-exporter.smithi019 (1/2) 2024-02-22T15:51:51.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:51 smithi062 ceph-mon[48130]: Deploying daemon node-exporter.smithi019 on smithi019 2024-02-22T15:51:52.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:52 smithi019 ceph-mon[67418]: pgmap v23: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-02-22T15:51:52.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:52 smithi062 ceph-mon[48130]: pgmap v23: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-02-22T15:51:53.603 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:53 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:53.604 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:53 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:53.604 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:53 smithi062 ceph-mon[48130]: pgmap v24: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 554 op/s 2024-02-22T15:51:53.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:53 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:53.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:53 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:51:53.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:53 smithi019 ceph-mon[67418]: pgmap v24: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 554 op/s 2024-02-22T15:51:54.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:54 smithi062 ceph-mon[48130]: Upgrade: Updating node-exporter.smithi062 (2/2) 2024-02-22T15:51:54.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:54 smithi062 ceph-mon[48130]: Deploying daemon node-exporter.smithi062 on smithi062 2024-02-22T15:51:54.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:54 smithi019 ceph-mon[67418]: Upgrade: Updating node-exporter.smithi062 (2/2) 2024-02-22T15:51:54.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:54 smithi019 ceph-mon[67418]: Deploying daemon node-exporter.smithi062 on smithi062 2024-02-22T15:51:55.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:55 smithi062 ceph-mon[48130]: pgmap v25: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 377 op/s 2024-02-22T15:51:55.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:55 smithi019 ceph-mon[67418]: pgmap v25: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 377 op/s 2024-02-22T15:51:55.895 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-22T15:51:55.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-22T15:51:56.262 DEBUG:teuthology.parallel:result is None 2024-02-22T15:51:56.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:56 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:56.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:56 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:51:57.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:57 smithi062 ceph-mon[48130]: pgmap v26: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 501 op/s 2024-02-22T15:51:57.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:57 smithi019 ceph-mon[67418]: pgmap v26: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 501 op/s 2024-02-22T15:52:00.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:51:59 smithi019 ceph-mon[67418]: pgmap v27: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 540 op/s 2024-02-22T15:52:00.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:51:59 smithi062 ceph-mon[48130]: pgmap v27: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 540 op/s 2024-02-22T15:52:01.406 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-22T15:52:01.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-22T15:52:01.689 DEBUG:teuthology.parallel:result is None 2024-02-22T15:52:01.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-22T15:52:01.734 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-22T15:52:01.735 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-22T15:52:01.800 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-22T15:52:01.801 DEBUG:teuthology.parallel:result is None 2024-02-22T15:52:01.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:01 smithi019 ceph-mon[67418]: pgmap v28: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 425 op/s 2024-02-22T15:52:02.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:01 smithi062 ceph-mon[48130]: pgmap v28: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 425 op/s 2024-02-22T15:52:04.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:03 smithi019 ceph-mon[67418]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 562 op/s 2024-02-22T15:52:04.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:03 smithi062 ceph-mon[48130]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 562 op/s 2024-02-22T15:52:05.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:05 smithi062 ceph-mon[48130]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-02-22T15:52:05.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:05 smithi019 ceph-mon[67418]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 351 op/s 2024-02-22T15:52:06.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 -- 172.21.15.19:0/1956394592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67440663e0 msgr2=0x7f6744101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 --2- 172.21.15.19:0/1956394592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67440663e0 0x7f6744101ef0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6738004940 tx=0x7f6738009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 -- 172.21.15.19:0/1956394592 shutdown_connections 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 --2- 172.21.15.19:0/1956394592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6744102430 0x7f6744104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 --2- 172.21.15.19:0/1956394592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67440663e0 0x7f6744101ef0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 -- 172.21.15.19:0/1956394592 >> 172.21.15.19:0/1956394592 conn(0x7f67440f74e0 msgr2=0x7f67440f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 -- 172.21.15.19:0/1956394592 shutdown_connections 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 -- 172.21.15.19:0/1956394592 wait complete. 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 Processor -- start 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.798+0000 7f674b13c700 1 -- start start 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f674b13c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67440663e0 0x7f674410fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f674b13c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 0x7f674410ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f674b13c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744110560 con 0x7f6744102430 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f674b13c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744110e00 con 0x7f67440663e0 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 0x7f674410ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 0x7f674410ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47178/0 (socket says 172.21.15.19:47178) 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 -- 172.21.15.19:0/3258956612 learned_addr learned my addr 172.21.15.19:0/3258956612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6748ed8700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67440663e0 0x7f674410fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 -- 172.21.15.19:0/3258956612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67440663e0 msgr2=0x7f674410fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67440663e0 0x7f674410fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 -- 172.21.15.19:0/3258956612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738005040 con 0x7f6744102430 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.799+0000 7f6743fff700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 0x7f674410ff60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f673000eb40 tx=0x7f673000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.800+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6730007cc0 con 0x7f6744102430 2024-02-22T15:52:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.800+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f673000f040 con 0x7f6744102430 2024-02-22T15:52:06.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.800+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744111080 con 0x7f6744102430 2024-02-22T15:52:06.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.800+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673000c8f0 con 0x7f6744102430 2024-02-22T15:52:06.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.800+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67441115c0 con 0x7f6744102430 2024-02-22T15:52:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.801+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674404bed0 con 0x7f6744102430 2024-02-22T15:52:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.801+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f673000cb60 con 0x7f6744102430 2024-02-22T15:52:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.802+0000 7f67417fa700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f673406c9c0 0x7f673406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.802+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f673008b7b0 con 0x7f6744102430 2024-02-22T15:52:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.802+0000 7f6748ed8700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f673406c9c0 0x7f673406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.802+0000 7f6748ed8700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f673406c9c0 0x7f673406ee70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6738000c00 tx=0x7f6738009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:06.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:06.806+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f67300595b0 con 0x7f6744102430 2024-02-22T15:52:07.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.029+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6744111a10 con 0x7f673406c9c0 2024-02-22T15:52:07.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.030+0000 7f67417fa700 1 -- 172.21.15.19:0/3258956612 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f6744111a10 con 0x7f673406c9c0 2024-02-22T15:52:07.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f673406c9c0 msgr2=0x7f673406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f673406c9c0 0x7f673406ee70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6738000c00 tx=0x7f6738009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:07.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 msgr2=0x7f674410ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 0x7f674410ff60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f673000eb40 tx=0x7f673000b040 comp rx=0 tx=0).stop 2024-02-22T15:52:07.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 shutdown_connections 2024-02-22T15:52:07.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f673406c9c0 0x7f673406ee70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67440663e0 0x7f674410fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 --2- 172.21.15.19:0/3258956612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6744102430 0x7f674410ff60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 >> 172.21.15.19:0/3258956612 conn(0x7f67440f74e0 msgr2=0x7f67440f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 shutdown_connections 2024-02-22T15:52:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.032+0000 7f674b13c700 1 -- 172.21.15.19:0/3258956612 wait complete. 2024-02-22T15:52:07.024 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:52:07.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.136+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/1295071701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 msgr2=0x7f7b040ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.136+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/1295071701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b040ffe60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b00004940 tx=0x7f7b00009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/1295071701 shutdown_connections 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/1295071701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b041003a0 0x7f7b04102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/1295071701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b040ffe60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/1295071701 >> 172.21.15.19:0/1295071701 conn(0x7f7b040f74e0 msgr2=0x7f7b040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/1295071701 shutdown_connections 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/1295071701 wait complete. 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 Processor -- start 2024-02-22T15:52:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.137+0000 7f7b0c5c0700 1 -- start start 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0c5c0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b0410d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0c5c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b041003a0 0x7f7b0410dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0c5c0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0410e390 con 0x7f7b040fda80 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0c5c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0410ec30 con 0x7f7b041003a0 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b0410d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b0410d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47202/0 (socket says 172.21.15.19:47202) 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 -- 172.21.15.19:0/738136331 learned_addr learned my addr 172.21.15.19:0/738136331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b09b5b700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b041003a0 0x7f7b0410dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 -- 172.21.15.19:0/738136331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b041003a0 msgr2=0x7f7b0410dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b041003a0 0x7f7b0410dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 -- 172.21.15.19:0/738136331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00005040 con 0x7f7b040fda80 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0a35c700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b0410d850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009fd0 tx=0x7f7b00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:07.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b0000e040 con 0x7f7b040fda80 2024-02-22T15:52:07.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0410eeb0 con 0x7f7b040fda80 2024-02-22T15:52:07.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b00004290 con 0x7f7b040fda80 2024-02-22T15:52:07.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b0001c3f0 con 0x7f7b040fda80 2024-02-22T15:52:07.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.138+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0410f3f0 con 0x7f7b040fda80 2024-02-22T15:52:07.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.139+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04107910 con 0x7f7b040fda80 2024-02-22T15:52:07.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.140+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f7b00013420 con 0x7f7b040fda80 2024-02-22T15:52:07.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.140+0000 7f7afaffd700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7af006c9c0 0x7f7af006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.140+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7b0008d1a0 con 0x7f7b040fda80 2024-02-22T15:52:07.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.141+0000 7f7b09b5b700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7af006c9c0 0x7f7af006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.141+0000 7f7b09b5b700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7af006c9c0 0x7f7af006ee70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4009040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:07.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.145+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f7b0005aef0 con 0x7f7b040fda80 2024-02-22T15:52:07.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:07 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:07 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:07 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:07 smithi062 ceph-mon[48130]: pgmap v31: 65 pgs: 65 active+clean; 256 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 378 op/s 2024-02-22T15:52:07.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:07 smithi062 ceph-mon[48130]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:07.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.371+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b0402b310 con 0x7f7af006c9c0 2024-02-22T15:52:07.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.372+0000 7f7afaffd700 1 -- 172.21.15.19:0/738136331 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f7b0402b310 con 0x7f7af006c9c0 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7af006c9c0 msgr2=0x7f7af006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7af006c9c0 0x7f7af006ee70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4009040 comp rx=0 tx=0).stop 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 msgr2=0x7f7b0410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b0410d850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009fd0 tx=0x7f7b00009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 shutdown_connections 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7af006c9c0 0x7f7af006ee70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b040fda80 0x7f7b0410d850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 --2- 172.21.15.19:0/738136331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b041003a0 0x7f7b0410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 >> 172.21.15.19:0/738136331 conn(0x7f7b040f74e0 msgr2=0x7f7b040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 shutdown_connections 2024-02-22T15:52:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.374+0000 7f7b0c5c0700 1 -- 172.21.15.19:0/738136331 wait complete. 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 -- 172.21.15.19:0/2848872572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fbe40 msgr2=0x7fed680fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 --2- 172.21.15.19:0/2848872572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fbe40 0x7fed680fc250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed64004940 tx=0x7fed64009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 -- 172.21.15.19:0/2848872572 shutdown_connections 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 --2- 172.21.15.19:0/2848872572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed680fcfb0 0x7fed680fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 --2- 172.21.15.19:0/2848872572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fbe40 0x7fed680fc250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 -- 172.21.15.19:0/2848872572 >> 172.21.15.19:0/2848872572 conn(0x7fed680f74e0 msgr2=0x7fed680f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 -- 172.21.15.19:0/2848872572 shutdown_connections 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.477+0000 7fed70790700 1 -- 172.21.15.19:0/2848872572 wait complete. 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed70790700 1 Processor -- start 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed70790700 1 -- start start 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed70790700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed680fbe40 0x7fed6810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed70790700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 0x7fed68110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed70790700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68110600 con 0x7fed680fcfb0 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed70790700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68110ea0 con 0x7fed680fbe40 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 0x7fed68110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6e52c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed680fbe40 0x7fed6810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 0x7fed68110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45464/0 (socket says 172.21.15.19:45464) 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 -- 172.21.15.19:0/697015572 learned_addr learned my addr 172.21.15.19:0/697015572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 -- 172.21.15.19:0/697015572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed680fbe40 msgr2=0x7fed6810fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed680fbe40 0x7fed6810fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 -- 172.21.15.19:0/697015572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64005040 con 0x7fed680fcfb0 2024-02-22T15:52:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed6dd2b700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 0x7fed68110000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed5c002a50 tx=0x7fed5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.478+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5c007cc0 con 0x7fed680fcfb0 2024-02-22T15:52:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.479+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed5c00f040 con 0x7fed680fcfb0 2024-02-22T15:52:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.479+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed68111180 con 0x7fed680fcfb0 2024-02-22T15:52:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.479+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5c00d9c0 con 0x7fed680fcfb0 2024-02-22T15:52:07.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.479+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed681116c0 con 0x7fed680fcfb0 2024-02-22T15:52:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.480+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6804bed0 con 0x7fed680fcfb0 2024-02-22T15:52:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.480+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fed5c00db20 con 0x7fed680fcfb0 2024-02-22T15:52:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.481+0000 7fed5affd700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fed5406c9c0 0x7fed5406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.481+0000 7fed6e52c700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fed5406c9c0 0x7fed5406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.481+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fed5c08b980 con 0x7fed680fcfb0 2024-02-22T15:52:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.481+0000 7fed6e52c700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fed5406c9c0 0x7fed5406ee70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed64004890 tx=0x7fed64009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.486+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fed5c059750 con 0x7fed680fcfb0 2024-02-22T15:52:07.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:07 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:07.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:07 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:07.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:07 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:07.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:07 smithi019 ceph-mon[67418]: pgmap v31: 65 pgs: 65 active+clean; 256 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 378 op/s 2024-02-22T15:52:07.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:07 smithi019 ceph-mon[67418]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:07.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.700+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed68111b60 con 0x7fed5406c9c0 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.710+0000 7fed5affd700 1 -- 172.21.15.19:0/697015572 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fed68111b60 con 0x7fed5406c9c0 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (2m) 20s ago 10m 17.1M - 0.25.0 c8568f914cd2 36f5804ea0aa 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (10m) 20s ago 10m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (9m) 25s ago 9m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (118s) 20s ago 10m 82.2M - 9.4.7 954c08fa6188 00e17776d4e6 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (4m) 20s ago 4m 136M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (4m) 20s ago 4m 82.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:52:07.691 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (4m) 25s ago 4m 71.9M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (4m) 25s ago 4m 136M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (70s) 20s ago 11m 490M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (29s) 25s ago 9m 45.9M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (11m) 20s ago 12m 54.7M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (9m) 25s ago 9m 45.2M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 starting - - - - 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 starting - - - - 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 20s ago 8m 204M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (8m) 20s ago 8m 221M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (7m) 20s ago 7m 178M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (7m) 25s ago 7m 245M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (6m) 25s ago 6m 216M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:52:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (6m) 25s ago 6m 213M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:52:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (35s) 20s ago 9m 41.5M - 2.43.0 a07b618ecd1d 93ff4711cdd9 2024-02-22T15:52:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fed5406c9c0 msgr2=0x7fed5406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fed5406c9c0 0x7fed5406ee70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed64004890 tx=0x7fed64009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 msgr2=0x7fed68110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 0x7fed68110000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed5c002a50 tx=0x7fed5c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 shutdown_connections 2024-02-22T15:52:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fed5406c9c0 0x7fed5406ee70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed680fbe40 0x7fed6810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 --2- 172.21.15.19:0/697015572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed680fcfb0 0x7fed68110000 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 >> 172.21.15.19:0/697015572 conn(0x7fed680f74e0 msgr2=0x7fed680656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 shutdown_connections 2024-02-22T15:52:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.712+0000 7fed70790700 1 -- 172.21.15.19:0/697015572 wait complete. 2024-02-22T15:52:07.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- 172.21.15.19:0/3348312041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 msgr2=0x7f90a8102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 --2- 172.21.15.19:0/3348312041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a8102780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9098004940 tx=0x7f9098009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:07.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- 172.21.15.19:0/3348312041 shutdown_connections 2024-02-22T15:52:07.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 --2- 172.21.15.19:0/3348312041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a8102780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 --2- 172.21.15.19:0/3348312041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90a80fda80 0x7f90a80ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- 172.21.15.19:0/3348312041 >> 172.21.15.19:0/3348312041 conn(0x7f90a80f74e0 msgr2=0x7f90a80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:07.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- 172.21.15.19:0/3348312041 shutdown_connections 2024-02-22T15:52:07.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- 172.21.15.19:0/3348312041 wait complete. 2024-02-22T15:52:07.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 Processor -- start 2024-02-22T15:52:07.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- start start 2024-02-22T15:52:07.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90a80fda80 0x7f90a80fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a80fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8111560 con 0x7f90a81003a0 2024-02-22T15:52:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90b03ec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81116d0 con 0x7f90a80fda80 2024-02-22T15:52:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90ad987700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a80fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90ad987700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a80fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45486/0 (socket says 172.21.15.19:45486) 2024-02-22T15:52:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.815+0000 7f90ad987700 1 -- 172.21.15.19:0/2657845522 learned_addr learned my addr 172.21.15.19:0/2657845522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90ae188700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90a80fda80 0x7f90a80fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90ad987700 1 -- 172.21.15.19:0/2657845522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90a80fda80 msgr2=0x7f90a80fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90ad987700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90a80fda80 0x7f90a80fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90ad987700 1 -- 172.21.15.19:0/2657845522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098005040 con 0x7f90a81003a0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90ad987700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a80fb990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f909800b3b0 tx=0x7f9098009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f909800e040 con 0x7f90a81003a0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f909800bcd0 con 0x7f90a81003a0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a80fbf90 con 0x7f90a81003a0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f909801c630 con 0x7f90a81003a0 2024-02-22T15:52:07.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.816+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a80fc4d0 con 0x7f90a81003a0 2024-02-22T15:52:07.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.817+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a8107910 con 0x7f90a81003a0 2024-02-22T15:52:07.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.818+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f9098013420 con 0x7f90a81003a0 2024-02-22T15:52:07.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.818+0000 7f909effd700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f909406c9c0 0x7f909406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:07.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.818+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f909808d390 con 0x7f90a81003a0 2024-02-22T15:52:07.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.818+0000 7f90ae188700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f909406c9c0 0x7f909406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:07.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.819+0000 7f90ae188700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f909406c9c0 0x7f909406ee70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90a80fd2a0 tx=0x7f90a4009040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:07.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:07.822+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f909805b140 con 0x7f90a81003a0 2024-02-22T15:52:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.115+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f90a804bed0 con 0x7f90a81003a0 2024-02-22T15:52:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.115+0000 7f909effd700 1 -- 172.21.15.19:0/2657845522 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f909805acd0 con 0x7f90a81003a0 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:08.097 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:52:08.098 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 12, 2024-02-22T15:52:08.098 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:52:08.098 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:52:08.098 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:52:08.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f909406c9c0 msgr2=0x7f909406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f909406c9c0 0x7f909406ee70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f90a80fd2a0 tx=0x7f90a4009040 comp rx=0 tx=0).stop 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 msgr2=0x7f90a80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a80fb990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f909800b3b0 tx=0x7f9098009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 shutdown_connections 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f909406c9c0 0x7f909406ee70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90a80fda80 0x7f90a80fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 --2- 172.21.15.19:0/2657845522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90a81003a0 0x7f90a80fb990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.117+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 >> 172.21.15.19:0/2657845522 conn(0x7f90a80f74e0 msgr2=0x7f90a80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.118+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 shutdown_connections 2024-02-22T15:52:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.118+0000 7f90b03ec700 1 -- 172.21.15.19:0/2657845522 wait complete. 2024-02-22T15:52:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 -- 172.21.15.19:0/180988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d341003a0 msgr2=0x7f2d34102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/180988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d341003a0 0x7f2d34102780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c004940 tx=0x7f2d1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 -- 172.21.15.19:0/180988616 shutdown_connections 2024-02-22T15:52:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/180988616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d341003a0 0x7f2d34102780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/180988616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d340fda80 0x7f2d340ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 -- 172.21.15.19:0/180988616 >> 172.21.15.19:0/180988616 conn(0x7f2d340f74e0 msgr2=0x7f2d340f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:08.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 -- 172.21.15.19:0/180988616 shutdown_connections 2024-02-22T15:52:08.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 -- 172.21.15.19:0/180988616 wait complete. 2024-02-22T15:52:08.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 Processor -- start 2024-02-22T15:52:08.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.225+0000 7f2d39cf3700 1 -- start start 2024-02-22T15:52:08.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d39cf3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 0x7f2d3410d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d39cf3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d341003a0 0x7f2d3410ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d39cf3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3410e3c0 con 0x7f2d340fda80 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d39cf3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3410ec60 con 0x7f2d341003a0 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 0x7f2d3410d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 0x7f2d3410d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45506/0 (socket says 172.21.15.19:45506) 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 -- 172.21.15.19:0/3759616432 learned_addr learned my addr 172.21.15.19:0/3759616432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d32ffd700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d341003a0 0x7f2d3410ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 -- 172.21.15.19:0/3759616432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d341003a0 msgr2=0x7f2d3410ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d341003a0 0x7f2d3410ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 -- 172.21.15.19:0/3759616432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c005040 con 0x7f2d340fda80 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d337fe700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 0x7f2d3410d880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c002a50 tx=0x7f2d2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2c007cc0 con 0x7f2d340fda80 2024-02-22T15:52:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3410ef40 con 0x7f2d340fda80 2024-02-22T15:52:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d2c00f040 con 0x7f2d340fda80 2024-02-22T15:52:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2c00d9c0 con 0x7f2d340fda80 2024-02-22T15:52:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.226+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3410f480 con 0x7f2d340fda80 2024-02-22T15:52:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.227+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d34107910 con 0x7f2d340fda80 2024-02-22T15:52:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.228+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f2d2c00db20 con 0x7f2d340fda80 2024-02-22T15:52:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.228+0000 7f2d2bfff700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f2d2006c9c0 0x7f2d2006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.228+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2d2c08ba40 con 0x7f2d340fda80 2024-02-22T15:52:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.228+0000 7f2d32ffd700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f2d2006c9c0 0x7f2d2006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.229+0000 7f2d32ffd700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f2d2006c9c0 0x7f2d2006ee70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b3b0 tx=0x7f2d1c009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.233+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f2d2c059790 con 0x7f2d340fda80 2024-02-22T15:52:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.460+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d3410f7b0 con 0x7f2d2006c9c0 2024-02-22T15:52:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.461+0000 7f2d2bfff700 1 -- 172.21.15.19:0/3759616432 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f2d3410f7b0 con 0x7f2d2006c9c0 2024-02-22T15:52:08.442 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "2/2 daemons upgraded", 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading node-exporter daemons", 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:52:08.443 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f2d2006c9c0 msgr2=0x7f2d2006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f2d2006c9c0 0x7f2d2006ee70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b3b0 tx=0x7f2d1c009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 msgr2=0x7f2d3410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 0x7f2d3410d880 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c002a50 tx=0x7f2d2c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 shutdown_connections 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f2d2006c9c0 0x7f2d2006ee70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d340fda80 0x7f2d3410d880 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 --2- 172.21.15.19:0/3759616432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d341003a0 0x7f2d3410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.463+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 >> 172.21.15.19:0/3759616432 conn(0x7f2d340f74e0 msgr2=0x7f2d340f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.464+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 shutdown_connections 2024-02-22T15:52:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:08.464+0000 7f2d39cf3700 1 -- 172.21.15.19:0/3759616432 wait complete. 2024-02-22T15:52:08.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:08 smithi062 ceph-mon[48130]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:08.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:08 smithi062 ceph-mon[48130]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:08.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:08 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2657845522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:08.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:08 smithi019 ceph-mon[67418]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:08.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:08 smithi019 ceph-mon[67418]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:08.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:08 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/2657845522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:10.167 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:09 smithi062 ceph-mon[48130]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:10.167 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:09 smithi062 ceph-mon[48130]: pgmap v32: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 806 KiB/s rd, 747 KiB/s wr, 272 op/s 2024-02-22T15:52:10.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.168 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:09 smithi019 ceph-mon[67418]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:09 smithi019 ceph-mon[67418]: pgmap v32: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 806 KiB/s rd, 747 KiB/s wr, 272 op/s 2024-02-22T15:52:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:10.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:11 smithi019 ceph-mon[67418]: pgmap v33: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 423 KiB/s rd, 410 KiB/s wr, 181 op/s 2024-02-22T15:52:12.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:12.435 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:11 smithi062 ceph-mon[48130]: pgmap v33: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 423 KiB/s rd, 410 KiB/s wr, 181 op/s 2024-02-22T15:52:12.435 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:13 smithi019 ceph-mon[67418]: pgmap v34: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 424 KiB/s rd, 411 KiB/s wr, 183 op/s 2024-02-22T15:52:14.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:13 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:13 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:13 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:13 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:13 smithi062 ceph-mon[48130]: pgmap v34: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 424 KiB/s rd, 411 KiB/s wr, 183 op/s 2024-02-22T15:52:14.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:13 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:13 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:13 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:14.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:13 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:15.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:15.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:15.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:15 smithi019 ceph-mon[67418]: pgmap v35: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 32 KiB/s wr, 47 op/s 2024-02-22T15:52:15.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:15.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:15.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:15 smithi062 ceph-mon[48130]: pgmap v35: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 32 KiB/s wr, 47 op/s 2024-02-22T15:52:18.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:17 smithi019 ceph-mon[67418]: pgmap v36: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 32 KiB/s wr, 48 op/s 2024-02-22T15:52:18.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:17 smithi062 ceph-mon[48130]: pgmap v36: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 32 KiB/s wr, 48 op/s 2024-02-22T15:52:19.704 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:19.704 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:19.704 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:19 smithi019 ceph-mon[67418]: pgmap v37: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 27 KiB/s wr, 22 op/s 2024-02-22T15:52:19.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:19.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:19 smithi062 ceph-mon[48130]: pgmap v37: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 27 KiB/s wr, 22 op/s 2024-02-22T15:52:22.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:21 smithi019 ceph-mon[67418]: Upgrade: Updating prometheus.smithi019 2024-02-22T15:52:22.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:21 smithi019 ceph-mon[67418]: Deploying daemon prometheus.smithi019 on smithi019 2024-02-22T15:52:22.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:21 smithi019 ceph-mon[67418]: pgmap v38: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-22T15:52:22.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:21 smithi062 ceph-mon[48130]: Upgrade: Updating prometheus.smithi019 2024-02-22T15:52:22.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:21 smithi062 ceph-mon[48130]: Deploying daemon prometheus.smithi019 on smithi019 2024-02-22T15:52:22.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:21 smithi062 ceph-mon[48130]: pgmap v38: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 4 op/s 2024-02-22T15:52:24.016 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:23 smithi019 ceph-mon[67418]: pgmap v39: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-22T15:52:24.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:23 smithi062 ceph-mon[48130]: pgmap v39: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-22T15:52:25.194 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:25 smithi019 ceph-mon[67418]: pgmap v40: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:25.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:25 smithi062 ceph-mon[48130]: pgmap v40: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:26.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:27.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:27.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:27 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:27 smithi062 ceph-mon[48130]: pgmap v41: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:27.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:27.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:27 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:27.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:27 smithi019 ceph-mon[67418]: pgmap v41: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:30.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:29 smithi019 ceph-mon[67418]: pgmap v42: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:30.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:29 smithi062 ceph-mon[48130]: pgmap v42: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:32.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:31 smithi019 ceph-mon[67418]: pgmap v43: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:32.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:31 smithi062 ceph-mon[48130]: pgmap v43: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:33.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:33 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:33.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:33 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:34.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:34 smithi019 ceph-mon[67418]: pgmap v44: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:34.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:34 smithi062 ceph-mon[48130]: pgmap v44: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:35.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:35 smithi062 ceph-mon[48130]: pgmap v45: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:35.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:35 smithi019 ceph-mon[67418]: pgmap v45: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:37.708 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:37.708 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:37.708 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.709 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.710 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:37 smithi019 ceph-mon[67418]: pgmap v46: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:52:37.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:37.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:37 smithi062 ceph-mon[48130]: pgmap v46: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 -- 172.21.15.19:0/3309310946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 msgr2=0x7fed0c0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 --2- 172.21.15.19:0/3309310946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0ffe60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004940 tx=0x7fecfc009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 -- 172.21.15.19:0/3309310946 shutdown_connections 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 --2- 172.21.15.19:0/3309310946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c1003a0 0x7fed0c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 --2- 172.21.15.19:0/3309310946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0ffe60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 -- 172.21.15.19:0/3309310946 >> 172.21.15.19:0/3309310946 conn(0x7fed0c0f74e0 msgr2=0x7fed0c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 -- 172.21.15.19:0/3309310946 shutdown_connections 2024-02-22T15:52:38.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 -- 172.21.15.19:0/3309310946 wait complete. 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.572+0000 7fed11ced700 1 Processor -- start 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed11ced700 1 -- start start 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed11ced700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed11ced700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c1003a0 0x7fed0c0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed11ced700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0fbf90 con 0x7fed0c0fda80 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed11ced700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0fc100 con 0x7fed0c1003a0 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41716/0 (socket says 172.21.15.19:41716) 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 -- 172.21.15.19:0/229137204 learned_addr learned my addr 172.21.15.19:0/229137204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0affd700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c1003a0 0x7fed0c0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 -- 172.21.15.19:0/229137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c1003a0 msgr2=0x7fed0c0fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c1003a0 0x7fed0c0fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 -- 172.21.15.19:0/229137204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4005040 con 0x7fed0c0fda80 2024-02-22T15:52:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.573+0000 7fed0b7fe700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0fd510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.574+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecfc021070 con 0x7fed0c0fda80 2024-02-22T15:52:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.574+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc005040 con 0x7fed0c0fda80 2024-02-22T15:52:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.574+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fecfc01c400 con 0x7fed0c0fda80 2024-02-22T15:52:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.574+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecfc00e070 con 0x7fed0c0fda80 2024-02-22T15:52:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.574+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c0fc6d0 con 0x7fed0c0fda80 2024-02-22T15:52:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.575+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c04bed0 con 0x7fed0c0fda80 2024-02-22T15:52:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.575+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fecfc01c570 con 0x7fed0c0fda80 2024-02-22T15:52:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.576+0000 7fed03fff700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fecf806c9c0 0x7fecf806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.576+0000 7fed0affd700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fecf806c9c0 0x7fecf806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.576+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fecfc08d050 con 0x7fed0c0fda80 2024-02-22T15:52:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.576+0000 7fed0affd700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fecf806c9c0 0x7fecf806ee70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fd0 tx=0x7fecf4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:38.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.581+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fecfc05ae20 con 0x7fed0c0fda80 2024-02-22T15:52:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.804+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed0c101000 con 0x7fecf806c9c0 2024-02-22T15:52:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.805+0000 7fed03fff700 1 -- 172.21.15.19:0/229137204 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fed0c101000 con 0x7fecf806c9c0 2024-02-22T15:52:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fecf806c9c0 msgr2=0x7fecf806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fecf806c9c0 0x7fecf806ee70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fd0 tx=0x7fecf4009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 msgr2=0x7fed0c0fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0fd510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 shutdown_connections 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fecf806c9c0 0x7fecf806ee70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed0c0fda80 0x7fed0c0fd510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 --2- 172.21.15.19:0/229137204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c1003a0 0x7fed0c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 >> 172.21.15.19:0/229137204 conn(0x7fed0c0f74e0 msgr2=0x7fed0c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 shutdown_connections 2024-02-22T15:52:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.807+0000 7fed11ced700 1 -- 172.21.15.19:0/229137204 wait complete. 2024-02-22T15:52:38.799 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:52:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 -- 172.21.15.19:0/3267383670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fcfb0 msgr2=0x7f34940fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 --2- 172.21.15.19:0/3267383670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fcfb0 0x7f34940fd420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3488004940 tx=0x7f3488009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 -- 172.21.15.19:0/3267383670 shutdown_connections 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 --2- 172.21.15.19:0/3267383670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fcfb0 0x7f34940fd420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 --2- 172.21.15.19:0/3267383670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940fbe40 0x7f34940fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 -- 172.21.15.19:0/3267383670 >> 172.21.15.19:0/3267383670 conn(0x7f34940f7520 msgr2=0x7f34940f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 -- 172.21.15.19:0/3267383670 shutdown_connections 2024-02-22T15:52:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.918+0000 7f34996ba700 1 -- 172.21.15.19:0/3267383670 wait complete. 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f34996ba700 1 Processor -- start 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f34996ba700 1 -- start start 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f34996ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 0x7f3494110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f34996ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940fcfb0 0x7f3494112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f34996ba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494066d10 con 0x7f34940fbe40 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f34996ba700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494066e50 con 0x7f34940fcfb0 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.919+0000 7f3492ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 0x7f3494110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f3492ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 0x7f3494110550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41740/0 (socket says 172.21.15.19:41740) 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f3492ffd700 1 -- 172.21.15.19:0/2933463308 learned_addr learned my addr 172.21.15.19:0/2933463308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f34927fc700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940fcfb0 0x7f3494112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f3492ffd700 1 -- 172.21.15.19:0/2933463308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940fcfb0 msgr2=0x7f3494112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f3492ffd700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940fcfb0 0x7f3494112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f3492ffd700 1 -- 172.21.15.19:0/2933463308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488005040 con 0x7f34940fbe40 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f3492ffd700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 0x7f3494110550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f348400c830 tx=0x7f3484004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348400a760 con 0x7f34940fbe40 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f348400ad70 con 0x7f34940fbe40 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3494113100 con 0x7f34940fbe40 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34840113a0 con 0x7f34940fbe40 2024-02-22T15:52:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.920+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34941135f0 con 0x7f34940fbe40 2024-02-22T15:52:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.921+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349404bed0 con 0x7f34940fbe40 2024-02-22T15:52:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.922+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f3484012400 con 0x7f34940fbe40 2024-02-22T15:52:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.922+0000 7f347b7fe700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f347c06c9c0 0x7f347c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.923+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f348408bbd0 con 0x7f34940fbe40 2024-02-22T15:52:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.923+0000 7f34927fc700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f347c06c9c0 0x7f347c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.923+0000 7f34927fc700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f347c06c9c0 0x7f347c06ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f348800b3b0 tx=0x7f3488009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:38.927+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f34840560c0 con 0x7f34940fbe40 2024-02-22T15:52:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.154+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34941138c0 con 0x7f347c06c9c0 2024-02-22T15:52:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.155+0000 7f347b7fe700 1 -- 172.21.15.19:0/2933463308 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f34941138c0 con 0x7f347c06c9c0 2024-02-22T15:52:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f347c06c9c0 msgr2=0x7f347c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f347c06c9c0 0x7f347c06ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f348800b3b0 tx=0x7f3488009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 msgr2=0x7f3494110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 0x7f3494110550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f348400c830 tx=0x7f3484004470 comp rx=0 tx=0).stop 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 shutdown_connections 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f347c06c9c0 0x7f347c06ee70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34940fbe40 0x7f3494110550 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 --2- 172.21.15.19:0/2933463308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34940fcfb0 0x7f3494112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.157+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 >> 172.21.15.19:0/2933463308 conn(0x7f34940f7520 msgr2=0x7f3494100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.158+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 shutdown_connections 2024-02-22T15:52:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.158+0000 7f34996ba700 1 -- 172.21.15.19:0/2933463308 wait complete. 2024-02-22T15:52:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/527179749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c8066c50 msgr2=0x7fc4c80ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/527179749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c8066c50 0x7fc4c80ffb10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4004940 tx=0x7fc4c4009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/527179749 shutdown_connections 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/527179749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c81000e0 0x7fc4c8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/527179749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c8066c50 0x7fc4c80ffb10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/527179749 >> 172.21.15.19:0/527179749 conn(0x7fc4c80f7500 msgr2=0x7fc4c80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/527179749 shutdown_connections 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.261+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/527179749 wait complete. 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cfd4e700 1 Processor -- start 2024-02-22T15:52:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cfd4e700 1 -- start start 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cfd4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c8066c50 0x7fc4c80fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cfd4e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 0x7fc4c80fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cfd4e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80fc010 con 0x7fc4c81000e0 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cfd4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80fc180 con 0x7fc4c8066c50 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 0x7fc4c80fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 0x7fc4c80fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41750/0 (socket says 172.21.15.19:41750) 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 -- 172.21.15.19:0/3835799220 learned_addr learned my addr 172.21.15.19:0/3835799220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cdaea700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c8066c50 0x7fc4c80fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 -- 172.21.15.19:0/3835799220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c8066c50 msgr2=0x7fc4c80fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c8066c50 0x7fc4c80fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 -- 172.21.15.19:0/3835799220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4005040 con 0x7fc4c81000e0 2024-02-22T15:52:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.262+0000 7fc4cd2e9700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 0x7fc4c80fba10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00eb40 tx=0x7fc4bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.263+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4bc007cc0 con 0x7fc4c81000e0 2024-02-22T15:52:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.263+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c80fc400 con 0x7fc4c81000e0 2024-02-22T15:52:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.263+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4bc00f040 con 0x7fc4c81000e0 2024-02-22T15:52:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.263+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4bc00c8f0 con 0x7fc4c81000e0 2024-02-22T15:52:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.263+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c8115860 con 0x7fc4c81000e0 2024-02-22T15:52:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.264+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c8107930 con 0x7fc4c81000e0 2024-02-22T15:52:39.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.264+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fc4bc00cb60 con 0x7fc4c81000e0 2024-02-22T15:52:39.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.265+0000 7fc4ba7fc700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fc4b406c9c0 0x7fc4b406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.265+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc4bc08b7b0 con 0x7fc4c81000e0 2024-02-22T15:52:39.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.265+0000 7fc4cdaea700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fc4b406c9c0 0x7fc4b406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.265+0000 7fc4cdaea700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fc4b406c9c0 0x7fc4b406ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4013680 tx=0x7fc4c4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.269+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fc4bc0595b0 con 0x7fc4c81000e0 2024-02-22T15:52:39.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.482+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c802b310 con 0x7fc4b406c9c0 2024-02-22T15:52:39.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.492+0000 7fc4ba7fc700 1 -- 172.21.15.19:0/3835799220 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc4c802b310 con 0x7fc4b406c9c0 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (2m) 7s ago 10m 26.0M - 0.25.0 c8568f914cd2 36f5804ea0aa 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (11m) 7s ago 11m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (10m) 30s ago 10m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (2m) 7s ago 10m 82.3M - 9.4.7 954c08fa6188 00e17776d4e6 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (5m) 7s ago 5m 114M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (5m) 7s ago 5m 67.2M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (5m) 30s ago 5m 61.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:52:39.473 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (5m) 30s ago 5m 117M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (101s) 7s ago 12m 498M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (61s) 30s ago 9m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (12m) 7s ago 12m 56.5M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (9m) 30s ago 9m 46.1M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (47s) 7s ago 11m 10.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (36s) 30s ago 9m 3565k - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (9m) 7s ago 9m 221M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (8m) 7s ago 8m 241M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (8m) 7s ago 8m 194M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (7m) 30s ago 7m 257M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (7m) 30s ago 7m 228M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:52:39.474 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (6m) 30s ago 6m 219M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:52:39.475 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (13s) 7s ago 10m 37.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:52:39.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fc4b406c9c0 msgr2=0x7fc4b406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fc4b406c9c0 0x7fc4b406ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4013680 tx=0x7fc4c4009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:39.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 msgr2=0x7fc4c80fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 0x7fc4c80fba10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00eb40 tx=0x7fc4bc00b040 comp rx=0 tx=0).stop 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 shutdown_connections 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fc4b406c9c0 0x7fc4b406ee70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c8066c50 0x7fc4c80fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 --2- 172.21.15.19:0/3835799220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4c81000e0 0x7fc4c80fba10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 >> 172.21.15.19:0/3835799220 conn(0x7fc4c80f7500 msgr2=0x7fc4c80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 shutdown_connections 2024-02-22T15:52:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.494+0000 7fc4cfd4e700 1 -- 172.21.15.19:0/3835799220 wait complete. 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 -- 172.21.15.19:0/3252568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 msgr2=0x7ff40c0ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 --2- 172.21.15.19:0/3252568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0ffe60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff408004940 tx=0x7ff408009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 -- 172.21.15.19:0/3252568195 shutdown_connections 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 --2- 172.21.15.19:0/3252568195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c1003a0 0x7ff40c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 --2- 172.21.15.19:0/3252568195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0ffe60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 -- 172.21.15.19:0/3252568195 >> 172.21.15.19:0/3252568195 conn(0x7ff40c0f74e0 msgr2=0x7ff40c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 -- 172.21.15.19:0/3252568195 shutdown_connections 2024-02-22T15:52:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.597+0000 7ff413814700 1 -- 172.21.15.19:0/3252568195 wait complete. 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff413814700 1 Processor -- start 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff413814700 1 -- start start 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff413814700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff413814700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c1003a0 0x7ff40c0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff413814700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0fbf90 con 0x7ff40c0fda80 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff413814700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c0fc100 con 0x7ff40c1003a0 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff4115b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff4115b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41768/0 (socket says 172.21.15.19:41768) 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff4115b0700 1 -- 172.21.15.19:0/4127187349 learned_addr learned my addr 172.21.15.19:0/4127187349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff410daf700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c1003a0 0x7ff40c0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff4115b0700 1 -- 172.21.15.19:0/4127187349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c1003a0 msgr2=0x7ff40c0fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff4115b0700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c1003a0 0x7ff40c0fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.598+0000 7ff4115b0700 1 -- 172.21.15.19:0/4127187349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408005040 con 0x7ff40c0fda80 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.599+0000 7ff4115b0700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0fd510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff40800b3b0 tx=0x7ff408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.599+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40800e040 con 0x7ff40c0fda80 2024-02-22T15:52:39.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.599+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff40800bc50 con 0x7ff40c0fda80 2024-02-22T15:52:39.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.599+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c0fc380 con 0x7ff40c0fda80 2024-02-22T15:52:39.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.599+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40801c630 con 0x7ff40c0fda80 2024-02-22T15:52:39.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.599+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c1158a0 con 0x7ff40c0fda80 2024-02-22T15:52:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.600+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c107930 con 0x7ff40c0fda80 2024-02-22T15:52:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.601+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7ff408013420 con 0x7ff40c0fda80 2024-02-22T15:52:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.601+0000 7ff3fdffb700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff3f806c9c0 0x7ff3f806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.601+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff40808d130 con 0x7ff40c0fda80 2024-02-22T15:52:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.601+0000 7ff410daf700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff3f806c9c0 0x7ff3f806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.601+0000 7ff410daf700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff3f806c9c0 0x7ff3f806ee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff400004f60 tx=0x7ff40000b040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.606+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7ff40805af00 con 0x7ff40c0fda80 2024-02-22T15:52:39.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.892+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff40c107350 con 0x7ff40c0fda80 2024-02-22T15:52:39.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.892+0000 7ff3fdffb700 1 -- 172.21.15.19:0/4127187349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7ff40805aa90 con 0x7ff40c0fda80 2024-02-22T15:52:39.873 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:52:39.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 12, 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:52:39.874 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:52:39.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff3f806c9c0 msgr2=0x7ff3f806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff3f806c9c0 0x7ff3f806ee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff400004f60 tx=0x7ff40000b040 comp rx=0 tx=0).stop 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 msgr2=0x7ff40c0fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0fd510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff40800b3b0 tx=0x7ff408009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 shutdown_connections 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ff3f806c9c0 0x7ff3f806ee70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c0fda80 0x7ff40c0fd510 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 --2- 172.21.15.19:0/4127187349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c1003a0 0x7ff40c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.894+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 >> 172.21.15.19:0/4127187349 conn(0x7ff40c0f74e0 msgr2=0x7ff40c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.895+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 shutdown_connections 2024-02-22T15:52:39.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:39.895+0000 7ff413814700 1 -- 172.21.15.19:0/4127187349 wait complete. 2024-02-22T15:52:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 -- 172.21.15.19:0/2843139960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fbe40 msgr2=0x7fd7ec0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2843139960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fbe40 0x7fd7ec0fc250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc004940 tx=0x7fd7dc009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 -- 172.21.15.19:0/2843139960 shutdown_connections 2024-02-22T15:52:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2843139960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2843139960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fbe40 0x7fd7ec0fc250 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 -- 172.21.15.19:0/2843139960 >> 172.21.15.19:0/2843139960 conn(0x7fd7ec0f74e0 msgr2=0x7fd7ec0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 -- 172.21.15.19:0/2843139960 shutdown_connections 2024-02-22T15:52:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.003+0000 7fd7f2841700 1 -- 172.21.15.19:0/2843139960 wait complete. 2024-02-22T15:52:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7f2841700 1 Processor -- start 2024-02-22T15:52:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7f2841700 1 -- start start 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7f2841700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0fbe40 0x7fd7ec10c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7f2841700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec10ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7f2841700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec10d340 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7f2841700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec10db80 con 0x7fd7ec0fbe40 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec10ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7eb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec10ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41794/0 (socket says 172.21.15.19:41794) 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7eb7fe700 1 -- 172.21.15.19:0/2169309036 learned_addr learned my addr 172.21.15.19:0/2169309036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.004+0000 7fd7ebfff700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0fbe40 0x7fd7ec10c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7eb7fe700 1 -- 172.21.15.19:0/2169309036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0fbe40 msgr2=0x7fd7ec10c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7eb7fe700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0fbe40 0x7fd7ec10c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7eb7fe700 1 -- 172.21.15.19:0/2169309036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc005040 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7eb7fe700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec10ccb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000b790 tx=0x7fd7e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e000f6f0 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec10de60 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7e000fd00 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e00103a0 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.005+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec10e3a0 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.006+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec04bed0 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.007+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 31) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fd7e000f850 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.008+0000 7fd7e8ff9700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7d406c9c0 0x7fd7d406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:52:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.008+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd7e008c6d0 con 0x7fd7ec0fcfb0 2024-02-22T15:52:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.008+0000 7fd7ebfff700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7d406c9c0 0x7fd7d406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:52:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.008+0000 7fd7ebfff700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7d406c9c0 0x7fd7d406ee70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00b3b0 tx=0x7fd7dc009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:52:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.012+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fd7e005a590 con 0x7fd7ec0fcfb0 2024-02-22T15:52:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.236+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7ec10e8e0 con 0x7fd7d406c9c0 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.237+0000 7fd7e8ff9700 1 -- 172.21.15.19:0/2169309036 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fd7ec10e8e0 con 0x7fd7d406c9c0 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:52:40.218 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T15:52:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:52:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "2/2 daemons upgraded", 2024-02-22T15:52:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading alertmanager daemons", 2024-02-22T15:52:40.219 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:52:40.219 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:52:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7d406c9c0 msgr2=0x7fd7d406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7d406c9c0 0x7fd7d406ee70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00b3b0 tx=0x7fd7dc009e30 comp rx=0 tx=0).stop 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 msgr2=0x7fd7ec10ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec10ccb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000b790 tx=0x7fd7e0009250 comp rx=0 tx=0).stop 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 shutdown_connections 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7d406c9c0 0x7fd7d406ee70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7ec0fbe40 0x7fd7ec10c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 --2- 172.21.15.19:0/2169309036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7ec0fcfb0 0x7fd7ec10ccb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 >> 172.21.15.19:0/2169309036 conn(0x7fd7ec0f74e0 msgr2=0x7fd7ec100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 shutdown_connections 2024-02-22T15:52:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:52:40.239+0000 7fd7f2841700 1 -- 172.21.15.19:0/2169309036 wait complete. 2024-02-22T15:52:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:39 smithi019 ceph-mon[67418]: Upgrade: Updating alertmanager.smithi019 2024-02-22T15:52:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:39 smithi019 ceph-mon[67418]: Deploying daemon alertmanager.smithi019 on smithi019 2024-02-22T15:52:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:39 smithi019 ceph-mon[67418]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:39 smithi019 ceph-mon[67418]: pgmap v47: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:40.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:39 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/4127187349' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:39 smithi062 ceph-mon[48130]: Upgrade: Updating alertmanager.smithi019 2024-02-22T15:52:40.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:39 smithi062 ceph-mon[48130]: Deploying daemon alertmanager.smithi019 on smithi019 2024-02-22T15:52:40.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:39 smithi062 ceph-mon[48130]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:40.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:39 smithi062 ceph-mon[48130]: pgmap v47: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:40.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:39 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/4127187349' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:52:41.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:40 smithi019 ceph-mon[67418]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:41.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:40 smithi019 ceph-mon[67418]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:41.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:41 smithi062 ceph-mon[48130]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:41.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:41 smithi062 ceph-mon[48130]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:41 smithi019 ceph-mon[67418]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:42.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:41 smithi019 ceph-mon[67418]: pgmap v48: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:42.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:41 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:42.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:41 smithi062 ceph-mon[48130]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:52:42.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:41 smithi062 ceph-mon[48130]: pgmap v48: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:42.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:41 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:44.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:44 smithi019 ceph-mon[67418]: pgmap v49: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:52:44.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:44 smithi019 ceph-mon[67418]: mgrmap e32: smithi019.yvjrkp(active, since 92s), standbys: smithi062.ecubmv 2024-02-22T15:52:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:44 smithi062 ceph-mon[48130]: pgmap v49: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:52:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:44 smithi062 ceph-mon[48130]: mgrmap e32: smithi019.yvjrkp(active, since 92s), standbys: smithi062.ecubmv 2024-02-22T15:52:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:45 smithi062 ceph-mon[48130]: pgmap v50: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-22T15:52:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:45 smithi019 ceph-mon[67418]: pgmap v50: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-22T15:52:48.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:47 smithi019 ceph-mon[67418]: pgmap v51: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:52:48.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:47 smithi062 ceph-mon[48130]: pgmap v51: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:52:50.128 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:49 smithi019 ceph-mon[67418]: pgmap v52: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-22T15:52:50.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:49 smithi062 ceph-mon[48130]: pgmap v52: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-22T15:52:52.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:52 smithi019 ceph-mon[67418]: pgmap v53: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-22T15:52:52.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:52 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:52.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:52 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:52.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:52 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:52.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:52 smithi062 ceph-mon[48130]: pgmap v53: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-22T15:52:52.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:52 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:52.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:52 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:52.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:52 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:52:54.176 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:54 smithi019 ceph-mon[67418]: pgmap v54: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:52:54.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:54 smithi062 ceph-mon[48130]: pgmap v54: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-22T15:52:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:55 smithi019 ceph-mon[67418]: pgmap v55: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:55.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:55 smithi062 ceph-mon[48130]: pgmap v55: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:56.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:56 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:56.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:56 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:52:57.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:57 smithi062 ceph-mon[48130]: pgmap v56: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:57.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:57 smithi019 ceph-mon[67418]: pgmap v56: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:52:58.592 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:58 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.592 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:58 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.592 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:58 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.592 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:58 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:58 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:58 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:58 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:58.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:58 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:52:59.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:52:59 smithi062 ceph-mon[48130]: pgmap v57: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:52:59.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:52:59 smithi019 ceph-mon[67418]: pgmap v57: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: pgmap v58: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:02.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:01 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: pgmap v58: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:02.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:01 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:03 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:53:03.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:03 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-22T15:53:04.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:04 smithi019 ceph-mon[67418]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:04 smithi062 ceph-mon[48130]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:05.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:05 smithi019 ceph-mon[67418]: Upgrade: Updating grafana.smithi019 2024-02-22T15:53:05.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:05 smithi019 ceph-mon[67418]: Deploying daemon grafana.smithi019 on smithi019 2024-02-22T15:53:05.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:05 smithi062 ceph-mon[48130]: Upgrade: Updating grafana.smithi019 2024-02-22T15:53:05.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:05 smithi062 ceph-mon[48130]: Deploying daemon grafana.smithi019 on smithi019 2024-02-22T15:53:06.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:06 smithi019 ceph-mon[67418]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:06.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:06 smithi062 ceph-mon[48130]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:08.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:08 smithi019 ceph-mon[67418]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:08.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:08 smithi062 ceph-mon[48130]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:09.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:09.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:09.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:09 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:09.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:09 smithi062 ceph-mon[48130]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:09.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:09.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:09.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:09 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:09.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:09 smithi019 ceph-mon[67418]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.346+0000 7f350bcad700 1 -- 172.21.15.19:0/1536574011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 msgr2=0x7f35040fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.346+0000 7f350bcad700 1 --2- 172.21.15.19:0/1536574011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f35040fd420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f34f4004940 tx=0x7f34f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- 172.21.15.19:0/1536574011 shutdown_connections 2024-02-22T15:53:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 --2- 172.21.15.19:0/1536574011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f35040fd420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 --2- 172.21.15.19:0/1536574011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35040fbe40 0x7f35040fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- 172.21.15.19:0/1536574011 >> 172.21.15.19:0/1536574011 conn(0x7f35040f74e0 msgr2=0x7f35040f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- 172.21.15.19:0/1536574011 shutdown_connections 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- 172.21.15.19:0/1536574011 wait complete. 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 Processor -- start 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- start start 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35040fbe40 0x7f350410c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f350410cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350410d330 con 0x7f35040fcfb0 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f350bcad700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350410dbd0 con 0x7f35040fbe40 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.347+0000 7f3509248700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f350410cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509248700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f350410cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41770/0 (socket says 172.21.15.19:41770) 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509248700 1 -- 172.21.15.19:0/2414926111 learned_addr learned my addr 172.21.15.19:0/2414926111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509a49700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35040fbe40 0x7f350410c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509248700 1 -- 172.21.15.19:0/2414926111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35040fbe40 msgr2=0x7f350410c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509248700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35040fbe40 0x7f350410c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509248700 1 -- 172.21.15.19:0/2414926111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4005040 con 0x7f35040fcfb0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f3509248700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f350410cd30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f34f4004f90 tx=0x7f34f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f400e040 con 0x7f35040fcfb0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34f400be60 con 0x7f35040fcfb0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f4013770 con 0x7f35040fcfb0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350410de50 con 0x7f35040fcfb0 2024-02-22T15:53:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.348+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350410e330 con 0x7f35040fcfb0 2024-02-22T15:53:10.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.349+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350404bed0 con 0x7f35040fcfb0 2024-02-22T15:53:10.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.350+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f34f401ead0 con 0x7f35040fcfb0 2024-02-22T15:53:10.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.350+0000 7f34fa7fc700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f34f006c8a0 0x7f34f006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:10.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.350+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f34f408fa60 con 0x7f35040fcfb0 2024-02-22T15:53:10.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.350+0000 7f3509a49700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f34f006c8a0 0x7f34f006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:10.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.350+0000 7f3509a49700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f34f006c8a0 0x7f34f006ed50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3500007fd0 tx=0x7f3500009040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:10.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.355+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f34f405da50 con 0x7f35040fcfb0 2024-02-22T15:53:10.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.581+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f350410e770 con 0x7f34f006c8a0 2024-02-22T15:53:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.583+0000 7f34fa7fc700 1 -- 172.21.15.19:0/2414926111 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f350410e770 con 0x7f34f006c8a0 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f34f006c8a0 msgr2=0x7f34f006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f34f006c8a0 0x7f34f006ed50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3500007fd0 tx=0x7f3500009040 comp rx=0 tx=0).stop 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 msgr2=0x7f350410cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f350410cd30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f34f4004f90 tx=0x7f34f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 shutdown_connections 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f34f006c8a0 0x7f34f006ed50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35040fbe40 0x7f350410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 --2- 172.21.15.19:0/2414926111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35040fcfb0 0x7f350410cd30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 >> 172.21.15.19:0/2414926111 conn(0x7f35040f74e0 msgr2=0x7f3504100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 shutdown_connections 2024-02-22T15:53:10.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.585+0000 7f350bcad700 1 -- 172.21.15.19:0/2414926111 wait complete. 2024-02-22T15:53:10.578 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:53:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 -- 172.21.15.19:0/1939353331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 msgr2=0x7f8d3c101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 --2- 172.21.15.19:0/1939353331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c101ef0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c004940 tx=0x7f8d2c009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 -- 172.21.15.19:0/1939353331 shutdown_connections 2024-02-22T15:53:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 --2- 172.21.15.19:0/1939353331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d3c102430 0x7f8d3c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 --2- 172.21.15.19:0/1939353331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c101ef0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 -- 172.21.15.19:0/1939353331 >> 172.21.15.19:0/1939353331 conn(0x7f8d3c0f7520 msgr2=0x7f8d3c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 -- 172.21.15.19:0/1939353331 shutdown_connections 2024-02-22T15:53:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.699+0000 7f8d44054700 1 -- 172.21.15.19:0/1939353331 wait complete. 2024-02-22T15:53:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d44054700 1 Processor -- start 2024-02-22T15:53:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d44054700 1 -- start start 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d44054700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c10fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d44054700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d3c102430 0x7f8d3c10ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d44054700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c110650 con 0x7f8d3c066c70 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d44054700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c110e90 con 0x7f8d3c102430 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d41df0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c10fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d41df0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c10fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41802/0 (socket says 172.21.15.19:41802) 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.700+0000 7f8d41df0700 1 -- 172.21.15.19:0/4218114653 learned_addr learned my addr 172.21.15.19:0/4218114653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d41df0700 1 -- 172.21.15.19:0/4218114653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d3c102430 msgr2=0x7f8d3c10ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d415ef700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d3c102430 0x7f8d3c10ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d41df0700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d3c102430 0x7f8d3c10ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d41df0700 1 -- 172.21.15.19:0/4218114653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c005040 con 0x7f8d3c066c70 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d41df0700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c10fa80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009360 tx=0x7f8d2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d2c00e040 con 0x7f8d3c066c70 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d2c00bcd0 con 0x7f8d3c066c70 2024-02-22T15:53:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c111110 con 0x7f8d3c066c70 2024-02-22T15:53:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d2c01c630 con 0x7f8d3c066c70 2024-02-22T15:53:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.701+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c111650 con 0x7f8d3c066c70 2024-02-22T15:53:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.702+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c04bed0 con 0x7f8d3c066c70 2024-02-22T15:53:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.702+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f8d2c013960 con 0x7f8d3c066c70 2024-02-22T15:53:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.703+0000 7f8d327fc700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8d2806c8a0 0x7f8d2806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.703+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8d2c08cfe0 con 0x7f8d3c066c70 2024-02-22T15:53:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.703+0000 7f8d415ef700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8d2806c8a0 0x7f8d2806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.703+0000 7f8d415ef700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8d2806c8a0 0x7f8d2806ed50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006fd0 tx=0x7f8d38008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:10.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.707+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f8d2c0c04d0 con 0x7f8d3c066c70 2024-02-22T15:53:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.930+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d3c103090 con 0x7f8d2806c8a0 2024-02-22T15:53:10.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.931+0000 7f8d327fc700 1 -- 172.21.15.19:0/4218114653 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f8d3c103090 con 0x7f8d2806c8a0 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8d2806c8a0 msgr2=0x7f8d2806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8d2806c8a0 0x7f8d2806ed50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006fd0 tx=0x7f8d38008040 comp rx=0 tx=0).stop 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 msgr2=0x7f8d3c10fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c10fa80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009360 tx=0x7f8d2c009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 shutdown_connections 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f8d2806c8a0 0x7f8d2806ed50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c066c70 0x7f8d3c10fa80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 --2- 172.21.15.19:0/4218114653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d3c102430 0x7f8d3c10ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 >> 172.21.15.19:0/4218114653 conn(0x7f8d3c0f7520 msgr2=0x7f8d3c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 shutdown_connections 2024-02-22T15:53:10.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:10.933+0000 7f8d44054700 1 -- 172.21.15.19:0/4218114653 wait complete. 2024-02-22T15:53:11.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 -- 172.21.15.19:0/1737506189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00fda80 msgr2=0x7fa4e00ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 --2- 172.21.15.19:0/1737506189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00fda80 0x7fa4e00ffe60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc004940 tx=0x7fa4dc009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:11.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 -- 172.21.15.19:0/1737506189 shutdown_connections 2024-02-22T15:53:11.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 --2- 172.21.15.19:0/1737506189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e01003a0 0x7fa4e0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 --2- 172.21.15.19:0/1737506189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00fda80 0x7fa4e00ffe60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 -- 172.21.15.19:0/1737506189 >> 172.21.15.19:0/1737506189 conn(0x7fa4e00f74e0 msgr2=0x7fa4e00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 -- 172.21.15.19:0/1737506189 shutdown_connections 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 -- 172.21.15.19:0/1737506189 wait complete. 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 Processor -- start 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 -- start start 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00fda80 0x7fa4e00fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.043+0000 7fa4e864d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 0x7fa4e00fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e864d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00fbf90 con 0x7fa4e01003a0 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e864d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e00fc100 con 0x7fa4e00fda80 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 0x7fa4e00fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 0x7fa4e00fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41814/0 (socket says 172.21.15.19:41814) 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 -- 172.21.15.19:0/2623317423 learned_addr learned my addr 172.21.15.19:0/2623317423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e63e9700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00fda80 0x7fa4e00fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 -- 172.21.15.19:0/2623317423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00fda80 msgr2=0x7fa4e00fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00fda80 0x7fa4e00fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 -- 172.21.15.19:0/2623317423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc005040 con 0x7fa4e01003a0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e63e9700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00fda80 0x7fa4e00fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e5be8700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 0x7fa4e00fb990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000c810 tx=0x7fa4d0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d0012810 con 0x7fa4e01003a0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4d0012e20 con 0x7fa4e01003a0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e00fc3e0 con 0x7fa4e01003a0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d000a370 con 0x7fa4e01003a0 2024-02-22T15:53:11.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.044+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e01158a0 con 0x7fa4e01003a0 2024-02-22T15:53:11.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.045+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e0107930 con 0x7fa4e01003a0 2024-02-22T15:53:11.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.046+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fa4d0011780 con 0x7fa4e01003a0 2024-02-22T15:53:11.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.046+0000 7fa4d6ffd700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa4cc06c8b0 0x7fa4cc06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.046+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa4d008be80 con 0x7fa4e01003a0 2024-02-22T15:53:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.046+0000 7fa4e63e9700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa4cc06c8b0 0x7fa4cc06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.047+0000 7fa4e63e9700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa4cc06c8b0 0x7fa4cc06ed60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc004f90 tx=0x7fa4dc009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.051+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fa4d0059e70 con 0x7fa4e01003a0 2024-02-22T15:53:11.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.267+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4e002b3b0 con 0x7fa4cc06c8b0 2024-02-22T15:53:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.277+0000 7fa4d6ffd700 1 -- 172.21.15.19:0/2623317423 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fa4e002b3b0 con 0x7fa4cc06c8b0 2024-02-22T15:53:11.259 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:53:11.259 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (20s) 13s ago 11m 16.5M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:53:11.259 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (11m) 13s ago 11m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:53:11.259 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (10m) 62s ago 10m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 starting - - - - 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (5m) 13s ago 5m 115M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (5m) 13s ago 5m 67.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (5m) 62s ago 5m 61.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (5m) 62s ago 5m 117M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (2m) 13s ago 13m 499M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (92s) 62s ago 10m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:53:11.260 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (13m) 13s ago 13m 56.8M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:53:11.263 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (10m) 62s ago 10m 46.1M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:53:11.263 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (79s) 13s ago 11m 10.7M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:53:11.263 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (67s) 62s ago 10m 3565k - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:53:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (9m) 13s ago 9m 222M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:53:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (9m) 13s ago 9m 241M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:53:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (8m) 13s ago 8m 195M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:53:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (8m) 62s ago 8m 257M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:53:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (7m) 62s ago 7m 228M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:53:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (7m) 62s ago 7m 219M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:53:11.265 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (45s) 13s ago 10m 48.5M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:53:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa4cc06c8b0 msgr2=0x7fa4cc06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa4cc06c8b0 0x7fa4cc06ed60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc004f90 tx=0x7fa4dc009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 msgr2=0x7fa4e00fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 0x7fa4e00fb990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000c810 tx=0x7fa4d0004470 comp rx=0 tx=0).stop 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 shutdown_connections 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa4cc06c8b0 0x7fa4cc06ed60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e00fda80 0x7fa4e00fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 --2- 172.21.15.19:0/2623317423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e01003a0 0x7fa4e00fb990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 >> 172.21.15.19:0/2623317423 conn(0x7fa4e00f74e0 msgr2=0x7fa4e00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 shutdown_connections 2024-02-22T15:53:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.280+0000 7fa4e864d700 1 -- 172.21.15.19:0/2623317423 wait complete. 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.383+0000 7fede5aeb700 1 -- 172.21.15.19:0/354767908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede01003a0 msgr2=0x7fede0102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.383+0000 7fede5aeb700 1 --2- 172.21.15.19:0/354767908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede01003a0 0x7fede0102780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fedd4004940 tx=0x7fedd4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.383+0000 7fede5aeb700 1 -- 172.21.15.19:0/354767908 shutdown_connections 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.383+0000 7fede5aeb700 1 --2- 172.21.15.19:0/354767908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede01003a0 0x7fede0102780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.383+0000 7fede5aeb700 1 --2- 172.21.15.19:0/354767908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede00fda80 0x7fede00ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 -- 172.21.15.19:0/354767908 >> 172.21.15.19:0/354767908 conn(0x7fede00f74e0 msgr2=0x7fede00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 -- 172.21.15.19:0/354767908 shutdown_connections 2024-02-22T15:53:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 -- 172.21.15.19:0/354767908 wait complete. 2024-02-22T15:53:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 Processor -- start 2024-02-22T15:53:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 -- start start 2024-02-22T15:53:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 0x7fede010d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede01003a0 0x7fede010ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede010e3c0 con 0x7fede00fda80 2024-02-22T15:53:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.384+0000 7fede5aeb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede010ec60 con 0x7fede01003a0 2024-02-22T15:53:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 0x7fede010d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 0x7fede010d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41826/0 (socket says 172.21.15.19:41826) 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 -- 172.21.15.19:0/4021688598 learned_addr learned my addr 172.21.15.19:0/4021688598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddeffd700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede01003a0 0x7fede010ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 -- 172.21.15.19:0/4021688598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede01003a0 msgr2=0x7fede010ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede01003a0 0x7fede010ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 -- 172.21.15.19:0/4021688598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4005040 con 0x7fede00fda80 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7feddf7fe700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 0x7fede010d880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c830 tx=0x7fedd0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedd00106f0 con 0x7fede00fda80 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede010ef40 con 0x7fede00fda80 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fedd0010d00 con 0x7fede00fda80 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedd0012e90 con 0x7fede00fda80 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.385+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede010f480 con 0x7fede00fda80 2024-02-22T15:53:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.386+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede0107910 con 0x7fede00fda80 2024-02-22T15:53:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.387+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fedd0010850 con 0x7fede00fda80 2024-02-22T15:53:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.387+0000 7fedc7fff700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedc806c8a0 0x7fedc806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.388+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fedd008c540 con 0x7fede00fda80 2024-02-22T15:53:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.388+0000 7feddeffd700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedc806c8a0 0x7fedc806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.388+0000 7feddeffd700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedc806c8a0 0x7fedc806ed50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b3b0 tx=0x7fedd4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.392+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fedd005a5d0 con 0x7fede00fda80 2024-02-22T15:53:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.704+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fede01058d0 con 0x7fede00fda80 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.705+0000 7fedc7fff700 1 -- 172.21.15.19:0/4021688598 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fedd005a160 con 0x7fede00fda80 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:53:11.686 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 12, 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:53:11.687 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:53:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedc806c8a0 msgr2=0x7fedc806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedc806c8a0 0x7fedc806ed50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b3b0 tx=0x7fedd4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 msgr2=0x7fede010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 0x7fede010d880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c830 tx=0x7fedd0004470 comp rx=0 tx=0).stop 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 shutdown_connections 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedc806c8a0 0x7fedc806ed50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fede00fda80 0x7fede010d880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 --2- 172.21.15.19:0/4021688598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede01003a0 0x7fede010ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 >> 172.21.15.19:0/4021688598 conn(0x7fede00f74e0 msgr2=0x7fede00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 shutdown_connections 2024-02-22T15:53:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.707+0000 7fede5aeb700 1 -- 172.21.15.19:0/4021688598 wait complete. 2024-02-22T15:53:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 -- 172.21.15.19:0/967234801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fcef0 msgr2=0x7f73100fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 --2- 172.21.15.19:0/967234801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fcef0 0x7f73100fd320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7304004940 tx=0x7f7304009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 -- 172.21.15.19:0/967234801 shutdown_connections 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 --2- 172.21.15.19:0/967234801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fe0e0 0x7f73100fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 --2- 172.21.15.19:0/967234801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fcef0 0x7f73100fd320 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 -- 172.21.15.19:0/967234801 >> 172.21.15.19:0/967234801 conn(0x7f73100f8620 msgr2=0x7f73100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 -- 172.21.15.19:0/967234801 shutdown_connections 2024-02-22T15:53:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 -- 172.21.15.19:0/967234801 wait complete. 2024-02-22T15:53:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 Processor -- start 2024-02-22T15:53:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.818+0000 7f731562d700 1 -- start start 2024-02-22T15:53:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f731562d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 0x7f731010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f731562d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fe0e0 0x7f731010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f731562d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731010e480 con 0x7f73100fe0e0 2024-02-22T15:53:11.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f731562d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731010ecc0 con 0x7f73100fcef0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 0x7f731010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 0x7f731010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:35878/0 (socket says 172.21.15.19:35878) 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730effd700 1 -- 172.21.15.19:0/2011425967 learned_addr learned my addr 172.21.15.19:0/2011425967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730e7fc700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fe0e0 0x7f731010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730e7fc700 1 -- 172.21.15.19:0/2011425967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 msgr2=0x7f731010d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730e7fc700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 0x7f731010d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730e7fc700 1 -- 172.21.15.19:0/2011425967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7304005040 con 0x7f73100fe0e0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730effd700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 0x7f731010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.819+0000 7f730e7fc700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fe0e0 0x7f731010ddf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f730000b580 tx=0x7f73000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.820+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f730000f6f0 con 0x7f73100fe0e0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.820+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731010efa0 con 0x7f73100fe0e0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.820+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f730000fd00 con 0x7f73100fe0e0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.820+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73000103a0 con 0x7f73100fe0e0 2024-02-22T15:53:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.820+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731010f4e0 con 0x7f73100fe0e0 2024-02-22T15:53:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.820+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731004bed0 con 0x7f73100fe0e0 2024-02-22T15:53:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.821+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f730000f850 con 0x7f73100fe0e0 2024-02-22T15:53:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.822+0000 7f72f77fe700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f72f806c8a0 0x7f72f806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.822+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f730008b820 con 0x7f73100fe0e0 2024-02-22T15:53:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.822+0000 7f730effd700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f72f806c8a0 0x7f72f806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:11.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.822+0000 7f730effd700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f72f806c8a0 0x7f72f806ed50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7304005010 tx=0x7f7304009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:11.826+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f7300059810 con 0x7f73100fe0e0 2024-02-22T15:53:11.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:12 smithi019 ceph-mon[67418]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:11.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:12 smithi019 ceph-mon[67418]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:11.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:12 smithi019 ceph-mon[67418]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:11.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:12 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:11.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:12 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/4021688598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:12.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.059+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f731010fa20 con 0x7f72f806c8a0 2024-02-22T15:53:12.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.060+0000 7f72f77fe700 1 -- 172.21.15.19:0/2011425967 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f731010fa20 con 0x7f72f806c8a0 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "2/2 daemons upgraded", 2024-02-22T15:53:12.041 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading grafana daemons", 2024-02-22T15:53:12.042 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:53:12.042 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f72f806c8a0 msgr2=0x7f72f806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f72f806c8a0 0x7f72f806ed50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7304005010 tx=0x7f7304009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fe0e0 msgr2=0x7f731010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fe0e0 0x7f731010ddf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f730000b580 tx=0x7f73000091a0 comp rx=0 tx=0).stop 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 shutdown_connections 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f72f806c8a0 0x7f72f806ed50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73100fcef0 0x7f731010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 --2- 172.21.15.19:0/2011425967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73100fe0e0 0x7f731010ddf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:12.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 >> 172.21.15.19:0/2011425967 conn(0x7f73100f8620 msgr2=0x7f73101013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:12.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 shutdown_connections 2024-02-22T15:53:12.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:12.062+0000 7f731562d700 1 -- 172.21.15.19:0/2011425967 wait complete. 2024-02-22T15:53:12.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:12 smithi062 ceph-mon[48130]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:12.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:12 smithi062 ceph-mon[48130]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:12.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:12 smithi062 ceph-mon[48130]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:12.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:12 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:12.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:12 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/4021688598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:13 smithi019 ceph-mon[67418]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:13.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:13 smithi062 ceph-mon[48130]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:14 smithi019 ceph-mon[67418]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:14 smithi019 ceph-mon[67418]: pgmap v64: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:14.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:14 smithi062 ceph-mon[48130]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:14.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:14 smithi062 ceph-mon[48130]: pgmap v64: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:15.285 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.285 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.285 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.285 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:15.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:16.434 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:16 smithi019 ceph-mon[67418]: pgmap v65: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:16.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:16 smithi062 ceph-mon[48130]: pgmap v65: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:18.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:18 smithi019 ceph-mon[67418]: pgmap v66: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:18.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:18 smithi062 ceph-mon[48130]: pgmap v66: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:19.646 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.646 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:19.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:19.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:53:19.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:53:19.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: Upgrade: Finalizing container_image settings 2024-02-22T15:53:19.649 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-22T15:53:19.650 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.651 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: Upgrade: Complete! 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: pgmap v67: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:19.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:19.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:19.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:19 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:19.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.958 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: Upgrade: Finalizing container_image settings 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-22T15:53:19.959 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.960 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: Upgrade: Complete! 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: pgmap v67: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:19.961 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:19.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:19.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:53:19.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:53:19.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:53:19.962 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:19 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:22.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:22 smithi019 ceph-mon[67418]: pgmap v68: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:22.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:22 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:22.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:22 smithi062 ceph-mon[48130]: pgmap v68: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:22.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:22 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:53:24.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:24 smithi019 ceph-mon[67418]: pgmap v69: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:24.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:24 smithi062 ceph-mon[48130]: pgmap v69: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:25.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:25 smithi062 ceph-mon[48130]: pgmap v70: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:25.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:25 smithi019 ceph-mon[67418]: pgmap v70: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:26.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:26 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:26.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:26 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:27.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:27 smithi062 ceph-mon[48130]: pgmap v71: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:27.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:27 smithi019 ceph-mon[67418]: pgmap v71: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:30.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:30 smithi019 ceph-mon[67418]: pgmap v72: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:30.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:30 smithi062 ceph-mon[48130]: pgmap v72: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:32.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:32 smithi019 ceph-mon[67418]: pgmap v73: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:32.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:32 smithi062 ceph-mon[48130]: pgmap v73: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:34.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:34 smithi019 ceph-mon[67418]: pgmap v74: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:34.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:34 smithi062 ceph-mon[48130]: pgmap v74: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:35.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:35 smithi062 ceph-mon[48130]: pgmap v75: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:35.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:35 smithi019 ceph-mon[67418]: pgmap v75: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:38.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:38 smithi019 ceph-mon[67418]: pgmap v76: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:38.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:38 smithi062 ceph-mon[48130]: pgmap v76: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:40 smithi019 ceph-mon[67418]: pgmap v77: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:40 smithi062 ceph-mon[48130]: pgmap v77: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:41.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:41 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:41.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:41 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 -- 172.21.15.19:0/1537111116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fbe40 msgr2=0x7f944c0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 --2- 172.21.15.19:0/1537111116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fbe40 0x7f944c0fc250 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9448004940 tx=0x7f9448009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 -- 172.21.15.19:0/1537111116 shutdown_connections 2024-02-22T15:53:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 --2- 172.21.15.19:0/1537111116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f944c0fcfb0 0x7f944c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 --2- 172.21.15.19:0/1537111116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fbe40 0x7f944c0fc250 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 -- 172.21.15.19:0/1537111116 >> 172.21.15.19:0/1537111116 conn(0x7f944c0f74e0 msgr2=0x7f944c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 -- 172.21.15.19:0/1537111116 shutdown_connections 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.181+0000 7f9453b18700 1 -- 172.21.15.19:0/1537111116 wait complete. 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f9453b18700 1 Processor -- start 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f9453b18700 1 -- start start 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f9453b18700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f944c0fbe40 0x7f944c10d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f9453b18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 0x7f944c10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f9453b18700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c10e440 con 0x7f944c0fcfb0 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f9453b18700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c10ec80 con 0x7f944c0fbe40 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f94510b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 0x7f944c10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f94518b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f944c0fbe40 0x7f944c10d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f94510b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 0x7f944c10ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33686/0 (socket says 172.21.15.19:33686) 2024-02-22T15:53:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.182+0000 7f94510b3700 1 -- 172.21.15.19:0/1157888851 learned_addr learned my addr 172.21.15.19:0/1157888851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f94510b3700 1 -- 172.21.15.19:0/1157888851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f944c0fbe40 msgr2=0x7f944c10d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f94510b3700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f944c0fbe40 0x7f944c10d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f94510b3700 1 -- 172.21.15.19:0/1157888851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448005040 con 0x7f944c0fcfb0 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f94510b3700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 0x7f944c10ddb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f944000b790 tx=0x7f9440009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f944000f6f0 con 0x7f944c0fcfb0 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f944000fd00 con 0x7f944c0fcfb0 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c10ef60 con 0x7f944c0fcfb0 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94400103a0 con 0x7f944c0fcfb0 2024-02-22T15:53:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.183+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c10f4a0 con 0x7f944c0fcfb0 2024-02-22T15:53:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.184+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c04bed0 con 0x7f944c0fcfb0 2024-02-22T15:53:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.185+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f944000f850 con 0x7f944c0fcfb0 2024-02-22T15:53:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.185+0000 7f943e7fc700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f943806c8a0 0x7f943806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.185+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f944008b820 con 0x7f944c0fcfb0 2024-02-22T15:53:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.185+0000 7f94518b4700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f943806c8a0 0x7f943806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.186+0000 7f94518b4700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f943806c8a0 0x7f943806ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f944800b3b0 tx=0x7f9448009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.190+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f94400598b0 con 0x7f944c0fcfb0 2024-02-22T15:53:42.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.413+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f944c104a00 con 0x7f943806c8a0 2024-02-22T15:53:42.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.414+0000 7f943e7fc700 1 -- 172.21.15.19:0/1157888851 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f944c104a00 con 0x7f943806c8a0 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f943806c8a0 msgr2=0x7f943806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f943806c8a0 0x7f943806ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f944800b3b0 tx=0x7f9448009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 msgr2=0x7f944c10ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 0x7f944c10ddb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f944000b790 tx=0x7f9440009250 comp rx=0 tx=0).stop 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 shutdown_connections 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f943806c8a0 0x7f943806ed50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f944c0fbe40 0x7f944c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 --2- 172.21.15.19:0/1157888851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c0fcfb0 0x7f944c10ddb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 >> 172.21.15.19:0/1157888851 conn(0x7f944c0f74e0 msgr2=0x7f944c0656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 shutdown_connections 2024-02-22T15:53:42.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:42.416+0000 7f9453b18700 1 -- 172.21.15.19:0/1157888851 wait complete. 2024-02-22T15:53:42.419 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:42 smithi019 ceph-mon[67418]: pgmap v78: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:42.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:42 smithi062 ceph-mon[48130]: pgmap v78: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:43.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-02-22T15:53:43.393 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:53:44.327 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:44 smithi019 ceph-mon[67418]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:44.327 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:44 smithi019 ceph-mon[67418]: pgmap v79: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:44 smithi062 ceph-mon[48130]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:44 smithi062 ceph-mon[48130]: pgmap v79: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:53:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 -- 172.21.15.19:0/920231079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa81027e0 msgr2=0x7f7fa80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 --2- 172.21.15.19:0/920231079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa81027e0 0x7f7fa80fbea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4004940 tx=0x7f7fa4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 -- 172.21.15.19:0/920231079 shutdown_connections 2024-02-22T15:53:45.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 --2- 172.21.15.19:0/920231079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa81027e0 0x7f7fa80fbea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 --2- 172.21.15.19:0/920231079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa8101e40 0x7f7fa8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 -- 172.21.15.19:0/920231079 >> 172.21.15.19:0/920231079 conn(0x7f7fa8071d90 msgr2=0x7f7fa8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 -- 172.21.15.19:0/920231079 shutdown_connections 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 -- 172.21.15.19:0/920231079 wait complete. 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 Processor -- start 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.243+0000 7f7faf573700 1 -- start start 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7faf573700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 0x7f7fa80fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7faf573700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa81027e0 0x7f7fa80fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7faf573700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80fe600 con 0x7f7fa8101e40 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7faf573700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80fe770 con 0x7f7fa81027e0 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 0x7f7fa80fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 0x7f7fa80fdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33708/0 (socket says 172.21.15.19:33708) 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 -- 172.21.15.19:0/351438849 learned_addr learned my addr 172.21.15.19:0/351438849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7facb0e700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa81027e0 0x7f7fa80fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:45.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 -- 172.21.15.19:0/351438849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa81027e0 msgr2=0x7f7fa80fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa81027e0 0x7f7fa80fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 -- 172.21.15.19:0/351438849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa4005040 con 0x7f7fa8101e40 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7fad30f700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 0x7f7fa80fdac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c002a50 tx=0x7f7f9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f9c007cc0 con 0x7f7fa8101e40 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f9c00f040 con 0x7f7fa8101e40 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa8119c80 con 0x7f7fa8101e40 2024-02-22T15:53:45.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.244+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f9c00d9c0 con 0x7f7fa8101e40 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.245+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa811a130 con 0x7f7fa8101e40 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.246+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa80738b0 con 0x7f7fa8101e40 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.246+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f7f9c00db20 con 0x7f7fa8101e40 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.246+0000 7f7f99ffb700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7f9406c8a0 0x7f7f9406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.246+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7f9c08b7c0 con 0x7f7fa8101e40 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.247+0000 7f7facb0e700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7f9406c8a0 0x7f7f9406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:45.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.247+0000 7f7facb0e700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7f9406c8a0 0x7f7f9406ed50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b3b0 tx=0x7f7fa4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:45.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.251+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f7f9c055ca0 con 0x7f7fa8101e40 2024-02-22T15:53:45.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:45 smithi019 ceph-mon[67418]: pgmap v80: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:45 smithi062 ceph-mon[48130]: pgmap v80: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:53:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.539+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7fa80ff2a0 con 0x7f7fa8101e40 2024-02-22T15:53:45.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.540+0000 7f7f99ffb700 1 -- 172.21.15.19:0/351438849 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f7f9c0592c0 con 0x7f7fa8101e40 2024-02-22T15:53:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7f9406c8a0 msgr2=0x7f7f9406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7f9406c8a0 0x7f7f9406ed50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b3b0 tx=0x7f7fa4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 msgr2=0x7f7fa80fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 0x7f7fa80fdac0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c002a50 tx=0x7f7f9c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 shutdown_connections 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f7f9406c8a0 0x7f7f9406ed50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa8101e40 0x7f7fa80fdac0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 --2- 172.21.15.19:0/351438849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa81027e0 0x7f7fa80fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 >> 172.21.15.19:0/351438849 conn(0x7f7fa8071d90 msgr2=0x7f7fa8065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 shutdown_connections 2024-02-22T15:53:45.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:45.542+0000 7f7faf573700 1 -- 172.21.15.19:0/351438849 wait complete. 2024-02-22T15:53:45.533 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:53:46.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:46 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/351438849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:46.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-02-22T15:53:46.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:46 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/351438849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:46.530 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:53:47.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:47 smithi019 ceph-mon[67418]: pgmap v81: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:47.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:47 smithi062 ceph-mon[48130]: pgmap v81: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:48.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 -- 172.21.15.19:0/964300211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 msgr2=0x7f67dc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/964300211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc102210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f67d0004940 tx=0x7f67d0009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 -- 172.21.15.19:0/964300211 shutdown_connections 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/964300211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 0x7f67dc0fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/964300211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc102210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 -- 172.21.15.19:0/964300211 >> 172.21.15.19:0/964300211 conn(0x7f67dc0f79d0 msgr2=0x7f67dc0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.400+0000 7f67e2ebe700 1 -- 172.21.15.19:0/964300211 shutdown_connections 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.401+0000 7f67e2ebe700 1 -- 172.21.15.19:0/964300211 wait complete. 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.401+0000 7f67e2ebe700 1 Processor -- start 2024-02-22T15:53:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.401+0000 7f67e2ebe700 1 -- start start 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e2ebe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc10b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e2ebe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 0x7f67dc10bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e2ebe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc10c160 con 0x7f67dc101e40 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e2ebe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc10c2a0 con 0x7f67dc1027e0 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc10b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc10b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53862/0 (socket says 172.21.15.19:53862) 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 -- 172.21.15.19:0/1723407894 learned_addr learned my addr 172.21.15.19:0/1723407894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67dbfff700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 0x7f67dc10bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 -- 172.21.15.19:0/1723407894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 msgr2=0x7f67dc10bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 0x7f67dc10bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:48.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 -- 172.21.15.19:0/1723407894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0005040 con 0x7f67dc101e40 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67dbfff700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 0x7f67dc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e0c5a700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc10b600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009180 tx=0x7f67d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d000e040 con 0x7f67dc101e40 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67d0013420 con 0x7f67dc101e40 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.402+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc11e050 con 0x7f67dc101e40 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.403+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d001c630 con 0x7f67dc101e40 2024-02-22T15:53:48.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.403+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc11e4b0 con 0x7f67dc101e40 2024-02-22T15:53:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.404+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc0fb5d0 con 0x7f67dc101e40 2024-02-22T15:53:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.404+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f67d00139e0 con 0x7f67dc101e40 2024-02-22T15:53:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.405+0000 7f67d97fa700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f67cc06c8a0 0x7f67cc06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.405+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f67d008cdc0 con 0x7f67dc101e40 2024-02-22T15:53:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.405+0000 7f67dbfff700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f67cc06c8a0 0x7f67cc06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.405+0000 7f67dbfff700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f67cc06c8a0 0x7f67cc06ed50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f67dc10d000 tx=0x7f67c8008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.409+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f67d0057320 con 0x7f67dc101e40 2024-02-22T15:53:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.700+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f67dc10cde0 con 0x7f67dc101e40 2024-02-22T15:53:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.701+0000 7f67d97fa700 1 -- 172.21.15.19:0/1723407894 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f67d005a940 con 0x7f67dc101e40 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f67cc06c8a0 msgr2=0x7f67cc06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f67cc06c8a0 0x7f67cc06ed50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f67dc10d000 tx=0x7f67c8008040 comp rx=0 tx=0).stop 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 msgr2=0x7f67dc10b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc10b600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009180 tx=0x7f67d0009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 shutdown_connections 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f67cc06c8a0 0x7f67cc06ed50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67dc101e40 0x7f67dc10b600 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 --2- 172.21.15.19:0/1723407894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67dc1027e0 0x7f67dc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 >> 172.21.15.19:0/1723407894 conn(0x7f67dc0f79d0 msgr2=0x7f67dc0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 shutdown_connections 2024-02-22T15:53:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:48.703+0000 7f67e2ebe700 1 -- 172.21.15.19:0/1723407894 wait complete. 2024-02-22T15:53:48.694 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)" 2024-02-22T15:53:48.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:48 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/1723407894' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:48 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/1723407894' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:49.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-02-22T15:53:49.757 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:53:49.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:49 smithi019 ceph-mon[67418]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:50.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:49 smithi062 ceph-mon[48130]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 -- 172.21.15.19:0/3420831464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c0fb570 msgr2=0x7f131c0fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 --2- 172.21.15.19:0/3420831464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c0fb570 0x7f131c0fb9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f130c004940 tx=0x7f130c009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 -- 172.21.15.19:0/3420831464 shutdown_connections 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 --2- 172.21.15.19:0/3420831464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c0fb570 0x7f131c0fb9c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 --2- 172.21.15.19:0/3420831464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f131c103140 0x7f131c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 -- 172.21.15.19:0/3420831464 >> 172.21.15.19:0/3420831464 conn(0x7f131c071560 msgr2=0x7f131c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 -- 172.21.15.19:0/3420831464 shutdown_connections 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.639+0000 7f13235fa700 1 -- 172.21.15.19:0/3420831464 wait complete. 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f13235fa700 1 Processor -- start 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f13235fa700 1 -- start start 2024-02-22T15:53:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f13235fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f131c0fb570 0x7f131c0fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f13235fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 0x7f131c0fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f13235fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0fd290 con 0x7f131c103140 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f13235fa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0fd400 con 0x7f131c0fb570 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1320b95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 0x7f131c0fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1320b95700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 0x7f131c0fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53880/0 (socket says 172.21.15.19:53880) 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1320b95700 1 -- 172.21.15.19:0/3190698200 learned_addr learned my addr 172.21.15.19:0/3190698200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1321396700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f131c0fb570 0x7f131c0fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1320b95700 1 -- 172.21.15.19:0/3190698200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f131c0fb570 msgr2=0x7f131c0fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1320b95700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f131c0fb570 0x7f131c0fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.640+0000 7f1320b95700 1 -- 172.21.15.19:0/3190698200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c005040 con 0x7f131c103140 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.641+0000 7f1320b95700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 0x7f131c0fcc90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b3b0 tx=0x7f130c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.641+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c00e040 con 0x7f131c103140 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.641+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c119d10 con 0x7f131c103140 2024-02-22T15:53:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.641+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f130c01c430 con 0x7f131c103140 2024-02-22T15:53:51.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.641+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c013640 con 0x7f131c103140 2024-02-22T15:53:51.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.641+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c11a170 con 0x7f131c103140 2024-02-22T15:53:51.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.642+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c1054d0 con 0x7f131c103140 2024-02-22T15:53:51.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.642+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f130c01c5a0 con 0x7f131c103140 2024-02-22T15:53:51.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.643+0000 7f1311ffb700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f130806c8a0 0x7f130806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:51.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.643+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f130c08c570 con 0x7f131c103140 2024-02-22T15:53:51.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.643+0000 7f1321396700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f130806c8a0 0x7f130806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:51.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.643+0000 7f1321396700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f130806c8a0 0x7f130806ed50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1318004f40 tx=0x7f131800b040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.648+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f130c056b80 con 0x7f131c103140 2024-02-22T15:53:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.932+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f131c04bed0 con 0x7f131c103140 2024-02-22T15:53:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.933+0000 7f1311ffb700 1 -- 172.21.15.19:0/3190698200 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f130c05a1a0 con 0x7f131c103140 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f130806c8a0 msgr2=0x7f130806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f130806c8a0 0x7f130806ed50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1318004f40 tx=0x7f131800b040 comp rx=0 tx=0).stop 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 msgr2=0x7f131c0fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 0x7f131c0fcc90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b3b0 tx=0x7f130c009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 shutdown_connections 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f130806c8a0 0x7f130806ed50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f131c0fb570 0x7f131c0fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 --2- 172.21.15.19:0/3190698200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f131c103140 0x7f131c0fcc90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 >> 172.21.15.19:0/3190698200 conn(0x7f131c071560 msgr2=0x7f131c0f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 shutdown_connections 2024-02-22T15:53:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:51.935+0000 7f13235fa700 1 -- 172.21.15.19:0/3190698200 wait complete. 2024-02-22T15:53:51.926 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:53:52.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:52 smithi019 ceph-mon[67418]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:52.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:52 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3190698200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:52.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:52 smithi062 ceph-mon[48130]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:52.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:52 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3190698200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:53:52.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-02-22T15:53:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:53:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:54 smithi019 ceph-mon[67418]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:54 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:53:54.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:54 smithi062 ceph-mon[48130]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:54.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:54 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.821+0000 7fbe3bd64700 1 -- 172.21.15.19:0/102402719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fe8f0 msgr2=0x7fbe34107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.821+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/102402719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fe8f0 0x7fbe34107050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbe28004970 tx=0x7fbe28009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.821+0000 7fbe3bd64700 1 -- 172.21.15.19:0/102402719 shutdown_connections 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.821+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/102402719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fe8f0 0x7fbe34107050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.821+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/102402719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.821+0000 7fbe3bd64700 1 -- 172.21.15.19:0/102402719 >> 172.21.15.19:0/102402719 conn(0x7fbe34071d90 msgr2=0x7fbe34072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 -- 172.21.15.19:0/102402719 shutdown_connections 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 -- 172.21.15.19:0/102402719 wait complete. 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 Processor -- start 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 -- start start 2024-02-22T15:53:54.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fe8f0 0x7fbe340fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340fc9e0 con 0x7fbe340fdfe0 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe3bd64700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340fcb50 con 0x7fbe340fe8f0 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe39b00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe39b00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53890/0 (socket says 172.21.15.19:53890) 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.822+0000 7fbe39b00700 1 -- 172.21.15.19:0/3566483428 learned_addr learned my addr 172.21.15.19:0/3566483428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe392ff700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fe8f0 0x7fbe340fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe39b00700 1 -- 172.21.15.19:0/3566483428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fe8f0 msgr2=0x7fbe340fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe39b00700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fe8f0 0x7fbe340fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe39b00700 1 -- 172.21.15.19:0/3566483428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28005040 con 0x7fbe340fdfe0 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe392ff700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fe8f0 0x7fbe340fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe39b00700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fbea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbe30002a50 tx=0x7fbe300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe30007cc0 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe3000f040 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe34119bb0 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe3000da70 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.823+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3411a090 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.824+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe341046c0 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.824+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fbe3000dc10 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.825+0000 7fbe267fc700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbe2006c8b0 0x7fbe2006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.825+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbe3008b140 con 0x7fbe340fdfe0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.825+0000 7fbe392ff700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbe2006c8b0 0x7fbe2006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.825+0000 7fbe392ff700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbe2006c8b0 0x7fbe2006ed60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fd0 tx=0x7fbe28009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:54.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:54.830+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fbe300556a0 con 0x7fbe340fdfe0 2024-02-22T15:53:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:55.051+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe340fd5e0 con 0x7fbe2006c8b0 2024-02-22T15:53:55.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:55 smithi062 ceph-mon[48130]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:55.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:55 smithi062 ceph-mon[48130]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:55.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:55 smithi019 ceph-mon[67418]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:53:55.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:55 smithi019 ceph-mon[67418]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:53:56.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:56 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.421+0000 7fbe267fc700 1 -- 172.21.15.19:0/3566483428 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5093 (secure 0 0 0) 0x7fbe340fd5e0 con 0x7fbe2006c8b0 2024-02-22T15:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbe2006c8b0 msgr2=0x7fbe2006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbe2006c8b0 0x7fbe2006ed60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fd0 tx=0x7fbe28009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 msgr2=0x7fbe340fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fbea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbe30002a50 tx=0x7fbe300046b0 comp rx=0 tx=0).stop 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 shutdown_connections 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fbe2006c8b0 0x7fbe2006ed60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe340fdfe0 0x7fbe340fbea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 --2- 172.21.15.19:0/3566483428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe340fe8f0 0x7fbe340fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 >> 172.21.15.19:0/3566483428 conn(0x7fbe34071d90 msgr2=0x7fbe34101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 shutdown_connections 2024-02-22T15:53:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:56.423+0000 7fbe3bd64700 1 -- 172.21.15.19:0/3566483428 wait complete. 2024-02-22T15:53:56.415 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:53:56.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:56 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:53:57.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch ps' 2024-02-22T15:53:57.410 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:53:57.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:53:57 smithi062 ceph-mon[48130]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:57.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:53:57 smithi019 ceph-mon[67418]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 -- 172.21.15.19:0/1007556604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c01038f0 msgr2=0x7f17c00fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 --2- 172.21.15.19:0/1007556604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c01038f0 0x7f17c00fcf60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f17b4004940 tx=0x7f17b4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 -- 172.21.15.19:0/1007556604 shutdown_connections 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 --2- 172.21.15.19:0/1007556604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c01038f0 0x7f17c00fcf60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 --2- 172.21.15.19:0/1007556604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c0102f50 0x7f17c0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 -- 172.21.15.19:0/1007556604 >> 172.21.15.19:0/1007556604 conn(0x7f17c00f8ae0 msgr2=0x7f17c00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 -- 172.21.15.19:0/1007556604 shutdown_connections 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.294+0000 7f17c843b700 1 -- 172.21.15.19:0/1007556604 wait complete. 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c843b700 1 Processor -- start 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c843b700 1 -- start start 2024-02-22T15:53:59.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c843b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 0x7f17c01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c843b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c01038f0 0x7f17c0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c843b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0114530 con 0x7f17c0102f50 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c843b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c010d860 con 0x7f17c01038f0 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c61d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 0x7f17c01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c61d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 0x7f17c01139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57904/0 (socket says 172.21.15.19:57904) 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.295+0000 7f17c61d7700 1 -- 172.21.15.19:0/3520902924 learned_addr learned my addr 172.21.15.19:0/3520902924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c59d6700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c01038f0 0x7f17c0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c61d7700 1 -- 172.21.15.19:0/3520902924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c01038f0 msgr2=0x7f17c0113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c61d7700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c01038f0 0x7f17c0113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c61d7700 1 -- 172.21.15.19:0/3520902924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4005040 con 0x7f17c0102f50 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c61d7700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 0x7f17c01139a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f17bc002a50 tx=0x7f17bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17bc007cc0 con 0x7f17c0102f50 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c010db40 con 0x7f17c0102f50 2024-02-22T15:53:59.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17bc00f040 con 0x7f17c0102f50 2024-02-22T15:53:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17bc009540 con 0x7f17c0102f50 2024-02-22T15:53:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.296+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c010e080 con 0x7f17c0102f50 2024-02-22T15:53:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.297+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0105420 con 0x7f17c0102f50 2024-02-22T15:53:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.298+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f17bc0096a0 con 0x7f17c0102f50 2024-02-22T15:53:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.298+0000 7f17b2ffd700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f17ac06c8a0 0x7f17ac06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:53:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.298+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f17bc08b8c0 con 0x7f17c0102f50 2024-02-22T15:53:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.298+0000 7f17c59d6700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f17ac06c8a0 0x7f17ac06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:53:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.299+0000 7f17c59d6700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f17ac06c8a0 0x7f17ac06ed50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17c010f1d0 tx=0x7f17b4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:53:59.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.303+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f17bc059830 con 0x7f17c0102f50 2024-02-22T15:53:59.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.514+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c010ef80 con 0x7f17ac06c8a0 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.524+0000 7f17b2ffd700 1 -- 172.21.15.19:0/3520902924 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f17c010ef80 con 0x7f17ac06c8a0 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (68s) 45s ago 12m 16.5M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (12m) 45s ago 12m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (11m) 110s ago 11m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (51s) 45s ago 11m 39.4M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (6m) 45s ago 6m 115M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (6m) 45s ago 6m 67.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (6m) 110s ago 6m 61.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:53:59.505 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (6m) 110s ago 6m 117M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (3m) 45s ago 13m 499M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (2m) 110s ago 11m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (13m) 45s ago 13m 57.2M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (11m) 110s ago 11m 46.1M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (2m) 45s ago 12m 11.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (116s) 110s ago 11m 3565k - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (10m) 45s ago 10m 222M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (9m) 45s ago 9m 241M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (9m) 45s ago 9m 195M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (9m) 110s ago 8m 257M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (8m) 110s ago 8m 228M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (8m) 110s ago 8m 219M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:53:59.506 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (94s) 45s ago 11m 48.6M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f17ac06c8a0 msgr2=0x7f17ac06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f17ac06c8a0 0x7f17ac06ed50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17c010f1d0 tx=0x7f17b4009e30 comp rx=0 tx=0).stop 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 msgr2=0x7f17c01139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 0x7f17c01139a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f17bc002a50 tx=0x7f17bc0046b0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 shutdown_connections 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f17ac06c8a0 0x7f17ac06ed50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c0102f50 0x7f17c01139a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 --2- 172.21.15.19:0/3520902924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c01038f0 0x7f17c0113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:53:59.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 >> 172.21.15.19:0/3520902924 conn(0x7f17c00f8ae0 msgr2=0x7f17c00fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:53:59.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 shutdown_connections 2024-02-22T15:53:59.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:53:59.526+0000 7f17c843b700 1 -- 172.21.15.19:0/3520902924 wait complete. 2024-02-22T15:54:00.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:00 smithi019 ceph-mon[67418]: pgmap v87: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:00.161 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-22T15:54:00.161 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:54:00.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-02-22T15:54:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:54:00.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:00 smithi062 ceph-mon[48130]: pgmap v87: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:01.401 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:01 smithi019 ceph-mon[67418]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:01.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:01 smithi062 ceph-mon[48130]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:02.046 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:02 smithi019 ceph-mon[67418]: pgmap v88: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/2853551007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fdfe0 msgr2=0x7ffb980fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/2853551007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fdfe0 0x7ffb980fe3b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c004970 tx=0x7ffb8c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/2853551007 shutdown_connections 2024-02-22T15:54:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/2853551007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb980fe8f0 0x7ffb98107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/2853551007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fdfe0 0x7ffb980fe3b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/2853551007 >> 172.21.15.19:0/2853551007 conn(0x7ffb98071d90 msgr2=0x7ffb98072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/2853551007 shutdown_connections 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/2853551007 wait complete. 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.319+0000 7ffb9d7f3700 1 Processor -- start 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb9d7f3700 1 -- start start 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb9d7f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb980fdfe0 0x7ffb98113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb9d7f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 0x7ffb98113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb9d7f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb981144c0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb9d7f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9810d7f0 con 0x7ffb980fdfe0 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 0x7ffb98113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 0x7ffb98113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57924/0 (socket says 172.21.15.19:57924) 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 -- 172.21.15.19:0/3192086657 learned_addr learned my addr 172.21.15.19:0/3192086657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb96ffd700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb980fdfe0 0x7ffb98113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 -- 172.21.15.19:0/3192086657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb980fdfe0 msgr2=0x7ffb98113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb980fdfe0 0x7ffb98113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 -- 172.21.15.19:0/3192086657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c005040 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb967fc700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 0x7ffb98113e70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c8c0 tx=0x7ffb880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb8800a980 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb8800ce60 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9810dad0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.320+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb880113a0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.321+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9810e010 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.321+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb981046c0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.322+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7ffb88012400 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.322+0000 7ffb7f7fe700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb8006c8a0 0x7ffb8006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.322+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ffb8808bbb0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.322+0000 7ffb96ffd700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb8006c8a0 0x7ffb8006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.323+0000 7ffb96ffd700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb8006c8a0 0x7ffb8006ed50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c004f50 tx=0x7ffb8c009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.327+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7ffb88092080 con 0x7ffb980fe8f0 2024-02-22T15:54:02.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:02 smithi062 ceph-mon[48130]: pgmap v88: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:02.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.527+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7ffb980738b0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.532+0000 7ffb7f7fe700 1 -- 172.21.15.19:0/3192086657 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v48)=0 v48) v1 ==== 125+0+0 (secure 0 0 0) 0x7ffb88059ba0 con 0x7ffb980fe8f0 2024-02-22T15:54:02.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb8006c8a0 msgr2=0x7ffb8006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb8006c8a0 0x7ffb8006ed50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c004f50 tx=0x7ffb8c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 msgr2=0x7ffb98113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 0x7ffb98113e70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c8c0 tx=0x7ffb880044c0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 shutdown_connections 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb8006c8a0 0x7ffb8006ed50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb980fdfe0 0x7ffb98113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 --2- 172.21.15.19:0/3192086657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb980fe8f0 0x7ffb98113e70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 >> 172.21.15.19:0/3192086657 conn(0x7ffb98071d90 msgr2=0x7ffb98101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 shutdown_connections 2024-02-22T15:54:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:02.534+0000 7ffb9d7f3700 1 -- 172.21.15.19:0/3192086657 wait complete. 2024-02-22T15:54:03.261 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-22T15:54:03.261 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:54:03.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-22T15:54:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:54:03.767 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:03 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/3192086657' entity='client.admin' 2024-02-22T15:54:03.767 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:03 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:03.768 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:03 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:03.768 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:03 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:03.768 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:03 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:03.768 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:03 smithi019 ceph-mon[67418]: pgmap v89: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:03.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:03 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/3192086657' entity='client.admin' 2024-02-22T15:54:03.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:03 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:03.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:03 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:03.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:03 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:03.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:03 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:03.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:03 smithi062 ceph-mon[48130]: pgmap v89: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.309+0000 7f9acec4d700 1 -- 172.21.15.19:0/1923700174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac81027e0 msgr2=0x7f9ac80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.309+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1923700174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac81027e0 0x7f9ac80fbea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4004940 tx=0x7f9ab4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 -- 172.21.15.19:0/1923700174 shutdown_connections 2024-02-22T15:54:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1923700174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac81027e0 0x7f9ac80fbea0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1923700174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 -- 172.21.15.19:0/1923700174 >> 172.21.15.19:0/1923700174 conn(0x7f9ac8071d90 msgr2=0x7f9ac8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 -- 172.21.15.19:0/1923700174 shutdown_connections 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 -- 172.21.15.19:0/1923700174 wait complete. 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 Processor -- start 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 -- start start 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.310+0000 7f9acec4d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac81027e0 0x7f9ac81031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acec4d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8103870 con 0x7f9ac8101e40 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acec4d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac81039e0 con 0x7f9ac81027e0 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57938/0 (socket says 172.21.15.19:57938) 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 -- 172.21.15.19:0/1009860495 learned_addr learned my addr 172.21.15.19:0/1009860495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9ac7fff700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac81027e0 0x7f9ac81031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 -- 172.21.15.19:0/1009860495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac81027e0 msgr2=0x7f9ac81031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac81027e0 0x7f9ac81031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 -- 172.21.15.19:0/1009860495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4005040 con 0x7f9ac8101e40 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acc9e9700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102ca0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b790 tx=0x7f9abc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9abc00f770 con 0x7f9ac8101e40 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac8119c40 con 0x7f9ac8101e40 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9abc00fd80 con 0x7f9ac8101e40 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9abc00d310 con 0x7f9ac8101e40 2024-02-22T15:54:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.311+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac811a130 con 0x7f9ac8101e40 2024-02-22T15:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.312+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac80738b0 con 0x7f9ac8101e40 2024-02-22T15:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.313+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f9abc010420 con 0x7f9ac8101e40 2024-02-22T15:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.314+0000 7f9ac57fa700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f9ab806c8a0 0x7f9ab806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.314+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9abc08c130 con 0x7f9ac8101e40 2024-02-22T15:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.314+0000 7f9ac7fff700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f9ab806c8a0 0x7f9ab806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.314+0000 7f9ac7fff700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f9ab806c8a0 0x7f9ab806ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ac81046a0 tx=0x7f9ab4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.318+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f9abc056730 con 0x7f9ac8101e40 2024-02-22T15:54:05.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:05 smithi062 ceph-mon[48130]: pgmap v90: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:05.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:05 smithi019 ceph-mon[67418]: pgmap v90: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:05.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.520+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f9ac8108e50 con 0x7f9ac8101e40 2024-02-22T15:54:05.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.520+0000 7f9ac57fa700 1 -- 172.21.15.19:0/1009860495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v48)=0 v48) v1 ==== 155+0+0 (secure 0 0 0) 0x7f9abc059d50 con 0x7f9ac8101e40 2024-02-22T15:54:05.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f9ab806c8a0 msgr2=0x7f9ab806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:05.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f9ab806c8a0 0x7f9ab806ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ac81046a0 tx=0x7f9ab4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 msgr2=0x7f9ac8102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102ca0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00b790 tx=0x7f9abc009250 comp rx=0 tx=0).stop 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 shutdown_connections 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f9ab806c8a0 0x7f9ab806ed50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ac8101e40 0x7f9ac8102ca0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 --2- 172.21.15.19:0/1009860495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ac81027e0 0x7f9ac81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 >> 172.21.15.19:0/1009860495 conn(0x7f9ac8071d90 msgr2=0x7f9ac8065800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 shutdown_connections 2024-02-22T15:54:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:05.522+0000 7f9acec4d700 1 -- 172.21.15.19:0/1009860495 wait complete. 2024-02-22T15:54:06.193 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:06 smithi019 ceph-mon[67418]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:54:06.215 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-22T15:54:06.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:06 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:54:06.497 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:07 smithi019 ceph-mon[67418]: pgmap v91: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:07.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:07 smithi062 ceph-mon[48130]: pgmap v91: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 -- 172.21.15.19:0/810566711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 msgr2=0x7ffb340fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/810566711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb340fe3b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffb28004970 tx=0x7ffb28009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 -- 172.21.15.19:0/810566711 shutdown_connections 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/810566711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb340fe8f0 0x7ffb34107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/810566711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb340fe3b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 -- 172.21.15.19:0/810566711 >> 172.21.15.19:0/810566711 conn(0x7ffb34071d90 msgr2=0x7ffb34072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.526+0000 7ffb3b23c700 1 -- 172.21.15.19:0/810566711 shutdown_connections 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.527+0000 7ffb3b23c700 1 -- 172.21.15.19:0/810566711 wait complete. 2024-02-22T15:54:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.527+0000 7ffb3b23c700 1 Processor -- start 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.527+0000 7ffb3b23c700 1 -- start start 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb3b23c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb34113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb3b23c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb340fe8f0 0x7ffb34113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb3b23c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb341144c0 con 0x7ffb340fdfe0 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb3b23c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3410d7f0 con 0x7ffb340fe8f0 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb34113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb34113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58248/0 (socket says 172.21.15.19:58248) 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 -- 172.21.15.19:0/675410965 learned_addr learned my addr 172.21.15.19:0/675410965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb33fff700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb340fe8f0 0x7ffb34113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 -- 172.21.15.19:0/675410965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb340fe8f0 msgr2=0x7ffb34113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb340fe8f0 0x7ffb34113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 -- 172.21.15.19:0/675410965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28005040 con 0x7ffb340fdfe0 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb38fd8700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb34113930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009380 tx=0x7ffb28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb2800e040 con 0x7ffb340fdfe0 2024-02-22T15:54:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb28004290 con 0x7ffb340fdfe0 2024-02-22T15:54:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3410da70 con 0x7ffb340fdfe0 2024-02-22T15:54:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb2801c630 con 0x7ffb340fdfe0 2024-02-22T15:54:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.528+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3410dfb0 con 0x7ffb340fdfe0 2024-02-22T15:54:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.530+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb341046c0 con 0x7ffb340fdfe0 2024-02-22T15:54:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.530+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7ffb2801c860 con 0x7ffb340fdfe0 2024-02-22T15:54:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.530+0000 7ffb317fa700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb2406c8a0 0x7ffb2406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.530+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ffb2808ce50 con 0x7ffb340fdfe0 2024-02-22T15:54:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.530+0000 7ffb33fff700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb2406c8a0 0x7ffb2406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.531+0000 7ffb33fff700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb2406c8a0 0x7ffb2406ed50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb3410f160 tx=0x7ffb20009ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.535+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7ffb2805aeb0 con 0x7ffb340fdfe0 2024-02-22T15:54:08.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.735+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7ffb3404bed0 con 0x7ffb340fdfe0 2024-02-22T15:54:08.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.736+0000 7ffb317fa700 1 -- 172.21.15.19:0/675410965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v48)=0 v48) v1 ==== 163+0+0 (secure 0 0 0) 0x7ffb2805aa40 con 0x7ffb340fdfe0 2024-02-22T15:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb2406c8a0 msgr2=0x7ffb2406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb2406c8a0 0x7ffb2406ed50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb3410f160 tx=0x7ffb20009ee0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 msgr2=0x7ffb34113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb34113930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009380 tx=0x7ffb28009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 shutdown_connections 2024-02-22T15:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7ffb2406c8a0 0x7ffb2406ed50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb340fdfe0 0x7ffb34113930 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 --2- 172.21.15.19:0/675410965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb340fe8f0 0x7ffb34113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 >> 172.21.15.19:0/675410965 conn(0x7ffb34071d90 msgr2=0x7ffb34101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 shutdown_connections 2024-02-22T15:54:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:08.738+0000 7ffb3b23c700 1 -- 172.21.15.19:0/675410965 wait complete. 2024-02-22T15:54:09.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-22T15:54:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:54:10.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:10 smithi019 ceph-mon[67418]: pgmap v92: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:10.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:10 smithi062 ceph-mon[48130]: pgmap v92: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:11.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:11 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:54:11.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:11 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:54:11.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.459+0000 7f0463ea7700 1 -- 172.21.15.19:0/4044551048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 msgr2=0x7f045c0fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.459+0000 7f0463ea7700 1 --2- 172.21.15.19:0/4044551048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c0fcf50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0450004940 tx=0x7f0450009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.459+0000 7f0463ea7700 1 -- 172.21.15.19:0/4044551048 shutdown_connections 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.459+0000 7f0463ea7700 1 --2- 172.21.15.19:0/4044551048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c0fcf50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.459+0000 7f0463ea7700 1 --2- 172.21.15.19:0/4044551048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c102f40 0x7f045c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.459+0000 7f0463ea7700 1 -- 172.21.15.19:0/4044551048 >> 172.21.15.19:0/4044551048 conn(0x7f045c0f8af0 msgr2=0x7f045c0faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.460+0000 7f0463ea7700 1 -- 172.21.15.19:0/4044551048 shutdown_connections 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.460+0000 7f0463ea7700 1 -- 172.21.15.19:0/4044551048 wait complete. 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.460+0000 7f0463ea7700 1 Processor -- start 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.460+0000 7f0463ea7700 1 -- start start 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0463ea7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c102f40 0x7f045c113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0463ea7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0463ea7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c1145d0 con 0x7f045c1038e0 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0463ea7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c10d900 con 0x7f045c102f40 2024-02-22T15:54:11.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58266/0 (socket says 172.21.15.19:58266) 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 -- 172.21.15.19:0/3154786723 learned_addr learned my addr 172.21.15.19:0/3154786723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461c43700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c102f40 0x7f045c113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 -- 172.21.15.19:0/3154786723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c102f40 msgr2=0x7f045c113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c102f40 0x7f045c113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 -- 172.21.15.19:0/3154786723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450005040 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0461442700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c113f80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0450009fd0 tx=0x7f0450009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045000e040 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0450013420 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c10db80 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045001c630 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.461+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c10e0c0 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.462+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c105410 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.463+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f0450013a00 con 0x7f045c1038e0 2024-02-22T15:54:11.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.463+0000 7f044e7fc700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f044806c8a0 0x7f044806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.463+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f045008ccd0 con 0x7f045c1038e0 2024-02-22T15:54:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.463+0000 7f0461c43700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f044806c8a0 0x7f044806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.464+0000 7f0461c43700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f044806c8a0 0x7f044806ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0458004f40 tx=0x7f045800b040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:11.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.468+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f045005acc0 con 0x7f045c1038e0 2024-02-22T15:54:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.670+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f045c0fc6f0 con 0x7f045c1038e0 2024-02-22T15:54:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.670+0000 7f044e7fc700 1 -- 172.21.15.19:0/3154786723 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v48)=0 v48) v1 ==== 135+0+0 (secure 0 0 0) 0x7f045005a850 con 0x7f045c1038e0 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f044806c8a0 msgr2=0x7f044806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f044806c8a0 0x7f044806ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0458004f40 tx=0x7f045800b040 comp rx=0 tx=0).stop 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 msgr2=0x7f045c113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c113f80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0450009fd0 tx=0x7f0450009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 shutdown_connections 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f044806c8a0 0x7f044806ed50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045c102f40 0x7f045c113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 --2- 172.21.15.19:0/3154786723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f045c1038e0 0x7f045c113f80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 >> 172.21.15.19:0/3154786723 conn(0x7f045c0f8af0 msgr2=0x7f045c0fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 shutdown_connections 2024-02-22T15:54:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:11.672+0000 7f0463ea7700 1 -- 172.21.15.19:0/3154786723 wait complete. 2024-02-22T15:54:12.363 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:12 smithi019 ceph-mon[67418]: pgmap v93: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:12.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-22T15:54:12.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:12 smithi062 ceph-mon[48130]: pgmap v93: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:12.672 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:54:14.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 -- 172.21.15.19:0/3150406920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8441027e0 msgr2=0x7fa8440fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:14.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 --2- 172.21.15.19:0/3150406920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8441027e0 0x7fa8440fbea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa834004940 tx=0x7fa834009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:14.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 -- 172.21.15.19:0/3150406920 shutdown_connections 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 --2- 172.21.15.19:0/3150406920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8441027e0 0x7fa8440fbea0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 --2- 172.21.15.19:0/3150406920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa844101e40 0x7fa844102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 -- 172.21.15.19:0/3150406920 >> 172.21.15.19:0/3150406920 conn(0x7fa844071d90 msgr2=0x7fa844072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 -- 172.21.15.19:0/3150406920 shutdown_connections 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.406+0000 7fa84b587700 1 -- 172.21.15.19:0/3150406920 wait complete. 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa84b587700 1 Processor -- start 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa84b587700 1 -- start start 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa84b587700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 0x7fa8441139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa84b587700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8441027e0 0x7fa844113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa84b587700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844114540 con 0x7fa844101e40 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa84b587700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84410d870 con 0x7fa8441027e0 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa849323700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 0x7fa8441139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa849323700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 0x7fa8441139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58288/0 (socket says 172.21.15.19:58288) 2024-02-22T15:54:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa849323700 1 -- 172.21.15.19:0/592724929 learned_addr learned my addr 172.21.15.19:0/592724929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.407+0000 7fa848b22700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8441027e0 0x7fa844113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa849323700 1 -- 172.21.15.19:0/592724929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8441027e0 msgr2=0x7fa844113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa849323700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8441027e0 0x7fa844113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa849323700 1 -- 172.21.15.19:0/592724929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834005040 con 0x7fa844101e40 2024-02-22T15:54:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa849323700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 0x7fa8441139b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c830 tx=0x7fa840004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8400106f0 con 0x7fa844101e40 2024-02-22T15:54:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84410db50 con 0x7fa844101e40 2024-02-22T15:54:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa840010d00 con 0x7fa844101e40 2024-02-22T15:54:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa840012e90 con 0x7fa844101e40 2024-02-22T15:54:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.408+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84410e060 con 0x7fa844101e40 2024-02-22T15:54:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.409+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8440738b0 con 0x7fa844101e40 2024-02-22T15:54:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.409+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fa840010850 con 0x7fa844101e40 2024-02-22T15:54:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.410+0000 7fa839ffb700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa83006c8a0 0x7fa83006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.410+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa84008bab0 con 0x7fa844101e40 2024-02-22T15:54:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.410+0000 7fa848b22700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa83006c8a0 0x7fa83006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.410+0000 7fa848b22700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa83006c8a0 0x7fa83006ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa84410eba0 tx=0x7fa834009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:14.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.415+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fa840059a40 con 0x7fa844101e40 2024-02-22T15:54:14.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:14 smithi062 ceph-mon[48130]: pgmap v94: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:14.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:14 smithi019 ceph-mon[67418]: pgmap v94: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:14.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.642+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84410f000 con 0x7fa83006c8a0 2024-02-22T15:54:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.657+0000 7fa839ffb700 1 -- 172.21.15.19:0/592724929 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fa84410f000 con 0x7fa83006c8a0 2024-02-22T15:54:14.640 INFO:teuthology.orchestra.run.smithi019.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:54:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.659+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa83006c8a0 msgr2=0x7fa83006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.659+0000 7fa84b587700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa83006c8a0 0x7fa83006ed50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa84410eba0 tx=0x7fa834009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 msgr2=0x7fa8441139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 0x7fa8441139b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c830 tx=0x7fa840004470 comp rx=0 tx=0).stop 2024-02-22T15:54:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 shutdown_connections 2024-02-22T15:54:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fa83006c8a0 0x7fa83006ed50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa844101e40 0x7fa8441139b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 --2- 172.21.15.19:0/592724929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8441027e0 0x7fa844113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:14.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 >> 172.21.15.19:0/592724929 conn(0x7fa844071d90 msgr2=0x7fa8440657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:14.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 shutdown_connections 2024-02-22T15:54:14.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:14.660+0000 7fa84b587700 1 -- 172.21.15.19:0/592724929 wait complete. 2024-02-22T15:54:15.395 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-22T15:54:15.396 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T15:54:15.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-22T15:54:15.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:15.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:54:15.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:15.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:15.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: pgmap v95: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:15.635 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:15.635 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:15.635 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:15 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:15.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:15.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:54:15.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:15.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:15.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: pgmap v95: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:15.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:15.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:15.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:15 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:16.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:16 smithi019 ceph-mon[67418]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:54:16.757 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T15:54:16.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:16 smithi062 ceph-mon[48130]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c1dc18f444603397768fcb7c34a8488b10284df6 2024-02-22T15:54:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: Upgrade: Target is version 18.2.1-638-gc1dc18f4 (reef) 2024-02-22T15:54:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0'] 2024-02-22T15:54:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:17.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:17.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:17.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: Upgrade: Setting container_image for all mgr 2024-02-22T15:54:17.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:17.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-22T15:54:17.995 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:17 smithi019 ceph-mon[67418]: pgmap v96: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:18.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:18.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: Upgrade: Target is version 18.2.1-638-gc1dc18f4 (reef) 2024-02-22T15:54:18.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0, digests ['quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0'] 2024-02-22T15:54:18.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:18.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:18.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:18.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: Upgrade: Setting container_image for all mgr 2024-02-22T15:54:18.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:18.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-22T15:54:18.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:17 smithi062 ceph-mon[48130]: pgmap v96: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.619+0000 7fb7b60eb700 1 -- 172.21.15.19:0/49597379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 msgr2=0x7fb7b0103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.619+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/49597379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b0103510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004940 tx=0x7fb7a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.619+0000 7fb7b60eb700 1 -- 172.21.15.19:0/49597379 shutdown_connections 2024-02-22T15:54:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.619+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/49597379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b00fb570 0x7fb7b00fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.619+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/49597379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b0103510 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.620+0000 7fb7b60eb700 1 -- 172.21.15.19:0/49597379 >> 172.21.15.19:0/49597379 conn(0x7fb7b0071560 msgr2=0x7fb7b0071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.620+0000 7fb7b60eb700 1 -- 172.21.15.19:0/49597379 shutdown_connections 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.620+0000 7fb7b60eb700 1 -- 172.21.15.19:0/49597379 wait complete. 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.620+0000 7fb7b60eb700 1 Processor -- start 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.620+0000 7fb7b60eb700 1 -- start start 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7b60eb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b00fb570 0x7fb7b00fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7b60eb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b00fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7b60eb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00fd290 con 0x7fb7b0103140 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7b60eb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00fd400 con 0x7fb7b00fb570 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b00fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b00fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40274/0 (socket says 172.21.15.19:40274) 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 -- 172.21.15.19:0/850641711 learned_addr learned my addr 172.21.15.19:0/850641711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7af7fe700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b00fb570 0x7fb7b00fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 -- 172.21.15.19:0/850641711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b00fb570 msgr2=0x7fb7b00fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b00fb570 0x7fb7b00fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 -- 172.21.15.19:0/850641711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a0005040 con 0x7fb7b0103140 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7aeffd700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b00fcc90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4002a50 tx=0x7fb7a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a4007cc0 con 0x7fb7b0103140 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7a400f040 con 0x7fb7b0103140 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b0119c80 con 0x7fb7b0103140 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.621+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a400d9c0 con 0x7fb7b0103140 2024-02-22T15:54:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.622+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b011a160 con 0x7fb7b0103140 2024-02-22T15:54:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.623+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b0100310 con 0x7fb7b0103140 2024-02-22T15:54:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.623+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fb7a4009070 con 0x7fb7b0103140 2024-02-22T15:54:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.623+0000 7fb797fff700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb79806c8a0 0x7fb79806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.624+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb7a408b800 con 0x7fb7b0103140 2024-02-22T15:54:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.624+0000 7fb7af7fe700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb79806c8a0 0x7fb79806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.624+0000 7fb7af7fe700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb79806c8a0 0x7fb79806ed50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004f90 tx=0x7fb7a0009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.628+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fb7a4055ce0 con 0x7fb7b0103140 2024-02-22T15:54:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.857+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7b00fde90 con 0x7fb79806c8a0 2024-02-22T15:54:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.858+0000 7fb797fff700 1 -- 172.21.15.19:0/850641711 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb7b00fde90 con 0x7fb79806c8a0 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.860+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb79806c8a0 msgr2=0x7fb79806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.860+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb79806c8a0 0x7fb79806ed50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004f90 tx=0x7fb7a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.860+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 msgr2=0x7fb7b00fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.860+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b00fcc90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4002a50 tx=0x7fb7a40046b0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 shutdown_connections 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb79806c8a0 0x7fb79806ed50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7b00fb570 0x7fb7b00fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 --2- 172.21.15.19:0/850641711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7b0103140 0x7fb7b00fcc90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 >> 172.21.15.19:0/850641711 conn(0x7fb7b0071560 msgr2=0x7fb7b00fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 shutdown_connections 2024-02-22T15:54:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.861+0000 7fb7b60eb700 1 -- 172.21.15.19:0/850641711 wait complete. 2024-02-22T15:54:19.853 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 -- 172.21.15.19:0/3463003558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fdfe0 msgr2=0x7fee100fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 --2- 172.21.15.19:0/3463003558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fdfe0 0x7fee100fe3b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fee00004970 tx=0x7fee00009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 -- 172.21.15.19:0/3463003558 shutdown_connections 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 --2- 172.21.15.19:0/3463003558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee100fe8f0 0x7fee10107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 --2- 172.21.15.19:0/3463003558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fdfe0 0x7fee100fe3b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 -- 172.21.15.19:0/3463003558 >> 172.21.15.19:0/3463003558 conn(0x7fee10071d90 msgr2=0x7fee10072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 -- 172.21.15.19:0/3463003558 shutdown_connections 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 -- 172.21.15.19:0/3463003558 wait complete. 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 Processor -- start 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.970+0000 7fee181f3700 1 -- start start 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee181f3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee100fdfe0 0x7fee10113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee181f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 0x7fee10113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee181f3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee101144c0 con 0x7fee100fe8f0 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee181f3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1010d7f0 con 0x7fee100fdfe0 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 0x7fee10113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 0x7fee10113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40290/0 (socket says 172.21.15.19:40290) 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 -- 172.21.15.19:0/247929743 learned_addr learned my addr 172.21.15.19:0/247929743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee15f8f700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee100fdfe0 0x7fee10113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 -- 172.21.15.19:0/247929743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee100fdfe0 msgr2=0x7fee10113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee100fdfe0 0x7fee10113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 -- 172.21.15.19:0/247929743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00005040 con 0x7fee100fe8f0 2024-02-22T15:54:19.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee1578e700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 0x7fee10113e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fee0c002a50 tx=0x7fee0c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:19.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee0c007cc0 con 0x7fee100fe8f0 2024-02-22T15:54:19.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee0c00f040 con 0x7fee100fe8f0 2024-02-22T15:54:19.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1010dad0 con 0x7fee100fe8f0 2024-02-22T15:54:19.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.971+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee0c00d9c0 con 0x7fee100fe8f0 2024-02-22T15:54:19.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.972+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1010e010 con 0x7fee100fe8f0 2024-02-22T15:54:19.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.972+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee101046c0 con 0x7fee100fe8f0 2024-02-22T15:54:19.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.973+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fee0c00db20 con 0x7fee100fe8f0 2024-02-22T15:54:19.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.973+0000 7fee067fc700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedfc06c8a0 0x7fedfc06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:19.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.973+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fee0c08b700 con 0x7fee100fe8f0 2024-02-22T15:54:19.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.974+0000 7fee15f8f700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedfc06c8a0 0x7fedfc06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:19.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.974+0000 7fee15f8f700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedfc06c8a0 0x7fedfc06ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee00004f50 tx=0x7fee00009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:19.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:19.978+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fee0c0596f0 con 0x7fee100fe8f0 2024-02-22T15:54:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.201+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee10109ef0 con 0x7fedfc06c8a0 2024-02-22T15:54:20.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.203+0000 7fee067fc700 1 -- 172.21.15.19:0/247929743 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fee10109ef0 con 0x7fedfc06c8a0 2024-02-22T15:54:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedfc06c8a0 msgr2=0x7fedfc06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedfc06c8a0 0x7fedfc06ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee00004f50 tx=0x7fee00009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 msgr2=0x7fee10113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 0x7fee10113e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fee0c002a50 tx=0x7fee0c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 shutdown_connections 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fedfc06c8a0 0x7fedfc06ed50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee100fdfe0 0x7fee10113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 --2- 172.21.15.19:0/247929743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee100fe8f0 0x7fee10113e70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 >> 172.21.15.19:0/247929743 conn(0x7fee10071d90 msgr2=0x7fee10101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 shutdown_connections 2024-02-22T15:54:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.205+0000 7fee181f3700 1 -- 172.21.15.19:0/247929743 wait complete. 2024-02-22T15:54:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:20 smithi019 ceph-mon[67418]: pgmap v97: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:20 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:20 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:54:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:20 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:54:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:20 smithi019 ceph-mon[67418]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 -- 172.21.15.19:0/777727179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fe8f0 msgr2=0x7fd804107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/777727179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fe8f0 0x7fd804107050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004970 tx=0x7fd7f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 -- 172.21.15.19:0/777727179 shutdown_connections 2024-02-22T15:54:20.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/777727179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fe8f0 0x7fd804107050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/777727179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8040fdfe0 0x7fd8040fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 -- 172.21.15.19:0/777727179 >> 172.21.15.19:0/777727179 conn(0x7fd804071d90 msgr2=0x7fd804072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 -- 172.21.15.19:0/777727179 shutdown_connections 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 -- 172.21.15.19:0/777727179 wait complete. 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.312+0000 7fd80b8e0700 1 Processor -- start 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80b8e0700 1 -- start start 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80b8e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 0x7fd804113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80b8e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8040fe8f0 0x7fd804113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80b8e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd804114500 con 0x7fd8040fdfe0 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80b8e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80410d830 con 0x7fd8040fe8f0 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 0x7fd804113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 0x7fd804113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40310/0 (socket says 172.21.15.19:40310) 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 -- 172.21.15.19:0/1815182233 learned_addr learned my addr 172.21.15.19:0/1815182233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:20.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd808e7b700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8040fe8f0 0x7fd804113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 -- 172.21.15.19:0/1815182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8040fe8f0 msgr2=0x7fd804113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8040fe8f0 0x7fd804113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 -- 172.21.15.19:0/1815182233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f4005040 con 0x7fd8040fdfe0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd80967c700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 0x7fd804113970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c8c0 tx=0x7fd8000044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.313+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd80000a760 con 0x7fd8040fdfe0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.314+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd80000ad70 con 0x7fd8040fdfe0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.314+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80410db10 con 0x7fd8040fdfe0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.314+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8000113a0 con 0x7fd8040fdfe0 2024-02-22T15:54:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.314+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80410e020 con 0x7fd8040fdfe0 2024-02-22T15:54:20.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.315+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8041046c0 con 0x7fd8040fdfe0 2024-02-22T15:54:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.315+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fd800012400 con 0x7fd8040fdfe0 2024-02-22T15:54:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.315+0000 7fd7f9ffb700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7f006c8a0 0x7fd7f006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.315+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd80008b9e0 con 0x7fd8040fdfe0 2024-02-22T15:54:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.316+0000 7fd808e7b700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7f006c8a0 0x7fd7f006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:20.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.316+0000 7fd808e7b700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7f006c8a0 0x7fd7f006ed50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004f50 tx=0x7fd7f4009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:20.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.320+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fd800059a70 con 0x7fd8040fdfe0 2024-02-22T15:54:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:20 smithi062 ceph-mon[48130]: pgmap v97: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:20 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:20 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:54:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:20 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:54:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:20 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:20.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.532+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd80404bed0 con 0x7fd7f006c8a0 2024-02-22T15:54:20.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.542+0000 7fd7f9ffb700 1 -- 172.21.15.19:0/1815182233 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd80404bed0 con 0x7fd7f006c8a0 2024-02-22T15:54:20.523 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:54:20.523 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (89s) 66s ago 12m 16.5M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (12m) 66s ago 12m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (11m) 2m ago 11m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (72s) 66s ago 12m 39.4M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (6m) 66s ago 6m 115M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (6m) 66s ago 6m 67.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (6m) 2m ago 6m 61.8M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (6m) 2m ago 6m 117M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (3m) 66s ago 14m 499M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (2m) 2m ago 11m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (14m) 66s ago 14m 57.2M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 5b269c418fef 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (11m) 2m ago 11m 46.1M 2048M 17.2.7-527-g6ea2a55e e2e4ad29904c 8dce36aa88eb 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (2m) 66s ago 12m 11.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:54:20.524 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (2m) 2m ago 11m 3565k - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (10m) 66s ago 10m 222M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (10m) 66s ago 10m 241M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (9m) 66s ago 9m 195M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (9m) 2m ago 9m 257M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (8m) 2m ago 8m 228M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (8m) 2m ago 8m 219M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:54:20.525 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (115s) 66s ago 12m 48.6M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7f006c8a0 msgr2=0x7fd7f006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7f006c8a0 0x7fd7f006ed50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004f50 tx=0x7fd7f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 msgr2=0x7fd804113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 0x7fd804113970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c8c0 tx=0x7fd8000044c0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 shutdown_connections 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd7f006c8a0 0x7fd7f006ed50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8040fdfe0 0x7fd804113970 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 --2- 172.21.15.19:0/1815182233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8040fe8f0 0x7fd804113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 >> 172.21.15.19:0/1815182233 conn(0x7fd804071d90 msgr2=0x7fd804101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 shutdown_connections 2024-02-22T15:54:20.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.544+0000 7fd80b8e0700 1 -- 172.21.15.19:0/1815182233 wait complete. 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 -- 172.21.15.19:0/175276568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 msgr2=0x7fb3701092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 --2- 172.21.15.19:0/175276568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701092c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb36c004970 tx=0x7fb36c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 -- 172.21.15.19:0/175276568 shutdown_connections 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 --2- 172.21.15.19:0/175276568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701092c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 --2- 172.21.15.19:0/175276568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 0x7fb3700706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 -- 172.21.15.19:0/175276568 >> 172.21.15.19:0/175276568 conn(0x7fb3700f8ae0 msgr2=0x7fb3700faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 -- 172.21.15.19:0/175276568 shutdown_connections 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 -- 172.21.15.19:0/175276568 wait complete. 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 Processor -- start 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 -- start start 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 0x7fb370115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.650+0000 7fb377a81700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb377a81700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370116700 con 0x7fb370070be0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb377a81700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37010fa30 con 0x7fb3700702d0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40326/0 (socket says 172.21.15.19:40326) 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 -- 172.21.15.19:0/125424092 learned_addr learned my addr 172.21.15.19:0/125424092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37581d700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 0x7fb370115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 -- 172.21.15.19:0/125424092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 msgr2=0x7fb370115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 0x7fb370115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 -- 172.21.15.19:0/125424092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c005040 con 0x7fb370070be0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37581d700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 0x7fb370115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb37501c700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701160b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb36c004f50 tx=0x7fb36c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb36c00e040 con 0x7fb370070be0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37010fcb0 con 0x7fb370070be0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb36c00bc10 con 0x7fb370070be0 2024-02-22T15:54:20.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb36c01c3f0 con 0x7fb370070be0 2024-02-22T15:54:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.651+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3701101f0 con 0x7fb370070be0 2024-02-22T15:54:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.652+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb370106980 con 0x7fb370070be0 2024-02-22T15:54:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.653+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fb36c013420 con 0x7fb370070be0 2024-02-22T15:54:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.653+0000 7fb3667fc700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb35c06c8b0 0x7fb35c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.653+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb36c08cf70 con 0x7fb370070be0 2024-02-22T15:54:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.653+0000 7fb37581d700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb35c06c8b0 0x7fb35c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.654+0000 7fb37581d700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb35c06c8b0 0x7fb35c06ed60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb360009fd0 tx=0x7fb360009ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:20.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.658+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fb36c0c04d0 con 0x7fb370070be0 2024-02-22T15:54:20.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.989+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb37010c110 con 0x7fb370070be0 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.990+0000 7fb3667fc700 1 -- 172.21.15.19:0/125424092 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fb36c05fe60 con 0x7fb370070be0 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:54:20.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 12, 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:54:20.973 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb35c06c8b0 msgr2=0x7fb35c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb35c06c8b0 0x7fb35c06ed60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb360009fd0 tx=0x7fb360009ee0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 msgr2=0x7fb3701160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701160b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb36c004f50 tx=0x7fb36c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 shutdown_connections 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fb35c06c8b0 0x7fb35c06ed60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3700702d0 0x7fb370115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 --2- 172.21.15.19:0/125424092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb370070be0 0x7fb3701160b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 >> 172.21.15.19:0/125424092 conn(0x7fb3700f8ae0 msgr2=0x7fb370103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 shutdown_connections 2024-02-22T15:54:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:20.992+0000 7fb377a81700 1 -- 172.21.15.19:0/125424092 wait complete. 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 -- 172.21.15.19:0/4054771840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 msgr2=0x7f67a40fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/4054771840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a40fcf50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6798004940 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 -- 172.21.15.19:0/4054771840 shutdown_connections 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/4054771840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a40fcf50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/4054771840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f67a4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 -- 172.21.15.19:0/4054771840 >> 172.21.15.19:0/4054771840 conn(0x7f67a40f8af0 msgr2=0x7f67a40faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 -- 172.21.15.19:0/4054771840 shutdown_connections 2024-02-22T15:54:21.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.103+0000 7f67aa5a1700 1 -- 172.21.15.19:0/4054771840 wait complete. 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.104+0000 7f67aa5a1700 1 Processor -- start 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.104+0000 7f67aa5a1700 1 -- start start 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.104+0000 7f67aa5a1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f67a4113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.104+0000 7f67aa5a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.104+0000 7f67aa5a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41144b0 con 0x7f67a41038e0 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.104+0000 7f67aa5a1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a410d7e0 con 0x7f67a4102f40 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40336/0 (socket says 172.21.15.19:40336) 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 learned_addr learned my addr 172.21.15.19:0/2746393600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a3fff700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f67a4113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 msgr2=0x7f67a4113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f67a4113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798005040 con 0x7f67a41038e0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6798009380 tx=0x7f6798009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679800e040 con 0x7f67a41038e0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a410da60 con 0x7f67a41038e0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f679800bc50 con 0x7f67a41038e0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679801c3f0 con 0x7f67a41038e0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.105+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a410dfa0 con 0x7f67a41038e0 2024-02-22T15:54:21.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.106+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4105410 con 0x7f67a41038e0 2024-02-22T15:54:21.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.107+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f6798013420 con 0x7f67a41038e0 2024-02-22T15:54:21.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.107+0000 7f67a0ff9700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f678c06c8a0 0x7f678c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.108+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f679808da30 con 0x7f67a41038e0 2024-02-22T15:54:21.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.108+0000 7f67a3fff700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f678c06c8a0 0x7f678c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.108+0000 7f67a3fff700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f678c06c8a0 0x7f678c06ed50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6794009b80 tx=0x7f679400b040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.112+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f679805ba00 con 0x7f67a41038e0 2024-02-22T15:54:21.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 ceph-mon[67418]: Upgrade: Updating mon.smithi019 2024-02-22T15:54:21.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 ceph-mon[67418]: Deploying daemon mon.smithi019 on smithi019 2024-02-22T15:54:21.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 ceph-mon[67418]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:21.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 ceph-mon[67418]: from='client.? 172.21.15.19:0/125424092' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:21.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 systemd[1]: Stopping Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.331+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 msgr2=0x7f67a4113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.331+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 msgr2=0x7f67a4113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.331+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6798009380 tx=0x7f6798009e30 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.331+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6798009380 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.331+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 msgr2=0x7f67a4113e60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.331+0000 7f67a0ff9700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41038e0 0x7f67a4113e60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a0ff9700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f678c076500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a0ff9700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f678c076c00 0x7f678c079080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c079af0 con 0x7f678c076c00 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c079c30 con 0x7f67a4102f40 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a8b3e700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f678c076c00 msgr2=0x7f678c079080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:3300/0 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a8b3e700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f678c076c00 0x7f678c079080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.332+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f678c076500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.333+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f678c076c00 msgr2=0x7f678c079080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:21.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.333+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f678c076c00 0x7f678c079080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.333+0000 7f67a37fe700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=3+}) v3 -- 0x7f678c07e1d0 con 0x7f67a4102f40 2024-02-22T15:54:21.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.334+0000 7f67a37fe700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f678c076500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67a410eb10 tx=0x7f6798009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.334+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6798024ec0 con 0x7f67a4102f40 2024-02-22T15:54:21.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.334+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6798059fd0 con 0x7f67a4102f40 2024-02-22T15:54:21.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.336+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f679805b1b0 con 0x7f67a4102f40 2024-02-22T15:54:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.370+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6784005120 con 0x7f67a4102f40 2024-02-22T15:54:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.371+0000 7f67a0ff9700 1 -- 172.21.15.19:0/2746393600 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f679805b770 con 0x7f67a4102f40 2024-02-22T15:54:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:54:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:54:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:54:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:21.354 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:21.355 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:21.355 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:21.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:54:21.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f678c06c8a0 msgr2=0x7f678c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f678c06c8a0 0x7f678c06ed50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6794009b80 tx=0x7f679400b040 comp rx=0 tx=0).stop 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 msgr2=0x7f678c076500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f678c076500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67a410eb10 tx=0x7f6798009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 shutdown_connections 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f678c076c00 0x7f678c079080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67a4102f40 0x7f678c076500 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 --2- 172.21.15.19:0/2746393600 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f678c06c8a0 0x7f678c06ed50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 >> 172.21.15.19:0/2746393600 conn(0x7f67a40f8af0 msgr2=0x7f67a40fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 shutdown_connections 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.373+0000 7f67aa5a1700 1 -- 172.21.15.19:0/2746393600 wait complete. 2024-02-22T15:54:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:54:21.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:21 smithi062 ceph-mon[48130]: Upgrade: Updating mon.smithi019 2024-02-22T15:54:21.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:21 smithi062 ceph-mon[48130]: Deploying daemon mon.smithi019 on smithi019 2024-02-22T15:54:21.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:21 smithi062 ceph-mon[48130]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:21.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:21 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/125424092' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 -- 172.21.15.19:0/683011190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6881038f0 msgr2=0x7fd6880fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 --2- 172.21.15.19:0/683011190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6881038f0 0x7fd6880fcf60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd670004940 tx=0x7fd670009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 -- 172.21.15.19:0/683011190 shutdown_connections 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 --2- 172.21.15.19:0/683011190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6881038f0 0x7fd6880fcf60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 --2- 172.21.15.19:0/683011190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd688102f50 0x7fd688103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 -- 172.21.15.19:0/683011190 >> 172.21.15.19:0/683011190 conn(0x7fd6880f8ae0 msgr2=0x7fd6880faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.478+0000 7fd68d55a700 1 -- 172.21.15.19:0/683011190 shutdown_connections 2024-02-22T15:54:21.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 -- 172.21.15.19:0/683011190 wait complete. 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 Processor -- start 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 -- start start 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 0x7fd688113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6881038f0 0x7fd688113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd688114690 con 0x7fd6881038f0 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd68d55a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68810d930 con 0x7fd688102f50 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.479+0000 7fd6867fc700 1 -- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6881038f0 msgr2=0x7fd688113fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:3300/0 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd6867fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6881038f0 0x7fd688113fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 0x7fd688113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 0x7fd688113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:36168/0 (socket says 172.21.15.19:36168) 2024-02-22T15:54:21.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 -- 172.21.15.19:0/1512448142 learned_addr learned my addr 172.21.15.19:0/1512448142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 -- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6881038f0 msgr2=0x7fd688113fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6881038f0 0x7fd688113fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 -- 172.21.15.19:0/1512448142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670005040 con 0x7fd688102f50 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd686ffd700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 0x7fd688113a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b760 tx=0x7fd678009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd678010790 con 0x7fd688102f50 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd678010da0 con 0x7fd688102f50 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68810dc10 con 0x7fd688102f50 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67800f300 con 0x7fd688102f50 2024-02-22T15:54:21.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.480+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68810e150 con 0x7fd688102f50 2024-02-22T15:54:21.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.481+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd688100260 con 0x7fd688102f50 2024-02-22T15:54:21.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.482+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7fd6780116d0 con 0x7fd688102f50 2024-02-22T15:54:21.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.482+0000 7fd67f7fe700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd67406c7f0 0x7fd67406eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.483+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd67808bde0 con 0x7fd688102f50 2024-02-22T15:54:21.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.483+0000 7fd6867fc700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd67406c7f0 0x7fd67406eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.483+0000 7fd6867fc700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd67406c7f0 0x7fd67406eca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd670000c00 tx=0x7fd670009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.488+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7fd678059aa0 con 0x7fd688102f50 2024-02-22T15:54:21.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.716+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 --> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd68810f050 con 0x7fd67406c7f0 2024-02-22T15:54:21.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.718+0000 7fd67f7fe700 1 -- 172.21.15.19:0/1512448142 <== mgr.14660 v2:172.21.15.19:6800/3932892700 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fd68810f050 con 0x7fd67406c7f0 2024-02-22T15:54:21.699 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:54:21.699 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:54:21.699 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:54:21.699 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "2/21 daemons upgraded", 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mon daemons", 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:54:21.700 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:54:21.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd67406c7f0 msgr2=0x7fd67406eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd67406c7f0 0x7fd67406eca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd670000c00 tx=0x7fd670009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 msgr2=0x7fd688113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 0x7fd688113a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b760 tx=0x7fd678009220 comp rx=0 tx=0).stop 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 shutdown_connections 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7fd67406c7f0 0x7fd67406eca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd688102f50 0x7fd688113a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 --2- 172.21.15.19:0/1512448142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6881038f0 0x7fd688113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 >> 172.21.15.19:0/1512448142 conn(0x7fd6880f8ae0 msgr2=0x7fd6880fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 shutdown_connections 2024-02-22T15:54:21.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.720+0000 7fd68d55a700 1 -- 172.21.15.19:0/1512448142 wait complete. 2024-02-22T15:54:21.740 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[67395]: 2024-02-22T15:54:21.330+0000 7fb663606700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi019 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:54:21.740 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[67395]: 2024-02-22T15:54:21.330+0000 7fb663606700 -1 mon.smithi019@0(leader) e2 *** Got Signal Terminated *** 2024-02-22T15:54:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.825+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/653754017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b68102f40 msgr2=0x7f4b68103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.825+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/653754017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b68102f40 0x7f4b68103310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c004940 tx=0x7f4b5c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.825+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/653754017 shutdown_connections 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.825+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/653754017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b681038e0 0x7f4b680fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.825+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/653754017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b68102f40 0x7f4b68103310 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.825+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/653754017 >> 172.21.15.19:0/653754017 conn(0x7f4b680f8af0 msgr2=0x7f4b680faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/653754017 shutdown_connections 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/653754017 wait complete. 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 Processor -- start 2024-02-22T15:54:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 -- start start 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68102f40 0x7f4b68113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 0x7f4b68113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68114670 con 0x7f4b68102f40 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b6d8a9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6810d910 con 0x7f4b681038e0 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.826+0000 7f4b66ffd700 1 -- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68102f40 msgr2=0x7f4b68113a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:3300/0 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b66ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68102f40 0x7f4b68113a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 0x7f4b68113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 0x7f4b68113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:36186/0 (socket says 172.21.15.19:36186) 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 -- 172.21.15.19:0/2448929333 learned_addr learned my addr 172.21.15.19:0/2448929333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 -- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68102f40 msgr2=0x7f4b68113a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68102f40 0x7f4b68113a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 -- 172.21.15.19:0/2448929333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c005040 con 0x7f4b681038e0 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b667fc700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 0x7f4b68113f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800b790 tx=0x7f4b58009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b58010790 con 0x7f4b681038e0 2024-02-22T15:54:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6810dbf0 con 0x7f4b681038e0 2024-02-22T15:54:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b58010da0 con 0x7f4b681038e0 2024-02-22T15:54:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5800f300 con 0x7f4b681038e0 2024-02-22T15:54:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.827+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6810e130 con 0x7f4b681038e0 2024-02-22T15:54:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.828+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b68100250 con 0x7f4b681038e0 2024-02-22T15:54:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.829+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 32) v1 ==== 90711+0+0 (secure 0 0 0) 0x7f4b58011440 con 0x7f4b681038e0 2024-02-22T15:54:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.829+0000 7f4b4f7fe700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4b5006c7f0 0x7f4b5006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.830+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4b5808bcb0 con 0x7f4b681038e0 2024-02-22T15:54:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.830+0000 7f4b66ffd700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4b5006c7f0 0x7f4b5006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.830+0000 7f4b66ffd700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4b5006c7f0 0x7f4b5006eca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c000c00 tx=0x7f4b5c009e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:21.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:21.834+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181436 (secure 0 0 0) 0x7f4b58059cc0 con 0x7f4b681038e0 2024-02-22T15:54:21.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:21 smithi019 bash[173327]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019 2024-02-22T15:54:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.125+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4b6804bed0 con 0x7f4b681038e0 2024-02-22T15:54:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.125+0000 7f4b4f7fe700 1 -- 172.21.15.19:0/2448929333 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4b58059850 con 0x7f4b681038e0 2024-02-22T15:54:22.107 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_OK 2024-02-22T15:54:22.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4b5006c7f0 msgr2=0x7f4b5006eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4b5006c7f0 0x7f4b5006eca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c000c00 tx=0x7f4b5c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 msgr2=0x7f4b68113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 0x7f4b68113f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800b790 tx=0x7f4b58009220 comp rx=0 tx=0).stop 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 shutdown_connections 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:6800/3932892700,v1:172.21.15.19:6801/3932892700] conn(0x7f4b5006c7f0 0x7f4b5006eca0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b68102f40 0x7f4b68113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 --2- 172.21.15.19:0/2448929333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b681038e0 0x7f4b68113f90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 >> 172.21.15.19:0/2448929333 conn(0x7f4b680f8af0 msgr2=0x7f4b680fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 shutdown_connections 2024-02-22T15:54:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:22.128+0000 7f4b6d8a9700 1 -- 172.21.15.19:0/2448929333 wait complete. 2024-02-22T15:54:22.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service: Succeeded. 2024-02-22T15:54:22.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 systemd[1]: Stopped Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:54:22.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 systemd[1]: Starting Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:54:23.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: set uid:gid to 167:167 (ceph:ceph) 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable), process ceph-mon, pid 7 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: pidfile_write: ignore empty --pid-file 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: load: jerasure load: lrc 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: RocksDB version: 7.9.2 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Git sha 0 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Compile date 2024-02-21 21:10:13 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: DB SUMMARY 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: DB Session ID: KY5I7IEOWDTH13KHMNH0 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: CURRENT file: CURRENT 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: IDENTITY file: IDENTITY 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1170 Bytes 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi019/store.db dir, Total Num: 1, files: 000033.sst 2024-02-22T15:54:23.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi019/store.db: 000031.log size: 3850766 ; 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.error_if_exists: 0 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.create_if_missing: 0 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.paranoid_checks: 1 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.env: 0x562fc7749720 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.fs: PosixFileSystem 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.info_log: 0x562fc9c81360 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_file_opening_threads: 16 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.statistics: (nil) 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.use_fsync: 0 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_log_file_size: 0 2024-02-22T15:54:23.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.keep_log_file_num: 1000 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.recycle_log_file_num: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.allow_fallocate: 1 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.allow_mmap_reads: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.allow_mmap_writes: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.use_direct_reads: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.create_missing_column_families: 0 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.db_log_dir: 2024-02-22T15:54:23.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.wal_dir: 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.advise_random_on_open: 1 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.db_write_buffer_size: 0 2024-02-22T15:54:23.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.write_buffer_manager: 0x562fc8ef2140 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.rate_limiter: (nil) 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.wal_recovery_mode: 2 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.enable_thread_tracking: 0 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.enable_pipelined_write: 0 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.unordered_write: 0 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-22T15:54:23.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.row_cache: None 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.wal_filter: None 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.allow_ingest_behind: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.two_write_queues: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.manual_wal_flush: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.wal_compression: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.atomic_flush: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.log_readahead_size: 0 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-22T15:54:23.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.best_efforts_recovery: 0 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.allow_data_in_errors: 0 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.db_host_id: __hostname__ 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_background_jobs: 2 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_background_compactions: -1 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_subcompactions: 1 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_total_wal_size: 0 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-22T15:54:23.252 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_open_files: -1 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bytes_per_sync: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_readahead_size: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_background_flushes: -1 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Compression algorithms supported: 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kZSTD supported: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kXpressCompression supported: 0 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kLZ4HCCompression supported: 1 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kZlibCompression supported: 1 2024-02-22T15:54:23.253 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kSnappyCompression supported: 1 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kLZ4Compression supported: 1 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: kBZip2Compression supported: 0 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000015 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.merge_operator: 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_filter: None 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_filter_factory: None 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.sst_partitioner_factory: None 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562fc9c81480) 2024-02-22T15:54:23.254 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_type: 0 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_index_type: 0 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_shortening: 1 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: checksum: 4 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: no_block_cache: 0 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache: 0x562fc8f931f0 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_options: 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: capacity : 536870912 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: num_shard_bits : 4 2024-02-22T15:54:23.255 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: strict_capacity_limit : 0 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_compressed: (nil) 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: persistent_cache: (nil) 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size: 4096 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size_deviation: 10 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_restart_interval: 16 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_block_restart_interval: 1 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: metadata_block_size: 4096 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: partition_filters: 0 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: use_delta_encoding: 1 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: filter_policy: bloomfilter 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: whole_key_filtering: 1 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: verify_compression: 0 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-02-22T15:54:23.256 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: format_version: 5 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_index_compression: 1 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_align: 0 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_auto_readahead_size: 262144 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: prepopulate_block_cache: 0 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: initial_auto_readahead_size: 8192 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.write_buffer_size: 33554432 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_write_buffer_number: 2 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression: NoCompression 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression: Disabled 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.prefix_extractor: nullptr 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.num_levels: 7 2024-02-22T15:54:23.257 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.level: 32767 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.strategy: 0 2024-02-22T15:54:23.258 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.enabled: false 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.target_file_size_base: 67108864 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-22T15:54:23.259 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.arena_block_size: 1048576 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.disable_auto_compactions: 0 2024-02-22T15:54:23.260 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.table_properties_collectors: 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.inplace_update_support: 0 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-22T15:54:23.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.bloom_locality: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.max_successive_merges: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.paranoid_file_checks: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.force_consistency_checks: 1 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.report_bg_io_stats: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.ttl: 2592000 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.enable_blob_files: false 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.min_blob_size: 0 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.blob_file_size: 268435456 2024-02-22T15:54:23.262 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.blob_file_starting_level: 0 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12131, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: adc65834-e262-4095-865d-7daf4e8a6abd 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708617262846824, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-02-22T15:54:23.263 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708617262858411, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 3373683, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12132, "largest_seqno": 15432, "table_properties": {"data_size": 3361047, "index_size": 7903, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3717, "raw_key_size": 32584, "raw_average_key_size": 22, "raw_value_size": 3332132, "raw_average_value_size": 2263, "num_data_blocks": 359, "num_entries": 1472, "num_filter_entries": 1472, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708617262, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "adc65834-e262-4095-865d-7daf4e8a6abd", "db_session_id": "KY5I7IEOWDTH13KHMNH0", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708617262858523, "job": 1, "event": "recovery_finished"} 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi019/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562fc9030000 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: DB pointer 0x562fc901e000 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** DB Stats ** 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T15:54:23.264 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L0 1/0 3.22 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 652.4 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L6 1/0 5.49 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Sum 2/0 8.71 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 652.4 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 652.4 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-22T15:54:23.265 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 652.4 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative compaction: 0.00 GB write, 175.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval compaction: 0.00 GB write, 175.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T15:54:23.266 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Block cache BinnedLRUCache@0x562fc8f931f0#7 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: starting mon.smithi019 rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi019 fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???) e2 preinit fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).mds e24 new map 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).mds e24 print_map 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: e24 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: legacy client fscid: 1 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Filesystem 'cephfs' (1) 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: fs_name cephfs 2024-02-22T15:54:23.267 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: epoch 24 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: tableserver 0 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: root 0 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: session_timeout 60 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: session_autoclose 300 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_file_size 1099511627776 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_xattr_size 65536 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: required_client_features {} 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: last_failure 0 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: last_failure_osd_epoch 47 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:23.268 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_mds 2 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: in 0,1 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: up {0=14472,1=24267} 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: failed 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: damaged 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: stopped 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_pools [3] 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: metadata_pool 2 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: inline_data disabled 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: balancer 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: bal_rank_mask -1 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: standby_count_wanted 1 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:23.269 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 ceph-mon[173579]: mon.smithi019@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 bash[173542]: a352b4940a74825491ce8d5a93d7eb62aa8039fdad67b1f68058aeb35f362b34 2024-02-22T15:54:23.270 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:22 smithi019 systemd[1]: Started Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: from='client.14850 172.21.15.19:0/2746393600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: from='client.? 172.21.15.19:0/2448929333' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: pgmap v99: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: pgmap v100: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: mon.smithi019 calling monitor election 2024-02-22T15:54:26.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: mon.smithi019 is new leader, mons smithi019,smithi062 in quorum (ranks 0,1) 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: monmap e2: 2 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],smithi062=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: osdmap e49: 6 total, 6 up, 6 in 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: mgrmap e32: smithi019.yvjrkp(active, since 3m), standbys: smithi062.ecubmv 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: mgrmap e33: smithi019.yvjrkp(active, since 3m), standbys: smithi062.ecubmv 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: overall HEALTH_OK 2024-02-22T15:54:26.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:25 smithi062 ceph-mon[48130]: from='mgr.14660 ' entity='' 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: from='mgr.14660 172.21.15.19:0/2708610957' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: from='client.14850 172.21.15.19:0/2746393600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2448929333' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: pgmap v99: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: pgmap v100: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: mon.smithi019 calling monitor election 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: mon.smithi019 is new leader, mons smithi019,smithi062 in quorum (ranks 0,1) 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: monmap e2: 2 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],smithi062=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: osdmap e49: 6 total, 6 up, 6 in 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: mgrmap e32: smithi019.yvjrkp(active, since 3m), standbys: smithi062.ecubmv 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: mgrmap e33: smithi019.yvjrkp(active, since 3m), standbys: smithi062.ecubmv 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: overall HEALTH_OK 2024-02-22T15:54:26.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:25 smithi019 ceph-mon[173579]: from='mgr.14660 ' entity='' 2024-02-22T15:54:33.189 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062[48107]: 2024-02-22T15:54:33.075+0000 7f3e9e55d700 -1 mon.smithi062@1(peon).osd e50 update_from_paxos full map CRC mismatch, resetting to canonical 2024-02-22T15:54:34.117 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: Active manager daemon smithi019.yvjrkp restarted 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: osdmap e50: 6 total, 6 up, 6 in 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: mgrmap e34: smithi019.yvjrkp(active, starting, since 0.005846s), standbys: smithi062.ecubmv 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:54:34.118 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv restarted 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:54:34.119 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: failed to encode map e50 with expected crc 2024-02-22T15:54:34.120 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:33 smithi062 ceph-mon[48130]: failed to encode map e50 with expected crc 2024-02-22T15:54:34.139 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: Active manager daemon smithi019.yvjrkp restarted 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: Activating manager daemon smithi019.yvjrkp 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: osdmap e50: 6 total, 6 up, 6 in 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: mgrmap e34: smithi019.yvjrkp(active, starting, since 0.005846s), standbys: smithi062.ecubmv 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.yvjrkp", "id": "smithi019.yvjrkp"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr metadata", "who": "smithi062.ecubmv", "id": "smithi062.ecubmv"}]: dispatch 2024-02-22T15:54:34.140 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: Standby manager daemon smithi062.ecubmv restarted 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: Standby manager daemon smithi062.ecubmv started 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: Manager daemon smithi019.yvjrkp is now available 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/crt"}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-22T15:54:34.141 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi062.ecubmv/key"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.? 172.21.15.62:0/1565410097' entity='mgr.smithi062.ecubmv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/mirror_snapshot_schedule"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.yvjrkp/trash_purge_schedule"}]: dispatch 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: failed to encode map e50 with expected crc 2024-02-22T15:54:34.142 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:33 smithi019 ceph-mon[173579]: failed to encode map e50 with expected crc 2024-02-22T15:54:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:34 smithi062 ceph-mon[48130]: failed to encode map e50 with expected crc 2024-02-22T15:54:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:34 smithi062 ceph-mon[48130]: [22/Feb/2024:15:54:33] ENGINE Bus STARTING 2024-02-22T15:54:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:34 smithi062 ceph-mon[48130]: [22/Feb/2024:15:54:33] ENGINE Serving on http://172.21.15.19:8765 2024-02-22T15:54:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:34 smithi062 ceph-mon[48130]: [22/Feb/2024:15:54:33] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:54:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:34 smithi062 ceph-mon[48130]: [22/Feb/2024:15:54:33] ENGINE Bus STARTED 2024-02-22T15:54:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:34 smithi062 ceph-mon[48130]: mgrmap e35: smithi019.yvjrkp(active, since 1.00957s), standbys: smithi062.ecubmv 2024-02-22T15:54:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:34 smithi019 ceph-mon[173579]: failed to encode map e50 with expected crc 2024-02-22T15:54:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:34 smithi019 ceph-mon[173579]: [22/Feb/2024:15:54:33] ENGINE Bus STARTING 2024-02-22T15:54:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:34 smithi019 ceph-mon[173579]: [22/Feb/2024:15:54:33] ENGINE Serving on http://172.21.15.19:8765 2024-02-22T15:54:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:34 smithi019 ceph-mon[173579]: [22/Feb/2024:15:54:33] ENGINE Serving on https://172.21.15.19:7150 2024-02-22T15:54:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:34 smithi019 ceph-mon[173579]: [22/Feb/2024:15:54:33] ENGINE Bus STARTED 2024-02-22T15:54:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:34 smithi019 ceph-mon[173579]: mgrmap e35: smithi019.yvjrkp(active, since 1.00957s), standbys: smithi062.ecubmv 2024-02-22T15:54:35.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:35 smithi019 ceph-mon[173579]: pgmap v3: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:36.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:35 smithi062 ceph-mon[48130]: pgmap v3: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:36 smithi062 ceph-mon[48130]: pgmap v4: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:36 smithi062 ceph-mon[48130]: mgrmap e36: smithi019.yvjrkp(active, since 3s), standbys: smithi062.ecubmv 2024-02-22T15:54:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:36 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:36 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:36 smithi019 ceph-mon[173579]: pgmap v4: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:37.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:36 smithi019 ceph-mon[173579]: mgrmap e36: smithi019.yvjrkp(active, since 3s), standbys: smithi062.ecubmv 2024-02-22T15:54:37.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:36 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:37.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:36 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:38.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:37 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:38.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:37 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:38.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:37 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:38.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:37 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:39.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:38 smithi062 ceph-mon[48130]: pgmap v5: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:39.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:38 smithi062 ceph-mon[48130]: mgrmap e37: smithi019.yvjrkp(active, since 4s), standbys: smithi062.ecubmv 2024-02-22T15:54:39.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:38 smithi019 ceph-mon[173579]: pgmap v5: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:39.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:38 smithi019 ceph-mon[173579]: mgrmap e37: smithi019.yvjrkp(active, since 4s), standbys: smithi062.ecubmv 2024-02-22T15:54:40.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:39 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:39 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:39 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:39 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:39 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:39 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:39 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:40.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:39 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:41.201 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:40 smithi019 ceph-mon[173579]: pgmap v6: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:41.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:40 smithi062 ceph-mon[48130]: pgmap v6: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:54:42.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:42 smithi019 ceph-mon[173579]: pgmap v7: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-22T15:54:43.228 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:42 smithi062 ceph-mon[48130]: pgmap v7: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: pgmap v8: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Detected new or changed devices on smithi019 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:54:44.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:54:44.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:54:44.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:54:44.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:44 smithi062 ceph-mon[48130]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:54:44.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: pgmap v8: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-02-22T15:54:44.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Detected new or changed devices on smithi019 2024-02-22T15:54:44.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:44.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:44.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:44.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi019:/etc/ceph/ceph.conf 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi062:/etc/ceph/ceph.conf 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.conf 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:54:44.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:44 smithi019 ceph-mon[173579]: Updating smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/config/ceph.client.admin.keyring 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:45.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:45 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:45.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:45 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-22T15:54:47.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: pgmap v9: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-22T15:54:47.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: Upgrade: Updating mon.smithi062 2024-02-22T15:54:47.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:47.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:54:47.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:54:47.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:47.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:46 smithi062 ceph-mon[48130]: Deploying daemon mon.smithi062 on smithi062 2024-02-22T15:54:47.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: pgmap v9: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-22T15:54:47.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: Upgrade: Updating mon.smithi062 2024-02-22T15:54:47.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:47.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:54:47.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:54:47.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:47.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:46 smithi019 ceph-mon[173579]: Deploying daemon mon.smithi062 on smithi062 2024-02-22T15:54:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:47 smithi062 systemd[1]: Stopping Ceph mon.smithi062 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:54:48.283 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:48 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062[48107]: 2024-02-22T15:54:48.002+0000 7f3ea3d68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi062 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:54:48.283 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:48 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062[48107]: 2024-02-22T15:54:48.002+0000 7f3ea3d68700 -1 mon.smithi062@1(peon) e2 *** Got Signal Terminated *** 2024-02-22T15:54:48.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:48 smithi062 bash[106665]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062 2024-02-22T15:54:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:48 smithi062 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi062.service: Succeeded. 2024-02-22T15:54:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:48 smithi062 systemd[1]: Stopped Ceph mon.smithi062 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:54:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:48 smithi062 systemd[1]: Starting Ceph mon.smithi062 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:54:49.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: set uid:gid to 167:167 (ceph:ceph) 2024-02-22T15:54:49.708 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable), process ceph-mon, pid 7 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: pidfile_write: ignore empty --pid-file 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: load: jerasure load: lrc 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: RocksDB version: 7.9.2 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Git sha 0 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Compile date 2024-02-21 21:10:13 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: DB SUMMARY 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: DB Session ID: W93PBJUFAQTMO2HWX8CA 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: CURRENT file: CURRENT 2024-02-22T15:54:49.709 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: IDENTITY file: IDENTITY 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1095 Bytes 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi062/store.db dir, Total Num: 1, files: 000027.sst 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi062/store.db: 000025.log size: 8337835 ; 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.error_if_exists: 0 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.create_if_missing: 0 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.paranoid_checks: 1 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-22T15:54:49.710 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.env: 0x557ceb840720 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.fs: PosixFileSystem 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.info_log: 0x557cee527360 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_file_opening_threads: 16 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.statistics: (nil) 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.use_fsync: 0 2024-02-22T15:54:49.711 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_log_file_size: 0 2024-02-22T15:54:49.712 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-22T15:54:49.712 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-22T15:54:49.712 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.keep_log_file_num: 1000 2024-02-22T15:54:49.712 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.recycle_log_file_num: 0 2024-02-22T15:54:49.712 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.allow_fallocate: 1 2024-02-22T15:54:49.712 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.allow_mmap_reads: 0 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.allow_mmap_writes: 0 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.use_direct_reads: 0 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.create_missing_column_families: 0 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.db_log_dir: 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.wal_dir: 2024-02-22T15:54:49.713 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-22T15:54:49.714 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-22T15:54:49.714 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-22T15:54:49.714 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-22T15:54:49.714 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-22T15:54:49.714 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-22T15:54:49.714 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.advise_random_on_open: 1 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.db_write_buffer_size: 0 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.write_buffer_manager: 0x557ced798140 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.rate_limiter: (nil) 2024-02-22T15:54:49.715 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.wal_recovery_mode: 2 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.enable_thread_tracking: 0 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.enable_pipelined_write: 0 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.unordered_write: 0 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.row_cache: None 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.wal_filter: None 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.allow_ingest_behind: 0 2024-02-22T15:54:49.716 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.two_write_queues: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.manual_wal_flush: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.wal_compression: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.atomic_flush: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.log_readahead_size: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.best_efforts_recovery: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.allow_data_in_errors: 0 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.db_host_id: __hostname__ 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-22T15:54:49.717 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_background_jobs: 2 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_background_compactions: -1 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_subcompactions: 1 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_total_wal_size: 0 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_open_files: -1 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bytes_per_sync: 0 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_readahead_size: 0 2024-02-22T15:54:49.718 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_background_flushes: -1 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Compression algorithms supported: 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kZSTD supported: 0 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kXpressCompression supported: 0 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kLZ4HCCompression supported: 1 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kZlibCompression supported: 1 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kSnappyCompression supported: 1 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kLZ4Compression supported: 1 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: kBZip2Compression supported: 0 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi062/store.db/MANIFEST-000009 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-22T15:54:49.719 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.merge_operator: 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_filter: None 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_filter_factory: None 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.sst_partitioner_factory: None 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557cee527480) 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: index_type: 0 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: data_block_index_type: 0 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: index_shortening: 1 2024-02-22T15:54:49.720 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: checksum: 4 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: no_block_cache: 0 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_cache: 0x557ced8391f0 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_cache_options: 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: capacity : 536870912 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: num_shard_bits : 4 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: strict_capacity_limit : 0 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_cache_compressed: (nil) 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: persistent_cache: (nil) 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_size: 4096 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_size_deviation: 10 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_restart_interval: 16 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: index_block_restart_interval: 1 2024-02-22T15:54:49.721 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: metadata_block_size: 4096 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: partition_filters: 0 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: use_delta_encoding: 1 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: filter_policy: bloomfilter 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: whole_key_filtering: 1 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: verify_compression: 0 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: format_version: 5 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: enable_index_compression: 1 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: block_align: 0 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: max_auto_readahead_size: 262144 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: prepopulate_block_cache: 0 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: initial_auto_readahead_size: 8192 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.write_buffer_size: 33554432 2024-02-22T15:54:49.722 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_write_buffer_number: 2 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression: NoCompression 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression: Disabled 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.prefix_extractor: nullptr 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.num_levels: 7 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-22T15:54:49.723 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.level: 32767 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.strategy: 0 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.enabled: false 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-22T15:54:49.724 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.target_file_size_base: 67108864 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-22T15:54:49.725 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.arena_block_size: 1048576 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.disable_auto_compactions: 0 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-22T15:54:49.726 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.table_properties_collectors: 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.inplace_update_support: 0 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-22T15:54:49.727 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.bloom_locality: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.max_successive_merges: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.paranoid_file_checks: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.force_consistency_checks: 1 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.report_bg_io_stats: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.ttl: 2592000 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.enable_blob_files: false 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.min_blob_size: 0 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.blob_file_size: 268435456 2024-02-22T15:54:49.728 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.blob_file_starting_level: 0 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-22T15:54:49.729 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi062/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12233, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e8c4956d-12a1-4862-b40e-a5c04eed9ca7 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708617289405217, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708617289426000, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4979027, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12238, "largest_seqno": 16427, "table_properties": {"data_size": 4964394, "index_size": 9388, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4229, "raw_key_size": 37500, "raw_average_key_size": 22, "raw_value_size": 4931018, "raw_average_value_size": 2929, "num_data_blocks": 424, "num_entries": 1683, "num_filter_entries": 1683, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708617289, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e8c4956d-12a1-4862-b40e-a5c04eed9ca7", "db_session_id": "W93PBJUFAQTMO2HWX8CA", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708617289426118, "job": 1, "event": "recovery_finished"} 2024-02-22T15:54:49.730 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi062/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557ced8d6000 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: DB pointer 0x557ced8c4000 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: ** DB Stats ** 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-22T15:54:49.731 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: ** Compaction Stats [default] ** 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: L0 1/0 4.75 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 576.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: L6 1/0 5.49 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Sum 2/0 10.24 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 576.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 576.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: ** Compaction Stats [default] ** 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 576.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.732 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Cumulative compaction: 0.00 GB write, 200.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Interval compaction: 0.00 GB write, 200.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Block cache BinnedLRUCache@0x557ced8391f0#7 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.733 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: starting mon.smithi062 rank 1 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi062 fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???) e2 preinit fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).mds e24 new map 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).mds e24 print_map 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: e24 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: legacy client fscid: 1 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: Filesystem 'cephfs' (1) 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: fs_name cephfs 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: epoch 24 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:54:49.734 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: tableserver 0 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: root 0 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: session_timeout 60 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: session_autoclose 300 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: max_file_size 1099511627776 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: max_xattr_size 65536 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: required_client_features {} 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: last_failure 0 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: last_failure_osd_epoch 47 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: max_mds 2 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: in 0,1 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: up {0=14472,1=24267} 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: failed 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: damaged 2024-02-22T15:54:49.735 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: stopped 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: data_pools [3] 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: metadata_pool 2 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: inline_data disabled 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: balancer 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: bal_rank_mask -1 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: standby_count_wanted 1 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: [mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: [mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: [mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: [mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout: 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-22T15:54:49.736 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: mon.smithi062@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 bash[106824]: ce641474408acc4d1b9147b9437987cf0d3ab9d4e2f6121e1e16e7d332173da1 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 systemd[1]: Started Ceph mon.smithi062 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: pgmap v10: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-22T15:54:49.737 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:54:49.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:49 smithi019 ceph-mon[173579]: pgmap v10: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-22T15:54:49.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:54:50.901 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: pgmap v11: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-22T15:54:50.901 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: overall HEALTH_OK 2024-02-22T15:54:50.901 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:54:50.901 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: mon.smithi019 calling monitor election 2024-02-22T15:54:50.901 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: mon.smithi062 calling monitor election 2024-02-22T15:54:50.901 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: mon.smithi019 is new leader, mons smithi019,smithi062 in quorum (ranks 0,1) 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: monmap e3: 2 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],smithi062=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: osdmap e50: 6 total, 6 up, 6 in 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: mgrmap e37: smithi019.yvjrkp(active, since 16s), standbys: smithi062.ecubmv 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: overall HEALTH_OK 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:50.902 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:50 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: pgmap v11: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: overall HEALTH_OK 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: mon.smithi019 calling monitor election 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: mon.smithi062 calling monitor election 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: mon.smithi019 is new leader, mons smithi019,smithi062 in quorum (ranks 0,1) 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mon metadata", "id": "smithi062"}]: dispatch 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: monmap e3: 2 mons at {smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0],smithi062=[v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2 up:standby-replay 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: osdmap e50: 6 total, 6 up, 6 in 2024-02-22T15:54:50.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: mgrmap e37: smithi019.yvjrkp(active, since 16s), standbys: smithi062.ecubmv 2024-02-22T15:54:50.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: overall HEALTH_OK 2024-02-22T15:54:50.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:50.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:50.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:50 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:54:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 -- 172.21.15.19:0/218734619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 msgr2=0x7fa810103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 --2- 172.21.15.19:0/218734619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810103310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa800004940 tx=0x7fa800009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 -- 172.21.15.19:0/218734619 shutdown_connections 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 --2- 172.21.15.19:0/218734619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 --2- 172.21.15.19:0/218734619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810103310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 -- 172.21.15.19:0/218734619 >> 172.21.15.19:0/218734619 conn(0x7fa8100f8af0 msgr2=0x7fa8100faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 -- 172.21.15.19:0/218734619 shutdown_connections 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 -- 172.21.15.19:0/218734619 wait complete. 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.250+0000 7fa8166a6700 1 Processor -- start 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa8166a6700 1 -- start start 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa8166a6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa8166a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa8166a6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa810070e40 con 0x7fa810102f40 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa8166a6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa810070fb0 con 0x7fa8101038e0 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35238/0 (socket says 172.21.15.19:35238) 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80ffff700 1 -- 172.21.15.19:0/948830536 learned_addr learned my addr 172.21.15.19:0/948830536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80f7fe700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80f7fe700 1 -- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 msgr2=0x7fa8100707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80f7fe700 1 -- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 msgr2=0x7fa8100707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80ffff700 1 -- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 msgr2=0x7fa8100707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80f7fe700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80f7fe700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100707f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80ffff700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100707f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.251+0000 7fa80ffff700 1 -- 172.21.15.19:0/948830536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa800005040 con 0x7fa810102f40 2024-02-22T15:54:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.252+0000 7fa80ffff700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810070290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b3b0 tx=0x7fa800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.252+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80000e040 con 0x7fa810102f40 2024-02-22T15:54:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.252+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa80000bc50 con 0x7fa810102f40 2024-02-22T15:54:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.252+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa810119be0 con 0x7fa810102f40 2024-02-22T15:54:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.252+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa800013420 con 0x7fa810102f40 2024-02-22T15:54:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.252+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81011a0c0 con 0x7fa810102f40 2024-02-22T15:54:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.253+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa810100250 con 0x7fa810102f40 2024-02-22T15:54:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.254+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fa80001c430 con 0x7fa810102f40 2024-02-22T15:54:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.255+0000 7fa80cff9700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa7f806cd40 0x7fa7f806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.255+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa80008d770 con 0x7fa810102f40 2024-02-22T15:54:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.255+0000 7fa80f7fe700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa7f806cd40 0x7fa7f806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.255+0000 7fa80f7fe700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa7f806cd40 0x7fa7f806f1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa810071c90 tx=0x7fa804009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.259+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa800057780 con 0x7fa810102f40 2024-02-22T15:54:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.483+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8100719d0 con 0x7fa7f806cd40 2024-02-22T15:54:52.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.484+0000 7fa80cff9700 1 -- 172.21.15.19:0/948830536 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fa8100719d0 con 0x7fa7f806cd40 2024-02-22T15:54:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa7f806cd40 msgr2=0x7fa7f806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa7f806cd40 0x7fa7f806f1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa810071c90 tx=0x7fa804009f70 comp rx=0 tx=0).stop 2024-02-22T15:54:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 msgr2=0x7fa810070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810070290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b3b0 tx=0x7fa800009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 shutdown_connections 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa7f806cd40 0x7fa7f806f1f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa810102f40 0x7fa810070290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 --2- 172.21.15.19:0/948830536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8101038e0 0x7fa8100707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 >> 172.21.15.19:0/948830536 conn(0x7fa8100f8af0 msgr2=0x7fa8100fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 shutdown_connections 2024-02-22T15:54:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.486+0000 7fa8166a6700 1 -- 172.21.15.19:0/948830536 wait complete. 2024-02-22T15:54:52.477 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:54:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 -- 172.21.15.19:0/1084873598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 msgr2=0x7fd6600fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 --2- 172.21.15.19:0/1084873598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fbc70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd654004940 tx=0x7fd654009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 -- 172.21.15.19:0/1084873598 shutdown_connections 2024-02-22T15:54:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 --2- 172.21.15.19:0/1084873598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fbc70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 --2- 172.21.15.19:0/1084873598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd660102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 -- 172.21.15.19:0/1084873598 >> 172.21.15.19:0/1084873598 conn(0x7fd660073ab0 msgr2=0x7fd660073eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 -- 172.21.15.19:0/1084873598 shutdown_connections 2024-02-22T15:54:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.592+0000 7fd664a02700 1 -- 172.21.15.19:0/1084873598 wait complete. 2024-02-22T15:54:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd664a02700 1 Processor -- start 2024-02-22T15:54:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd664a02700 1 -- start start 2024-02-22T15:54:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd664a02700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd6600fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd664a02700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd664a02700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600fe880 con 0x7fd660102600 2024-02-22T15:54:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd664a02700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600fe9f0 con 0x7fd660101c60 2024-02-22T15:54:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd65e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd65e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fe280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35256/0 (socket says 172.21.15.19:35256) 2024-02-22T15:54:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd65e59c700 1 -- 172.21.15.19:0/99781348 learned_addr learned my addr 172.21.15.19:0/99781348 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.593+0000 7fd65ed9d700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd6600fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65ed9d700 1 -- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 msgr2=0x7fd6600fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-22T15:54:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65ed9d700 1 -- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 msgr2=0x7fd6600fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:54:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65ed9d700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd6600fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:54:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65ed9d700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd6600fdd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65e59c700 1 -- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 msgr2=0x7fd6600fdd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65e59c700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd6600fdd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65e59c700 1 -- 172.21.15.19:0/99781348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654005040 con 0x7fd660102600 2024-02-22T15:54:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd65e59c700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fe280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd654009fd0 tx=0x7fd654009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65400e040 con 0x7fd660102600 2024-02-22T15:54:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd65400bc50 con 0x7fd660102600 2024-02-22T15:54:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6601199e0 con 0x7fd660102600 2024-02-22T15:54:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd65401c3f0 con 0x7fd660102600 2024-02-22T15:54:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.594+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd660119e40 con 0x7fd660102600 2024-02-22T15:54:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.595+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66005f5c0 con 0x7fd660102600 2024-02-22T15:54:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.595+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fd654013420 con 0x7fd660102600 2024-02-22T15:54:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.596+0000 7fd6477fe700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd64806cd40 0x7fd64806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.596+0000 7fd65ed9d700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd64806cd40 0x7fd64806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.596+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd65408d510 con 0x7fd660102600 2024-02-22T15:54:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.596+0000 7fd65ed9d700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd64806cd40 0x7fd64806f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd650009e50 tx=0x7fd650009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.601+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd654057470 con 0x7fd660102600 2024-02-22T15:54:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.821+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6600656c0 con 0x7fd64806cd40 2024-02-22T15:54:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.823+0000 7fd6477fe700 1 -- 172.21.15.19:0/99781348 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd6600656c0 con 0x7fd64806cd40 2024-02-22T15:54:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd64806cd40 msgr2=0x7fd64806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd64806cd40 0x7fd64806f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd650009e50 tx=0x7fd650009f10 comp rx=0 tx=0).stop 2024-02-22T15:54:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 msgr2=0x7fd6600fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fe280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd654009fd0 tx=0x7fd654009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 shutdown_connections 2024-02-22T15:54:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd64806cd40 0x7fd64806f1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd660101c60 0x7fd6600fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 --2- 172.21.15.19:0/99781348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd660102600 0x7fd6600fe280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 >> 172.21.15.19:0/99781348 conn(0x7fd660073ab0 msgr2=0x7fd6600fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 shutdown_connections 2024-02-22T15:54:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.825+0000 7fd664a02700 1 -- 172.21.15.19:0/99781348 wait complete. 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 -- 172.21.15.19:0/1762186724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 msgr2=0x7f89040fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1762186724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f89040fbea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f88f0004940 tx=0x7f88f0009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 -- 172.21.15.19:0/1762186724 shutdown_connections 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1762186724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f89040fbea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1762186724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 -- 172.21.15.19:0/1762186724 >> 172.21.15.19:0/1762186724 conn(0x7f8904071d90 msgr2=0x7f8904072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 -- 172.21.15.19:0/1762186724 shutdown_connections 2024-02-22T15:54:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.933+0000 7f890ac1f700 1 -- 172.21.15.19:0/1762186724 wait complete. 2024-02-22T15:54:52.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f890ac1f700 1 Processor -- start 2024-02-22T15:54:52.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f890ac1f700 1 -- start start 2024-02-22T15:54:52.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f890ac1f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f890ac1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f8904113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f890ac1f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89041145d0 con 0x7f89041027e0 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f890ac1f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890410d900 con 0x7f8904101e40 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f8903fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f8904113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f8903fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f8904113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35272/0 (socket says 172.21.15.19:35272) 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f8903fff700 1 -- 172.21.15.19:0/1813806454 learned_addr learned my addr 172.21.15.19:0/1813806454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.934+0000 7f89089bb700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89089bb700 1 -- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 msgr2=0x7f8904113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89089bb700 1 -- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 msgr2=0x7f8904113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89089bb700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89089bb700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904113a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f8903fff700 1 -- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 msgr2=0x7f8904113a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:52.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f8903fff700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904113a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f8903fff700 1 -- 172.21.15.19:0/1813806454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f0005040 con 0x7f89041027e0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f8903fff700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f8904113f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009fd0 tx=0x7f88f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f000e040 con 0x7f89041027e0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88f000bc50 con 0x7f89041027e0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890410db80 con 0x7f89041027e0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f001c3f0 con 0x7f89041027e0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.935+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890410e0c0 con 0x7f89041027e0 2024-02-22T15:54:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.936+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89040738b0 con 0x7f89041027e0 2024-02-22T15:54:52.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.937+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f88f0013420 con 0x7f89041027e0 2024-02-22T15:54:52.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.937+0000 7f89017fa700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88f406cd40 0x7f88f406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:52.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.937+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f88f008d590 con 0x7f89041027e0 2024-02-22T15:54:52.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.937+0000 7f89089bb700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88f406cd40 0x7f88f406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:52.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.938+0000 7f89089bb700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88f406cd40 0x7f88f406f1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009e50 tx=0x7f88f8009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:52.942+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f88f005af80 con 0x7f89041027e0 2024-02-22T15:54:52.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:52 smithi062 ceph-mon[106861]: pgmap v12: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-22T15:54:52.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:52.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:52.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:52 smithi019 ceph-mon[173579]: pgmap v12: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-02-22T15:54:52.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:52.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:53.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.159+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f890404bed0 con 0x7f88f406cd40 2024-02-22T15:54:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.169+0000 7f89017fa700 1 -- 172.21.15.19:0/1813806454 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f890404bed0 con 0x7f88f406cd40 2024-02-22T15:54:53.149 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:54:53.149 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (2m) 14s ago 13m 16.6M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:54:53.149 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (13m) 14s ago 13m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:54:53.149 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (12m) 0s ago 12m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (105s) 14s ago 12m 78.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (7m) 14s ago 7m 114M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (7m) 14s ago 7m 67.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (7m) 0s ago 7m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (7m) 0s ago 7m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (3m) 14s ago 14m 468M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (3m) 0s ago 12m 446M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (30s) 14s ago 14m 27.6M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (4s) 0s ago 12m 26.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (3m) 14s ago 13m 18.1M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (2m) 0s ago 12m 16.8M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (11m) 14s ago 11m 224M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (10m) 14s ago 10m 243M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:54:53.150 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (10m) 14s ago 10m 196M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:54:53.151 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (9m) 0s ago 9m 259M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:54:53.151 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (9m) 0s ago 9m 229M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:54:53.151 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (9m) 0s ago 9m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:54:53.151 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (2m) 14s ago 12m 59.6M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:54:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88f406cd40 msgr2=0x7f88f406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88f406cd40 0x7f88f406f1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009e50 tx=0x7f88f8009f10 comp rx=0 tx=0).stop 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 msgr2=0x7f8904113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f8904113f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009fd0 tx=0x7f88f0009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 shutdown_connections 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88f406cd40 0x7f88f406f1f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8904101e40 0x7f8904113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 --2- 172.21.15.19:0/1813806454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89041027e0 0x7f8904113f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 >> 172.21.15.19:0/1813806454 conn(0x7f8904071d90 msgr2=0x7f8904065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 shutdown_connections 2024-02-22T15:54:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.171+0000 7f890ac1f700 1 -- 172.21.15.19:0/1813806454 wait complete. 2024-02-22T15:54:53.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 -- 172.21.15.19:0/3831705229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2000fb570 msgr2=0x7fa2000fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 --2- 172.21.15.19:0/3831705229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2000fb570 0x7fa2000fb9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4004940 tx=0x7fa1f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 -- 172.21.15.19:0/3831705229 shutdown_connections 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 --2- 172.21.15.19:0/3831705229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2000fb570 0x7fa2000fb9c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 --2- 172.21.15.19:0/3831705229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa200103140 0x7fa200103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 -- 172.21.15.19:0/3831705229 >> 172.21.15.19:0/3831705229 conn(0x7fa200071560 msgr2=0x7fa200071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 -- 172.21.15.19:0/3831705229 shutdown_connections 2024-02-22T15:54:53.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.274+0000 7fa207852700 1 -- 172.21.15.19:0/3831705229 wait complete. 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa207852700 1 Processor -- start 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa207852700 1 -- start start 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa207852700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 0x7fa200113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa207852700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 0x7fa200113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa207852700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200114600 con 0x7fa200103140 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa207852700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20010d930 con 0x7fa2000fb570 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa204ded700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 0x7fa200113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:53.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa204ded700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 0x7fa200113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35282/0 (socket says 172.21.15.19:35282) 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa2055ee700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 0x7fa200113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa204ded700 1 -- 172.21.15.19:0/1773180086 learned_addr learned my addr 172.21.15.19:0/1773180086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa2055ee700 1 -- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 msgr2=0x7fa200113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa2055ee700 1 -- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 msgr2=0x7fa200113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa2055ee700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 0x7fa200113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.275+0000 7fa2055ee700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 0x7fa200113a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa204ded700 1 -- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 msgr2=0x7fa200113a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa204ded700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 0x7fa200113a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa204ded700 1 -- 172.21.15.19:0/1773180086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4005040 con 0x7fa200103140 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa204ded700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 0x7fa200113fb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400b3b0 tx=0x7fa1f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f400e040 con 0x7fa200103140 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1f400bc50 con 0x7fa200103140 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f4013420 con 0x7fa200103140 2024-02-22T15:54:53.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20010dbb0 con 0x7fa200103140 2024-02-22T15:54:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.276+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20010e0f0 con 0x7fa200103140 2024-02-22T15:54:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.277+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2001054d0 con 0x7fa200103140 2024-02-22T15:54:53.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.277+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fa1f401c430 con 0x7fa200103140 2024-02-22T15:54:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.278+0000 7fa1f1ffb700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1ec06cd40 0x7fa1ec06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.278+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa1f408cfa0 con 0x7fa200103140 2024-02-22T15:54:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.278+0000 7fa2055ee700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1ec06cd40 0x7fa1ec06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:53.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.278+0000 7fa2055ee700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1ec06cd40 0x7fa1ec06f1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009ce0 tx=0x7fa1fc009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:53.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.283+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa1f40c0960 con 0x7fa200103140 2024-02-22T15:54:53.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.570+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa20010f050 con 0x7fa200103140 2024-02-22T15:54:53.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.571+0000 7fa1f1ffb700 1 -- 172.21.15.19:0/1773180086 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fa1f405a9c0 con 0x7fa200103140 2024-02-22T15:54:53.551 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:54:53.552 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 10, 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1ec06cd40 msgr2=0x7fa1ec06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1ec06cd40 0x7fa1ec06f1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009ce0 tx=0x7fa1fc009f70 comp rx=0 tx=0).stop 2024-02-22T15:54:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 msgr2=0x7fa200113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 0x7fa200113fb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400b3b0 tx=0x7fa1f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 shutdown_connections 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1ec06cd40 0x7fa1ec06f1f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2000fb570 0x7fa200113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 --2- 172.21.15.19:0/1773180086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200103140 0x7fa200113fb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 >> 172.21.15.19:0/1773180086 conn(0x7fa200071560 msgr2=0x7fa2000f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 shutdown_connections 2024-02-22T15:54:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.573+0000 7fa207852700 1 -- 172.21.15.19:0/1773180086 wait complete. 2024-02-22T15:54:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 -- 172.21.15.19:0/471276350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 msgr2=0x7f16d8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 --2- 172.21.15.19:0/471276350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8102210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16c8004940 tx=0x7f16c8009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 -- 172.21.15.19:0/471276350 shutdown_connections 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 --2- 172.21.15.19:0/471276350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d81027e0 0x7f16d80fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 --2- 172.21.15.19:0/471276350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8102210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 -- 172.21.15.19:0/471276350 >> 172.21.15.19:0/471276350 conn(0x7f16d8071d90 msgr2=0x7f16d8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 -- 172.21.15.19:0/471276350 shutdown_connections 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 -- 172.21.15.19:0/471276350 wait complete. 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 Processor -- start 2024-02-22T15:54:53.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.676+0000 7f16dd845700 1 -- start start 2024-02-22T15:54:53.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16dd845700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16dd845700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d81027e0 0x7f16d8113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16dd845700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8114630 con 0x7f16d8101e40 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16dd845700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d810d960 con 0x7f16d81027e0 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35298/0 (socket says 172.21.15.19:35298) 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 -- 172.21.15.19:0/1660946767 learned_addr learned my addr 172.21.15.19:0/1660946767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d67fc700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d81027e0 0x7f16d8113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 -- 172.21.15.19:0/1660946767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d81027e0 msgr2=0x7f16d8113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d81027e0 0x7f16d8113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 -- 172.21.15.19:0/1660946767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c8005040 con 0x7f16d8101e40 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16d6ffd700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8113aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009fd0 tx=0x7f16c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16c800e040 con 0x7f16d8101e40 2024-02-22T15:54:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16c800bc50 con 0x7f16d8101e40 2024-02-22T15:54:53.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d810dbe0 con 0x7f16d8101e40 2024-02-22T15:54:53.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.677+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16c8013ea0 con 0x7f16d8101e40 2024-02-22T15:54:53.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.678+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d810e120 con 0x7f16d8101e40 2024-02-22T15:54:53.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.678+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80ff150 con 0x7f16d8101e40 2024-02-22T15:54:53.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.679+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f16c8013420 con 0x7f16d8101e40 2024-02-22T15:54:53.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.679+0000 7f16bf7fe700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16c006cd40 0x7f16c006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:53.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.680+0000 7f16d67fc700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16c006cd40 0x7f16c006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:53.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.680+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f16c808d790 con 0x7f16d8101e40 2024-02-22T15:54:53.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.680+0000 7f16d67fc700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16c006cd40 0x7f16c006f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009fd0 tx=0x7f16cc009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:53.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.684+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f16c805b060 con 0x7f16d8101e40 2024-02-22T15:54:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.931+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f16d804bed0 con 0x7f16d8101e40 2024-02-22T15:54:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.931+0000 7f16bf7fe700 1 -- 172.21.15.19:0/1660946767 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f16c808be60 con 0x7f16d8101e40 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:54:53.912 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:54:53.913 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:54:53.914 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:54:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16c006cd40 msgr2=0x7f16c006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16c006cd40 0x7f16c006f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009fd0 tx=0x7f16cc009f20 comp rx=0 tx=0).stop 2024-02-22T15:54:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 msgr2=0x7f16d8113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8113aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009fd0 tx=0x7f16c8009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 shutdown_connections 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16c006cd40 0x7f16c006f1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16d8101e40 0x7f16d8113aa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 --2- 172.21.15.19:0/1660946767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d81027e0 0x7f16d8113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.933+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 >> 172.21.15.19:0/1660946767 conn(0x7f16d8071d90 msgr2=0x7f16d80faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.934+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 shutdown_connections 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:53.934+0000 7f16dd845700 1 -- 172.21.15.19:0/1660946767 wait complete. 2024-02-22T15:54:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:54:54.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 -- 172.21.15.19:0/1071965508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 msgr2=0x7ffab0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 --2- 172.21.15.19:0/1071965508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0103320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004940 tx=0x7ffaa4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 -- 172.21.15.19:0/1071965508 shutdown_connections 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 --2- 172.21.15.19:0/1071965508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 --2- 172.21.15.19:0/1071965508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0103320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 -- 172.21.15.19:0/1071965508 >> 172.21.15.19:0/1071965508 conn(0x7ffab00f8ae0 msgr2=0x7ffab00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 -- 172.21.15.19:0/1071965508 shutdown_connections 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 -- 172.21.15.19:0/1071965508 wait complete. 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 Processor -- start 2024-02-22T15:54:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.038+0000 7ffab6c28700 1 -- start start 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab6c28700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab6c28700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab6c28700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0114500 con 0x7ffab0102f50 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab6c28700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab010d830 con 0x7ffab01038f0 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35316/0 (socket says 172.21.15.19:35316) 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 -- 172.21.15.19:0/3742299346 learned_addr learned my addr 172.21.15.19:0/3742299346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffaaffff700 1 -- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 msgr2=0x7ffab0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffaaffff700 1 -- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 msgr2=0x7ffab0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab0113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab0113eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-22T15:54:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 -- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 msgr2=0x7ffab0113eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab0113eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 -- 172.21.15.19:0/3742299346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4005040 con 0x7ffab0102f50 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.039+0000 7ffab49c4700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0113970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400b3b0 tx=0x7ffaa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.040+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa400e040 con 0x7ffab0102f50 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.040+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab010dab0 con 0x7ffab0102f50 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.040+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffaa400bc50 con 0x7ffab0102f50 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.040+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa4013420 con 0x7ffab0102f50 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.040+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab010dff0 con 0x7ffab0102f50 2024-02-22T15:54:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.041+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab0100260 con 0x7ffab0102f50 2024-02-22T15:54:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.041+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7ffaa401c430 con 0x7ffab0102f50 2024-02-22T15:54:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.042+0000 7ffaad7fa700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffaa006cd40 0x7ffaa006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.042+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffaa006cd40 0x7ffaa006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.042+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffaa408d020 con 0x7ffab0102f50 2024-02-22T15:54:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.042+0000 7ffaaffff700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffaa006cd40 0x7ffaa006f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffab010f1a0 tx=0x7ffa9c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:54.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.046+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ffaa405aa20 con 0x7ffab0102f50 2024-02-22T15:54:54.186 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:53 smithi062 ceph-mon[106861]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:54.186 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:53 smithi062 ceph-mon[106861]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:54.186 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:54.187 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:54.187 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:53 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1773180086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:54.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:53 smithi019 ceph-mon[173579]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:53 smithi019 ceph-mon[173579]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:53 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1773180086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:54:54.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.269+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab004bf70 con 0x7ffaa006cd40 2024-02-22T15:54:54.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.270+0000 7ffaad7fa700 1 -- 172.21.15.19:0/3742299346 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ffa9c004e30 con 0x7ffaa006cd40 2024-02-22T15:54:54.250 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:54:54.250 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "4/21 daemons upgraded", 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mon daemons", 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:54:54.251 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:54:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffaa006cd40 msgr2=0x7ffaa006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffaa006cd40 0x7ffaa006f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffab010f1a0 tx=0x7ffa9c009ee0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 msgr2=0x7ffab0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0113970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400b3b0 tx=0x7ffaa4009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 shutdown_connections 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffaa006cd40 0x7ffaa006f1f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffab0102f50 0x7ffab0113970 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 --2- 172.21.15.19:0/3742299346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab01038f0 0x7ffab0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 >> 172.21.15.19:0/3742299346 conn(0x7ffab00f8ae0 msgr2=0x7ffab00fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 shutdown_connections 2024-02-22T15:54:54.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.272+0000 7ffab6c28700 1 -- 172.21.15.19:0/3742299346 wait complete. 2024-02-22T15:54:54.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.375+0000 7f0a626fe700 1 -- 172.21.15.19:0/3471912504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 msgr2=0x7f0a5c0fe300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.375+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3471912504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fe300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c004940 tx=0x7f0a4c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.375+0000 7f0a626fe700 1 -- 172.21.15.19:0/3471912504 shutdown_connections 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.375+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3471912504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a5c0fe8d0 0x7f0a5c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.375+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3471912504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fe300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.375+0000 7f0a626fe700 1 -- 172.21.15.19:0/3471912504 >> 172.21.15.19:0/3471912504 conn(0x7f0a5c0f79d0 msgr2=0x7f0a5c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 -- 172.21.15.19:0/3471912504 shutdown_connections 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 -- 172.21.15.19:0/3471912504 wait complete. 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 Processor -- start 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 -- start start 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a5c0fe8d0 0x7f0a5c0fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0fc930 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.376+0000 7f0a626fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0fcaa0 con 0x7f0a5c0fe8d0 2024-02-22T15:54:54.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35330/0 (socket says 172.21.15.19:35330) 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 -- 172.21.15.19:0/3797977563 learned_addr learned my addr 172.21.15.19:0/3797977563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a5c0fe8d0 0x7f0a5c0fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 -- 172.21.15.19:0/3797977563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a5c0fe8d0 msgr2=0x7f0a5c0fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a5c0fe8d0 0x7f0a5c0fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 -- 172.21.15.19:0/3797977563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c005040 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a5bfff700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fbdf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009390 tx=0x7f0a4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a4c00e040 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a4c004290 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c119c60 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a4c01c630 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.377+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c11a0c0 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.378+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c104600 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.379+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f0a4c00bc50 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.379+0000 7f0a58ff9700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0a4406cd40 0x7f0a4406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:54:54.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.379+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0a4406cd40 0x7f0a4406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:54:54.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.379+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0a4c08d4a0 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.380+0000 7f0a5b7fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0a4406cd40 0x7f0a4406f1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007fd0 tx=0x7f0a50009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:54:54.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.384+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0a4c057400 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.666+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0a5c0fb5d0 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.666+0000 7f0a58ff9700 1 -- 172.21.15.19:0/3797977563 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f0a4c05aa20 con 0x7f0a5c0fdf30 2024-02-22T15:54:54.648 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_OK 2024-02-22T15:54:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0a4406cd40 msgr2=0x7f0a4406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0a4406cd40 0x7f0a4406f1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a50007fd0 tx=0x7f0a50009040 comp rx=0 tx=0).stop 2024-02-22T15:54:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 msgr2=0x7f0a5c0fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:54:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fbdf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009390 tx=0x7f0a4c009e30 comp rx=0 tx=0).stop 2024-02-22T15:54:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 shutdown_connections 2024-02-22T15:54:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0a4406cd40 0x7f0a4406f1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a5c0fdf30 0x7f0a5c0fbdf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 --2- 172.21.15.19:0/3797977563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a5c0fe8d0 0x7f0a5c0fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:54:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.668+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 >> 172.21.15.19:0/3797977563 conn(0x7f0a5c0f79d0 msgr2=0x7f0a5c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:54:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.669+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 shutdown_connections 2024-02-22T15:54:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:54:54.669+0000 7f0a626fe700 1 -- 172.21.15.19:0/3797977563 wait complete. 2024-02-22T15:54:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:54 smithi062 ceph-mon[106861]: pgmap v13: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:54 smithi062 ceph-mon[106861]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:54 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1660946767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:54:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:54 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3797977563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:54:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:54 smithi019 ceph-mon[173579]: pgmap v13: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:54 smithi019 ceph-mon[173579]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:54 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1660946767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:54:55.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:54 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3797977563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:54:56.169 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:55 smithi062 ceph-mon[106861]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:55 smithi019 ceph-mon[173579]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:54:57.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: pgmap v14: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:57.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:57.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:54:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:54:57.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:57.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: pgmap v14: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:54:57.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:57.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:57.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:57.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:54:57.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:57.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:54:57.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:54:57.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:58.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:57 smithi062 ceph-mon[106861]: Detected new or changed devices on smithi062 2024-02-22T15:54:58.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:57 smithi062 ceph-mon[106861]: Reconfiguring mon.smithi019 (monmap changed)... 2024-02-22T15:54:58.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:57 smithi062 ceph-mon[106861]: Reconfiguring daemon mon.smithi019 on smithi019 2024-02-22T15:54:58.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:57 smithi019 ceph-mon[173579]: Detected new or changed devices on smithi062 2024-02-22T15:54:58.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:57 smithi019 ceph-mon[173579]: Reconfiguring mon.smithi019 (monmap changed)... 2024-02-22T15:54:58.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:57 smithi019 ceph-mon[173579]: Reconfiguring daemon mon.smithi019 on smithi019 2024-02-22T15:54:58.906 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:58 smithi019 ceph-mon[173579]: pgmap v15: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:58.906 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:58.906 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:58.906 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:54:58.906 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:54:58.906 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:54:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:58 smithi062 ceph-mon[106861]: pgmap v15: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:54:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:54:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.yvjrkp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:54:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:54:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:59 smithi062 ceph-mon[106861]: Reconfiguring mgr.smithi019.yvjrkp (monmap changed)... 2024-02-22T15:55:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:54:59 smithi062 ceph-mon[106861]: Reconfiguring daemon mgr.smithi019.yvjrkp on smithi019 2024-02-22T15:55:00.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:59 smithi019 ceph-mon[173579]: Reconfiguring mgr.smithi019.yvjrkp (monmap changed)... 2024-02-22T15:55:00.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:54:59 smithi019 ceph-mon[173579]: Reconfiguring daemon mgr.smithi019.yvjrkp on smithi019 2024-02-22T15:55:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:00 smithi062 ceph-mon[106861]: pgmap v16: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:00 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:00 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:00 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:00 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:00 smithi019 ceph-mon[173579]: pgmap v16: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:00 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:00 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:00 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:00 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:01 smithi062 ceph-mon[106861]: Reconfiguring crash.smithi019 (monmap changed)... 2024-02-22T15:55:02.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:01 smithi062 ceph-mon[106861]: Reconfiguring daemon crash.smithi019 on smithi019 2024-02-22T15:55:02.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:01 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:02.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:01 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:02.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:01 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T15:55:02.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:01 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:02.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:01 smithi019 ceph-mon[173579]: Reconfiguring crash.smithi019 (monmap changed)... 2024-02-22T15:55:02.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:01 smithi019 ceph-mon[173579]: Reconfiguring daemon crash.smithi019 on smithi019 2024-02-22T15:55:02.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:01 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:02.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:01 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:02.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:01 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T15:55:02.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:01 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:03.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:02 smithi062 ceph-mon[106861]: pgmap v17: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:02 smithi062 ceph-mon[106861]: Reconfiguring osd.0 (monmap changed)... 2024-02-22T15:55:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:02 smithi062 ceph-mon[106861]: Reconfiguring daemon osd.0 on smithi019 2024-02-22T15:55:03.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:02 smithi019 ceph-mon[173579]: pgmap v17: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:02 smithi019 ceph-mon[173579]: Reconfiguring osd.0 (monmap changed)... 2024-02-22T15:55:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:02 smithi019 ceph-mon[173579]: Reconfiguring daemon osd.0 on smithi019 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: Reconfiguring osd.1 (monmap changed)... 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: Reconfiguring daemon osd.1 on smithi019 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: pgmap v18: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:04.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: Reconfiguring osd.1 (monmap changed)... 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: Reconfiguring daemon osd.1 on smithi019 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: pgmap v18: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:04.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:05.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:05.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:05.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:05 smithi019 ceph-mon[173579]: Reconfiguring osd.2 (monmap changed)... 2024-02-22T15:55:05.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T15:55:05.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:05.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:05 smithi019 ceph-mon[173579]: Reconfiguring daemon osd.2 on smithi019 2024-02-22T15:55:06.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:06.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:06.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:05 smithi062 ceph-mon[106861]: Reconfiguring osd.2 (monmap changed)... 2024-02-22T15:55:06.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T15:55:06.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:06.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:05 smithi062 ceph-mon[106861]: Reconfiguring daemon osd.2 on smithi019 2024-02-22T15:55:07.011 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:06 smithi019 ceph-mon[173579]: pgmap v19: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:07.011 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:07.011 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:07.011 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:07.011 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:07.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:06 smithi062 ceph-mon[106861]: pgmap v19: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:07.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:07.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:07.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:07.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:07.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:07 smithi019 ceph-mon[173579]: Reconfiguring mds.cephfs.smithi019.lkzblb (monmap changed)... 2024-02-22T15:55:07.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:07 smithi019 ceph-mon[173579]: Reconfiguring daemon mds.cephfs.smithi019.lkzblb on smithi019 2024-02-22T15:55:08.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:07 smithi062 ceph-mon[106861]: Reconfiguring mds.cephfs.smithi019.lkzblb (monmap changed)... 2024-02-22T15:55:08.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:07 smithi062 ceph-mon[106861]: Reconfiguring daemon mds.cephfs.smithi019.lkzblb on smithi019 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: pgmap v20: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: Reconfiguring mds.cephfs.smithi019.fesnwx (monmap changed)... 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:08.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:08 smithi019 ceph-mon[173579]: Reconfiguring daemon mds.cephfs.smithi019.fesnwx on smithi019 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: pgmap v20: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: Reconfiguring mds.cephfs.smithi019.fesnwx (monmap changed)... 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:09.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:08 smithi062 ceph-mon[106861]: Reconfiguring daemon mds.cephfs.smithi019.fesnwx on smithi019 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: pgmap v21: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: Reconfiguring crash.smithi062 (monmap changed)... 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:10.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:10 smithi062 ceph-mon[106861]: Reconfiguring daemon crash.smithi062 on smithi062 2024-02-22T15:55:10.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: pgmap v21: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:10.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:10.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:10.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: Reconfiguring crash.smithi062 (monmap changed)... 2024-02-22T15:55:10.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:10.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:10.744 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:10 smithi019 ceph-mon[173579]: Reconfiguring daemon crash.smithi062 on smithi062 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: Reconfiguring mgr.smithi062.ecubmv (monmap changed)... 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:12.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:11 smithi062 ceph-mon[106861]: Reconfiguring daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:55:12.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: Reconfiguring mgr.smithi062.ecubmv (monmap changed)... 2024-02-22T15:55:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi062.ecubmv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-22T15:55:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-22T15:55:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:12.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:11 smithi019 ceph-mon[173579]: Reconfiguring daemon mgr.smithi062.ecubmv on smithi062 2024-02-22T15:55:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:12 smithi062 ceph-mon[106861]: pgmap v22: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:12 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:12 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:12 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:55:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:12 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:55:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:12 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:12 smithi019 ceph-mon[173579]: pgmap v22: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:12 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:12 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:12 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-22T15:55:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:12 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-22T15:55:13.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:12 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:13.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:13 smithi062 ceph-mon[106861]: Reconfiguring mon.smithi062 (monmap changed)... 2024-02-22T15:55:13.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:13 smithi062 ceph-mon[106861]: Reconfiguring daemon mon.smithi062 on smithi062 2024-02-22T15:55:13.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:13 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:13.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:13 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:13.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:13 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T15:55:13.957 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:13 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:14.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:13 smithi019 ceph-mon[173579]: Reconfiguring mon.smithi062 (monmap changed)... 2024-02-22T15:55:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:13 smithi019 ceph-mon[173579]: Reconfiguring daemon mon.smithi062 on smithi062 2024-02-22T15:55:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:13 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:13 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:13 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T15:55:14.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:13 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:14 smithi062 ceph-mon[106861]: pgmap v23: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:14 smithi062 ceph-mon[106861]: Reconfiguring osd.3 (monmap changed)... 2024-02-22T15:55:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:14 smithi062 ceph-mon[106861]: Reconfiguring daemon osd.3 on smithi062 2024-02-22T15:55:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:14 smithi019 ceph-mon[173579]: pgmap v23: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:14 smithi019 ceph-mon[173579]: Reconfiguring osd.3 (monmap changed)... 2024-02-22T15:55:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:14 smithi019 ceph-mon[173579]: Reconfiguring daemon osd.3 on smithi062 2024-02-22T15:55:16.439 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: pgmap v24: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:16.439 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:16.439 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:16.439 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: Reconfiguring osd.4 (monmap changed)... 2024-02-22T15:55:16.439 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T15:55:16.440 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:16.440 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:16 smithi062 ceph-mon[106861]: Reconfiguring daemon osd.4 on smithi062 2024-02-22T15:55:16.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: pgmap v24: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:16.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:16.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:16.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: Reconfiguring osd.4 (monmap changed)... 2024-02-22T15:55:16.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T15:55:16.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:16.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:16 smithi019 ceph-mon[173579]: Reconfiguring daemon osd.4 on smithi062 2024-02-22T15:55:17.852 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:17 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:17.852 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:17 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:17.852 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:17 smithi062 ceph-mon[106861]: Reconfiguring osd.5 (monmap changed)... 2024-02-22T15:55:17.852 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:17 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T15:55:17.852 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:17 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:17.852 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:17 smithi062 ceph-mon[106861]: Reconfiguring daemon osd.5 on smithi062 2024-02-22T15:55:17.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:17 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:17 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:17 smithi019 ceph-mon[173579]: Reconfiguring osd.5 (monmap changed)... 2024-02-22T15:55:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:17 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T15:55:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:17 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:17.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:17 smithi019 ceph-mon[173579]: Reconfiguring daemon osd.5 on smithi062 2024-02-22T15:55:18.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:18 smithi019 ceph-mon[173579]: pgmap v25: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:18.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:18.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:18.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:18.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:18.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:19.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:18 smithi062 ceph-mon[106861]: pgmap v25: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:19.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:19.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:19.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:19.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:19.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:19.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:19 smithi062 ceph-mon[106861]: Reconfiguring mds.cephfs.smithi062.oyfagg (monmap changed)... 2024-02-22T15:55:19.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:19 smithi062 ceph-mon[106861]: Reconfiguring daemon mds.cephfs.smithi062.oyfagg on smithi062 2024-02-22T15:55:19.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:19.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:19.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:19.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:20.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:19 smithi019 ceph-mon[173579]: Reconfiguring mds.cephfs.smithi062.oyfagg (monmap changed)... 2024-02-22T15:55:20.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:19 smithi019 ceph-mon[173579]: Reconfiguring daemon mds.cephfs.smithi062.oyfagg on smithi062 2024-02-22T15:55:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:55:20.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:21.176 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:20 smithi062 ceph-mon[106861]: pgmap v26: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:21.176 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:20 smithi062 ceph-mon[106861]: Reconfiguring mds.cephfs.smithi062.eyhruo (monmap changed)... 2024-02-22T15:55:21.176 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:20 smithi062 ceph-mon[106861]: Reconfiguring daemon mds.cephfs.smithi062.eyhruo on smithi062 2024-02-22T15:55:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:20 smithi019 ceph-mon[173579]: pgmap v26: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:55:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:20 smithi019 ceph-mon[173579]: Reconfiguring mds.cephfs.smithi062.eyhruo (monmap changed)... 2024-02-22T15:55:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:20 smithi019 ceph-mon[173579]: Reconfiguring daemon mds.cephfs.smithi062.eyhruo on smithi062 2024-02-22T15:55:22.441 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: pgmap v27: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:22.441 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:22.441 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:22.441 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:22.441 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all mon 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]: dispatch 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]': finished 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi062"}]: dispatch 2024-02-22T15:55:22.442 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:22 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi062"}]': finished 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: pgmap v27: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all mon 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]: dispatch 2024-02-22T15:55:22.706 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]': finished 2024-02-22T15:55:22.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi062"}]: dispatch 2024-02-22T15:55:22.707 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:22 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi062"}]': finished 2024-02-22T15:55:24.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:23 smithi062 ceph-mon[106861]: Upgrade: Updating crash.smithi019 (1/2) 2024-02-22T15:55:24.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:23 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:24.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:23 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:24.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:23 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:24.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:23 smithi062 ceph-mon[106861]: Deploying daemon crash.smithi019 on smithi019 2024-02-22T15:55:24.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:23 smithi019 ceph-mon[173579]: Upgrade: Updating crash.smithi019 (1/2) 2024-02-22T15:55:24.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:23 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:24.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:23 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:24.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:23 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:24.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:23 smithi019 ceph-mon[173579]: Deploying daemon crash.smithi019 on smithi019 2024-02-22T15:55:24.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 -- 172.21.15.19:0/3388522061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 msgr2=0x7fea48103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:24.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 --2- 172.21.15.19:0/3388522061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48103320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea38004940 tx=0x7fea38009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:24.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 -- 172.21.15.19:0/3388522061 shutdown_connections 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 --2- 172.21.15.19:0/3388522061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea481038f0 0x7fea480fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 --2- 172.21.15.19:0/3388522061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48103320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 -- 172.21.15.19:0/3388522061 >> 172.21.15.19:0/3388522061 conn(0x7fea480f8ae0 msgr2=0x7fea480faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 -- 172.21.15.19:0/3388522061 shutdown_connections 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.783+0000 7fea4e696700 1 -- 172.21.15.19:0/3388522061 wait complete. 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.784+0000 7fea4e696700 1 Processor -- start 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.784+0000 7fea4e696700 1 -- start start 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.784+0000 7fea4e696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.784+0000 7fea4e696700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea481038f0 0x7fea480707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.784+0000 7fea4e696700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48070dd0 con 0x7fea48102f50 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.784+0000 7fea4e696700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48070f40 con 0x7fea481038f0 2024-02-22T15:55:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57746/0 (socket says 172.21.15.19:57746) 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 -- 172.21.15.19:0/1212796189 learned_addr learned my addr 172.21.15.19:0/1212796189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea477fe700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea481038f0 0x7fea480707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 -- 172.21.15.19:0/1212796189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea481038f0 msgr2=0x7fea480707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea481038f0 0x7fea480707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 -- 172.21.15.19:0/1212796189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38005040 con 0x7fea48102f50 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea47fff700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48070290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea38009180 tx=0x7fea38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3800e040 con 0x7fea48102f50 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea38013420 con 0x7fea48102f50 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea48119bb0 con 0x7fea48102f50 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3801c630 con 0x7fea48102f50 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.785+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4811a010 con 0x7fea48102f50 2024-02-22T15:55:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.786+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48100260 con 0x7fea48102f50 2024-02-22T15:55:24.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.787+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fea380139e0 con 0x7fea48102f50 2024-02-22T15:55:24.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.787+0000 7fea44ff9700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea3006cd40 0x7fea3006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:24.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.787+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fea3808d550 con 0x7fea48102f50 2024-02-22T15:55:24.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.787+0000 7fea477fe700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea3006cd40 0x7fea3006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:24.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.788+0000 7fea477fe700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea3006cd40 0x7fea3006f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea48071c90 tx=0x7fea3c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:24.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:24.792+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fea38057410 con 0x7fea48102f50 2024-02-22T15:55:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.016+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea480719d0 con 0x7fea3006cd40 2024-02-22T15:55:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.018+0000 7fea44ff9700 1 -- 172.21.15.19:0/1212796189 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fea480719d0 con 0x7fea3006cd40 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea3006cd40 msgr2=0x7fea3006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea3006cd40 0x7fea3006f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea48071c90 tx=0x7fea3c009ee0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 msgr2=0x7fea48070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48070290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea38009180 tx=0x7fea38009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 shutdown_connections 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea3006cd40 0x7fea3006f1f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea48102f50 0x7fea48070290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 --2- 172.21.15.19:0/1212796189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea481038f0 0x7fea480707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 >> 172.21.15.19:0/1212796189 conn(0x7fea480f8ae0 msgr2=0x7fea480fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 shutdown_connections 2024-02-22T15:55:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.020+0000 7fea4e696700 1 -- 172.21.15.19:0/1212796189 wait complete. 2024-02-22T15:55:25.010 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:55:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 -- 172.21.15.19:0/2565072925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a81027e0 msgr2=0x7f31a80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 --2- 172.21.15.19:0/2565072925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a81027e0 0x7f31a80fbea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f319c004940 tx=0x7f319c009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 -- 172.21.15.19:0/2565072925 shutdown_connections 2024-02-22T15:55:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 --2- 172.21.15.19:0/2565072925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a81027e0 0x7f31a80fbea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 --2- 172.21.15.19:0/2565072925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31a8101e40 0x7f31a8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 -- 172.21.15.19:0/2565072925 >> 172.21.15.19:0/2565072925 conn(0x7f31a8071d90 msgr2=0x7f31a8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.127+0000 7f31afe1c700 1 -- 172.21.15.19:0/2565072925 shutdown_connections 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 -- 172.21.15.19:0/2565072925 wait complete. 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 Processor -- start 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 -- start start 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 0x7f31a8113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31a81027e0 0x7f31a8113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a81145d0 con 0x7f31a8101e40 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31afe1c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a810d900 con 0x7f31a81027e0 2024-02-22T15:55:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31adbb8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 0x7f31a8113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31adbb8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 0x7f31a8113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57758/0 (socket says 172.21.15.19:57758) 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.128+0000 7f31adbb8700 1 -- 172.21.15.19:0/890562808 learned_addr learned my addr 172.21.15.19:0/890562808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31ad3b7700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31a81027e0 0x7f31a8113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31adbb8700 1 -- 172.21.15.19:0/890562808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31a81027e0 msgr2=0x7f31a8113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31adbb8700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31a81027e0 0x7f31a8113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31adbb8700 1 -- 172.21.15.19:0/890562808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c005040 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31adbb8700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 0x7f31a8113a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31a4002a50 tx=0x7f31a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a4007cc0 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31a400f040 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a810dbe0 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a4009540 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.129+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a810e120 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.130+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a80738b0 con 0x7f31a8101e40 2024-02-22T15:55:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.130+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f31a40096a0 con 0x7f31a8101e40 2024-02-22T15:55:25.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.131+0000 7f319a7fc700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f319406cd40 0x7f319406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.131+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f31a408bf50 con 0x7f31a8101e40 2024-02-22T15:55:25.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.131+0000 7f31ad3b7700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f319406cd40 0x7f319406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.132+0000 7f31ad3b7700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f319406cd40 0x7f319406f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31a810f270 tx=0x7f319c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:25.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.136+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f31a40bf960 con 0x7f31a8101e40 2024-02-22T15:55:25.154 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:24 smithi019 ceph-mon[173579]: pgmap v28: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:25.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:24 smithi062 ceph-mon[106861]: pgmap v28: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:55:25.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.362+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31a810f020 con 0x7f319406cd40 2024-02-22T15:55:25.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.363+0000 7f319a7fc700 1 -- 172.21.15.19:0/890562808 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f31a810f020 con 0x7f319406cd40 2024-02-22T15:55:25.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f319406cd40 msgr2=0x7f319406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f319406cd40 0x7f319406f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31a810f270 tx=0x7f319c009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:25.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 msgr2=0x7f31a8113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 0x7f31a8113a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31a4002a50 tx=0x7f31a40046b0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 shutdown_connections 2024-02-22T15:55:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f319406cd40 0x7f319406f1f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a8101e40 0x7f31a8113a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 --2- 172.21.15.19:0/890562808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31a81027e0 0x7f31a8113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 >> 172.21.15.19:0/890562808 conn(0x7f31a8071d90 msgr2=0x7f31a8065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 shutdown_connections 2024-02-22T15:55:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.365+0000 7f31afe1c700 1 -- 172.21.15.19:0/890562808 wait complete. 2024-02-22T15:55:25.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/1211524896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 msgr2=0x7f8bc00fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/1211524896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc00fcf50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8bac004940 tx=0x7f8bac009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:25.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/1211524896 shutdown_connections 2024-02-22T15:55:25.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/1211524896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc00fcf50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/1211524896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc0102f40 0x7f8bc0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/1211524896 >> 172.21.15.19:0/1211524896 conn(0x7f8bc00f8af0 msgr2=0x7f8bc00faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.468+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/1211524896 shutdown_connections 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/1211524896 wait complete. 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 Processor -- start 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 -- start start 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc0102f40 0x7f8bc0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0114510 con 0x7f8bc01038e0 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.469+0000 7f8bc6ee9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc010d840 con 0x7f8bc0102f40 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc0113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57776/0 (socket says 172.21.15.19:57776) 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 -- 172.21.15.19:0/3372222039 learned_addr learned my addr 172.21.15.19:0/3372222039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bc4c85700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc0102f40 0x7f8bc0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 -- 172.21.15.19:0/3372222039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc0102f40 msgr2=0x7f8bc0113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc0102f40 0x7f8bc0113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 -- 172.21.15.19:0/3372222039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac005040 con 0x7f8bc01038e0 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbffff700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc0113ec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009180 tx=0x7f8bac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bac00e040 con 0x7f8bc01038e0 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc010dac0 con 0x7f8bc01038e0 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8bac004290 con 0x7f8bc01038e0 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bac01c3f0 con 0x7f8bc01038e0 2024-02-22T15:55:25.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.470+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc010e000 con 0x7f8bc01038e0 2024-02-22T15:55:25.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.471+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc0105410 con 0x7f8bc01038e0 2024-02-22T15:55:25.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.472+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f8bac013770 con 0x7f8bc01038e0 2024-02-22T15:55:25.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.472+0000 7f8bbd7fa700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8bb006cd40 0x7f8bb006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.472+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8bac08d530 con 0x7f8bc01038e0 2024-02-22T15:55:25.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.472+0000 7f8bc4c85700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8bb006cd40 0x7f8bb006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.473+0000 7f8bc4c85700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8bb006cd40 0x7f8bb006f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40041d0 tx=0x7f8bb40043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:25.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.477+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8bac05af30 con 0x7f8bc01038e0 2024-02-22T15:55:25.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.719+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bc0109ee0 con 0x7f8bb006cd40 2024-02-22T15:55:25.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.730+0000 7f8bbd7fa700 1 -- 172.21.15.19:0/3372222039 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f8bc0109ee0 con 0x7f8bb006cd40 2024-02-22T15:55:25.710 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:55:25.710 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (2m) 46s ago 13m 16.6M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:55:25.710 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (13m) 46s ago 13m 6656k - 17.2.7-527-g6ea2a55e e2e4ad29904c dce38493b069 2024-02-22T15:55:25.710 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (12m) 33s ago 12m 6660k - 17.2.7-527-g6ea2a55e e2e4ad29904c ae9470556e6d 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (2m) 46s ago 13m 78.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (7m) 46s ago 7m 114M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (7m) 46s ago 7m 67.3M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (7m) 33s ago 7m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (7m) 33s ago 7m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (4m) 46s ago 15m 468M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (3m) 33s ago 12m 446M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (63s) 46s ago 15m 27.6M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (36s) 33s ago 12m 26.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (3m) 46s ago 13m 18.1M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (3m) 33s ago 12m 16.8M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (11m) 46s ago 11m 224M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c a17e601b37ec 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (11m) 46s ago 11m 243M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (10m) 46s ago 10m 196M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (10m) 33s ago 10m 259M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:55:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (10m) 33s ago 9m 229M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:55:25.712 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (9m) 33s ago 9m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:55:25.712 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (3m) 46s ago 13m 59.6M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:55:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8bb006cd40 msgr2=0x7f8bb006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8bb006cd40 0x7f8bb006f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40041d0 tx=0x7f8bb40043c0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 msgr2=0x7f8bc0113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc0113ec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009180 tx=0x7f8bac009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 shutdown_connections 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8bb006cd40 0x7f8bb006f1f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bc0102f40 0x7f8bc0113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 --2- 172.21.15.19:0/3372222039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc01038e0 0x7f8bc0113ec0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 >> 172.21.15.19:0/3372222039 conn(0x7f8bc00f8af0 msgr2=0x7f8bc00fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 shutdown_connections 2024-02-22T15:55:25.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.732+0000 7f8bc6ee9700 1 -- 172.21.15.19:0/3372222039 wait complete. 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.841+0000 7f9b53efa700 1 -- 172.21.15.19:0/3312515080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c101e40 msgr2=0x7f9b4c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.841+0000 7f9b53efa700 1 --2- 172.21.15.19:0/3312515080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c101e40 0x7f9b4c102210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b48004940 tx=0x7f9b48009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.841+0000 7f9b53efa700 1 -- 172.21.15.19:0/3312515080 shutdown_connections 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.841+0000 7f9b53efa700 1 --2- 172.21.15.19:0/3312515080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c0fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.841+0000 7f9b53efa700 1 --2- 172.21.15.19:0/3312515080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c101e40 0x7f9b4c102210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.841+0000 7f9b53efa700 1 -- 172.21.15.19:0/3312515080 >> 172.21.15.19:0/3312515080 conn(0x7f9b4c0f79d0 msgr2=0x7f9b4c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 -- 172.21.15.19:0/3312515080 shutdown_connections 2024-02-22T15:55:25.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 -- 172.21.15.19:0/3312515080 wait complete. 2024-02-22T15:55:25.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 Processor -- start 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 -- start start 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b4c101e40 0x7f9b4c102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c1037e0 con 0x7f9b4c1027e0 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.842+0000 7f9b53efa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c103950 con 0x7f9b4c101e40 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c1031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57794/0 (socket says 172.21.15.19:57794) 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 -- 172.21.15.19:0/2852674459 learned_addr learned my addr 172.21.15.19:0/2852674459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 -- 172.21.15.19:0/2852674459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b4c101e40 msgr2=0x7f9b4c102ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b4c101e40 0x7f9b4c102ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 -- 172.21.15.19:0/2852674459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b48005040 con 0x7f9b4c1027e0 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b51495700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c1031e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c002a50 tx=0x7f9b3c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:25.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c007cc0 con 0x7f9b4c1027e0 2024-02-22T15:55:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b3c00f040 con 0x7f9b4c1027e0 2024-02-22T15:55:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c119c60 con 0x7f9b4c1027e0 2024-02-22T15:55:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c00d9a0 con 0x7f9b4c1027e0 2024-02-22T15:55:25.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.843+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c11a140 con 0x7f9b4c1027e0 2024-02-22T15:55:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.844+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c0fb5d0 con 0x7f9b4c1027e0 2024-02-22T15:55:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.845+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f9b3c00dc60 con 0x7f9b4c1027e0 2024-02-22T15:55:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.845+0000 7f9b427fc700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9b3806cd40 0x7f9b3806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.845+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9b3c08c060 con 0x7f9b4c1027e0 2024-02-22T15:55:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.846+0000 7f9b51c96700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9b3806cd40 0x7f9b3806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:25.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.846+0000 7f9b51c96700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9b3806cd40 0x7f9b3806f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b48004f90 tx=0x7f9b48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:25.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:25.850+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9b3c055ea0 con 0x7f9b4c1027e0 2024-02-22T15:55:26.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.153+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b4c104480 con 0x7f9b4c1027e0 2024-02-22T15:55:26.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.153+0000 7f9b427fc700 1 -- 172.21.15.19:0/2852674459 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f9b3c0594c0 con 0x7f9b4c1027e0 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:26.134 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 10, 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9b3806cd40 msgr2=0x7f9b3806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9b3806cd40 0x7f9b3806f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b48004f90 tx=0x7f9b48009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 msgr2=0x7f9b4c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c1031e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c002a50 tx=0x7f9b3c004600 comp rx=0 tx=0).stop 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 shutdown_connections 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9b3806cd40 0x7f9b3806f1f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b4c101e40 0x7f9b4c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 --2- 172.21.15.19:0/2852674459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b4c1027e0 0x7f9b4c1031e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 >> 172.21.15.19:0/2852674459 conn(0x7f9b4c0f79d0 msgr2=0x7f9b4c066070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 shutdown_connections 2024-02-22T15:55:26.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.155+0000 7f9b53efa700 1 -- 172.21.15.19:0/2852674459 wait complete. 2024-02-22T15:55:26.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 -- 172.21.15.19:0/1928712510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50101e40 msgr2=0x7f0c50102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 --2- 172.21.15.19:0/1928712510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50101e40 0x7f0c50102210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c44004940 tx=0x7f0c44009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:26.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 -- 172.21.15.19:0/1928712510 shutdown_connections 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 --2- 172.21.15.19:0/1928712510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c501027e0 0x7f0c500fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 --2- 172.21.15.19:0/1928712510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50101e40 0x7f0c50102210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 -- 172.21.15.19:0/1928712510 >> 172.21.15.19:0/1928712510 conn(0x7f0c50071d90 msgr2=0x7f0c50072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 -- 172.21.15.19:0/1928712510 shutdown_connections 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.263+0000 7f0c586cc700 1 -- 172.21.15.19:0/1928712510 wait complete. 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c586cc700 1 Processor -- start 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c586cc700 1 -- start start 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c586cc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c50101e40 0x7f0c50113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c586cc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 0x7f0c50113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c586cc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c501144f0 con 0x7f0c501027e0 2024-02-22T15:55:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c586cc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5010d820 con 0x7f0c50101e40 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 0x7f0c50113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 0x7f0c50113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57812/0 (socket says 172.21.15.19:57812) 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 -- 172.21.15.19:0/2638146521 learned_addr learned my addr 172.21.15.19:0/2638146521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c56468700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c50101e40 0x7f0c50113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 -- 172.21.15.19:0/2638146521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c50101e40 msgr2=0x7f0c50113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c50101e40 0x7f0c50113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 -- 172.21.15.19:0/2638146521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44005040 con 0x7f0c501027e0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.264+0000 7f0c55c67700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 0x7f0c50113ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c002a50 tx=0x7f0c4c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.265+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c4c007cc0 con 0x7f0c501027e0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.265+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c4c00f040 con 0x7f0c501027e0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.265+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5010db00 con 0x7f0c501027e0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.265+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c4c0095d0 con 0x7f0c501027e0 2024-02-22T15:55:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.265+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5010e040 con 0x7f0c501027e0 2024-02-22T15:55:26.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.266+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c500738b0 con 0x7f0c501027e0 2024-02-22T15:55:26.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.266+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f0c4c009730 con 0x7f0c501027e0 2024-02-22T15:55:26.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.266+0000 7f0c42ffd700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0c3c06cd40 0x7f0c3c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.267+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0c4c08be50 con 0x7f0c501027e0 2024-02-22T15:55:26.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.267+0000 7f0c56468700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0c3c06cd40 0x7f0c3c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.267+0000 7f0c56468700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0c3c06cd40 0x7f0c3c06f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0c44004f90 tx=0x7f0c44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:26.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.271+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0c4c059850 con 0x7f0c501027e0 2024-02-22T15:55:26.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.536+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c5004bed0 con 0x7f0c501027e0 2024-02-22T15:55:26.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.537+0000 7f0c42ffd700 1 -- 172.21.15.19:0/2638146521 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f0c4c0593e0 con 0x7f0c501027e0 2024-02-22T15:55:26.517 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:55:26.517 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:55:26.517 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:55:26.517 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:55:26.518 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:55:26.519 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:26.520 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:26.520 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:26.520 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:26.520 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:55:26.520 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0c3c06cd40 msgr2=0x7f0c3c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0c3c06cd40 0x7f0c3c06f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0c44004f90 tx=0x7f0c44009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 msgr2=0x7f0c50113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 0x7f0c50113ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c002a50 tx=0x7f0c4c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 shutdown_connections 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0c3c06cd40 0x7f0c3c06f1f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c50101e40 0x7f0c50113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 --2- 172.21.15.19:0/2638146521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c501027e0 0x7f0c50113ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 >> 172.21.15.19:0/2638146521 conn(0x7f0c50071d90 msgr2=0x7f0c501052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 shutdown_connections 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.539+0000 7f0c586cc700 1 -- 172.21.15.19:0/2638146521 wait complete. 2024-02-22T15:55:26.521 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:55:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 -- 172.21.15.19:0/3755226709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 msgr2=0x7f80ac0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 --2- 172.21.15.19:0/3755226709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0fcf60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80a0004940 tx=0x7f80a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 -- 172.21.15.19:0/3755226709 shutdown_connections 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 --2- 172.21.15.19:0/3755226709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0fcf60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 --2- 172.21.15.19:0/3755226709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80ac102f50 0x7f80ac103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 -- 172.21.15.19:0/3755226709 >> 172.21.15.19:0/3755226709 conn(0x7f80ac0f8ae0 msgr2=0x7f80ac0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 -- 172.21.15.19:0/3755226709 shutdown_connections 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 -- 172.21.15.19:0/3755226709 wait complete. 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.648+0000 7f80b2941700 1 Processor -- start 2024-02-22T15:55:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80b2941700 1 -- start start 2024-02-22T15:55:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80b2941700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80ac102f50 0x7f80ac070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80b2941700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80b2941700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac070dd0 con 0x7f80ac1038f0 2024-02-22T15:55:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80b2941700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac070f40 con 0x7f80ac102f50 2024-02-22T15:55:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57832/0 (socket says 172.21.15.19:57832) 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 -- 172.21.15.19:0/747719448 learned_addr learned my addr 172.21.15.19:0/747719448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80abfff700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80ac102f50 0x7f80ac070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 -- 172.21.15.19:0/747719448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80ac102f50 msgr2=0x7f80ac070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80ac102f50 0x7f80ac070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 -- 172.21.15.19:0/747719448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0005040 con 0x7f80ac1038f0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.649+0000 7f80ab7fe700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0707d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.650+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a0021070 con 0x7f80ac1038f0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.650+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80a00093b0 con 0x7f80ac1038f0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.650+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac119b60 con 0x7f80ac1038f0 2024-02-22T15:55:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.650+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a000e070 con 0x7f80ac1038f0 2024-02-22T15:55:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.650+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac119f60 con 0x7f80ac1038f0 2024-02-22T15:55:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.651+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac105420 con 0x7f80ac1038f0 2024-02-22T15:55:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.651+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f80a001c670 con 0x7f80ac1038f0 2024-02-22T15:55:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.651+0000 7f80a8ff9700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f809406cd40 0x7f809406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.652+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f80a008d3b0 con 0x7f80ac1038f0 2024-02-22T15:55:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.652+0000 7f80abfff700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f809406cd40 0x7f809406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.652+0000 7f80abfff700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f809406cd40 0x7f809406f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f809c009b80 tx=0x7f809c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:26.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.656+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f80a0057270 con 0x7f80ac1038f0 2024-02-22T15:55:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.879+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80ac0719d0 con 0x7f809406cd40 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.880+0000 7f80a8ff9700 1 -- 172.21.15.19:0/747719448 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f80ac0719d0 con 0x7f809406cd40 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "4/21 daemons upgraded", 2024-02-22T15:55:26.860 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading crash daemons", 2024-02-22T15:55:26.861 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:55:26.861 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f809406cd40 msgr2=0x7f809406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f809406cd40 0x7f809406f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f809c009b80 tx=0x7f809c00b040 comp rx=0 tx=0).stop 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 msgr2=0x7f80ac0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0707d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 shutdown_connections 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f809406cd40 0x7f809406f1f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80ac102f50 0x7f80ac070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 --2- 172.21.15.19:0/747719448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80ac1038f0 0x7f80ac0707d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 >> 172.21.15.19:0/747719448 conn(0x7f80ac0f8ae0 msgr2=0x7f80ac0fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 shutdown_connections 2024-02-22T15:55:26.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.882+0000 7f80b2941700 1 -- 172.21.15.19:0/747719448 wait complete. 2024-02-22T15:55:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 -- 172.21.15.19:0/1591260441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c1027f0 msgr2=0x7fed9c0fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 --2- 172.21.15.19:0/1591260441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c1027f0 0x7fed9c0fbe60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed90004940 tx=0x7fed90009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 -- 172.21.15.19:0/1591260441 shutdown_connections 2024-02-22T15:55:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 --2- 172.21.15.19:0/1591260441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c1027f0 0x7fed9c0fbe60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 --2- 172.21.15.19:0/1591260441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c101e50 0x7fed9c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 -- 172.21.15.19:0/1591260441 >> 172.21.15.19:0/1591260441 conn(0x7fed9c0f79e0 msgr2=0x7fed9c0f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 -- 172.21.15.19:0/1591260441 shutdown_connections 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 -- 172.21.15.19:0/1591260441 wait complete. 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 Processor -- start 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.986+0000 7feda0b1f700 1 -- start start 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7feda0b1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 0x7fed9c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7feda0b1f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c1027f0 0x7fed9c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7feda0b1f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1144c0 con 0x7fed9c101e50 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7feda0b1f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c10d7f0 con 0x7fed9c1027f0 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 0x7fed9c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 0x7fed9c113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57846/0 (socket says 172.21.15.19:57846) 2024-02-22T15:55:26.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 -- 172.21.15.19:0/3520887817 learned_addr learned my addr 172.21.15.19:0/3520887817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed99d9b700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c1027f0 0x7fed9c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 -- 172.21.15.19:0/3520887817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c1027f0 msgr2=0x7fed9c113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c1027f0 0x7fed9c113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 -- 172.21.15.19:0/3520887817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed90005040 con 0x7fed9c101e50 2024-02-22T15:55:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed99d9b700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c1027f0 0x7fed9c113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:55:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed9a59c700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 0x7fed9c113930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00eb40 tx=0x7fed8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.987+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8c007cc0 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.988+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c10da70 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.988+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed8c00f040 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.988+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8c00c990 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.988+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c10df80 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.989+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c064b00 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.989+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fed8c00cc00 con 0x7fed9c101e50 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.989+0000 7fed8affd700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed8406cd40 0x7fed8406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.989+0000 7fed99d9b700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed8406cd40 0x7fed8406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.989+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fed8c08be40 con 0x7fed9c101e50 2024-02-22T15:55:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.990+0000 7fed99d9b700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed8406cd40 0x7fed8406f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed90000c00 tx=0x7fed90009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:26.994+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fed8c0597c0 con 0x7fed9c101e50 2024-02-22T15:55:27.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: pgmap v29: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2852674459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2638146521' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:27.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:26 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:27.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:27.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: pgmap v29: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:27.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:27.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:27.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2852674459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:27.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2638146521' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:55:27.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:27.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:26 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.278+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fed9c108df0 con 0x7fed9c101e50 2024-02-22T15:55:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.278+0000 7fed8affd700 1 -- 172.21.15.19:0/3520887817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fed8c059350 con 0x7fed9c101e50 2024-02-22T15:55:27.258 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_OK 2024-02-22T15:55:27.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed8406cd40 msgr2=0x7fed8406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed8406cd40 0x7fed8406f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed90000c00 tx=0x7fed90009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 msgr2=0x7fed9c113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 0x7fed9c113930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00eb40 tx=0x7fed8c00b040 comp rx=0 tx=0).stop 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 shutdown_connections 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed8406cd40 0x7fed8406f1f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c101e50 0x7fed9c113930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 --2- 172.21.15.19:0/3520887817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed9c1027f0 0x7fed9c113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.280+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 >> 172.21.15.19:0/3520887817 conn(0x7fed9c0f79e0 msgr2=0x7fed9c065ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.281+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 shutdown_connections 2024-02-22T15:55:27.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:27.281+0000 7feda0b1f700 1 -- 172.21.15.19:0/3520887817 wait complete. 2024-02-22T15:55:28.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:27 smithi062 ceph-mon[106861]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:28.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:27 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3520887817' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:55:28.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:27 smithi019 ceph-mon[173579]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:28.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:27 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3520887817' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:55:29.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:28 smithi062 ceph-mon[106861]: pgmap v30: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:29.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:28 smithi062 ceph-mon[106861]: Upgrade: Updating crash.smithi062 (2/2) 2024-02-22T15:55:29.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:29.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:29.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:29.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:28 smithi062 ceph-mon[106861]: Deploying daemon crash.smithi062 on smithi062 2024-02-22T15:55:29.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:28 smithi019 ceph-mon[173579]: pgmap v30: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:29.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:28 smithi019 ceph-mon[173579]: Upgrade: Updating crash.smithi062 (2/2) 2024-02-22T15:55:29.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:29.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi062", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-22T15:55:29.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:29.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:28 smithi019 ceph-mon[173579]: Deploying daemon crash.smithi062 on smithi062 2024-02-22T15:55:31.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:30 smithi062 ceph-mon[106861]: pgmap v31: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:31.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:30 smithi019 ceph-mon[173579]: pgmap v31: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:32.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:32 smithi062 ceph-mon[106861]: pgmap v32: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:32.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:32.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:32.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:32.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:32 smithi019 ceph-mon[173579]: pgmap v32: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:32.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:32.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:32.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:34.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:34.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:34 smithi062 ceph-mon[106861]: pgmap v33: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:34 smithi019 ceph-mon[173579]: pgmap v33: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:35.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:36 smithi062 ceph-mon[106861]: pgmap v34: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-22T15:55:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:36 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:36 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:36 smithi019 ceph-mon[173579]: pgmap v34: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-02-22T15:55:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:36 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:36 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:38 smithi019 ceph-mon[173579]: pgmap v35: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:38.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:38 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:38 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:38 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:38 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:38 smithi062 ceph-mon[106861]: pgmap v35: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:38.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:38 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:38 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:38 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:38.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:38 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:40.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:40 smithi062 ceph-mon[106861]: pgmap v36: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:40 smithi019 ceph-mon[173579]: pgmap v36: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]: dispatch 2024-02-22T15:55:42.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]': finished 2024-02-22T15:55:42.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi062"}]: dispatch 2024-02-22T15:55:42.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi062"}]': finished 2024-02-22T15:55:42.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-22T15:55:42.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]: dispatch 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]': finished 2024-02-22T15:55:42.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi062"}]: dispatch 2024-02-22T15:55:42.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi062"}]': finished 2024-02-22T15:55:42.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all crash 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: Upgrade: osd.0 is safe to restart 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: pgmap v37: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T15:55:43.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:42 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:43.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all crash 2024-02-22T15:55:43.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-22T15:55:43.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: Upgrade: osd.0 is safe to restart 2024-02-22T15:55:43.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: pgmap v37: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:43.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:43.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-22T15:55:43.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:42 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:55:44.110 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:43 smithi019 ceph-mon[173579]: Upgrade: Updating osd.0 2024-02-22T15:55:44.110 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:43 smithi019 ceph-mon[173579]: Deploying daemon osd.0 on smithi019 2024-02-22T15:55:44.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:43 smithi062 ceph-mon[106861]: Upgrade: Updating osd.0 2024-02-22T15:55:44.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:43 smithi062 ceph-mon[106861]: Deploying daemon osd.0 on smithi019 2024-02-22T15:55:45.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:44 smithi062 ceph-mon[106861]: pgmap v38: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:45.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:44 smithi019 ceph-mon[173579]: pgmap v38: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:55:46.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:45 smithi019 systemd[1]: Stopping Ceph osd.0 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:55:46.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:46 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[91179]: 2024-02-22T15:55:46.113+0000 7effa56b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:55:46.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:46 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[91179]: 2024-02-22T15:55:46.113+0000 7effa56b0700 -1 osd.0 50 *** Got signal Terminated *** 2024-02-22T15:55:46.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:46 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[91179]: 2024-02-22T15:55:46.113+0000 7effa56b0700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T15:55:47.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:46 smithi062 ceph-mon[106861]: pgmap v39: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:47.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:46 smithi062 ceph-mon[106861]: osd.0 marked itself down and dead 2024-02-22T15:55:47.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:46 smithi019 ceph-mon[173579]: pgmap v39: 65 pgs: 65 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T15:55:47.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:46 smithi019 ceph-mon[173579]: osd.0 marked itself down and dead 2024-02-22T15:55:47.935 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:47 smithi019 bash[188301]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0 2024-02-22T15:55:48.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:47 smithi062 ceph-mon[106861]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:55:48.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:47 smithi062 ceph-mon[106861]: failed to encode map e51 with expected crc 2024-02-22T15:55:48.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:47 smithi062 ceph-mon[106861]: osdmap e51: 6 total, 5 up, 6 in 2024-02-22T15:55:48.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:47 smithi019 ceph-mon[173579]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:55:48.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:47 smithi019 ceph-mon[173579]: failed to encode map e51 with expected crc 2024-02-22T15:55:48.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:47 smithi019 ceph-mon[173579]: osdmap e51: 6 total, 5 up, 6 in 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: osdmap e52: 6 total, 5 up, 6 in 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:49.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: osdmap e52: 6 total, 5 up, 6 in 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: failed to encode map e52 with expected crc 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:49.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:55:49.988 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:49 smithi019 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.0.service: Succeeded. 2024-02-22T15:55:49.988 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:49 smithi019 systemd[1]: Stopped Ceph osd.0 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:55:49.988 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:49 smithi019 systemd[1]: Starting Ceph osd.0 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:55:50.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:50 smithi019 ceph-mon[173579]: pgmap v43: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:50.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:50 smithi019 ceph-mon[173579]: Health check failed: Degraded data redundancy: 63/318 objects degraded (19.811%), 19 pgs degraded (PG_DEGRADED) 2024-02-22T15:55:50.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:50 smithi062 ceph-mon[106861]: pgmap v43: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:50.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:50 smithi062 ceph-mon[106861]: Health check failed: Degraded data redundancy: 63/318 objects degraded (19.811%), 19 pgs degraded (PG_DEGRADED) 2024-02-22T15:55:51.166 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-22T15:55:51.167 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-22T15:55:51.167 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-22T15:55:51.167 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-22T15:55:51.167 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-22T15:55:51.167 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-22T15:55:51.167 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:50 smithi019 bash[188576]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-22T15:55:52.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:52 smithi019 ceph-mon[173579]: pgmap v44: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:52.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:51 smithi019 ceph-osd[188881]: -- 172.21.15.19:0/2655724990 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55acf61184e0 con 0x55acf60e9000 2024-02-22T15:55:52.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:52 smithi019 bash[188843]: b15c807d7724538f471f0178f22aea05904610ea13c4197e6c357de4070bbd1b 2024-02-22T15:55:52.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:52 smithi019 systemd[1]: Started Ceph osd.0 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:55:52.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:52 smithi062 ceph-mon[106861]: pgmap v44: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:52.743 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:52 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T15:55:52.501+0000 7fd678dec640 -1 Falling back to public interface 2024-02-22T15:55:53.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:53.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:53.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:53.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:53.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:53.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:55:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:54 smithi062 ceph-mon[106861]: pgmap v45: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/318 objects degraded (19.811%) 2024-02-22T15:55:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:54 smithi019 ceph-mon[173579]: pgmap v45: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 63/318 objects degraded (19.811%) 2024-02-22T15:55:57.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:56 smithi062 ceph-mon[106861]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:56 smithi019 ceph-mon[173579]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:57.243 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:56 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T15:55:56.976+0000 7fd678dec640 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-22T15:55:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 -- 172.21.15.19:0/1330202779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 msgr2=0x7fa4200fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 --2- 172.21.15.19:0/1330202779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa4200fbb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa414004940 tx=0x7fa414009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 -- 172.21.15.19:0/1330202779 shutdown_connections 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 --2- 172.21.15.19:0/1330202779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa4200fbb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 --2- 172.21.15.19:0/1330202779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4201016b0 0x7fa420101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 -- 172.21.15.19:0/1330202779 >> 172.21.15.19:0/1330202779 conn(0x7fa420071560 msgr2=0x7fa420071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 -- 172.21.15.19:0/1330202779 shutdown_connections 2024-02-22T15:55:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.397+0000 7fa425727700 1 -- 172.21.15.19:0/1330202779 wait complete. 2024-02-22T15:55:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.398+0000 7fa425727700 1 Processor -- start 2024-02-22T15:55:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.398+0000 7fa425727700 1 -- start start 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa425727700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa420113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa425727700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4201016b0 0x7fa420113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa425727700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa420114500 con 0x7fa4200fb6c0 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa425727700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42010d830 con 0x7fa4201016b0 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa420113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa420113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53402/0 (socket says 172.21.15.19:53402) 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 -- 172.21.15.19:0/1311201292 learned_addr learned my addr 172.21.15.19:0/1311201292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41e7fc700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4201016b0 0x7fa420113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 -- 172.21.15.19:0/1311201292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4201016b0 msgr2=0x7fa420113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4201016b0 0x7fa420113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 -- 172.21.15.19:0/1311201292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa414005040 con 0x7fa4200fb6c0 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.399+0000 7fa41effd700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa420113970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa41000a6e0 tx=0x7fa41000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.400+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa41000f040 con 0x7fa4200fb6c0 2024-02-22T15:55:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.400+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa41000de50 con 0x7fa4200fb6c0 2024-02-22T15:55:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.400+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42010db10 con 0x7fa4200fb6c0 2024-02-22T15:55:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.400+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa410014e80 con 0x7fa4200fb6c0 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.400+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42010e050 con 0x7fa4200fb6c0 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.401+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420103b30 con 0x7fa4200fb6c0 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.401+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fa4100133e0 con 0x7fa4200fb6c0 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.402+0000 7fa4077fe700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa40806cd40 0x7fa40806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.402+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa41008be50 con 0x7fa4200fb6c0 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.402+0000 7fa41e7fc700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa40806cd40 0x7fa40806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.402+0000 7fa41e7fc700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa40806cd40 0x7fa40806f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa42010f1a0 tx=0x7fa414009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.410+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa4100598f0 con 0x7fa4200fb6c0 2024-02-22T15:55:57.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.640+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa42004bf40 con 0x7fa40806cd40 2024-02-22T15:55:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.642+0000 7fa4077fe700 1 -- 172.21.15.19:0/1311201292 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa42004bf40 con 0x7fa40806cd40 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa40806cd40 msgr2=0x7fa40806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa40806cd40 0x7fa40806f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa42010f1a0 tx=0x7fa414009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 msgr2=0x7fa420113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa420113970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa41000a6e0 tx=0x7fa41000c040 comp rx=0 tx=0).stop 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 shutdown_connections 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa40806cd40 0x7fa40806f1f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4200fb6c0 0x7fa420113970 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 --2- 172.21.15.19:0/1311201292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4201016b0 0x7fa420113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 >> 172.21.15.19:0/1311201292 conn(0x7fa420071560 msgr2=0x7fa4200fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 shutdown_connections 2024-02-22T15:55:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.644+0000 7fa425727700 1 -- 172.21.15.19:0/1311201292 wait complete. 2024-02-22T15:55:57.640 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:55:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 -- 172.21.15.19:0/769879224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa4101e40 msgr2=0x7fcfa4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/769879224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa4101e40 0x7fcfa4102210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0004940 tx=0x7fcfa0009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 -- 172.21.15.19:0/769879224 shutdown_connections 2024-02-22T15:55:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/769879224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa41027e0 0x7fcfa40fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/769879224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa4101e40 0x7fcfa4102210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 -- 172.21.15.19:0/769879224 >> 172.21.15.19:0/769879224 conn(0x7fcfa40f79d0 msgr2=0x7fcfa40f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 -- 172.21.15.19:0/769879224 shutdown_connections 2024-02-22T15:55:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.761+0000 7fcfaba0a700 1 -- 172.21.15.19:0/769879224 wait complete. 2024-02-22T15:55:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfaba0a700 1 Processor -- start 2024-02-22T15:55:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfaba0a700 1 -- start start 2024-02-22T15:55:57.744 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:57 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T15:55:57.424+0000 7fd678dec640 -1 osd.0 50 log_to_monitors true 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfaba0a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa4101e40 0x7fcfa410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfaba0a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 0x7fcfa410ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfaba0a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa410f520 con 0x7fcfa41027e0 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfaba0a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa410f660 con 0x7fcfa4101e40 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfa8fa5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 0x7fcfa410ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfa8fa5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 0x7fcfa410ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53436/0 (socket says 172.21.15.19:53436) 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.762+0000 7fcfa8fa5700 1 -- 172.21.15.19:0/2394232070 learned_addr learned my addr 172.21.15.19:0/2394232070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcfa97a6700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa4101e40 0x7fcfa410ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcfa8fa5700 1 -- 172.21.15.19:0/2394232070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa4101e40 msgr2=0x7fcfa410ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcfa8fa5700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa4101e40 0x7fcfa410ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcfa8fa5700 1 -- 172.21.15.19:0/2394232070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa0005040 con 0x7fcfa41027e0 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcfa8fa5700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 0x7fcfa410ef90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400eb40 tx=0x7fcf9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf94007cc0 con 0x7fcfa41027e0 2024-02-22T15:55:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcf9400f040 con 0x7fcfa41027e0 2024-02-22T15:55:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa4117960 con 0x7fcfa41027e0 2024-02-22T15:55:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.763+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9400ca30 con 0x7fcfa41027e0 2024-02-22T15:55:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.764+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa4117e70 con 0x7fcfa41027e0 2024-02-22T15:55:57.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.765+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa40ff150 con 0x7fcfa41027e0 2024-02-22T15:55:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.766+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fcf9400cb90 con 0x7fcfa41027e0 2024-02-22T15:55:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.767+0000 7fcf99ffb700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcf9006cd40 0x7fcf9006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.767+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf9408c060 con 0x7fcfa41027e0 2024-02-22T15:55:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.767+0000 7fcfa97a6700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcf9006cd40 0x7fcf9006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.767+0000 7fcfa97a6700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcf9006cd40 0x7fcf9006f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0000c00 tx=0x7fcfa0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:57.771+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fcf94055f20 con 0x7fcfa41027e0 2024-02-22T15:55:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.009+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfa4066750 con 0x7fcf9006cd40 2024-02-22T15:55:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.010+0000 7fcf99ffb700 1 -- 172.21.15.19:0/2394232070 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcfa4066750 con 0x7fcf9006cd40 2024-02-22T15:55:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcf9006cd40 msgr2=0x7fcf9006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcf9006cd40 0x7fcf9006f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0000c00 tx=0x7fcfa0009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 msgr2=0x7fcfa410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 0x7fcfa410ef90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400eb40 tx=0x7fcf9400b040 comp rx=0 tx=0).stop 2024-02-22T15:55:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 shutdown_connections 2024-02-22T15:55:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcf9006cd40 0x7fcf9006f1f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfa4101e40 0x7fcfa410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 --2- 172.21.15.19:0/2394232070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcfa41027e0 0x7fcfa410ef90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 >> 172.21.15.19:0/2394232070 conn(0x7fcfa40f79d0 msgr2=0x7fcfa40f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 shutdown_connections 2024-02-22T15:55:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.012+0000 7fcfaba0a700 1 -- 172.21.15.19:0/2394232070 wait complete. 2024-02-22T15:55:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 -- 172.21.15.19:0/933612020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c101e40 msgr2=0x7fd80c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 --2- 172.21.15.19:0/933612020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c101e40 0x7fd80c102210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd808004940 tx=0x7fd808009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 -- 172.21.15.19:0/933612020 shutdown_connections 2024-02-22T15:55:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 --2- 172.21.15.19:0/933612020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c1027e0 0x7fd80c0fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 --2- 172.21.15.19:0/933612020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c101e40 0x7fd80c102210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 -- 172.21.15.19:0/933612020 >> 172.21.15.19:0/933612020 conn(0x7fd80c0f79d0 msgr2=0x7fd80c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 -- 172.21.15.19:0/933612020 shutdown_connections 2024-02-22T15:55:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.120+0000 7fd813539700 1 -- 172.21.15.19:0/933612020 wait complete. 2024-02-22T15:55:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd813539700 1 Processor -- start 2024-02-22T15:55:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd813539700 1 -- start start 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd813539700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c101e40 0x7fd80c112810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd813539700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 0x7fd80c112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd813539700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c1133a0 con 0x7fd80c1027e0 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd813539700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c10c740 con 0x7fd80c101e40 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd810ad4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 0x7fd80c112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd810ad4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 0x7fd80c112d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53452/0 (socket says 172.21.15.19:53452) 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.121+0000 7fd810ad4700 1 -- 172.21.15.19:0/857338181 learned_addr learned my addr 172.21.15.19:0/857338181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd8112d5700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c101e40 0x7fd80c112810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd810ad4700 1 -- 172.21.15.19:0/857338181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c101e40 msgr2=0x7fd80c112810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd810ad4700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c101e40 0x7fd80c112810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd810ad4700 1 -- 172.21.15.19:0/857338181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd808005040 con 0x7fd80c1027e0 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd810ad4700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 0x7fd80c112d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c800 tx=0x7fd7fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7fc0106f0 con 0x7fd80c1027e0 2024-02-22T15:55:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7fc010d00 con 0x7fd80c1027e0 2024-02-22T15:55:58.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c10ca20 con 0x7fd80c1027e0 2024-02-22T15:55:58.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7fc0113a0 con 0x7fd80c1027e0 2024-02-22T15:55:58.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.122+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c10cf60 con 0x7fd80c1027e0 2024-02-22T15:55:58.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.123+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c0ff150 con 0x7fd80c1027e0 2024-02-22T15:55:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.124+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fd7fc012400 con 0x7fd80c1027e0 2024-02-22T15:55:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.124+0000 7fd801ffb700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7f806cd40 0x7fd7f806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.124+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd7fc08c0b0 con 0x7fd80c1027e0 2024-02-22T15:55:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.124+0000 7fd8112d5700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7f806cd40 0x7fd7f806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.125+0000 7fd8112d5700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7f806cd40 0x7fd7f806f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd808004f90 tx=0x7fd808009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:58.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.129+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd7fc059b10 con 0x7fd80c1027e0 2024-02-22T15:55:58.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:57 smithi062 ceph-mon[106861]: from='osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T15:55:58.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:57 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:58.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:57 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:58.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:57 smithi019 ceph-mon[173579]: from='osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-22T15:55:58.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:57 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:58.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:57 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:58.244 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 15:55:57 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T15:55:57.917+0000 7fd66a245700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:55:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.344+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd80c108e70 con 0x7fd7f806cd40 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.354+0000 7fd801ffb700 1 -- 172.21.15.19:0/857338181 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fd80c108e70 con 0x7fd7f806cd40 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (3m) 0s ago 14m 26.8M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (32s) 0s ago 14m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (27s) 24s ago 13m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (2m) 0s ago 13m 78.4M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (8m) 0s ago 8m 114M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:55:58.335 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (8m) 0s ago 8m 67.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (8m) 24s ago 8m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (8m) 24s ago 8m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (5m) 0s ago 15m 494M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (4m) 24s ago 13m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (95s) 0s ago 15m 43.0M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (69s) 24s ago 13m 37.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (4m) 0s ago 14m 18.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (3m) 24s ago 13m 17.0M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:55:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (6s) 0s ago 12m 10.4M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:55:58.337 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (11m) 0s ago 11m 243M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c 184bbb26d39f 2024-02-22T15:55:58.337 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (11m) 0s ago 11m 197M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:55:58.337 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (10m) 24s ago 10m 259M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:55:58.337 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (10m) 24s ago 10m 229M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:55:58.337 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (10m) 24s ago 10m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:55:58.337 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (3m) 0s ago 13m 59.7M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7f806cd40 msgr2=0x7fd7f806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7f806cd40 0x7fd7f806f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd808004f90 tx=0x7fd808009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 msgr2=0x7fd80c112d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 0x7fd80c112d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c800 tx=0x7fd7fc004470 comp rx=0 tx=0).stop 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 shutdown_connections 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7f806cd40 0x7fd7f806f1f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd80c101e40 0x7fd80c112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 --2- 172.21.15.19:0/857338181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd80c1027e0 0x7fd80c112d50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 >> 172.21.15.19:0/857338181 conn(0x7fd80c0f79d0 msgr2=0x7fd80c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 shutdown_connections 2024-02-22T15:55:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.356+0000 7fd813539700 1 -- 172.21.15.19:0/857338181 wait complete. 2024-02-22T15:55:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/4126671908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 msgr2=0x7f4fd006fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/4126671908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd006fa40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0004940 tx=0x7f4fc0009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/4126671908 shutdown_connections 2024-02-22T15:55:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/4126671908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 0x7f4fd0105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/4126671908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd006fa40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/4126671908 >> 172.21.15.19:0/4126671908 conn(0x7f4fd0069850 msgr2=0x7f4fd0069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/4126671908 shutdown_connections 2024-02-22T15:55:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.464+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/4126671908 wait complete. 2024-02-22T15:55:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.465+0000 7f4fd4f9b700 1 Processor -- start 2024-02-22T15:55:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.465+0000 7f4fd4f9b700 1 -- start start 2024-02-22T15:55:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.465+0000 7f4fd4f9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd011c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.465+0000 7f4fd4f9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 0x7f4fd011c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.465+0000 7f4fd4f9b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd011cdf0 con 0x7f4fd006f670 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.465+0000 7f4fd4f9b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0116070 con 0x7f4fd0104f90 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd011c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd011c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53464/0 (socket says 172.21.15.19:53464) 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 -- 172.21.15.19:0/1867081221 learned_addr learned my addr 172.21.15.19:0/1867081221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fcdd9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 0x7f4fd011c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 -- 172.21.15.19:0/1867081221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 msgr2=0x7f4fd011c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 0x7f4fd011c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 -- 172.21.15.19:0/1867081221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0005040 con 0x7f4fd006f670 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fcdd9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 0x7f4fd011c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fce59c700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd011c1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000b3b0 tx=0x7f4fc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc000e040 con 0x7f4fd006f670 2024-02-22T15:55:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4fc000bd20 con 0x7f4fd006f670 2024-02-22T15:55:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd0116290 con 0x7f4fd006f670 2024-02-22T15:55:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc0013660 con 0x7f4fd006f670 2024-02-22T15:55:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.466+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd01167d0 con 0x7f4fd006f670 2024-02-22T15:55:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.467+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd0109ec0 con 0x7f4fd006f670 2024-02-22T15:55:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.468+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f4fc001c430 con 0x7f4fd006f670 2024-02-22T15:55:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.468+0000 7f4fbeffd700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4fb806cd40 0x7f4fb806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.469+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4fc008cf70 con 0x7f4fd006f670 2024-02-22T15:55:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.469+0000 7f4fcdd9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4fb806cd40 0x7f4fb806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.469+0000 7f4fcdd9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4fb806cd40 0x7f4fb806f1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0117a70 tx=0x7f4fc4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:58.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.474+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4fc0092060 con 0x7f4fd006f670 2024-02-22T15:55:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.779+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4fd004bed0 con 0x7f4fd006f670 2024-02-22T15:55:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.780+0000 7f4fbeffd700 1 -- 172.21.15.19:0/1867081221 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+738 (secure 0 0 0) 0x7f4fc005a830 con 0x7f4fd006f670 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 5 2024-02-22T15:55:58.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 9, 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:55:58.762 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4fb806cd40 msgr2=0x7f4fb806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4fb806cd40 0x7f4fb806f1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0117a70 tx=0x7f4fc4008040 comp rx=0 tx=0).stop 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 msgr2=0x7f4fd011c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd011c1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000b3b0 tx=0x7f4fc0009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 shutdown_connections 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4fb806cd40 0x7f4fb806f1f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4fd006f670 0x7f4fd011c1d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 --2- 172.21.15.19:0/1867081221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fd0104f90 0x7f4fd011c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 >> 172.21.15.19:0/1867081221 conn(0x7f4fd0069850 msgr2=0x7f4fd010e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 shutdown_connections 2024-02-22T15:55:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.782+0000 7f4fd4f9b700 1 -- 172.21.15.19:0/1867081221 wait complete. 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.900+0000 7f9fb0370700 1 -- 172.21.15.19:0/255106848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fe900 msgr2=0x7f9fa8106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.900+0000 7f9fb0370700 1 --2- 172.21.15.19:0/255106848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fe900 0x7f9fa8106f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f98004940 tx=0x7f9f98009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.900+0000 7f9fb0370700 1 -- 172.21.15.19:0/255106848 shutdown_connections 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.900+0000 7f9fb0370700 1 --2- 172.21.15.19:0/255106848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fe900 0x7f9fa8106f40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.900+0000 7f9fb0370700 1 --2- 172.21.15.19:0/255106848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80fdf60 0x7f9fa80fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.900+0000 7f9fb0370700 1 -- 172.21.15.19:0/255106848 >> 172.21.15.19:0/255106848 conn(0x7f9fa80f7a10 msgr2=0x7f9fa80f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 -- 172.21.15.19:0/255106848 shutdown_connections 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 -- 172.21.15.19:0/255106848 wait complete. 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 Processor -- start 2024-02-22T15:55:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 -- start start 2024-02-22T15:55:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 0x7f9fa81139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80fe900 0x7f9fa8113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa81145d0 con 0x7f9fa80fdf60 2024-02-22T15:55:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fb0370700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa810d870 con 0x7f9fa80fe900 2024-02-22T15:55:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.901+0000 7f9fae10c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 0x7f9fa81139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fae10c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 0x7f9fa81139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53474/0 (socket says 172.21.15.19:53474) 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fae10c700 1 -- 172.21.15.19:0/3362935190 learned_addr learned my addr 172.21.15.19:0/3362935190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fad90b700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80fe900 0x7f9fa8113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fae10c700 1 -- 172.21.15.19:0/3362935190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80fe900 msgr2=0x7f9fa8113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fae10c700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80fe900 0x7f9fa8113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fae10c700 1 -- 172.21.15.19:0/3362935190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98005040 con 0x7f9fa80fdf60 2024-02-22T15:55:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.902+0000 7f9fae10c700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 0x7f9fa81139b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4011dc0 tx=0x7f9fa4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.905+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fa4014d00 con 0x7f9fa80fdf60 2024-02-22T15:55:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.905+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9fa4014e60 con 0x7f9fa80fdf60 2024-02-22T15:55:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.905+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa810db50 con 0x7f9fa80fdf60 2024-02-22T15:55:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.905+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fa401c610 con 0x7f9fa80fdf60 2024-02-22T15:55:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.905+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa810e060 con 0x7f9fa80fdf60 2024-02-22T15:55:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.906+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa8104550 con 0x7f9fa80fdf60 2024-02-22T15:55:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.913+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f9fa401a020 con 0x7f9fa80fdf60 2024-02-22T15:55:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.914+0000 7f9f9effd700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9f9406cd40 0x7f9f9406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.915+0000 7f9fad90b700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9f9406cd40 0x7f9f9406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.918+0000 7f9fad90b700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9f9406cd40 0x7f9f9406f1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fa810eba0 tx=0x7f9f98009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.921+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9fa40614c0 con 0x7f9fa80fdf60 2024-02-22T15:55:58.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:58.929+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9fa4098060 con 0x7f9fa80fdf60 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.200+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9fa804bf30 con 0x7f9fa80fdf60 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.201+0000 7f9f9effd700 1 -- 172.21.15.19:0/3362935190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f9fa4060b90 con 0x7f9fa80fdf60 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:55:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:55:59.183 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:55:59.184 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9f9406cd40 msgr2=0x7f9f9406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9f9406cd40 0x7f9f9406f1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fa810eba0 tx=0x7f9f98009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:59.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 msgr2=0x7f9fa81139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 0x7f9fa81139b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4011dc0 tx=0x7f9fa4011af0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 shutdown_connections 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9f9406cd40 0x7f9f9406f1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa80fdf60 0x7f9fa81139b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 --2- 172.21.15.19:0/3362935190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80fe900 0x7f9fa8113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 >> 172.21.15.19:0/3362935190 conn(0x7f9fa80f7a10 msgr2=0x7f9fa81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 shutdown_connections 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.203+0000 7f9fb0370700 1 -- 172.21.15.19:0/3362935190 wait complete. 2024-02-22T15:55:59.186 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:55:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: failed to encode map e53 with expected crc 2024-02-22T15:55:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: from='osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T15:55:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: osdmap e53: 6 total, 5 up, 6 in 2024-02-22T15:55:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: from='osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:55:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: failed to encode map e53 with expected crc 2024-02-22T15:55:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: failed to encode map e53 with expected crc 2024-02-22T15:55:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:59.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:58 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1867081221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 254 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 63/318 objects degraded (19.811%) 2024-02-22T15:55:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:55:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: failed to encode map e53 with expected crc 2024-02-22T15:55:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: from='osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-22T15:55:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: osdmap e53: 6 total, 5 up, 6 in 2024-02-22T15:55:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: from='osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:55:59.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: failed to encode map e53 with expected crc 2024-02-22T15:55:59.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: failed to encode map e53 with expected crc 2024-02-22T15:55:59.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:59.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:55:59.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:58 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1867081221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 -- 172.21.15.19:0/2117577033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 msgr2=0x7ff6600fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 --2- 172.21.15.19:0/2117577033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fbea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff648004940 tx=0x7ff648009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 -- 172.21.15.19:0/2117577033 shutdown_connections 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 --2- 172.21.15.19:0/2117577033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fbea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 --2- 172.21.15.19:0/2117577033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 0x7ff660102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 -- 172.21.15.19:0/2117577033 >> 172.21.15.19:0/2117577033 conn(0x7ff660071d90 msgr2=0x7ff660072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 -- 172.21.15.19:0/2117577033 shutdown_connections 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 -- 172.21.15.19:0/2117577033 wait complete. 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.312+0000 7ff666126700 1 Processor -- start 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff666126700 1 -- start start 2024-02-22T15:55:59.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff666126700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 0x7ff6600fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff666126700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff666126700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600fe600 con 0x7ff6601027e0 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff666126700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600fe770 con 0x7ff660101e40 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff656ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff656ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53490/0 (socket says 172.21.15.19:53490) 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff656ffd700 1 -- 172.21.15.19:0/1533932537 learned_addr learned my addr 172.21.15.19:0/1533932537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff65f7fe700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 0x7ff6600fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff656ffd700 1 -- 172.21.15.19:0/1533932537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 msgr2=0x7ff6600fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff656ffd700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 0x7ff6600fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff656ffd700 1 -- 172.21.15.19:0/1533932537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648005040 con 0x7ff6601027e0 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.313+0000 7ff65f7fe700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 0x7ff6600fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.314+0000 7ff656ffd700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fe000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff648009fd0 tx=0x7ff648009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.314+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64800e040 con 0x7ff6601027e0 2024-02-22T15:55:59.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.314+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff660119c80 con 0x7ff6601027e0 2024-02-22T15:55:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.314+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66011a0e0 con 0x7ff6601027e0 2024-02-22T15:55:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.314+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff64800bc50 con 0x7ff6601027e0 2024-02-22T15:55:59.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.314+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64801c3f0 con 0x7ff6601027e0 2024-02-22T15:55:59.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.315+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6600738b0 con 0x7ff6601027e0 2024-02-22T15:55:59.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.315+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7ff648013420 con 0x7ff6601027e0 2024-02-22T15:55:59.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.316+0000 7ff65cff9700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff64c06cd40 0x7ff64c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:59.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.316+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff64808d550 con 0x7ff6601027e0 2024-02-22T15:55:59.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.316+0000 7ff65f7fe700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff64c06cd40 0x7ff64c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:59.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.316+0000 7ff65f7fe700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff64c06cd40 0x7ff64c06f1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff650007fd0 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.320+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff6480574b0 con 0x7ff6601027e0 2024-02-22T15:55:59.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.543+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6600ff270 con 0x7ff64c06cd40 2024-02-22T15:55:59.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.545+0000 7ff65cff9700 1 -- 172.21.15.19:0/1533932537 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff6600ff270 con 0x7ff64c06cd40 2024-02-22T15:55:59.525 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "7/21 daemons upgraded", 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading osd daemons", 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:55:59.526 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:55:59.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff64c06cd40 msgr2=0x7ff64c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff64c06cd40 0x7ff64c06f1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff650007fd0 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 msgr2=0x7ff6600fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fe000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff648009fd0 tx=0x7ff648009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 shutdown_connections 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff64c06cd40 0x7ff64c06f1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff660101e40 0x7ff6600fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 --2- 172.21.15.19:0/1533932537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6601027e0 0x7ff6600fe000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 >> 172.21.15.19:0/1533932537 conn(0x7ff660071d90 msgr2=0x7ff660065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 shutdown_connections 2024-02-22T15:55:59.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.547+0000 7ff666126700 1 -- 172.21.15.19:0/1533932537 wait complete. 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.650+0000 7f0686d01700 1 -- 172.21.15.19:0/1691196800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680101e40 msgr2=0x7f0680102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.650+0000 7f0686d01700 1 --2- 172.21.15.19:0/1691196800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680101e40 0x7f0680102210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0674004940 tx=0x7f0674009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- 172.21.15.19:0/1691196800 shutdown_connections 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 --2- 172.21.15.19:0/1691196800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06801027e0 0x7f06800fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 --2- 172.21.15.19:0/1691196800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680101e40 0x7f0680102210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- 172.21.15.19:0/1691196800 >> 172.21.15.19:0/1691196800 conn(0x7f0680071d90 msgr2=0x7f0680072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- 172.21.15.19:0/1691196800 shutdown_connections 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- 172.21.15.19:0/1691196800 wait complete. 2024-02-22T15:55:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 Processor -- start 2024-02-22T15:55:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- start start 2024-02-22T15:55:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0680101e40 0x7f0680115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 0x7f0680116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06801166a0 con 0x7f06801027e0 2024-02-22T15:55:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.651+0000 7f0686d01700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068010f9d0 con 0x7f0680101e40 2024-02-22T15:55:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 0x7f0680116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 0x7f0680116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53516/0 (socket says 172.21.15.19:53516) 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 -- 172.21.15.19:0/1438753745 learned_addr learned my addr 172.21.15.19:0/1438753745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f0684a9d700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0680101e40 0x7f0680115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 -- 172.21.15.19:0/1438753745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0680101e40 msgr2=0x7f0680115b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0680101e40 0x7f0680115b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 -- 172.21.15.19:0/1438753745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0674005040 con 0x7f06801027e0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067ffff700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 0x7f0680116050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f066c002a50 tx=0x7f066c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066c007cc0 con 0x7f06801027e0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f066c00f040 con 0x7f06801027e0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068010fcb0 con 0x7f06801027e0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066c00da30 con 0x7f06801027e0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.652+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06801101f0 con 0x7f06801027e0 2024-02-22T15:55:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.653+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06800738b0 con 0x7f06801027e0 2024-02-22T15:55:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.654+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 37) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f066c00db90 con 0x7f06801027e0 2024-02-22T15:55:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.655+0000 7f067d7fa700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067006cd40 0x7f067006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:55:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.655+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f066c08bce0 con 0x7f06801027e0 2024-02-22T15:55:59.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.655+0000 7f0684a9d700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067006cd40 0x7f067006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:55:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.655+0000 7f0684a9d700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067006cd40 0x7f067006f1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0674004f90 tx=0x7f0674009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:55:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.659+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f066c059590 con 0x7f06801027e0 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.974+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f06801110f0 con 0x7f06801027e0 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.975+0000 7f067d7fa700 1 -- 172.21.15.19:0/1438753745 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+349 (secure 0 0 0) 0x7f066c059120 con 0x7f06801027e0 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-02-22T15:55:59.956 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067006cd40 msgr2=0x7f067006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067006cd40 0x7f067006f1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0674004f90 tx=0x7f0674009e30 comp rx=0 tx=0).stop 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 msgr2=0x7f0680116050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 0x7f0680116050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f066c002a50 tx=0x7f066c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 shutdown_connections 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067006cd40 0x7f067006f1f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0680101e40 0x7f0680115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 --2- 172.21.15.19:0/1438753745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06801027e0 0x7f0680116050 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 >> 172.21.15.19:0/1438753745 conn(0x7f0680071d90 msgr2=0x7f0680065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 shutdown_connections 2024-02-22T15:55:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:55:59.977+0000 7f0686d01700 1 -- 172.21.15.19:0/1438753745 wait complete. 2024-02-22T15:56:00.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:00.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:00.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:56:00.169 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: failed to encode map e54 with expected crc 2024-02-22T15:56:00.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349] boot 2024-02-22T15:56:00.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: osdmap e54: 6 total, 6 up, 6 in 2024-02-22T15:56:00.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:56:00.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3362935190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:56:00.170 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:55:59 smithi019 ceph-mon[173579]: osdmap e55: 6 total, 6 up, 6 in 2024-02-22T15:56:00.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:00.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:00.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:56:00.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: failed to encode map e54 with expected crc 2024-02-22T15:56:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: osd.0 [v2:172.21.15.19:6802/2998470349,v1:172.21.15.19:6803/2998470349] boot 2024-02-22T15:56:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: osdmap e54: 6 total, 6 up, 6 in 2024-02-22T15:56:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-22T15:56:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3362935190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:56:00.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:55:59 smithi062 ceph-mon[106861]: osdmap e55: 6 total, 6 up, 6 in 2024-02-22T15:56:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: pgmap v50: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 11/318 objects degraded (3.459%) 2024-02-22T15:56:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T15:56:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:00 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1438753745' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:56:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: pgmap v50: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 11/318 objects degraded (3.459%) 2024-02-22T15:56:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: failed to encode map e55 with expected crc 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T15:56:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:00 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1438753745' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:56:03.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: pgmap v52: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 11/318 objects degraded (3.459%) 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:03.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: pgmap v52: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 11/318 objects degraded (3.459%) 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:03.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:04.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:03 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:04.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:03 smithi062 ceph-mon[106861]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-22T15:56:04.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:04.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:03 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:03 smithi019 ceph-mon[173579]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-22T15:56:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:05.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:04 smithi062 ceph-mon[106861]: pgmap v53: 65 pgs: 25 peering, 40 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-22T15:56:05.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:04 smithi062 ceph-mon[106861]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded) 2024-02-22T15:56:05.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:04 smithi062 ceph-mon[106861]: Cluster is now healthy 2024-02-22T15:56:05.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:04 smithi062 ceph-mon[106861]: mgrmap e38: smithi019.yvjrkp(active, since 91s), standbys: smithi062.ecubmv 2024-02-22T15:56:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:04 smithi019 ceph-mon[173579]: pgmap v53: 65 pgs: 25 peering, 40 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-22T15:56:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:04 smithi019 ceph-mon[173579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/318 objects degraded (3.459%), 3 pgs degraded) 2024-02-22T15:56:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:04 smithi019 ceph-mon[173579]: Cluster is now healthy 2024-02-22T15:56:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:04 smithi019 ceph-mon[173579]: mgrmap e38: smithi019.yvjrkp(active, since 91s), standbys: smithi062.ecubmv 2024-02-22T15:56:07.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:06 smithi062 ceph-mon[106861]: pgmap v54: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 568 B/s wr, 5 op/s 2024-02-22T15:56:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:06 smithi019 ceph-mon[173579]: pgmap v54: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 568 B/s wr, 5 op/s 2024-02-22T15:56:09.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:08 smithi062 ceph-mon[106861]: pgmap v55: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 499 B/s wr, 5 op/s 2024-02-22T15:56:09.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:08 smithi019 ceph-mon[173579]: pgmap v55: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 499 B/s wr, 5 op/s 2024-02-22T15:56:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:10 smithi062 ceph-mon[106861]: pgmap v56: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 409 B/s wr, 4 op/s 2024-02-22T15:56:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:10 smithi019 ceph-mon[173579]: pgmap v56: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 409 B/s wr, 4 op/s 2024-02-22T15:56:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:12 smithi062 ceph-mon[106861]: pgmap v57: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 365 B/s wr, 4 op/s 2024-02-22T15:56:13.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:12 smithi019 ceph-mon[173579]: pgmap v57: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 365 B/s wr, 4 op/s 2024-02-22T15:56:14.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:14 smithi062 ceph-mon[106861]: pgmap v58: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 341 B/s wr, 3 op/s 2024-02-22T15:56:14.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:14.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:14 smithi019 ceph-mon[173579]: pgmap v58: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 341 B/s wr, 3 op/s 2024-02-22T15:56:14.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:16.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:16 smithi062 ceph-mon[106861]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-22T15:56:16.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:16 smithi019 ceph-mon[173579]: pgmap v59: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-02-22T15:56:18.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:18 smithi019 ceph-mon[173579]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:18.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:18.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:18 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:18.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:18 smithi019 ceph-mon[173579]: Upgrade: osd.1 is safe to restart 2024-02-22T15:56:18.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:18.391 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:18.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:18 smithi062 ceph-mon[106861]: pgmap v60: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:18.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:18.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:18 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-22T15:56:18.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:18 smithi062 ceph-mon[106861]: Upgrade: osd.1 is safe to restart 2024-02-22T15:56:18.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:18.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:20.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:20 smithi019 ceph-mon[173579]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 5 op/s 2024-02-22T15:56:20.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:20 smithi019 ceph-mon[173579]: Upgrade: Updating osd.1 2024-02-22T15:56:20.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:20.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T15:56:20.401 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:20.401 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:20 smithi019 ceph-mon[173579]: Deploying daemon osd.1 on smithi019 2024-02-22T15:56:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:20 smithi062 ceph-mon[106861]: pgmap v61: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 5 op/s 2024-02-22T15:56:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:20 smithi062 ceph-mon[106861]: Upgrade: Updating osd.1 2024-02-22T15:56:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-22T15:56:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:20 smithi062 ceph-mon[106861]: Deploying daemon osd.1 on smithi019 2024-02-22T15:56:22.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:22 smithi062 ceph-mon[106861]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:22.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:22 smithi019 ceph-mon[173579]: pgmap v62: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:22.911 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:22 smithi019 systemd[1]: Stopping Ceph osd.1 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:56:23.243 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[100582]: 2024-02-22T15:56:22.930+0000 7fca015a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:56:23.243 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[100582]: 2024-02-22T15:56:22.930+0000 7fca015a0700 -1 osd.1 55 *** Got signal Terminated *** 2024-02-22T15:56:23.244 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[100582]: 2024-02-22T15:56:22.930+0000 7fca015a0700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T15:56:24.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:23 smithi062 ceph-mon[106861]: osd.1 marked itself down and dead 2024-02-22T15:56:24.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:23 smithi019 ceph-mon[173579]: osd.1 marked itself down and dead 2024-02-22T15:56:24.993 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:24 smithi019 bash[195719]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1 2024-02-22T15:56:24.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:24.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:56:24.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: osdmap e56: 6 total, 5 up, 6 in 2024-02-22T15:56:24.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: failed to encode map e56 with expected crc 2024-02-22T15:56:24.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: failed to encode map e56 with expected crc 2024-02-22T15:56:24.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: failed to encode map e56 with expected crc 2024-02-22T15:56:24.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: failed to encode map e56 with expected crc 2024-02-22T15:56:24.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:24 smithi019 ceph-mon[173579]: failed to encode map e56 with expected crc 2024-02-22T15:56:25.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: pgmap v63: 65 pgs: 65 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: osdmap e56: 6 total, 5 up, 6 in 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: failed to encode map e56 with expected crc 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: failed to encode map e56 with expected crc 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: failed to encode map e56 with expected crc 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: failed to encode map e56 with expected crc 2024-02-22T15:56:25.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:24 smithi062 ceph-mon[106861]: failed to encode map e56 with expected crc 2024-02-22T15:56:26.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:25 smithi062 ceph-mon[106861]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:25 smithi062 ceph-mon[106861]: osdmap e57: 6 total, 5 up, 6 in 2024-02-22T15:56:26.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:25 smithi062 ceph-mon[106861]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:25 smithi062 ceph-mon[106861]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:25 smithi062 ceph-mon[106861]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:25 smithi062 ceph-mon[106861]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:25 smithi019 ceph-mon[173579]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:25 smithi019 ceph-mon[173579]: osdmap e57: 6 total, 5 up, 6 in 2024-02-22T15:56:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:25 smithi019 ceph-mon[173579]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:25 smithi019 ceph-mon[173579]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:25 smithi019 ceph-mon[173579]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:25 smithi019 ceph-mon[173579]: failed to encode map e57 with expected crc 2024-02-22T15:56:26.894 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:26 smithi019 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.1.service: Succeeded. 2024-02-22T15:56:26.895 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:26 smithi019 systemd[1]: Stopped Ceph osd.1 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:56:26.895 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:26 smithi019 systemd[1]: Starting Ceph osd.1 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:56:27.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:26 smithi062 ceph-mon[106861]: pgmap v66: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-22T15:56:27.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:26 smithi062 ceph-mon[106861]: Health check failed: Reduced data availability: 8 pgs peering (PG_AVAILABILITY) 2024-02-22T15:56:27.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:26 smithi019 ceph-mon[173579]: pgmap v66: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-22T15:56:27.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:26 smithi019 ceph-mon[173579]: Health check failed: Reduced data availability: 8 pgs peering (PG_AVAILABILITY) 2024-02-22T15:56:28.243 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-22T15:56:28.243 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-22T15:56:28.243 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-22T15:56:28.244 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-22T15:56:28.244 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-22T15:56:28.244 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-22T15:56:28.244 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:27 smithi019 bash[195990]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-22T15:56:29.157 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:29 smithi019 ceph-osd[196308]: -- 172.21.15.19:0/645078865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a59444e4e0 con 0x55a59441f000 2024-02-22T15:56:29.157 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:28 smithi019 ceph-mon[173579]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:56:29.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:28 smithi062 ceph-mon[106861]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-22T15:56:29.493 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:29 smithi019 bash[196270]: bf25d550ab0c920bfddf98d9a501423ee33eaa3fbe6c69b290e31e4a7e2f2044 2024-02-22T15:56:29.493 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:29 smithi019 systemd[1]: Started Ceph osd.1 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:56:29.949 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:29 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T15:56:29.624+0000 7f16237bc640 -1 Falling back to public interface 2024-02-22T15:56:30.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 -- 172.21.15.19:0/435912509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fe8f0 msgr2=0x7f574c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 --2- 172.21.15.19:0/435912509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fe8f0 0x7f574c107050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5740004970 tx=0x7f5740009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 -- 172.21.15.19:0/435912509 shutdown_connections 2024-02-22T15:56:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 --2- 172.21.15.19:0/435912509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fe8f0 0x7f574c107050 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 --2- 172.21.15.19:0/435912509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 -- 172.21.15.19:0/435912509 >> 172.21.15.19:0/435912509 conn(0x7f574c071d90 msgr2=0x7f574c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 -- 172.21.15.19:0/435912509 shutdown_connections 2024-02-22T15:56:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 -- 172.21.15.19:0/435912509 wait complete. 2024-02-22T15:56:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.086+0000 7f5752286700 1 Processor -- start 2024-02-22T15:56:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f5752286700 1 -- start start 2024-02-22T15:56:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f5752286700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f5752286700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c0fe8f0 0x7f574c0fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f5752286700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0fc9e0 con 0x7f574c0fdfe0 2024-02-22T15:56:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f5752286700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0fcb50 con 0x7f574c0fe8f0 2024-02-22T15:56:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46248/0 (socket says 172.21.15.19:46248) 2024-02-22T15:56:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574bfff700 1 -- 172.21.15.19:0/2219990674 learned_addr learned my addr 172.21.15.19:0/2219990674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574b7fe700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c0fe8f0 0x7f574c0fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574bfff700 1 -- 172.21.15.19:0/2219990674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c0fe8f0 msgr2=0x7f574c0fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574bfff700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c0fe8f0 0x7f574c0fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.087+0000 7f574bfff700 1 -- 172.21.15.19:0/2219990674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740005040 con 0x7f574c0fdfe0 2024-02-22T15:56:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.088+0000 7f574bfff700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fbea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f573c002a50 tx=0x7f573c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.088+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c007cc0 con 0x7f574c0fdfe0 2024-02-22T15:56:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.088+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f573c00f040 con 0x7f574c0fdfe0 2024-02-22T15:56:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.088+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c119bb0 con 0x7f574c0fdfe0 2024-02-22T15:56:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.088+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c00d9c0 con 0x7f574c0fdfe0 2024-02-22T15:56:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.088+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c11a060 con 0x7f574c0fdfe0 2024-02-22T15:56:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.089+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c1046c0 con 0x7f574c0fdfe0 2024-02-22T15:56:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.089+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f573c00db60 con 0x7f574c0fdfe0 2024-02-22T15:56:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.090+0000 7f5748ff9700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f573406cc20 0x7f573406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.090+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f573c08b570 con 0x7f574c0fdfe0 2024-02-22T15:56:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.090+0000 7f574b7fe700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f573406cc20 0x7f573406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.090+0000 7f574b7fe700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f573406cc20 0x7f573406f0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f574000b370 tx=0x7f5740009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.094+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f573c055650 con 0x7f574c0fdfe0 2024-02-22T15:56:30.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.329+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f574c0fd5e0 con 0x7f573406cc20 2024-02-22T15:56:30.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.331+0000 7f5748ff9700 1 -- 172.21.15.19:0/2219990674 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f574c0fd5e0 con 0x7f573406cc20 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f573406cc20 msgr2=0x7f573406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f573406cc20 0x7f573406f0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f574000b370 tx=0x7f5740009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 msgr2=0x7f574c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fbea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f573c002a50 tx=0x7f573c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 shutdown_connections 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f573406cc20 0x7f573406f0d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f574c0fdfe0 0x7f574c0fbea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 --2- 172.21.15.19:0/2219990674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f574c0fe8f0 0x7f574c0fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 >> 172.21.15.19:0/2219990674 conn(0x7f574c071d90 msgr2=0x7f574c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 shutdown_connections 2024-02-22T15:56:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.333+0000 7f5752286700 1 -- 172.21.15.19:0/2219990674 wait complete. 2024-02-22T15:56:30.324 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:56:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 -- 172.21.15.19:0/3856670732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c102f40 msgr2=0x7fa68c103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3856670732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c102f40 0x7fa68c103310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa67c004940 tx=0x7fa67c009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 -- 172.21.15.19:0/3856670732 shutdown_connections 2024-02-22T15:56:30.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3856670732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c1038e0 0x7fa68c0fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3856670732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c102f40 0x7fa68c103310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 -- 172.21.15.19:0/3856670732 >> 172.21.15.19:0/3856670732 conn(0x7fa68c0f8af0 msgr2=0x7fa68c0faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 -- 172.21.15.19:0/3856670732 shutdown_connections 2024-02-22T15:56:30.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.452+0000 7fa6921c8700 1 -- 172.21.15.19:0/3856670732 wait complete. 2024-02-22T15:56:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.453+0000 7fa6921c8700 1 Processor -- start 2024-02-22T15:56:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.453+0000 7fa6921c8700 1 -- start start 2024-02-22T15:56:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa6921c8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c102f40 0x7fa68c113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa6921c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 0x7fa68c113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa6921c8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c1144b0 con 0x7fa68c1038e0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa6921c8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c10d7e0 con 0x7fa68c102f40 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 0x7fa68c113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 0x7fa68c113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46272/0 (socket says 172.21.15.19:46272) 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 -- 172.21.15.19:0/3166550833 learned_addr learned my addr 172.21.15.19:0/3166550833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68b7fe700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c102f40 0x7fa68c113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 -- 172.21.15.19:0/3166550833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c102f40 msgr2=0x7fa68c113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c102f40 0x7fa68c113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 -- 172.21.15.19:0/3166550833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c005040 con 0x7fa68c1038e0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa68affd700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 0x7fa68c113e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa68000eb40 tx=0x7fa68000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.454+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa680007cc0 con 0x7fa68c1038e0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.455+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c10da60 con 0x7fa68c1038e0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.455+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa68000f040 con 0x7fa68c1038e0 2024-02-22T15:56:30.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.455+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa68000c8f0 con 0x7fa68c1038e0 2024-02-22T15:56:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.455+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c10dfa0 con 0x7fa68c1038e0 2024-02-22T15:56:30.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.456+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c0fc6f0 con 0x7fa68c1038e0 2024-02-22T15:56:30.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.457+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa68000cb60 con 0x7fa68c1038e0 2024-02-22T15:56:30.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.458+0000 7fa673fff700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa67406cc80 0x7fa67406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.458+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa68008bd00 con 0x7fa68c1038e0 2024-02-22T15:56:30.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.458+0000 7fa68b7fe700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa67406cc80 0x7fa67406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.458+0000 7fa68b7fe700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa67406cc80 0x7fa67406f130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009fd0 tx=0x7fa67c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:30.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.462+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa680059880 con 0x7fa68c1038e0 2024-02-22T15:56:30.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:30 smithi019 ceph-mon[173579]: pgmap v68: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-02-22T15:56:30.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:30 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:30.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:30 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:30.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:30 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:56:30.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:30 smithi019 ceph-mon[173579]: Health check failed: Degraded data redundancy: 51/318 objects degraded (16.038%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T15:56:30.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:30 smithi019 ceph-mon[173579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs peering) 2024-02-22T15:56:30.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.692+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa68c10e1c0 con 0x7fa67406cc80 2024-02-22T15:56:30.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.693+0000 7fa673fff700 1 -- 172.21.15.19:0/3166550833 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa68c10e1c0 con 0x7fa67406cc80 2024-02-22T15:56:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa67406cc80 msgr2=0x7fa67406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa67406cc80 0x7fa67406f130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009fd0 tx=0x7fa67c009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:30.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 msgr2=0x7fa68c113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 0x7fa68c113e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa68000eb40 tx=0x7fa68000b040 comp rx=0 tx=0).stop 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 shutdown_connections 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa67406cc80 0x7fa67406f130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa68c102f40 0x7fa68c113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 --2- 172.21.15.19:0/3166550833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa68c1038e0 0x7fa68c113e60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.695+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 >> 172.21.15.19:0/3166550833 conn(0x7fa68c0f8af0 msgr2=0x7fa68c0fa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.696+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 shutdown_connections 2024-02-22T15:56:30.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.696+0000 7fa6921c8700 1 -- 172.21.15.19:0/3166550833 wait complete. 2024-02-22T15:56:30.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:30 smithi062 ceph-mon[106861]: pgmap v68: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-02-22T15:56:30.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:30 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:30.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:30 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:30.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:30 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:56:30.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:30 smithi062 ceph-mon[106861]: Health check failed: Degraded data redundancy: 51/318 objects degraded (16.038%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T15:56:30.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:30 smithi062 ceph-mon[106861]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs peering) 2024-02-22T15:56:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 -- 172.21.15.19:0/1314395015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 msgr2=0x7f1a380fe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 --2- 172.21.15.19:0/1314395015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a380fe360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a28004940 tx=0x7f1a28009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:30.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 -- 172.21.15.19:0/1314395015 shutdown_connections 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 --2- 172.21.15.19:0/1314395015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a380fe930 0x7f1a38107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 --2- 172.21.15.19:0/1314395015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a380fe360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 -- 172.21.15.19:0/1314395015 >> 172.21.15.19:0/1314395015 conn(0x7f1a380f7a10 msgr2=0x7f1a380f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 -- 172.21.15.19:0/1314395015 shutdown_connections 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.806+0000 7f1a3d977700 1 -- 172.21.15.19:0/1314395015 wait complete. 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a3d977700 1 Processor -- start 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a3d977700 1 -- start start 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a3d977700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a38113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a3d977700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a380fe930 0x7f1a38113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a3d977700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a381145a0 con 0x7f1a380fdf90 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a3d977700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3810d8d0 con 0x7f1a380fe930 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a36ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a38113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a36ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a38113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46296/0 (socket says 172.21.15.19:46296) 2024-02-22T15:56:30.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a36ffd700 1 -- 172.21.15.19:0/2768986649 learned_addr learned my addr 172.21.15.19:0/2768986649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a367fc700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a380fe930 0x7f1a38113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a36ffd700 1 -- 172.21.15.19:0/2768986649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a380fe930 msgr2=0x7f1a38113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a36ffd700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a380fe930 0x7f1a38113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.807+0000 7f1a36ffd700 1 -- 172.21.15.19:0/2768986649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28005040 con 0x7f1a380fdf90 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.808+0000 7f1a36ffd700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a38113a10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009150 tx=0x7f1a28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.808+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2800e040 con 0x7f1a380fdf90 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.808+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3810db50 con 0x7f1a380fdf90 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.808+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a28013420 con 0x7f1a380fdf90 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.808+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2801c630 con 0x7f1a380fdf90 2024-02-22T15:56:30.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.808+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3810e090 con 0x7f1a380fdf90 2024-02-22T15:56:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.809+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a381046c0 con 0x7f1a380fdf90 2024-02-22T15:56:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.809+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1a28013990 con 0x7f1a380fdf90 2024-02-22T15:56:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.810+0000 7f1a1f7fe700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2006cc20 0x7f1a2006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.810+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a2808d200 con 0x7f1a380fdf90 2024-02-22T15:56:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.810+0000 7f1a367fc700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2006cc20 0x7f1a2006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:30.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.810+0000 7f1a367fc700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2006cc20 0x7f1a2006f0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a3810f240 tx=0x7f1a2c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:30.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:30.814+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1a2805ae10 con 0x7f1a380fdf90 2024-02-22T15:56:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.031+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a3804bed0 con 0x7f1a2006cc20 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.041+0000 7f1a1f7fe700 1 -- 172.21.15.19:0/2768986649 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1a3804bed0 con 0x7f1a2006cc20 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (3m) 33s ago 14m 26.8M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (65s) 33s ago 15m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (60s) 57s ago 13m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (3m) 33s ago 14m 78.4M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:56:31.021 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (8m) 33s ago 8m 114M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (9m) 33s ago 9m 67.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (8m) 57s ago 8m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (9m) 57s ago 9m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (5m) 33s ago 16m 494M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (4m) 57s ago 13m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (2m) 33s ago 16m 43.0M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (102s) 57s ago 13m 37.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (4m) 33s ago 14m 18.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (4m) 57s ago 13m 17.0M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (39s) 33s ago 12m 10.4M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 starting - - - 1570M 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (11m) 33s ago 11m 197M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (11m) 57s ago 11m 259M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (11m) 57s ago 11m 229M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:56:31.022 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (10m) 57s ago 10m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:56:31.023 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (4m) 33s ago 14m 59.7M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:56:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2006cc20 msgr2=0x7f1a2006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2006cc20 0x7f1a2006f0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a3810f240 tx=0x7f1a2c00b040 comp rx=0 tx=0).stop 2024-02-22T15:56:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 msgr2=0x7f1a38113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a38113a10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009150 tx=0x7f1a28009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 shutdown_connections 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2006cc20 0x7f1a2006f0d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a380fdf90 0x7f1a38113a10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.043+0000 7f1a3d977700 1 --2- 172.21.15.19:0/2768986649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a380fe930 0x7f1a38113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.044+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 >> 172.21.15.19:0/2768986649 conn(0x7f1a380f7a10 msgr2=0x7f1a38101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.044+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 shutdown_connections 2024-02-22T15:56:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.044+0000 7f1a3d977700 1 -- 172.21.15.19:0/2768986649 wait complete. 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 -- 172.21.15.19:0/2884875673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 msgr2=0x7f379c0fe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 --2- 172.21.15.19:0/2884875673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c0fe220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3798004940 tx=0x7f3798009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 -- 172.21.15.19:0/2884875673 shutdown_connections 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 --2- 172.21.15.19:0/2884875673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 0x7f379c0fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 --2- 172.21.15.19:0/2884875673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c0fe220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 -- 172.21.15.19:0/2884875673 >> 172.21.15.19:0/2884875673 conn(0x7f379c071560 msgr2=0x7f379c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 -- 172.21.15.19:0/2884875673 shutdown_connections 2024-02-22T15:56:31.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.164+0000 7f37a3551700 1 -- 172.21.15.19:0/2884875673 wait complete. 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a3551700 1 Processor -- start 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a3551700 1 -- start start 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a3551700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 0x7f379c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a3551700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a3551700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c116710 con 0x7f379c0fde50 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a3551700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c10fa40 con 0x7f379c0fb570 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a0aec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a0aec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c1160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46316/0 (socket says 172.21.15.19:46316) 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a0aec700 1 -- 172.21.15.19:0/3986044025 learned_addr learned my addr 172.21.15.19:0/3986044025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:31.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a12ed700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 0x7f379c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.165+0000 7f37a0aec700 1 -- 172.21.15.19:0/3986044025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 msgr2=0x7f379c115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f37a0aec700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 0x7f379c115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f37a0aec700 1 -- 172.21.15.19:0/3986044025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798005040 con 0x7f379c0fde50 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f37a12ed700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 0x7f379c115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f37a0aec700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c1160c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f378c002a50 tx=0x7f378c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f378c007cc0 con 0x7f379c0fde50 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f378c00f040 con 0x7f379c0fde50 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c10fd20 con 0x7f379c0fde50 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f378c009540 con 0x7f379c0fde50 2024-02-22T15:56:31.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.166+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c110260 con 0x7f379c0fde50 2024-02-22T15:56:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.167+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c107420 con 0x7f379c0fde50 2024-02-22T15:56:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.168+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f378c0096a0 con 0x7f379c0fde50 2024-02-22T15:56:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.168+0000 7f3791ffb700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f378806cc20 0x7f378806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.168+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f378c08bdd0 con 0x7f379c0fde50 2024-02-22T15:56:31.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.168+0000 7f37a12ed700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f378806cc20 0x7f378806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.169+0000 7f37a12ed700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f378806cc20 0x7f378806f0d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3798004f90 tx=0x7f3798009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:31.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.173+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f378c0598c0 con 0x7f379c0fde50 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.471+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f379c10c180 con 0x7f379c0fde50 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.471+0000 7f3791ffb700 1 -- 172.21.15.19:0/3986044025 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f378c059450 con 0x7f379c0fde50 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:56:31.451 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4, 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:56:31.452 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:56:31.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:56:31.456 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:56:31.456 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 8, 2024-02-22T15:56:31.456 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 5 2024-02-22T15:56:31.456 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:56:31.456 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f378806cc20 msgr2=0x7f378806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f378806cc20 0x7f378806f0d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3798004f90 tx=0x7f3798009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 msgr2=0x7f379c1160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c1160c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f378c002a50 tx=0x7f378c0046b0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 shutdown_connections 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f378806cc20 0x7f378806f0d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379c0fb570 0x7f379c115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 --2- 172.21.15.19:0/3986044025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c0fde50 0x7f379c1160c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 >> 172.21.15.19:0/3986044025 conn(0x7f379c071560 msgr2=0x7f379c0f9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 shutdown_connections 2024-02-22T15:56:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.474+0000 7f37a3551700 1 -- 172.21.15.19:0/3986044025 wait complete. 2024-02-22T15:56:31.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:31 smithi019 ceph-mon[173579]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:31.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:31 smithi019 ceph-mon[173579]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.579+0000 7fd705466700 1 -- 172.21.15.19:0/2265124644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fe8f0 msgr2=0x7fd700107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.579+0000 7fd705466700 1 --2- 172.21.15.19:0/2265124644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fe8f0 0x7fd700107050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4004970 tx=0x7fd6f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 -- 172.21.15.19:0/2265124644 shutdown_connections 2024-02-22T15:56:31.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 --2- 172.21.15.19:0/2265124644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fe8f0 0x7fd700107050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 --2- 172.21.15.19:0/2265124644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 -- 172.21.15.19:0/2265124644 >> 172.21.15.19:0/2265124644 conn(0x7fd700071d90 msgr2=0x7fd700072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:31.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 -- 172.21.15.19:0/2265124644 shutdown_connections 2024-02-22T15:56:31.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 -- 172.21.15.19:0/2265124644 wait complete. 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 Processor -- start 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 -- start start 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.580+0000 7fd705466700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd705466700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fe8f0 0x7fd7000fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd705466700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000fc9e0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd705466700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7000fcb50 con 0x7fd7000fe8f0 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46342/0 (socket says 172.21.15.19:46342) 2024-02-22T15:56:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 -- 172.21.15.19:0/2426380350 learned_addr learned my addr 172.21.15.19:0/2426380350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6fe7fc700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fe8f0 0x7fd7000fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 -- 172.21.15.19:0/2426380350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fe8f0 msgr2=0x7fd7000fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fe8f0 0x7fd7000fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 -- 172.21.15.19:0/2426380350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4005040 con 0x7fd7000fdfe0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6fe7fc700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fe8f0 0x7fd7000fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6feffd700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fbea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c8c0 tx=0x7fd6f00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6f000a980 con 0x7fd7000fdfe0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd700119bb0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6f000ce60 con 0x7fd7000fdfe0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6f00113a0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.581+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70011a060 con 0x7fd7000fdfe0 2024-02-22T15:56:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.582+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7001046c0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.583+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd6f0012400 con 0x7fd7000fdfe0 2024-02-22T15:56:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.583+0000 7fd6e77fe700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6e806cc20 0x7fd6e806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.583+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd6f008c0c0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.583+0000 7fd6fe7fc700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6e806cc20 0x7fd6e806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.584+0000 7fd6fe7fc700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6e806cc20 0x7fd6e806f0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4004f50 tx=0x7fd6f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:31.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.588+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd6f00561a0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:31 smithi062 ceph-mon[106861]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:31.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:31 smithi062 ceph-mon[106861]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.835+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd7000738b0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.835+0000 7fd6e77fe700 1 -- 172.21.15.19:0/2426380350 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fd6f00597c0 con 0x7fd7000fdfe0 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:56:31.815 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:56:31.816 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:56:31.817 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:56:31.818 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:56:31.818 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:56:31.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:56:31.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6e806cc20 msgr2=0x7fd6e806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6e806cc20 0x7fd6e806f0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4004f50 tx=0x7fd6f4009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 msgr2=0x7fd7000fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fbea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c8c0 tx=0x7fd6f00044c0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 shutdown_connections 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6e806cc20 0x7fd6e806f0d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7000fdfe0 0x7fd7000fbea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 --2- 172.21.15.19:0/2426380350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7000fe8f0 0x7fd7000fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 >> 172.21.15.19:0/2426380350 conn(0x7fd700071d90 msgr2=0x7fd700101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 shutdown_connections 2024-02-22T15:56:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.837+0000 7fd705466700 1 -- 172.21.15.19:0/2426380350 wait complete. 2024-02-22T15:56:31.820 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:56:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 -- 172.21.15.19:0/3097675393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 msgr2=0x7f7374102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 --2- 172.21.15.19:0/3097675393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f7374102210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7364004940 tx=0x7f7364009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 -- 172.21.15.19:0/3097675393 shutdown_connections 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 --2- 172.21.15.19:0/3097675393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 0x7f73740fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 --2- 172.21.15.19:0/3097675393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f7374102210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 -- 172.21.15.19:0/3097675393 >> 172.21.15.19:0/3097675393 conn(0x7f73740f79d0 msgr2=0x7f73740f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 -- 172.21.15.19:0/3097675393 shutdown_connections 2024-02-22T15:56:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.953+0000 7f7378ee0700 1 -- 172.21.15.19:0/3097675393 wait complete. 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7378ee0700 1 Processor -- start 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7378ee0700 1 -- start start 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7378ee0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f737410e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7378ee0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 0x7f737410eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7378ee0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737410f470 con 0x7f7374101e40 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7378ee0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737410f5b0 con 0x7f73741027e0 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f737259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f737410e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f737259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f737410e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46360/0 (socket says 172.21.15.19:46360) 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f737259c700 1 -- 172.21.15.19:0/2349677664 learned_addr learned my addr 172.21.15.19:0/2349677664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7371d9b700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 0x7f737410eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f737259c700 1 -- 172.21.15.19:0/2349677664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 msgr2=0x7f737410eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f737259c700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 0x7f737410eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f737259c700 1 -- 172.21.15.19:0/2349677664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7364005040 con 0x7f7374101e40 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.954+0000 7f7371d9b700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 0x7f737410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.955+0000 7f737259c700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f737410e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7364000c00 tx=0x7f7364009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.955+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7364021070 con 0x7f7374101e40 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.955+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f736401c400 con 0x7f7374101e40 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.955+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73741178b0 con 0x7f7374101e40 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.955+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736400e070 con 0x7f7374101e40 2024-02-22T15:56:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.955+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7374117d60 con 0x7f7374101e40 2024-02-22T15:56:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.956+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73740ff150 con 0x7f7374101e40 2024-02-22T15:56:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.956+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f736401c970 con 0x7f7374101e40 2024-02-22T15:56:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.957+0000 7f736affd700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f736006cc20 0x7f736006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:31.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.957+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f736408d130 con 0x7f7374101e40 2024-02-22T15:56:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.957+0000 7f7371d9b700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f736006cc20 0x7f736006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:31.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.957+0000 7f7371d9b700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f736006cc20 0x7f736006f0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7374066a10 tx=0x7f735c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:31.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:31.962+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7364057210 con 0x7f7374101e40 2024-02-22T15:56:32.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.193+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7374066750 con 0x7f736006cc20 2024-02-22T15:56:32.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.194+0000 7f736affd700 1 -- 172.21.15.19:0/2349677664 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7374066750 con 0x7f736006cc20 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "7/21 daemons upgraded", 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading osd daemons", 2024-02-22T15:56:32.174 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:56:32.175 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f736006cc20 msgr2=0x7f736006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f736006cc20 0x7f736006f0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7374066a10 tx=0x7f735c00b040 comp rx=0 tx=0).stop 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 msgr2=0x7f737410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f737410e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7364000c00 tx=0x7f7364009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 shutdown_connections 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f736006cc20 0x7f736006f0d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.196+0000 7f7378ee0700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7374101e40 0x7f737410e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.197+0000 7f7378ee0700 1 --2- 172.21.15.19:0/2349677664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73741027e0 0x7f737410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.197+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 >> 172.21.15.19:0/2349677664 conn(0x7f73740f79d0 msgr2=0x7f73740f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.197+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 shutdown_connections 2024-02-22T15:56:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.197+0000 7f7378ee0700 1 -- 172.21.15.19:0/2349677664 wait complete. 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/3778114564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e0102f50 msgr2=0x7fe7e0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/3778114564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e0102f50 0x7fe7e0103320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004940 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/3778114564 shutdown_connections 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/3778114564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e01038f0 0x7fe7e00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/3778114564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e0102f50 0x7fe7e0103320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/3778114564 >> 172.21.15.19:0/3778114564 conn(0x7fe7e00f8ae0 msgr2=0x7fe7e00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/3778114564 shutdown_connections 2024-02-22T15:56:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.315+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/3778114564 wait complete. 2024-02-22T15:56:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7e6ae4700 1 Processor -- start 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7e6ae4700 1 -- start start 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7e6ae4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0102f50 0x7fe7e0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7e6ae4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 0x7fe7e00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7e6ae4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0070dd0 con 0x7fe7e01038f0 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7e6ae4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0070f40 con 0x7fe7e0102f50 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 0x7fe7e00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 0x7fe7e00707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46382/0 (socket says 172.21.15.19:46382) 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.316+0000 7fe7dffff700 1 -- 172.21.15.19:0/4234990959 learned_addr learned my addr 172.21.15.19:0/4234990959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dffff700 1 -- 172.21.15.19:0/4234990959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0102f50 msgr2=0x7fe7e0070290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dffff700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0102f50 0x7fe7e0070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dffff700 1 -- 172.21.15.19:0/4234990959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4005040 con 0x7fe7e01038f0 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dffff700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 0x7fe7e00707d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc002a50 tx=0x7fe7cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7cc007cc0 con 0x7fe7e01038f0 2024-02-22T15:56:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e0119be0 con 0x7fe7e01038f0 2024-02-22T15:56:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e011a0c0 con 0x7fe7e01038f0 2024-02-22T15:56:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7cc00f040 con 0x7fe7e01038f0 2024-02-22T15:56:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.317+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7cc0095d0 con 0x7fe7e01038f0 2024-02-22T15:56:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.318+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c40052e0 con 0x7fe7e01038f0 2024-02-22T15:56:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.319+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe7cc0097d0 con 0x7fe7e01038f0 2024-02-22T15:56:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.320+0000 7fe7dd7fa700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe7d006cc30 0x7fe7d006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:56:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.320+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe7cc08bcf0 con 0x7fe7e01038f0 2024-02-22T15:56:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.320+0000 7fe7e4880700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe7d006cc30 0x7fe7d006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:56:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.321+0000 7fe7e4880700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe7d006cc30 0x7fe7d006f0e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004f90 tx=0x7fe7d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:56:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.325+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe7cc055dd0 con 0x7fe7e01038f0 2024-02-22T15:56:32.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:32 smithi019 ceph-mon[173579]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:32.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:32 smithi019 ceph-mon[173579]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-02-22T15:56:32.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:32 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3986044025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:32.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:32 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2426380350' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:56:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.621+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7c4005590 con 0x7fe7e01038f0 2024-02-22T15:56:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.622+0000 7fe7dd7fa700 1 -- 172.21.15.19:0/4234990959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1160 (secure 0 0 0) 0x7fe7cc0593f0 con 0x7fe7e01038f0 2024-02-22T15:56:32.601 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 51/318 objects degraded (16.038%), 16 pgs degraded 2024-02-22T15:56:32.601 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: osd.1 (root=default,host=smithi019) is down 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 51/318 objects degraded (16.038%), 16 pgs degraded 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-02-22T15:56:32.602 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-02-22T15:56:32.603 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-02-22T15:56:32.603 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-02-22T15:56:32.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe7d006cc30 msgr2=0x7fe7d006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:32.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe7d006cc30 0x7fe7d006f0e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004f90 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 msgr2=0x7fe7e00707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 0x7fe7e00707d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc002a50 tx=0x7fe7cc0046b0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 shutdown_connections 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe7d006cc30 0x7fe7d006f0e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7e0102f50 0x7fe7e0070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 --2- 172.21.15.19:0/4234990959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7e01038f0 0x7fe7e00707d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 >> 172.21.15.19:0/4234990959 conn(0x7fe7e00f8ae0 msgr2=0x7fe7e00fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 shutdown_connections 2024-02-22T15:56:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:56:32.624+0000 7fe7e6ae4700 1 -- 172.21.15.19:0/4234990959 wait complete. 2024-02-22T15:56:32.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:32 smithi062 ceph-mon[106861]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:32.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:32 smithi062 ceph-mon[106861]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-02-22T15:56:32.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:32 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3986044025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:32.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:32 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2426380350' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:56:33.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:33 smithi019 ceph-mon[173579]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:33.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:33 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4234990959' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:56:33.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:33.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:33.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:33 smithi062 ceph-mon[106861]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:56:33.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:33 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4234990959' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:56:33.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:33.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:34.243 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:34 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T15:56:34.133+0000 7f16237bc640 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-22T15:56:34.993 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:34 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T15:56:34.679+0000 7f16237bc640 -1 osd.1 55 log_to_monitors true 2024-02-22T15:56:34.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:34 smithi019 ceph-mon[173579]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-02-22T15:56:34.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:34.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:34.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:34 smithi019 ceph-mon[173579]: from='osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T15:56:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:34 smithi062 ceph-mon[106861]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 51/318 objects degraded (16.038%) 2024-02-22T15:56:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:35.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:34 smithi062 ceph-mon[106861]: from='osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-22T15:56:35.739 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 15:56:35 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T15:56:35.505+0000 7f1612c11700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:56:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:36 smithi019 ceph-mon[173579]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 51/318 objects degraded (16.038%) 2024-02-22T15:56:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:36 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:36 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:36 smithi019 ceph-mon[173579]: from='osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T15:56:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:36 smithi019 ceph-mon[173579]: osdmap e58: 6 total, 5 up, 6 in 2024-02-22T15:56:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:36 smithi019 ceph-mon[173579]: from='osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:56:36.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:36 smithi062 ceph-mon[106861]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 51/318 objects degraded (16.038%) 2024-02-22T15:56:36.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:36 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:36.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:36 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:36.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:36 smithi062 ceph-mon[106861]: from='osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-22T15:56:36.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:36 smithi062 ceph-mon[106861]: osdmap e58: 6 total, 5 up, 6 in 2024-02-22T15:56:36.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:36 smithi062 ceph-mon[106861]: from='osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: failed to encode map e58 with expected crc 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: failed to encode map e58 with expected crc 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: failed to encode map e58 with expected crc 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: failed to encode map e59 with expected crc 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852] boot 2024-02-22T15:56:37.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: osdmap e59: 6 total, 6 up, 6 in 2024-02-22T15:56:37.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:37 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: failed to encode map e58 with expected crc 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: failed to encode map e58 with expected crc 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: failed to encode map e58 with expected crc 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: failed to encode map e59 with expected crc 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: osd.1 [v2:172.21.15.19:6810/1876741852,v1:172.21.15.19:6811/1876741852] boot 2024-02-22T15:56:37.496 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: osdmap e59: 6 total, 6 up, 6 in 2024-02-22T15:56:37.497 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:37 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-22T15:56:38.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: pgmap v74: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 51/318 objects degraded (16.038%) 2024-02-22T15:56:38.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: osdmap e60: 6 total, 6 up, 6 in 2024-02-22T15:56:38.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.652 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.653 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:38 smithi019 ceph-mon[173579]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: pgmap v74: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 51/318 objects degraded (16.038%) 2024-02-22T15:56:38.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: osdmap e60: 6 total, 6 up, 6 in 2024-02-22T15:56:38.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:38.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:38 smithi062 ceph-mon[106861]: failed to encode map e60 with expected crc 2024-02-22T15:56:40.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:40.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:40.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: pgmap v76: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s; 22/318 objects degraded (6.918%) 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:40.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:40.457 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:40 smithi062 ceph-mon[106861]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-02-22T15:56:40.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:40.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:56:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: pgmap v76: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s; 22/318 objects degraded (6.918%) 2024-02-22T15:56:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:56:40.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:40.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:40.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:56:40.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:40.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:40.495 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:40 smithi019 ceph-mon[173579]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-02-22T15:56:41.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:41 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T15:56:41.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:41 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T15:56:42.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:42 smithi062 ceph-mon[106861]: pgmap v77: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-22T15:56:42.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:42 smithi019 ceph-mon[173579]: pgmap v77: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-22T15:56:43.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:43 smithi062 ceph-mon[106861]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded) 2024-02-22T15:56:43.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:43 smithi062 ceph-mon[106861]: Cluster is now healthy 2024-02-22T15:56:43.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:43 smithi019 ceph-mon[173579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/318 objects degraded (6.918%), 7 pgs degraded) 2024-02-22T15:56:43.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:43 smithi019 ceph-mon[173579]: Cluster is now healthy 2024-02-22T15:56:44.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:44 smithi062 ceph-mon[106861]: pgmap v78: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 940 B/s rd, 0 op/s 2024-02-22T15:56:44.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:44 smithi019 ceph-mon[173579]: pgmap v78: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 940 B/s rd, 0 op/s 2024-02-22T15:56:46.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:46 smithi062 ceph-mon[106861]: pgmap v79: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-22T15:56:46.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:46 smithi019 ceph-mon[173579]: pgmap v79: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-22T15:56:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:48 smithi062 ceph-mon[106861]: pgmap v80: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-22T15:56:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:48 smithi019 ceph-mon[173579]: pgmap v80: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-22T15:56:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:56:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:51.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:50 smithi062 ceph-mon[106861]: pgmap v81: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-02-22T15:56:51.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:50 smithi019 ceph-mon[173579]: pgmap v81: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-02-22T15:56:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:52 smithi062 ceph-mon[106861]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-22T15:56:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:52 smithi019 ceph-mon[173579]: pgmap v82: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-22T15:56:54.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:54 smithi019 ceph-mon[173579]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:54.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:54 smithi062 ceph-mon[106861]: pgmap v83: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:55.399 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:56.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:55 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:56.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:55 smithi062 ceph-mon[106861]: Upgrade: osd.2 is safe to restart 2024-02-22T15:56:56.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:56.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T15:56:56.208 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:55 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-22T15:56:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:55 smithi019 ceph-mon[173579]: Upgrade: osd.2 is safe to restart 2024-02-22T15:56:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:56:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-22T15:56:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:56:57.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:56 smithi062 ceph-mon[106861]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:56:57.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:56 smithi062 ceph-mon[106861]: Upgrade: Updating osd.2 2024-02-22T15:56:57.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:56 smithi062 ceph-mon[106861]: Deploying daemon osd.2 on smithi019 2024-02-22T15:56:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:56 smithi019 ceph-mon[173579]: pgmap v84: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:56:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:56 smithi019 ceph-mon[173579]: Upgrade: Updating osd.2 2024-02-22T15:56:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:56 smithi019 ceph-mon[173579]: Deploying daemon osd.2 on smithi019 2024-02-22T15:56:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:58 smithi062 ceph-mon[106861]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:59.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:58 smithi019 ceph-mon[173579]: pgmap v85: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:56:59.493 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:56:59 smithi019 systemd[1]: Stopping Ceph osd.2 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:56:59.908 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:56:59 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[110364]: 2024-02-22T15:56:59.565+0000 7f8562511700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:56:59.908 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:56:59 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[110364]: 2024-02-22T15:56:59.565+0000 7f8562511700 -1 osd.2 60 *** Got signal Terminated *** 2024-02-22T15:56:59.908 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:56:59 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[110364]: 2024-02-22T15:56:59.565+0000 7f8562511700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T15:57:00.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:56:59 smithi062 ceph-mon[106861]: osd.2 marked itself down and dead 2024-02-22T15:57:00.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:56:59 smithi019 ceph-mon[173579]: osd.2 marked itself down and dead 2024-02-22T15:57:00.909 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:00 smithi019 bash[202999]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2 2024-02-22T15:57:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:57:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:57:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: osdmap e61: 6 total, 5 up, 6 in 2024-02-22T15:57:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:00 smithi062 ceph-mon[106861]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: pgmap v86: 65 pgs: 65 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:57:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:57:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: osdmap e61: 6 total, 5 up, 6 in 2024-02-22T15:57:01.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: failed to encode map e61 with expected crc 2024-02-22T15:57:01.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:00 smithi019 ceph-mon[173579]: failed to encode map e61 with expected crc 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: osdmap e62: 6 total, 5 up, 6 in 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:01 smithi062 ceph-mon[106861]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: osdmap e62: 6 total, 5 up, 6 in 2024-02-22T15:57:02.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:01 smithi019 ceph-mon[173579]: failed to encode map e62 with expected crc 2024-02-22T15:57:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 -- 172.21.15.19:0/115407706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 msgr2=0x7f97f8103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 --2- 172.21.15.19:0/115407706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8103310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97e8004940 tx=0x7f97e8009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 -- 172.21.15.19:0/115407706 shutdown_connections 2024-02-22T15:57:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 --2- 172.21.15.19:0/115407706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97f81038e0 0x7f97f80fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 --2- 172.21.15.19:0/115407706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8103310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 -- 172.21.15.19:0/115407706 >> 172.21.15.19:0/115407706 conn(0x7f97f80f8af0 msgr2=0x7f97f80faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 -- 172.21.15.19:0/115407706 shutdown_connections 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.744+0000 7f97fce6a700 1 -- 172.21.15.19:0/115407706 wait complete. 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97fce6a700 1 Processor -- start 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97fce6a700 1 -- start start 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97fce6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97fce6a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97f81038e0 0x7f97f80707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97fce6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8070e40 con 0x7f97f8102f40 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97fce6a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8070fb0 con 0x7f97f81038e0 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97f659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97f659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44090/0 (socket says 172.21.15.19:44090) 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97f659c700 1 -- 172.21.15.19:0/803209248 learned_addr learned my addr 172.21.15.19:0/803209248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.745+0000 7f97f5d9b700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97f81038e0 0x7f97f80707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:02.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97f659c700 1 -- 172.21.15.19:0/803209248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97f81038e0 msgr2=0x7f97f80707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97f659c700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97f81038e0 0x7f97f80707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97f659c700 1 -- 172.21.15.19:0/803209248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8005040 con 0x7f97f8102f40 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97f659c700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8070290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009180 tx=0x7f97e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e800e040 con 0x7f97f8102f40 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f8119ba0 con 0x7f97f8102f40 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97e8004290 con 0x7f97f8102f40 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e801c3f0 con 0x7f97f8102f40 2024-02-22T15:57:02.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.746+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f811a080 con 0x7f97f8102f40 2024-02-22T15:57:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.747+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f80fc6f0 con 0x7f97f8102f40 2024-02-22T15:57:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.748+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f97e8013420 con 0x7f97f8102f40 2024-02-22T15:57:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.748+0000 7f97e6ffd700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f97e006cc20 0x7f97e006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.748+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f97e808d210 con 0x7f97f8102f40 2024-02-22T15:57:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.748+0000 7f97f5d9b700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f97e006cc20 0x7f97e006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.749+0000 7f97f5d9b700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f97e006cc20 0x7f97e006f0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97f8071c90 tx=0x7f97ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.753+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f97e80572f0 con 0x7f97f8102f40 2024-02-22T15:57:02.743 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:02 smithi019 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.2.service: Succeeded. 2024-02-22T15:57:02.743 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:02 smithi019 systemd[1]: Stopped Ceph osd.2 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:57:02.743 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:02 smithi019 systemd[1]: Starting Ceph osd.2 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:57:02.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.980+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97f8071ae0 con 0x7f97e006cc20 2024-02-22T15:57:02.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.981+0000 7f97e6ffd700 1 -- 172.21.15.19:0/803209248 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f97f8071ae0 con 0x7f97e006cc20 2024-02-22T15:57:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f97e006cc20 msgr2=0x7f97e006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f97e006cc20 0x7f97e006f0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97f8071c90 tx=0x7f97ec00b040 comp rx=0 tx=0).stop 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 msgr2=0x7f97f8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8070290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009180 tx=0x7f97e8009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 shutdown_connections 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f97e006cc20 0x7f97e006f0d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f8102f40 0x7f97f8070290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 --2- 172.21.15.19:0/803209248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97f81038e0 0x7f97f80707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 >> 172.21.15.19:0/803209248 conn(0x7f97f80f8af0 msgr2=0x7f97f80fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 shutdown_connections 2024-02-22T15:57:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:02.983+0000 7f97fce6a700 1 -- 172.21.15.19:0/803209248 wait complete. 2024-02-22T15:57:02.975 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:57:03.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/2182253114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fe8f0 msgr2=0x7f8b48107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/2182253114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fe8f0 0x7f8b48107050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c004970 tx=0x7f8b3c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/2182253114 shutdown_connections 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/2182253114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fe8f0 0x7f8b48107050 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/2182253114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fdfe0 0x7f8b480fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/2182253114 >> 172.21.15.19:0/2182253114 conn(0x7f8b48071d90 msgr2=0x7f8b48072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/2182253114 shutdown_connections 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.102+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/2182253114 wait complete. 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4c9bf700 1 Processor -- start 2024-02-22T15:57:03.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4c9bf700 1 -- start start 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4c9bf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 0x7f8b48115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4c9bf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fe8f0 0x7f8b481160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4c9bf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48116700 con 0x7f8b480fdfe0 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4c9bf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4810fa30 con 0x7f8b480fe8f0 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 0x7f8b48115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 0x7f8b48115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44100/0 (socket says 172.21.15.19:44100) 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.103+0000 7f8b4659c700 1 -- 172.21.15.19:0/103656053 learned_addr learned my addr 172.21.15.19:0/103656053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b45d9b700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fe8f0 0x7f8b481160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b4659c700 1 -- 172.21.15.19:0/103656053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fe8f0 msgr2=0x7f8b481160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b4659c700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fe8f0 0x7f8b481160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b4659c700 1 -- 172.21.15.19:0/103656053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c005040 con 0x7f8b480fdfe0 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b45d9b700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fe8f0 0x7f8b481160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:57:03.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b4659c700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 0x7f8b48115b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c8c0 tx=0x7f8b380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b3800a980 con 0x7f8b480fdfe0 2024-02-22T15:57:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b3800ce60 con 0x7f8b480fdfe0 2024-02-22T15:57:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4810fd10 con 0x7f8b480fdfe0 2024-02-22T15:57:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b380113a0 con 0x7f8b480fdfe0 2024-02-22T15:57:03.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.104+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b48110250 con 0x7f8b480fdfe0 2024-02-22T15:57:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.105+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48106ac0 con 0x7f8b480fdfe0 2024-02-22T15:57:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.106+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8b38012400 con 0x7f8b480fdfe0 2024-02-22T15:57:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.106+0000 7f8b36ffd700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8b3006cc20 0x7f8b3006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.106+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8b3808cb10 con 0x7f8b480fdfe0 2024-02-22T15:57:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.106+0000 7f8b45d9b700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8b3006cc20 0x7f8b3006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.106+0000 7f8b45d9b700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8b3006cc20 0x7f8b3006f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00b370 tx=0x7f8b3c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.110+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8b3805a720 con 0x7f8b480fdfe0 2024-02-22T15:57:03.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:02 smithi062 ceph-mon[106861]: pgmap v89: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-02-22T15:57:03.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:02 smithi062 ceph-mon[106861]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-02-22T15:57:03.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:02 smithi019 ceph-mon[173579]: pgmap v89: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-02-22T15:57:03.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:02 smithi019 ceph-mon[173579]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-02-22T15:57:03.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.335+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b4804bf40 con 0x7f8b3006cc20 2024-02-22T15:57:03.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.336+0000 7f8b36ffd700 1 -- 172.21.15.19:0/103656053 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8b4804bf40 con 0x7f8b3006cc20 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8b3006cc20 msgr2=0x7f8b3006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8b3006cc20 0x7f8b3006f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00b370 tx=0x7f8b3c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 msgr2=0x7f8b48115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 0x7f8b48115b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c8c0 tx=0x7f8b380044c0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 shutdown_connections 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8b3006cc20 0x7f8b3006f0d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b480fdfe0 0x7f8b48115b70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 --2- 172.21.15.19:0/103656053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b480fe8f0 0x7f8b481160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 >> 172.21.15.19:0/103656053 conn(0x7f8b48071d90 msgr2=0x7f8b48103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:03.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 shutdown_connections 2024-02-22T15:57:03.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.338+0000 7f8b4c9bf700 1 -- 172.21.15.19:0/103656053 wait complete. 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.444+0000 7f36d1402700 1 -- 172.21.15.19:0/2454011528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 msgr2=0x7f36cc102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 --2- 172.21.15.19:0/2454011528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc102230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36bc004940 tx=0x7f36bc009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 -- 172.21.15.19:0/2454011528 shutdown_connections 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 --2- 172.21.15.19:0/2454011528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 0x7f36cc0fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 --2- 172.21.15.19:0/2454011528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc102230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 -- 172.21.15.19:0/2454011528 >> 172.21.15.19:0/2454011528 conn(0x7f36cc0f7a10 msgr2=0x7f36cc0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 -- 172.21.15.19:0/2454011528 shutdown_connections 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 -- 172.21.15.19:0/2454011528 wait complete. 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 Processor -- start 2024-02-22T15:57:03.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.445+0000 7f36d1402700 1 -- start start 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36d1402700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc10ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36d1402700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 0x7f36cc10ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36d1402700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc10f500 con 0x7f36cc101e60 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36d1402700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc10f640 con 0x7f36cc102800 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc10ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc10ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44122/0 (socket says 172.21.15.19:44122) 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 -- 172.21.15.19:0/3664232656 learned_addr learned my addr 172.21.15.19:0/3664232656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36ca7fc700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 0x7f36cc10ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 -- 172.21.15.19:0/3664232656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 msgr2=0x7f36cc10ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 0x7f36cc10ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 -- 172.21.15.19:0/3664232656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc005040 con 0x7f36cc101e60 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36ca7fc700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 0x7f36cc10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36caffd700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc10ea30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b3b0 tx=0x7f36bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.446+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36bc00e040 con 0x7f36cc101e60 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.447+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36bc004290 con 0x7f36cc101e60 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.447+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc117940 con 0x7f36cc101e60 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.447+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36bc01c3f0 con 0x7f36cc101e60 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.447+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc117e50 con 0x7f36cc101e60 2024-02-22T15:57:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.448+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36cc0ff170 con 0x7f36cc101e60 2024-02-22T15:57:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.448+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f36bc00bc50 con 0x7f36cc101e60 2024-02-22T15:57:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.448+0000 7f36b37fe700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f36b406cc20 0x7f36b406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.449+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f36bc08d3f0 con 0x7f36cc101e60 2024-02-22T15:57:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.449+0000 7f36ca7fc700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f36b406cc20 0x7f36b406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.449+0000 7f36ca7fc700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f36b406cc20 0x7f36b406f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009fd0 tx=0x7f36c0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.453+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f36bc0574d0 con 0x7f36cc101e60 2024-02-22T15:57:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.676+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f36cc066750 con 0x7f36b406cc20 2024-02-22T15:57:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.686+0000 7f36b37fe700 1 -- 172.21.15.19:0/3664232656 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f36cc066750 con 0x7f36b406cc20 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (4m) 29s ago 15m 26.8M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (97s) 29s ago 15m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (93s) 89s ago 14m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (3m) 29s ago 15m 78.4M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (9m) 29s ago 9m 113M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (9m) 29s ago 9m 67.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (9m) 89s ago 9m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:57:03.666 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (9m) 89s ago 9m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (6m) 29s ago 16m 498M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (5m) 89s ago 14m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (2m) 29s ago 17m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (2m) 89s ago 14m 37.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (5m) 29s ago 15m 18.7M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (5m) 89s ago 14m 17.0M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (72s) 29s ago 13m 87.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (34s) 29s ago 12m 11.3M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (12m) 29s ago 12m 197M 1570M 17.2.7-527-g6ea2a55e e2e4ad29904c c6c0b8ce61e8 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (12m) 89s ago 12m 259M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (11m) 89s ago 11m 229M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (11m) 89s ago 11m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:57:03.667 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (4m) 29s ago 14m 59.7M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:57:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f36b406cc20 msgr2=0x7f36b406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f36b406cc20 0x7f36b406f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009fd0 tx=0x7f36c0009ee0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 msgr2=0x7f36cc10ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc10ea30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b3b0 tx=0x7f36bc009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 shutdown_connections 2024-02-22T15:57:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f36b406cc20 0x7f36b406f0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36cc101e60 0x7f36cc10ea30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 --2- 172.21.15.19:0/3664232656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36cc102800 0x7f36cc10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 >> 172.21.15.19:0/3664232656 conn(0x7f36cc0f7a10 msgr2=0x7f36cc0f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.688+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 shutdown_connections 2024-02-22T15:57:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.689+0000 7f36d1402700 1 -- 172.21.15.19:0/3664232656 wait complete. 2024-02-22T15:57:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 -- 172.21.15.19:0/62783416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 msgr2=0x7f001c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 --2- 172.21.15.19:0/62783416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c103320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f000c004940 tx=0x7f000c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 -- 172.21.15.19:0/62783416 shutdown_connections 2024-02-22T15:57:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 --2- 172.21.15.19:0/62783416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001c1038f0 0x7f001c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 --2- 172.21.15.19:0/62783416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c103320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 -- 172.21.15.19:0/62783416 >> 172.21.15.19:0/62783416 conn(0x7f001c0f8ae0 msgr2=0x7f001c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 -- 172.21.15.19:0/62783416 shutdown_connections 2024-02-22T15:57:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.793+0000 7f00226c7700 1 -- 172.21.15.19:0/62783416 wait complete. 2024-02-22T15:57:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f00226c7700 1 Processor -- start 2024-02-22T15:57:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f00226c7700 1 -- start start 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f00226c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f00226c7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001c1038f0 0x7f001c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f00226c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c114500 con 0x7f001c102f50 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f00226c7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c10d830 con 0x7f001c1038f0 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f001bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f001bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44138/0 (socket says 172.21.15.19:44138) 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.794+0000 7f001bfff700 1 -- 172.21.15.19:0/1544323318 learned_addr learned my addr 172.21.15.19:0/1544323318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f001b7fe700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001c1038f0 0x7f001c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f001bfff700 1 -- 172.21.15.19:0/1544323318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001c1038f0 msgr2=0x7f001c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f001bfff700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001c1038f0 0x7f001c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f001bfff700 1 -- 172.21.15.19:0/1544323318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c005040 con 0x7f001c102f50 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f001bfff700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c113970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f000c009180 tx=0x7f000c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000c00e040 con 0x7f001c102f50 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f000c013420 con 0x7f001c102f50 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c10dab0 con 0x7f001c102f50 2024-02-22T15:57:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000c01c630 con 0x7f001c102f50 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.795+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c10dff0 con 0x7f001c102f50 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.796+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c100260 con 0x7f001c102f50 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.796+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f000c0139e0 con 0x7f001c102f50 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.797+0000 7f0018ff9700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f000406cc20 0x7f000406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.797+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f000c08d2d0 con 0x7f001c102f50 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.797+0000 7f001b7fe700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f000406cc20 0x7f000406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.797+0000 7f001b7fe700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f000406cc20 0x7f000406f0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f001c10f1a0 tx=0x7f0010009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:03.801+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f000c05ae40 con 0x7f001c102f50 2024-02-22T15:57:03.929 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-22T15:57:03.929 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-22T15:57:03.929 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-22T15:57:03.929 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-22T15:57:03.930 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-22T15:57:03.930 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-22T15:57:03.930 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:03 smithi019 bash[203291]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-22T15:57:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.096+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f001c0fc6e0 con 0x7f001c102f50 2024-02-22T15:57:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.097+0000 7f0018ff9700 1 -- 172.21.15.19:0/1544323318 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f000c05a9d0 con 0x7f001c102f50 2024-02-22T15:57:04.076 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:57:04.076 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 3, 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 7, 2024-02-22T15:57:04.077 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T15:57:04.078 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:57:04.078 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:57:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f000406cc20 msgr2=0x7f000406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f000406cc20 0x7f000406f0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f001c10f1a0 tx=0x7f0010009ee0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 msgr2=0x7f001c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c113970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f000c009180 tx=0x7f000c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 shutdown_connections 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f000406cc20 0x7f000406f0d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f001c102f50 0x7f001c113970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 --2- 172.21.15.19:0/1544323318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001c1038f0 0x7f001c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 >> 172.21.15.19:0/1544323318 conn(0x7f001c0f8ae0 msgr2=0x7f001c0fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 shutdown_connections 2024-02-22T15:57:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.099+0000 7f00226c7700 1 -- 172.21.15.19:0/1544323318 wait complete. 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 -- 172.21.15.19:0/4046429515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 msgr2=0x7fc9480fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 --2- 172.21.15.19:0/4046429515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc9480fbea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc930004940 tx=0x7fc930009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 -- 172.21.15.19:0/4046429515 shutdown_connections 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 --2- 172.21.15.19:0/4046429515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc9480fbea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 --2- 172.21.15.19:0/4046429515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc948101e40 0x7fc948102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 -- 172.21.15.19:0/4046429515 >> 172.21.15.19:0/4046429515 conn(0x7fc948071d90 msgr2=0x7fc948072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 -- 172.21.15.19:0/4046429515 shutdown_connections 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.203+0000 7fc94dd55700 1 -- 172.21.15.19:0/4046429515 wait complete. 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc94dd55700 1 Processor -- start 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc94dd55700 1 -- start start 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc94dd55700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc948101e40 0x7fc948112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc94dd55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc948112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc94dd55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc948113390 con 0x7fc9481027e0 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc94dd55700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94810c6c0 con 0x7fc948101e40 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc93effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc948112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc93effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc948112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44160/0 (socket says 172.21.15.19:44160) 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.204+0000 7fc93effd700 1 -- 172.21.15.19:0/2859120826 learned_addr learned my addr 172.21.15.19:0/2859120826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:04.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc9477fe700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc948101e40 0x7fc948112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc93effd700 1 -- 172.21.15.19:0/2859120826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc948101e40 msgr2=0x7fc948112800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc93effd700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc948101e40 0x7fc948112800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc93effd700 1 -- 172.21.15.19:0/2859120826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc930005040 con 0x7fc9481027e0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc93effd700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc948112d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc930009390 tx=0x7fc930009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc93000e040 con 0x7fc9481027e0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc93000be60 con 0x7fc9481027e0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94810c940 con 0x7fc9481027e0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc930013730 con 0x7fc9481027e0 2024-02-22T15:57:04.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.205+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94810ce20 con 0x7fc9481027e0 2024-02-22T15:57:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.206+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9480738b0 con 0x7fc9481027e0 2024-02-22T15:57:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.207+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc93001eb60 con 0x7fc9481027e0 2024-02-22T15:57:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.207+0000 7fc944ff9700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc93406cc30 0x7fc93406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.207+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc930090020 con 0x7fc9481027e0 2024-02-22T15:57:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.207+0000 7fc9477fe700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc93406cc30 0x7fc93406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.208+0000 7fc9477fe700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc93406cc30 0x7fc93406f0e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc938009fd0 tx=0x7fc938009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.212+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc93005db10 con 0x7fc9481027e0 2024-02-22T15:57:04.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:03 smithi062 ceph-mon[106861]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:04.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:04.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:03 smithi019 ceph-mon[173579]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:04.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.471+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc94804bed0 con 0x7fc9481027e0 2024-02-22T15:57:04.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.472+0000 7fc944ff9700 1 -- 172.21.15.19:0/2859120826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fc93005d6a0 con 0x7fc9481027e0 2024-02-22T15:57:04.451 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:57:04.452 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:57:04.453 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:04.454 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:04.455 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:57:04.455 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:57:04.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc93406cc30 msgr2=0x7fc93406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc93406cc30 0x7fc93406f0e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc938009fd0 tx=0x7fc938009f60 comp rx=0 tx=0).stop 2024-02-22T15:57:04.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 msgr2=0x7fc948112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc948112d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc930009390 tx=0x7fc930009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 shutdown_connections 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc93406cc30 0x7fc93406f0e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc948101e40 0x7fc948112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 --2- 172.21.15.19:0/2859120826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9481027e0 0x7fc948112d40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.474+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 >> 172.21.15.19:0/2859120826 conn(0x7fc948071d90 msgr2=0x7fc9480fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.475+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 shutdown_connections 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.475+0000 7fc94dd55700 1 -- 172.21.15.19:0/2859120826 wait complete. 2024-02-22T15:57:04.456 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:57:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 -- 172.21.15.19:0/2610416313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fe8f0 msgr2=0x7f068c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 --2- 172.21.15.19:0/2610416313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fe8f0 0x7f068c107050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0680004970 tx=0x7f0680009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 -- 172.21.15.19:0/2610416313 shutdown_connections 2024-02-22T15:57:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 --2- 172.21.15.19:0/2610416313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fe8f0 0x7f068c107050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 --2- 172.21.15.19:0/2610416313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f068c0fdfe0 0x7f068c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 -- 172.21.15.19:0/2610416313 >> 172.21.15.19:0/2610416313 conn(0x7f068c071d90 msgr2=0x7f068c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 -- 172.21.15.19:0/2610416313 shutdown_connections 2024-02-22T15:57:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.581+0000 7f06912a2700 1 -- 172.21.15.19:0/2610416313 wait complete. 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f06912a2700 1 Processor -- start 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f06912a2700 1 -- start start 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f06912a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 0x7f068c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f06912a2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f068c0fe8f0 0x7f068c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f06912a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c114500 con 0x7f068c0fdfe0 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f06912a2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c10d830 con 0x7f068c0fe8f0 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f068affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 0x7f068c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f068affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 0x7f068c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44184/0 (socket says 172.21.15.19:44184) 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.582+0000 7f068affd700 1 -- 172.21.15.19:0/3728729675 learned_addr learned my addr 172.21.15.19:0/3728729675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f068a7fc700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f068c0fe8f0 0x7f068c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f068affd700 1 -- 172.21.15.19:0/3728729675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f068c0fe8f0 msgr2=0x7f068c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f068affd700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f068c0fe8f0 0x7f068c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f068affd700 1 -- 172.21.15.19:0/3728729675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0680005040 con 0x7f068c0fdfe0 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f068affd700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 0x7f068c113970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f067c00eb40 tx=0x7f067c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f067c007cc0 con 0x7f068c0fdfe0 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f067c00f040 con 0x7f068c0fdfe0 2024-02-22T15:57:04.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c10dab0 con 0x7f068c0fdfe0 2024-02-22T15:57:04.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c10dfc0 con 0x7f068c0fdfe0 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.583+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f067c009570 con 0x7f068c0fdfe0 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.584+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06780052e0 con 0x7f068c0fdfe0 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.585+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f067c0096d0 con 0x7f068c0fdfe0 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.585+0000 7f06737fe700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067406cc30 0x7f067406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.585+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f067c08bba0 con 0x7f068c0fdfe0 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.586+0000 7f068a7fc700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067406cc30 0x7f067406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.586+0000 7f068a7fc700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067406cc30 0x7f067406f0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f068c10eb60 tx=0x7f0680009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.590+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f067c059740 con 0x7f068c0fdfe0 2024-02-22T15:57:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.815+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0678000bf0 con 0x7f067406cc30 2024-02-22T15:57:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.817+0000 7f06737fe700 1 -- 172.21.15.19:0/3728729675 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0678000bf0 con 0x7f067406cc30 2024-02-22T15:57:04.796 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:57:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:57:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:57:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:57:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "8/21 daemons upgraded", 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading osd daemons", 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:57:04.797 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:57:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067406cc30 msgr2=0x7f067406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067406cc30 0x7f067406f0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f068c10eb60 tx=0x7f0680009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 msgr2=0x7f068c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 0x7f068c113970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f067c00eb40 tx=0x7f067c00b040 comp rx=0 tx=0).stop 2024-02-22T15:57:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 shutdown_connections 2024-02-22T15:57:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f067406cc30 0x7f067406f0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c0fdfe0 0x7f068c113970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 --2- 172.21.15.19:0/3728729675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f068c0fe8f0 0x7f068c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 >> 172.21.15.19:0/3728729675 conn(0x7f068c071d90 msgr2=0x7f068c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 shutdown_connections 2024-02-22T15:57:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.819+0000 7f06912a2700 1 -- 172.21.15.19:0/3728729675 wait complete. 2024-02-22T15:57:04.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 -- 172.21.15.19:0/1031365160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 msgr2=0x7fa6380fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 --2- 172.21.15.19:0/1031365160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa6380fbea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa620004940 tx=0x7fa620009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 -- 172.21.15.19:0/1031365160 shutdown_connections 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 --2- 172.21.15.19:0/1031365160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa6380fbea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 --2- 172.21.15.19:0/1031365160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638101e40 0x7fa638102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 -- 172.21.15.19:0/1031365160 >> 172.21.15.19:0/1031365160 conn(0x7fa638071d90 msgr2=0x7fa638072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 -- 172.21.15.19:0/1031365160 shutdown_connections 2024-02-22T15:57:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.926+0000 7fa63d766700 1 -- 172.21.15.19:0/1031365160 wait complete. 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa63d766700 1 Processor -- start 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa63d766700 1 -- start start 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa63d766700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638101e40 0x7fa6381128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa63d766700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa638112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa63d766700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa638113460 con 0x7fa6381027e0 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa63d766700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63810c790 con 0x7fa638101e40 2024-02-22T15:57:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa6367fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa638112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa6367fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa638112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44204/0 (socket says 172.21.15.19:44204) 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa6367fc700 1 -- 172.21.15.19:0/2576256263 learned_addr learned my addr 172.21.15.19:0/2576256263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa636ffd700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638101e40 0x7fa6381128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa6367fc700 1 -- 172.21.15.19:0/2576256263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638101e40 msgr2=0x7fa6381128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa6367fc700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638101e40 0x7fa6381128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.927+0000 7fa6367fc700 1 -- 172.21.15.19:0/2576256263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620005040 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.928+0000 7fa6367fc700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa638112e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa620009380 tx=0x7fa620009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.928+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62000e040 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.928+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa620013420 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.928+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63810ca10 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.928+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62001c630 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.928+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63810cf50 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.929+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa638104310 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.929+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa620013590 con 0x7fa6381027e0 2024-02-22T15:57:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.930+0000 7fa62f7fe700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa62406cc20 0x7fa62406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:04.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.930+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa62008d200 con 0x7fa6381027e0 2024-02-22T15:57:04.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.930+0000 7fa636ffd700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa62406cc20 0x7fa62406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:04.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.930+0000 7fa636ffd700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa62406cc20 0x7fa62406f0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa628007a40 tx=0x7fa628009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:04.934+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa62005ae00 con 0x7fa6381027e0 2024-02-22T15:57:05.204 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 4 pgs peering; Degraded data redundancy: 11/318 objects degraded (3.459%), 4 pgs degraded 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: osd.2 (root=default,host=smithi019) is down 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 4 pgs peering 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.1 is stuck peering for 4m, current state peering, last acting [1,0] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.10 is stuck peering for 5m, current state peering, last acting [1,0] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 3.7 is stuck peering for 5m, current state peering, last acting [1,4] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 3.8 is stuck peering for 5m, current state peering, last acting [1,5] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 11/318 objects degraded (3.459%), 4 pgs degraded 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-02-22T15:57:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-02-22T15:57:05.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.224+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa638108de0 con 0x7fa6381027e0 2024-02-22T15:57:05.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.225+0000 7fa62f7fe700 1 -- 172.21.15.19:0/2576256263 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+908 (secure 0 0 0) 0x7fa62005a990 con 0x7fa6381027e0 2024-02-22T15:57:05.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:04 smithi062 ceph-mon[106861]: pgmap v90: 65 pgs: 4 active+undersized+degraded, 7 peering, 2 stale+active+clean, 4 active+undersized, 48 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 11/318 objects degraded (3.459%) 2024-02-22T15:57:05.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:04 smithi062 ceph-mon[106861]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:05.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:04 smithi062 ceph-mon[106861]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:05.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:04 smithi062 ceph-mon[106861]: Health check failed: Degraded data redundancy: 11/318 objects degraded (3.459%), 4 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:05.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:04 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1544323318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:05.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:04 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2859120826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:57:05.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa62406cc20 msgr2=0x7fa62406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:05.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa62406cc20 0x7fa62406f0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa628007a40 tx=0x7fa628009da0 comp rx=0 tx=0).stop 2024-02-22T15:57:05.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 msgr2=0x7fa638112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:05.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa638112e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa620009380 tx=0x7fa620009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 shutdown_connections 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa62406cc20 0x7fa62406f0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638101e40 0x7fa6381128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 --2- 172.21.15.19:0/2576256263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6381027e0 0x7fa638112e10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 >> 172.21.15.19:0/2576256263 conn(0x7fa638071d90 msgr2=0x7fa6380faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 shutdown_connections 2024-02-22T15:57:05.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:05.227+0000 7fa63d766700 1 -- 172.21.15.19:0/2576256263 wait complete. 2024-02-22T15:57:05.243 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-osd[203735]: -- 172.21.15.19:0/2676903393 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595ed9444e0 con 0x5595ed914c00 2024-02-22T15:57:05.243 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:04 smithi019 bash[203671]: 6d01d37aaf6c4e065aa097a3733a9bac85a4cbdabe271dbe625d4eef751dee7c 2024-02-22T15:57:05.243 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:04 smithi019 systemd[1]: Started Ceph osd.2 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:57:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-mon[173579]: pgmap v90: 65 pgs: 4 active+undersized+degraded, 7 peering, 2 stale+active+clean, 4 active+undersized, 48 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 11/318 objects degraded (3.459%) 2024-02-22T15:57:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-mon[173579]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-mon[173579]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:05.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-mon[173579]: Health check failed: Degraded data redundancy: 11/318 objects degraded (3.459%), 4 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:05.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1544323318' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:05.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:04 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2859120826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:57:05.696 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:05 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T15:57:05.382+0000 7ff29c0ef640 -1 Falling back to public interface 2024-02-22T15:57:06.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:05 smithi019 ceph-mon[173579]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:06.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:06.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:06.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:06.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:05 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2576256263' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:57:06.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:05 smithi062 ceph-mon[106861]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:06.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:06.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:06.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:06.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:05 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2576256263' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:57:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:06 smithi019 ceph-mon[173579]: pgmap v91: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 24/318 objects degraded (7.547%) 2024-02-22T15:57:07.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:06 smithi062 ceph-mon[106861]: pgmap v91: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 24/318 objects degraded (7.547%) 2024-02-22T15:57:09.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:08 smithi062 ceph-mon[106861]: pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-02-22T15:57:09.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:08 smithi062 ceph-mon[106861]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-02-22T15:57:09.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:08 smithi019 ceph-mon[173579]: pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-02-22T15:57:09.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:08 smithi019 ceph-mon[173579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-02-22T15:57:09.993 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:09 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T15:57:09.563+0000 7ff29c0ef640 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-22T15:57:10.493 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T15:57:10.059+0000 7ff29c0ef640 -1 osd.2 60 log_to_monitors true 2024-02-22T15:57:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:10 smithi062 ceph-mon[106861]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-02-22T15:57:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:10 smithi062 ceph-mon[106861]: from='osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T15:57:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.242 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T15:57:10.941+0000 7ff28b544700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:57:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-mon[173579]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 254 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 37/318 objects degraded (11.635%) 2024-02-22T15:57:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-mon[173579]: from='osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-22T15:57:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:12.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:11 smithi062 ceph-mon[106861]: from='osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T15:57:12.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:11 smithi062 ceph-mon[106861]: osdmap e63: 6 total, 5 up, 6 in 2024-02-22T15:57:12.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:11 smithi062 ceph-mon[106861]: from='osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:57:12.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:11 smithi062 ceph-mon[106861]: failed to encode map e63 with expected crc 2024-02-22T15:57:12.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:11 smithi062 ceph-mon[106861]: failed to encode map e63 with expected crc 2024-02-22T15:57:12.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:11 smithi019 ceph-mon[173579]: from='osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-22T15:57:12.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:11 smithi019 ceph-mon[173579]: osdmap e63: 6 total, 5 up, 6 in 2024-02-22T15:57:12.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:11 smithi019 ceph-mon[173579]: from='osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-02-22T15:57:12.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:11 smithi019 ceph-mon[173579]: failed to encode map e63 with expected crc 2024-02-22T15:57:12.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:11 smithi019 ceph-mon[173579]: failed to encode map e63 with expected crc 2024-02-22T15:57:12.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/318 objects degraded (11.635%) 2024-02-22T15:57:12.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:57:12.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: failed to encode map e64 with expected crc 2024-02-22T15:57:12.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: failed to encode map e64 with expected crc 2024-02-22T15:57:12.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918] boot 2024-02-22T15:57:12.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: osdmap e64: 6 total, 6 up, 6 in 2024-02-22T15:57:12.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: failed to encode map e64 with expected crc 2024-02-22T15:57:12.912 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: failed to encode map e64 with expected crc 2024-02-22T15:57:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/318 objects degraded (11.635%) 2024-02-22T15:57:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:57:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: failed to encode map e64 with expected crc 2024-02-22T15:57:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: failed to encode map e64 with expected crc 2024-02-22T15:57:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: osd.2 [v2:172.21.15.19:6818/947220918,v1:172.21.15.19:6819/947220918] boot 2024-02-22T15:57:13.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: osdmap e64: 6 total, 6 up, 6 in 2024-02-22T15:57:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: failed to encode map e64 with expected crc 2024-02-22T15:57:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: failed to encode map e64 with expected crc 2024-02-22T15:57:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:57:13.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:12 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 37/318 objects degraded (11.635%), 12 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:13.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-22T15:57:13.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:12 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 37/318 objects degraded (11.635%), 12 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: osdmap e65: 6 total, 6 up, 6 in 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:13 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: osdmap e65: 6 total, 6 up, 6 in 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: failed to encode map e65 with expected crc 2024-02-22T15:57:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:13 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: pgmap v98: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 29/318 objects degraded (9.119%) 2024-02-22T15:57:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:57:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:57:15.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:15.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: pgmap v98: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 29/318 objects degraded (9.119%) 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:15.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:15.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:16.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:15 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:16.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:15 smithi062 ceph-mon[106861]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-22T15:57:16.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:15 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:16.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:15 smithi019 ceph-mon[173579]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-22T15:57:17.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:16 smithi062 ceph-mon[106861]: pgmap v99: 65 pgs: 5 peering, 5 active+undersized+degraded, 7 active+undersized, 48 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 16/318 objects degraded (5.031%) 2024-02-22T15:57:17.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:16 smithi019 ceph-mon[173579]: pgmap v99: 65 pgs: 5 peering, 5 active+undersized+degraded, 7 active+undersized, 48 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 16/318 objects degraded (5.031%) 2024-02-22T15:57:18.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:17 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 16/318 objects degraded (5.031%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:18.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:17 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 16/318 objects degraded (5.031%), 5 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:19.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:18 smithi062 ceph-mon[106861]: pgmap v100: 65 pgs: 5 peering, 60 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-02-22T15:57:19.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:18 smithi062 ceph-mon[106861]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/318 objects degraded (5.031%), 5 pgs degraded) 2024-02-22T15:57:19.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:18 smithi062 ceph-mon[106861]: Cluster is now healthy 2024-02-22T15:57:19.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:19.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:18 smithi019 ceph-mon[173579]: pgmap v100: 65 pgs: 5 peering, 60 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-02-22T15:57:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:18 smithi019 ceph-mon[173579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/318 objects degraded (5.031%), 5 pgs degraded) 2024-02-22T15:57:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:18 smithi019 ceph-mon[173579]: Cluster is now healthy 2024-02-22T15:57:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:21.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:20 smithi062 ceph-mon[106861]: pgmap v101: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:57:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:20 smithi019 ceph-mon[173579]: pgmap v101: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:57:23.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:22 smithi062 ceph-mon[106861]: pgmap v102: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-22T15:57:23.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:22 smithi019 ceph-mon[173579]: pgmap v102: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-02-22T15:57:24.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:24 smithi062 ceph-mon[106861]: pgmap v103: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:24.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:24 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:24.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:24 smithi019 ceph-mon[173579]: pgmap v103: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:24.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:24 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:26.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:26 smithi062 ceph-mon[106861]: pgmap v104: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-02-22T15:57:26.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:26 smithi019 ceph-mon[173579]: pgmap v104: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-02-22T15:57:29.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:28 smithi062 ceph-mon[106861]: pgmap v105: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:29.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:28 smithi019 ceph-mon[173579]: pgmap v105: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:30.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:29 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:30.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:29 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:31.187 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:30 smithi062 ceph-mon[106861]: pgmap v106: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:57:31.187 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:30 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:31.187 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:30 smithi062 ceph-mon[106861]: Upgrade: osd.3 is safe to restart 2024-02-22T15:57:31.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:30 smithi019 ceph-mon[173579]: pgmap v106: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T15:57:31.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:30 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-22T15:57:31.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:30 smithi019 ceph-mon[173579]: Upgrade: osd.3 is safe to restart 2024-02-22T15:57:32.692 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:32 smithi062 ceph-mon[106861]: pgmap v107: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:32.693 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:32 smithi062 ceph-mon[106861]: Upgrade: Updating osd.3 2024-02-22T15:57:32.693 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:32.693 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T15:57:32.693 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:57:32.693 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:32 smithi062 ceph-mon[106861]: Deploying daemon osd.3 on smithi062 2024-02-22T15:57:32.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:32 smithi019 ceph-mon[173579]: pgmap v107: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:32.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:32 smithi019 ceph-mon[173579]: Upgrade: Updating osd.3 2024-02-22T15:57:32.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:32.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-22T15:57:32.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:57:32.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:32 smithi019 ceph-mon[173579]: Deploying daemon osd.3 on smithi062 2024-02-22T15:57:34.084 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:34.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:34.705 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:34 smithi062 systemd[1]: Stopping Ceph osd.3 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:57:34.706 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:34 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[54690]: 2024-02-22T15:57:34.630+0000 7f4c2a119700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:57:34.706 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:34 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[54690]: 2024-02-22T15:57:34.630+0000 7f4c2a119700 -1 osd.3 65 *** Got signal Terminated *** 2024-02-22T15:57:34.706 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:34 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[54690]: 2024-02-22T15:57:34.630+0000 7f4c2a119700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T15:57:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:34 smithi062 ceph-mon[106861]: pgmap v108: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:34 smithi062 ceph-mon[106861]: osd.3 marked itself down and dead 2024-02-22T15:57:35.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:34 smithi019 ceph-mon[173579]: pgmap v108: 65 pgs: 65 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:57:35.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:34 smithi019 ceph-mon[173579]: osd.3 marked itself down and dead 2024-02-22T15:57:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 -- 172.21.15.19:0/855344584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c101e40 msgr2=0x7f0d7c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 --2- 172.21.15.19:0/855344584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c101e40 0x7f0d7c102210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c004940 tx=0x7f0d6c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 -- 172.21.15.19:0/855344584 shutdown_connections 2024-02-22T15:57:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 --2- 172.21.15.19:0/855344584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 --2- 172.21.15.19:0/855344584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c101e40 0x7f0d7c102210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 -- 172.21.15.19:0/855344584 >> 172.21.15.19:0/855344584 conn(0x7f0d7c071d90 msgr2=0x7f0d7c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 -- 172.21.15.19:0/855344584 shutdown_connections 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.349+0000 7f0d826a3700 1 -- 172.21.15.19:0/855344584 wait complete. 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d826a3700 1 Processor -- start 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d826a3700 1 -- start start 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d826a3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d7c101e40 0x7f0d7c112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d826a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d826a3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c113410 con 0x7f0d7c1027e0 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d826a3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c10c740 con 0x7f0d7c101e40 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d7b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d7b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60476/0 (socket says 172.21.15.19:60476) 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.350+0000 7f0d7b7fe700 1 -- 172.21.15.19:0/3012108145 learned_addr learned my addr 172.21.15.19:0/3012108145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d7bfff700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d7c101e40 0x7f0d7c112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d7b7fe700 1 -- 172.21.15.19:0/3012108145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d7c101e40 msgr2=0x7f0d7c112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d7b7fe700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d7c101e40 0x7f0d7c112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d7b7fe700 1 -- 172.21.15.19:0/3012108145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c005040 con 0x7f0d7c1027e0 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d7b7fe700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c112dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000fce0 tx=0x7f0d70004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d70007ea0 con 0x7f0d7c1027e0 2024-02-22T15:57:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d7000dcd0 con 0x7f0d7c1027e0 2024-02-22T15:57:35.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c10c9c0 con 0x7f0d7c1027e0 2024-02-22T15:57:35.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d70012460 con 0x7f0d7c1027e0 2024-02-22T15:57:35.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.351+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c10cf00 con 0x7f0d7c1027e0 2024-02-22T15:57:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.352+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c0738b0 con 0x7f0d7c1027e0 2024-02-22T15:57:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.353+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0d7000d820 con 0x7f0d7c1027e0 2024-02-22T15:57:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.353+0000 7f0d78ff9700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0d6406cc20 0x7f0d6406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.353+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0d7008e6f0 con 0x7f0d7c1027e0 2024-02-22T15:57:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.353+0000 7f0d7bfff700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0d6406cc20 0x7f0d6406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.354+0000 7f0d7bfff700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0d6406cc20 0x7f0d6406f0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c004f90 tx=0x7f0d6c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.358+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0d7005c1e0 con 0x7f0d7c1027e0 2024-02-22T15:57:35.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.581+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d7c10d120 con 0x7f0d6406cc20 2024-02-22T15:57:35.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.582+0000 7f0d78ff9700 1 -- 172.21.15.19:0/3012108145 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0d7c10d120 con 0x7f0d6406cc20 2024-02-22T15:57:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0d6406cc20 msgr2=0x7f0d6406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0d6406cc20 0x7f0d6406f0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c004f90 tx=0x7f0d6c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:35.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 msgr2=0x7f0d7c112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c112dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000fce0 tx=0x7f0d70004600 comp rx=0 tx=0).stop 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 shutdown_connections 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0d6406cc20 0x7f0d6406f0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d7c101e40 0x7f0d7c112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 --2- 172.21.15.19:0/3012108145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d7c1027e0 0x7f0d7c112dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 >> 172.21.15.19:0/3012108145 conn(0x7f0d7c071d90 msgr2=0x7f0d7c0fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 shutdown_connections 2024-02-22T15:57:35.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.584+0000 7f0d826a3700 1 -- 172.21.15.19:0/3012108145 wait complete. 2024-02-22T15:57:35.574 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 -- 172.21.15.19:0/365693251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 msgr2=0x7f09d0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 --2- 172.21.15.19:0/365693251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d0102210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09c4004940 tx=0x7f09c4009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 -- 172.21.15.19:0/365693251 shutdown_connections 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 --2- 172.21.15.19:0/365693251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09d01027e0 0x7f09d00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 --2- 172.21.15.19:0/365693251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d0102210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 -- 172.21.15.19:0/365693251 >> 172.21.15.19:0/365693251 conn(0x7f09d0071d90 msgr2=0x7f09d0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 -- 172.21.15.19:0/365693251 shutdown_connections 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.689+0000 7f09d6e55700 1 -- 172.21.15.19:0/365693251 wait complete. 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d6e55700 1 Processor -- start 2024-02-22T15:57:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d6e55700 1 -- start start 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d6e55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d6e55700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09d01027e0 0x7f09d010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d6e55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d010f630 con 0x7f09d0101e40 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d6e55700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d010f7a0 con 0x7f09d01027e0 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d4bf1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d4bf1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d010eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60486/0 (socket says 172.21.15.19:60486) 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09d4bf1700 1 -- 172.21.15.19:0/2556259181 learned_addr learned my addr 172.21.15.19:0/2556259181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.690+0000 7f09cffff700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09d01027e0 0x7f09d010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09d4bf1700 1 -- 172.21.15.19:0/2556259181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09d01027e0 msgr2=0x7f09d010f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09d4bf1700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09d01027e0 0x7f09d010f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09d4bf1700 1 -- 172.21.15.19:0/2556259181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4005040 con 0x7f09d0101e40 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09d4bf1700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d010eb40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b3b0 tx=0x7f09c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c400e040 con 0x7f09d0101e40 2024-02-22T15:57:35.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09c401c430 con 0x7f09d0101e40 2024-02-22T15:57:35.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d0117ab0 con 0x7f09d0101e40 2024-02-22T15:57:35.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c4013640 con 0x7f09d0101e40 2024-02-22T15:57:35.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.691+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d0117fc0 con 0x7f09d0101e40 2024-02-22T15:57:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.692+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d00ff150 con 0x7f09d0101e40 2024-02-22T15:57:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.693+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f09c400bb80 con 0x7f09d0101e40 2024-02-22T15:57:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.693+0000 7f09cd7fa700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f09c006cc20 0x7f09c006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.693+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f09c408cae0 con 0x7f09d0101e40 2024-02-22T15:57:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.693+0000 7f09cffff700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f09c006cc20 0x7f09c006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.693+0000 7f09cffff700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f09c006cc20 0x7f09c006f0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09d0066180 tx=0x7f09bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:35.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.698+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f09c4056c70 con 0x7f09d0101e40 2024-02-22T15:57:35.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.915+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09d0065ec0 con 0x7f09c006cc20 2024-02-22T15:57:35.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.916+0000 7f09cd7fa700 1 -- 172.21.15.19:0/2556259181 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f09d0065ec0 con 0x7f09c006cc20 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f09c006cc20 msgr2=0x7f09c006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f09c006cc20 0x7f09c006f0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f09d0066180 tx=0x7f09bc00b040 comp rx=0 tx=0).stop 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 msgr2=0x7f09d010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d010eb40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b3b0 tx=0x7f09c4009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 shutdown_connections 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f09c006cc20 0x7f09c006f0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09d0101e40 0x7f09d010eb40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 --2- 172.21.15.19:0/2556259181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09d01027e0 0x7f09d010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 >> 172.21.15.19:0/2556259181 conn(0x7f09d0071d90 msgr2=0x7f09d00faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:35.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 shutdown_connections 2024-02-22T15:57:35.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:35.919+0000 7f09d6e55700 1 -- 172.21.15.19:0/2556259181 wait complete. 2024-02-22T15:57:35.931 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:35 smithi062 bash[119485]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3 2024-02-22T15:57:35.931 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:35 smithi062 ceph-mon[106861]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:57:35.931 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:35 smithi062 ceph-mon[106861]: failed to encode map e66 with expected crc 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 -- 172.21.15.19:0/1595842346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 msgr2=0x7f4c84109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 --2- 172.21.15.19:0/1595842346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 0x7f4c84109390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c78004940 tx=0x7f4c78009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 -- 172.21.15.19:0/1595842346 shutdown_connections 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 --2- 172.21.15.19:0/1595842346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 0x7f4c84109390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 --2- 172.21.15.19:0/1595842346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c840fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 -- 172.21.15.19:0/1595842346 >> 172.21.15.19:0/1595842346 conn(0x7f4c840f7a10 msgr2=0x7f4c840f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 -- 172.21.15.19:0/1595842346 shutdown_connections 2024-02-22T15:57:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.041+0000 7f4c8942b700 1 -- 172.21.15.19:0/1595842346 wait complete. 2024-02-22T15:57:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.042+0000 7f4c8942b700 1 Processor -- start 2024-02-22T15:57:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.042+0000 7f4c8942b700 1 -- start start 2024-02-22T15:57:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c8942b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c84115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c84115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c8942b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 0x7f4c84116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c84115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:52542/0 (socket says 172.21.15.19:52542) 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 -- 172.21.15.19:0/3147683768 learned_addr learned my addr 172.21.15.19:0/3147683768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c841166a0 con 0x7f4c840fc630 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8410f9d0 con 0x7f4c840fbd20 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c827fc700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 0x7f4c84116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 -- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 msgr2=0x7f4c84116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 0x7f4c84116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78005040 con 0x7f4c840fbd20 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.043+0000 7f4c82ffd700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c84115b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400eb40 tx=0x7f4c7400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.044+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c74007cc0 con 0x7f4c840fbd20 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.044+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8410fc50 con 0x7f4c840fbd20 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.044+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c7400f040 con 0x7f4c840fbd20 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.044+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7400c990 con 0x7f4c840fbd20 2024-02-22T15:57:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.044+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c84110190 con 0x7f4c840fbd20 2024-02-22T15:57:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.045+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c84106ac0 con 0x7f4c840fbd20 2024-02-22T15:57:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.045+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4c7400cc00 con 0x7f4c840fbd20 2024-02-22T15:57:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.046+0000 7f4c6b7fe700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4c6c06cc20 0x7f4c6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.046+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4c7408bc60 con 0x7f4c840fbd20 2024-02-22T15:57:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.046+0000 7f4c827fc700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4c6c06cc20 0x7f4c6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.046+0000 7f4c827fc700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4c6c06cc20 0x7f4c6c06f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c84111340 tx=0x7f4c78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.051+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4c74059760 con 0x7f4c840fbd20 2024-02-22T15:57:36.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:35 smithi062 ceph-mon[106861]: osdmap e66: 6 total, 5 up, 6 in 2024-02-22T15:57:36.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:35 smithi062 ceph-mon[106861]: osdmap e67: 6 total, 5 up, 6 in 2024-02-22T15:57:36.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.263+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c8404bed0 con 0x7f4c6c06cc20 2024-02-22T15:57:36.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:35 smithi019 ceph-mon[173579]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:57:36.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:35 smithi019 ceph-mon[173579]: failed to encode map e66 with expected crc 2024-02-22T15:57:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:35 smithi019 ceph-mon[173579]: osdmap e66: 6 total, 5 up, 6 in 2024-02-22T15:57:36.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:35 smithi019 ceph-mon[173579]: osdmap e67: 6 total, 5 up, 6 in 2024-02-22T15:57:36.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.273+0000 7f4c6b7fe700 1 -- 172.21.15.19:0/3147683768 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4c8404bed0 con 0x7f4c6c06cc20 2024-02-22T15:57:36.252 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:57:36.252 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (4m) 26s ago 15m 26.8M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:57:36.252 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (2m) 26s ago 16m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (2m) 2m ago 15m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (4m) 26s ago 15m 85.8M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (10m) 26s ago 10m 113M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (10m) 26s ago 10m 67.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (10m) 2m ago 10m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (10m) 2m ago 10m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (6m) 26s ago 17m 499M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (5m) 2m ago 14m 447M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (3m) 26s ago 17m 47.5M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (2m) 2m ago 14m 37.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (5m) 26s ago 16m 18.4M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (5m) 2m ago 14m 17.0M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:57:36.253 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (104s) 26s ago 14m 87.3M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:57:36.254 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (67s) 26s ago 13m 93.0M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T15:57:36.254 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (31s) 26s ago 13m 11.4M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T15:57:36.254 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (12m) 2m ago 12m 259M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c ab51b7bb6b8d 2024-02-22T15:57:36.254 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (12m) 2m ago 12m 229M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:57:36.254 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (11m) 2m ago 11m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:57:36.254 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (5m) 26s ago 15m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:57:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4c6c06cc20 msgr2=0x7f4c6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4c6c06cc20 0x7f4c6c06f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c84111340 tx=0x7f4c78009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 msgr2=0x7f4c84115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c84115b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400eb40 tx=0x7f4c7400b040 comp rx=0 tx=0).stop 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 shutdown_connections 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4c6c06cc20 0x7f4c6c06f0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c840fbd20 0x7f4c84115b10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 --2- 172.21.15.19:0/3147683768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c840fc630 0x7f4c84116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 >> 172.21.15.19:0/3147683768 conn(0x7f4c840f7a10 msgr2=0x7f4c840f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 shutdown_connections 2024-02-22T15:57:36.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.275+0000 7f4c8942b700 1 -- 172.21.15.19:0/3147683768 wait complete. 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 -- 172.21.15.19:0/1606474072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fdfe0 msgr2=0x7f97200fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 --2- 172.21.15.19:0/1606474072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fdfe0 0x7f97200fe3b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9708004970 tx=0x7f9708009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 -- 172.21.15.19:0/1606474072 shutdown_connections 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 --2- 172.21.15.19:0/1606474072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97200fe8f0 0x7f9720107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 --2- 172.21.15.19:0/1606474072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fdfe0 0x7f97200fe3b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 -- 172.21.15.19:0/1606474072 >> 172.21.15.19:0/1606474072 conn(0x7f9720071d90 msgr2=0x7f9720072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 -- 172.21.15.19:0/1606474072 shutdown_connections 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 -- 172.21.15.19:0/1606474072 wait complete. 2024-02-22T15:57:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.378+0000 7f9724d53700 1 Processor -- start 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9724d53700 1 -- start start 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9724d53700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97200fdfe0 0x7f97200fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9724d53700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 0x7f97200fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9724d53700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200fc9e0 con 0x7f97200fe8f0 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9724d53700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200fcb50 con 0x7f97200fdfe0 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 0x7f97200fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 0x7f97200fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60536/0 (socket says 172.21.15.19:60536) 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 -- 172.21.15.19:0/3231342359 learned_addr learned my addr 172.21.15.19:0/3231342359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:36.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971e59c700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97200fdfe0 0x7f97200fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 -- 172.21.15.19:0/3231342359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97200fdfe0 msgr2=0x7f97200fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97200fdfe0 0x7f97200fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 -- 172.21.15.19:0/3231342359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708005040 con 0x7f97200fe8f0 2024-02-22T15:57:36.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f971dd9b700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 0x7f97200fc3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f971000eb40 tx=0x7f971000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9710007cc0 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9720119b70 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f971000f040 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.379+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f971000c990 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.380+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9720119fd0 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.380+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97201046c0 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.381+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f971000cc00 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.381+0000 7f9716ffd700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f970c06cc20 0x7f970c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.381+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f971008bbc0 con 0x7f97200fe8f0 2024-02-22T15:57:36.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.381+0000 7f971e59c700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f970c06cc20 0x7f970c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.382+0000 7f971e59c700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f970c06cc20 0x7f970c06f0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fd0 tx=0x7f9708009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:36.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.386+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9710055cd0 con 0x7f97200fe8f0 2024-02-22T15:57:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.672+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f97200738b0 con 0x7f97200fe8f0 2024-02-22T15:57:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.672+0000 7f9716ffd700 1 -- 172.21.15.19:0/3231342359 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f97100592f0 con 0x7f97200fe8f0 2024-02-22T15:57:36.651 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 2, 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 3 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:57:36.652 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 6, 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 7 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f970c06cc20 msgr2=0x7f970c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f970c06cc20 0x7f970c06f0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fd0 tx=0x7f9708009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 msgr2=0x7f97200fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 0x7f97200fc3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f971000eb40 tx=0x7f971000b040 comp rx=0 tx=0).stop 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 shutdown_connections 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f970c06cc20 0x7f970c06f0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97200fdfe0 0x7f97200fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 --2- 172.21.15.19:0/3231342359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97200fe8f0 0x7f97200fc3e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 >> 172.21.15.19:0/3231342359 conn(0x7f9720071d90 msgr2=0x7f9720101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 shutdown_connections 2024-02-22T15:57:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.674+0000 7f9724d53700 1 -- 172.21.15.19:0/3231342359 wait complete. 2024-02-22T15:57:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 -- 172.21.15.19:0/336411984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c1038f0 msgr2=0x7f856c0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 --2- 172.21.15.19:0/336411984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c1038f0 0x7f856c0fcf60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f855c004940 tx=0x7f855c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 -- 172.21.15.19:0/336411984 shutdown_connections 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 --2- 172.21.15.19:0/336411984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c1038f0 0x7f856c0fcf60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 --2- 172.21.15.19:0/336411984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c102f50 0x7f856c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 -- 172.21.15.19:0/336411984 >> 172.21.15.19:0/336411984 conn(0x7f856c0f8ae0 msgr2=0x7f856c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 -- 172.21.15.19:0/336411984 shutdown_connections 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 -- 172.21.15.19:0/336411984 wait complete. 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 Processor -- start 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.778+0000 7f857430f700 1 -- start start 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f857430f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 0x7f856c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f857430f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c1038f0 0x7f856c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f857430f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c1144c0 con 0x7f856c102f50 2024-02-22T15:57:36.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f857430f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c10d7f0 con 0x7f856c1038f0 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 0x7f856c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 0x7f856c113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60560/0 (socket says 172.21.15.19:60560) 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 -- 172.21.15.19:0/606647815 learned_addr learned my addr 172.21.15.19:0/606647815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85718aa700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c1038f0 0x7f856c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 -- 172.21.15.19:0/606647815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c1038f0 msgr2=0x7f856c113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c1038f0 0x7f856c113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 -- 172.21.15.19:0/606647815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c005040 con 0x7f856c102f50 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85718aa700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c1038f0 0x7f856c113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:57:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f85720ab700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 0x7f856c113930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f856800eb40 tx=0x7f856800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8568007cc0 con 0x7f856c102f50 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f856800f040 con 0x7f856c102f50 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856c10da70 con 0x7f856c102f50 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f856800c990 con 0x7f856c102f50 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.779+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c10df80 con 0x7f856c102f50 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.780+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856c0fc6e0 con 0x7f856c102f50 2024-02-22T15:57:36.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.781+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f856800caf0 con 0x7f856c102f50 2024-02-22T15:57:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.781+0000 7f8562ffd700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f855806cc30 0x7f855806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.781+0000 7f85718aa700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f855806cc30 0x7f855806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.781+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f856808bb90 con 0x7f856c102f50 2024-02-22T15:57:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.782+0000 7f85718aa700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f855806cc30 0x7f855806f0e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f856c10eb20 tx=0x7f855c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:36.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:36.786+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8568059710 con 0x7f856c102f50 2024-02-22T15:57:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.035+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f856c10ef10 con 0x7f856c102f50 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.036+0000 7f8562ffd700 1 -- 172.21.15.19:0/606647815 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f8568019760 con 0x7f856c102f50 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:57:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:57:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:57:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:57:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f855806cc30 msgr2=0x7f855806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f855806cc30 0x7f855806f0e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f856c10eb20 tx=0x7f855c009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 msgr2=0x7f856c113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 0x7f856c113930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f856800eb40 tx=0x7f856800b040 comp rx=0 tx=0).stop 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 shutdown_connections 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f855806cc30 0x7f855806f0e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f856c102f50 0x7f856c113930 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 --2- 172.21.15.19:0/606647815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f856c1038f0 0x7f856c113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 >> 172.21.15.19:0/606647815 conn(0x7f856c0f8ae0 msgr2=0x7f856c0fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 shutdown_connections 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.038+0000 7f857430f700 1 -- 172.21.15.19:0/606647815 wait complete. 2024-02-22T15:57:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:57:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 -- 172.21.15.19:0/401864798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 msgr2=0x7fed740fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 --2- 172.21.15.19:0/401864798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed740fbea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fed64004940 tx=0x7fed64009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 -- 172.21.15.19:0/401864798 shutdown_connections 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 --2- 172.21.15.19:0/401864798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed740fbea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 --2- 172.21.15.19:0/401864798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed74101e40 0x7fed74102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 -- 172.21.15.19:0/401864798 >> 172.21.15.19:0/401864798 conn(0x7fed74071d90 msgr2=0x7fed74072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 -- 172.21.15.19:0/401864798 shutdown_connections 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 -- 172.21.15.19:0/401864798 wait complete. 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 Processor -- start 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.143+0000 7fed7b655700 1 -- start start 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed7b655700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed74101e40 0x7fed74113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed7b655700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed74113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed7b655700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed741145f0 con 0x7fed741027e0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed7b655700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7410d920 con 0x7fed74101e40 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed74113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed74113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60572/0 (socket says 172.21.15.19:60572) 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 -- 172.21.15.19:0/3469428893 learned_addr learned my addr 172.21.15.19:0/3469428893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed793f1700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed74101e40 0x7fed74113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 -- 172.21.15.19:0/3469428893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed74101e40 msgr2=0x7fed74113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed74101e40 0x7fed74113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 -- 172.21.15.19:0/3469428893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64005040 con 0x7fed741027e0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed78bf0700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed74113fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed64009fd0 tx=0x7fed64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6400e040 con 0x7fed741027e0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed6400bc50 con 0x7fed741027e0 2024-02-22T15:57:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7410dba0 con 0x7fed741027e0 2024-02-22T15:57:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.144+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6401c3f0 con 0x7fed741027e0 2024-02-22T15:57:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.145+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7410e0e0 con 0x7fed741027e0 2024-02-22T15:57:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.146+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed740738b0 con 0x7fed741027e0 2024-02-22T15:57:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.146+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fed64013420 con 0x7fed741027e0 2024-02-22T15:57:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.146+0000 7fed69ffb700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed6006cc20 0x7fed6006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.146+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fed6408d2b0 con 0x7fed741027e0 2024-02-22T15:57:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.146+0000 7fed793f1700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed6006cc20 0x7fed6006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.147+0000 7fed793f1700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed6006cc20 0x7fed6006f0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed70009fd0 tx=0x7fed70009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.151+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fed6405aec0 con 0x7fed741027e0 2024-02-22T15:57:37.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: pgmap v110: 65 pgs: 16 stale+active+clean, 49 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:57:37.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:37.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:36 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3231342359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:37.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: pgmap v110: 65 pgs: 16 stale+active+clean, 49 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:57:37.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: failed to encode map e67 with expected crc 2024-02-22T15:57:37.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:36 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3231342359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.373+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed7404bed0 con 0x7fed6006cc20 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.375+0000 7fed69ffb700 1 -- 172.21.15.19:0/3469428893 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fed7404bed0 con 0x7fed6006cc20 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:57:37.354 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:57:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:57:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:57:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "9/21 daemons upgraded", 2024-02-22T15:57:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading osd daemons", 2024-02-22T15:57:37.355 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:57:37.355 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed6006cc20 msgr2=0x7fed6006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed6006cc20 0x7fed6006f0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed70009fd0 tx=0x7fed70009f20 comp rx=0 tx=0).stop 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 msgr2=0x7fed74113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed74113fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed64009fd0 tx=0x7fed64009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 shutdown_connections 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fed6006cc20 0x7fed6006f0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed74101e40 0x7fed74113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 --2- 172.21.15.19:0/3469428893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed741027e0 0x7fed74113fa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 >> 172.21.15.19:0/3469428893 conn(0x7fed74071d90 msgr2=0x7fed74065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 shutdown_connections 2024-02-22T15:57:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.377+0000 7fed7b655700 1 -- 172.21.15.19:0/3469428893 wait complete. 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 -- 172.21.15.19:0/2775521448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c1027e0 msgr2=0x7f9c0c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 --2- 172.21.15.19:0/2775521448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c1027e0 0x7f9c0c0fbea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4004940 tx=0x7f9bf4009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 -- 172.21.15.19:0/2775521448 shutdown_connections 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 --2- 172.21.15.19:0/2775521448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c1027e0 0x7f9c0c0fbea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 --2- 172.21.15.19:0/2775521448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c101e40 0x7f9c0c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 -- 172.21.15.19:0/2775521448 >> 172.21.15.19:0/2775521448 conn(0x7f9c0c071d90 msgr2=0x7f9c0c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 -- 172.21.15.19:0/2775521448 shutdown_connections 2024-02-22T15:57:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.481+0000 7f9c10dec700 1 -- 172.21.15.19:0/2775521448 wait complete. 2024-02-22T15:57:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c10dec700 1 Processor -- start 2024-02-22T15:57:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c10dec700 1 -- start start 2024-02-22T15:57:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c10dec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 0x7f9c0c115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c10dec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c1027e0 0x7f9c0c116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c10dec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c116830 con 0x7f9c0c101e40 2024-02-22T15:57:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c10dec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c10faf0 con 0x7f9c0c1027e0 2024-02-22T15:57:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c0a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 0x7f9c0c115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c0a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 0x7f9c0c115c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46746/0 (socket says 172.21.15.19:46746) 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c0a59c700 1 -- 172.21.15.19:0/3524418657 learned_addr learned my addr 172.21.15.19:0/3524418657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.482+0000 7f9c09d9b700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c1027e0 0x7f9c0c116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c0a59c700 1 -- 172.21.15.19:0/3524418657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c1027e0 msgr2=0x7f9c0c116150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c0a59c700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c1027e0 0x7f9c0c116150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c0a59c700 1 -- 172.21.15.19:0/3524418657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4005040 con 0x7f9c0c101e40 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c0a59c700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 0x7f9c0c115c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00b790 tx=0x7f9bfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bfc00f6f0 con 0x7f9c0c101e40 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9bfc00fd00 con 0x7f9c0c101e40 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0c10fdd0 con 0x7f9c0c101e40 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bfc0103a0 con 0x7f9c0c101e40 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.483+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c110310 con 0x7f9c0c101e40 2024-02-22T15:57:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.484+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0c0738b0 con 0x7f9c0c101e40 2024-02-22T15:57:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.485+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9bfc00f850 con 0x7f9c0c101e40 2024-02-22T15:57:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.485+0000 7f9c02ffd700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9bf806cc30 0x7f9bf806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:57:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.485+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9bfc08be20 con 0x7f9c0c101e40 2024-02-22T15:57:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.485+0000 7f9c09d9b700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9bf806cc30 0x7f9bf806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:57:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.486+0000 7f9c09d9b700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9bf806cc30 0x7f9bf806f0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c1114b0 tx=0x7f9bf4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:57:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.490+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9bfc055f00 con 0x7f9c0c101e40 2024-02-22T15:57:37.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.774+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9c0c108e50 con 0x7f9c0c101e40 2024-02-22T15:57:37.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.774+0000 7f9c02ffd700 1 -- 172.21.15.19:0/3524418657 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f9bfc059520 con 0x7f9c0c101e40 2024-02-22T15:57:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN 1 osds down 2024-02-22T15:57:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-22T15:57:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: osd.3 (root=default,host=smithi062) is down 2024-02-22T15:57:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9bf806cc30 msgr2=0x7f9bf806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9bf806cc30 0x7f9bf806f0e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c1114b0 tx=0x7f9bf4009e30 comp rx=0 tx=0).stop 2024-02-22T15:57:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 msgr2=0x7f9c0c115c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:57:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 0x7f9c0c115c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00b790 tx=0x7f9bfc009250 comp rx=0 tx=0).stop 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 shutdown_connections 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9bf806cc30 0x7f9bf806f0e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c0c101e40 0x7f9c0c115c10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 --2- 172.21.15.19:0/3524418657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0c1027e0 0x7f9c0c116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 >> 172.21.15.19:0/3524418657 conn(0x7f9c0c071d90 msgr2=0x7f9c0c065840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 shutdown_connections 2024-02-22T15:57:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:57:37.776+0000 7f9c10dec700 1 -- 172.21.15.19:0/3524418657 wait complete. 2024-02-22T15:57:37.935 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:37 smithi062 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.3.service: Succeeded. 2024-02-22T15:57:37.936 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:37 smithi062 systemd[1]: Stopped Ceph osd.3 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:57:37.936 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:37 smithi062 systemd[1]: Starting Ceph osd.3 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:57:38.198 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:37 smithi062 ceph-mon[106861]: from='client.44211 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:38.198 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:37 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/606647815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:57:38.198 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:37 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3524418657' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:57:38.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:37 smithi019 ceph-mon[173579]: from='client.44211 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:38.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:37 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/606647815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:57:38.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:37 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3524418657' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:57:39.097 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:38 smithi062 ceph-mon[106861]: pgmap v112: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-22T15:57:39.097 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:38 smithi062 ceph-mon[106861]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:39.097 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:38 smithi062 ceph-mon[106861]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-22T15:57:39.098 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:38 smithi062 bash[119759]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-22T15:57:39.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:38 smithi019 ceph-mon[173579]: pgmap v112: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-22T15:57:39.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:38 smithi019 ceph-mon[173579]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:57:39.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:38 smithi019 ceph-mon[173579]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-22T15:57:40.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:40 smithi062 ceph-mon[106861]: pgmap v113: 65 pgs: 12 active+undersized+degraded, 13 peering, 13 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 44/318 objects degraded (13.836%) 2024-02-22T15:57:40.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:40 smithi062 ceph-mon[106861]: Health check failed: Degraded data redundancy: 44/318 objects degraded (13.836%), 12 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:40.205 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:39 smithi062 ceph-osd[120077]: -- 172.21.15.62:0/2382299051 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d9b1b44e0 con 0x560d9b184c00 2024-02-22T15:57:40.205 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:39 smithi062 bash[120039]: d462bbda7639c0c5c5d252279ddd17f6272e7f4ceff5d5f6e514bec96fba7cbe 2024-02-22T15:57:40.205 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:39 smithi062 systemd[1]: Started Ceph osd.3 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:57:40.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:40 smithi019 ceph-mon[173579]: pgmap v113: 65 pgs: 12 active+undersized+degraded, 13 peering, 13 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 44/318 objects degraded (13.836%) 2024-02-22T15:57:40.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:40 smithi019 ceph-mon[173579]: Health check failed: Degraded data redundancy: 44/318 objects degraded (13.836%), 12 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:40.705 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:40 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T15:57:40.411+0000 7fae0cf25640 -1 Falling back to public interface 2024-02-22T15:57:41.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:41.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:41.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:41 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:41.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:41.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:41 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:42.400 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:42 smithi062 ceph-mon[106861]: pgmap v114: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-02-22T15:57:42.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:42 smithi019 ceph-mon[173579]: pgmap v114: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-02-22T15:57:43.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:43 smithi062 ceph-mon[106861]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-22T15:57:43.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:43 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:43.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:43 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:43.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:43 smithi019 ceph-mon[173579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-22T15:57:43.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:43 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:43.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:43 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:44.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:44 smithi062 ceph-mon[106861]: pgmap v115: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-02-22T15:57:44.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:44 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:44.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:44 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:44.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:44 smithi019 ceph-mon[173579]: pgmap v115: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-02-22T15:57:44.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:44.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:45.066 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:44 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T15:57:44.952+0000 7fae0cf25640 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-22T15:57:45.614 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:45 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T15:57:45.556+0000 7fae0cf25640 -1 osd.3 65 log_to_monitors true 2024-02-22T15:57:45.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:45 smithi062 ceph-mon[106861]: from='osd.3 [v2:172.21.15.62:6800/1420983624,v1:172.21.15.62:6801/1420983624]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:57:45.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:45 smithi062 ceph-mon[106861]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:57:45.955 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 15:57:45 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T15:57:45.653+0000 7fadfab77700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:57:45.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:45 smithi019 ceph-mon[173579]: from='osd.3 [v2:172.21.15.62:6800/1420983624,v1:172.21.15.62:6801/1420983624]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:57:45.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:45 smithi019 ceph-mon[173579]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-22T15:57:46.684 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-02-22T15:57:46.684 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T15:57:46.684 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: osdmap e68: 6 total, 5 up, 6 in 2024-02-22T15:57:46.684 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: from='osd.3 [v2:172.21.15.62:6800/1420983624,v1:172.21.15.62:6801/1420983624]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:57:46.685 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:57:46.685 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: failed to encode map e68 with expected crc 2024-02-22T15:57:46.685 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:46 smithi062 ceph-mon[106861]: failed to encode map e68 with expected crc 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/318 objects degraded (20.440%) 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: osdmap e68: 6 total, 5 up, 6 in 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: from='osd.3 [v2:172.21.15.62:6800/1420983624,v1:172.21.15.62:6801/1420983624]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: failed to encode map e68 with expected crc 2024-02-22T15:57:46.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:46 smithi019 ceph-mon[173579]: failed to encode map e68 with expected crc 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: failed to encode map e69 with expected crc 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: osd.3 [v2:172.21.15.62:6800/1420983624,v1:172.21.15.62:6801/1420983624] boot 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: osdmap e69: 6 total, 6 up, 6 in 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:57:47.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:57:47.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:47 smithi062 ceph-mon[106861]: osdmap e70: 6 total, 6 up, 6 in 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: failed to encode map e69 with expected crc 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: osd.3 [v2:172.21.15.62:6800/1420983624,v1:172.21.15.62:6801/1420983624] boot 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: osdmap e69: 6 total, 6 up, 6 in 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:57:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:47.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:57:47.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:47.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:47.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:57:47.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:57:47.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:47 smithi019 ceph-mon[173579]: osdmap e70: 6 total, 6 up, 6 in 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 65/318 objects degraded (20.440%) 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 65/318 objects degraded (20.440%), 19 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:48.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:48.956 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 65/318 objects degraded (20.440%) 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: failed to encode map e70 with expected crc 2024-02-22T15:57:48.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 65/318 objects degraded (20.440%), 19 pgs degraded (PG_DEGRADED) 2024-02-22T15:57:48.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:48.994 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:57:50.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:50 smithi019 ceph-mon[173579]: pgmap v121: 65 pgs: 15 active+undersized+degraded, 11 active+undersized, 39 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 54/318 objects degraded (16.981%) 2024-02-22T15:57:50.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:50 smithi062 ceph-mon[106861]: pgmap v121: 65 pgs: 15 active+undersized+degraded, 11 active+undersized, 39 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 54/318 objects degraded (16.981%) 2024-02-22T15:57:51.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:51 smithi062 ceph-mon[106861]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/318 objects degraded (16.981%), 15 pgs degraded) 2024-02-22T15:57:51.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:51 smithi062 ceph-mon[106861]: Cluster is now healthy 2024-02-22T15:57:51.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:51 smithi019 ceph-mon[173579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/318 objects degraded (16.981%), 15 pgs degraded) 2024-02-22T15:57:51.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:51 smithi019 ceph-mon[173579]: Cluster is now healthy 2024-02-22T15:57:52.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:52 smithi062 ceph-mon[106861]: pgmap v122: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T15:57:52.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:52 smithi019 ceph-mon[173579]: pgmap v122: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-22T15:57:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:54 smithi062 ceph-mon[106861]: pgmap v123: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-02-22T15:57:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:54 smithi019 ceph-mon[173579]: pgmap v123: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-02-22T15:57:57.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:56 smithi062 ceph-mon[106861]: pgmap v124: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-22T15:57:57.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:56 smithi019 ceph-mon[173579]: pgmap v124: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-22T15:57:59.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:58 smithi062 ceph-mon[106861]: pgmap v125: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-02-22T15:57:59.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:57:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:57:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:58 smithi019 ceph-mon[173579]: pgmap v125: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-02-22T15:57:59.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:57:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:00 smithi062 ceph-mon[106861]: pgmap v126: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-22T15:58:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:00 smithi019 ceph-mon[173579]: pgmap v126: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-22T15:58:02.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:02 smithi062 ceph-mon[106861]: pgmap v127: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:58:02.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:02 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:58:03.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:02 smithi019 ceph-mon[173579]: pgmap v127: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:58:03.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:02 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: Upgrade: osd.4 is safe to restart 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T15:58:03.920 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:04.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-22T15:58:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: Upgrade: osd.4 is safe to restart 2024-02-22T15:58:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-22T15:58:04.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:05.071 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:04 smithi062 ceph-mon[106861]: pgmap v128: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:58:05.071 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:04 smithi062 ceph-mon[106861]: Upgrade: Updating osd.4 2024-02-22T15:58:05.071 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:04 smithi062 ceph-mon[106861]: Deploying daemon osd.4 on smithi062 2024-02-22T15:58:05.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:04 smithi019 ceph-mon[173579]: pgmap v128: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T15:58:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:04 smithi019 ceph-mon[173579]: Upgrade: Updating osd.4 2024-02-22T15:58:05.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:04 smithi019 ceph-mon[173579]: Deploying daemon osd.4 on smithi062 2024-02-22T15:58:06.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:06 smithi062 ceph-mon[106861]: pgmap v129: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:58:06.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:06 smithi062 systemd[1]: Stopping Ceph osd.4 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:58:06.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:06 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[63236]: 2024-02-22T15:58:06.948+0000 7f358f191700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:58:06.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:06 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[63236]: 2024-02-22T15:58:06.948+0000 7f358f191700 -1 osd.4 70 *** Got signal Terminated *** 2024-02-22T15:58:06.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:06 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[63236]: 2024-02-22T15:58:06.948+0000 7f358f191700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T15:58:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:06 smithi019 ceph-mon[173579]: pgmap v129: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.894+0000 7f139069e700 1 -- 172.21.15.19:0/3374014229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1388101e40 msgr2=0x7f1388102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.894+0000 7f139069e700 1 --2- 172.21.15.19:0/3374014229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1388101e40 0x7f1388102210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f137c004940 tx=0x7f137c009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 -- 172.21.15.19:0/3374014229 shutdown_connections 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 --2- 172.21.15.19:0/3374014229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13881027e0 0x7f13880fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 --2- 172.21.15.19:0/3374014229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1388101e40 0x7f1388102210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 -- 172.21.15.19:0/3374014229 >> 172.21.15.19:0/3374014229 conn(0x7f1388071d90 msgr2=0x7f1388072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:07.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 -- 172.21.15.19:0/3374014229 shutdown_connections 2024-02-22T15:58:07.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 -- 172.21.15.19:0/3374014229 wait complete. 2024-02-22T15:58:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 Processor -- start 2024-02-22T15:58:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.895+0000 7f139069e700 1 -- start start 2024-02-22T15:58:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f139069e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1388101e40 0x7f1388112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f139069e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 0x7f1388112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f139069e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881134a0 con 0x7f13881027e0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f139069e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138810c740 con 0x7f1388101e40 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 0x7f1388112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 0x7f1388112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56982/0 (socket says 172.21.15.19:56982) 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 -- 172.21.15.19:0/2181192951 learned_addr learned my addr 172.21.15.19:0/2181192951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138e43a700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1388101e40 0x7f1388112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 -- 172.21.15.19:0/2181192951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1388101e40 msgr2=0x7f1388112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1388101e40 0x7f1388112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 -- 172.21.15.19:0/2181192951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c005040 con 0x7f13881027e0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.896+0000 7f138dc39700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 0x7f1388112dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f138400daf0 tx=0x7f138400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.897+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138400ecf0 con 0x7f13881027e0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.897+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f138400ee50 con 0x7f13881027e0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.897+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138810c9c0 con 0x7f13881027e0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.897+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1384016840 con 0x7f13881027e0 2024-02-22T15:58:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.897+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138810cf00 con 0x7f13881027e0 2024-02-22T15:58:07.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.898+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880738b0 con 0x7f13881027e0 2024-02-22T15:58:07.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.898+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1384014070 con 0x7f13881027e0 2024-02-22T15:58:07.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.899+0000 7f137affd700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f137406cc20 0x7f137406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:07.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.899+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f138408df80 con 0x7f13881027e0 2024-02-22T15:58:07.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.899+0000 7f138e43a700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f137406cc20 0x7f137406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:07.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.899+0000 7f138e43a700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f137406cc20 0x7f137406f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f137c004f90 tx=0x7f137c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:07.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:07.903+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f138405bb90 con 0x7f13881027e0 2024-02-22T15:58:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.132+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f138804bf40 con 0x7f137406cc20 2024-02-22T15:58:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.133+0000 7f137affd700 1 -- 172.21.15.19:0/2181192951 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f138804bf40 con 0x7f137406cc20 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f137406cc20 msgr2=0x7f137406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f137406cc20 0x7f137406f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f137c004f90 tx=0x7f137c009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 msgr2=0x7f1388112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 0x7f1388112dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f138400daf0 tx=0x7f138400b4c0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 shutdown_connections 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f137406cc20 0x7f137406f0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1388101e40 0x7f1388112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 --2- 172.21.15.19:0/2181192951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13881027e0 0x7f1388112dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 >> 172.21.15.19:0/2181192951 conn(0x7f1388071d90 msgr2=0x7f13880fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 shutdown_connections 2024-02-22T15:58:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.135+0000 7f139069e700 1 -- 172.21.15.19:0/2181192951 wait complete. 2024-02-22T15:58:08.125 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:58:08.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:07 smithi062 ceph-mon[106861]: osd.4 marked itself down and dead 2024-02-22T15:58:08.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 -- 172.21.15.19:0/2732921922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 msgr2=0x7f3c400fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 --2- 172.21.15.19:0/2732921922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c400fcf60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3c34004940 tx=0x7f3c34009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:08.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 -- 172.21.15.19:0/2732921922 shutdown_connections 2024-02-22T15:58:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 --2- 172.21.15.19:0/2732921922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c400fcf60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 --2- 172.21.15.19:0/2732921922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c40102f50 0x7f3c40103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 -- 172.21.15.19:0/2732921922 >> 172.21.15.19:0/2732921922 conn(0x7f3c400f8ae0 msgr2=0x7f3c400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 -- 172.21.15.19:0/2732921922 shutdown_connections 2024-02-22T15:58:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.239+0000 7f3c45051700 1 -- 172.21.15.19:0/2732921922 wait complete. 2024-02-22T15:58:08.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c45051700 1 Processor -- start 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c45051700 1 -- start start 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c45051700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c40102f50 0x7f3c401139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c45051700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c40113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c45051700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40114530 con 0x7f3c401038f0 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c45051700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4010d860 con 0x7f3c40102f50 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c3e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c40113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c3e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c40113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57008/0 (socket says 172.21.15.19:57008) 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.240+0000 7f3c3e59c700 1 -- 172.21.15.19:0/1259834609 learned_addr learned my addr 172.21.15.19:0/1259834609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c3ed9d700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c40102f50 0x7f3c401139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c3e59c700 1 -- 172.21.15.19:0/1259834609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c40102f50 msgr2=0x7f3c401139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c3e59c700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c40102f50 0x7f3c401139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c3e59c700 1 -- 172.21.15.19:0/1259834609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34005040 con 0x7f3c401038f0 2024-02-22T15:58:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c3e59c700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c40113ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009180 tx=0x7f3c34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:08.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3400e040 con 0x7f3c401038f0 2024-02-22T15:58:08.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c34004290 con 0x7f3c401038f0 2024-02-22T15:58:08.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4010dae0 con 0x7f3c401038f0 2024-02-22T15:58:08.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3401c3f0 con 0x7f3c401038f0 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.241+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4010e020 con 0x7f3c401038f0 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.242+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c40105420 con 0x7f3c401038f0 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.243+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3c3400bc50 con 0x7f3c401038f0 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.243+0000 7f3c277fe700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c2806cc30 0x7f3c2806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.243+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3c3408d330 con 0x7f3c401038f0 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.243+0000 7f3c3ed9d700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c2806cc30 0x7f3c2806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.244+0000 7f3c3ed9d700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c2806cc30 0x7f3c2806f0e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004990 tx=0x7f3c30009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.248+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3c3405af50 con 0x7f3c401038f0 2024-02-22T15:58:08.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:07 smithi019 ceph-mon[173579]: osd.4 marked itself down and dead 2024-02-22T15:58:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.470+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c40109ef0 con 0x7f3c2806cc30 2024-02-22T15:58:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.471+0000 7f3c277fe700 1 -- 172.21.15.19:0/1259834609 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3c40109ef0 con 0x7f3c2806cc30 2024-02-22T15:58:08.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c2806cc30 msgr2=0x7f3c2806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c2806cc30 0x7f3c2806f0e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004990 tx=0x7f3c30009ee0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 msgr2=0x7f3c40113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c40113ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009180 tx=0x7f3c34009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:08.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 shutdown_connections 2024-02-22T15:58:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c2806cc30 0x7f3c2806f0e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c40102f50 0x7f3c401139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 --2- 172.21.15.19:0/1259834609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401038f0 0x7f3c40113ee0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 >> 172.21.15.19:0/1259834609 conn(0x7f3c400f8ae0 msgr2=0x7f3c400fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 shutdown_connections 2024-02-22T15:58:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.473+0000 7f3c45051700 1 -- 172.21.15.19:0/1259834609 wait complete. 2024-02-22T15:58:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1142665107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 msgr2=0x7f2cec102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1142665107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec102210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc004940 tx=0x7f2cdc009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1142665107 shutdown_connections 2024-02-22T15:58:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1142665107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cec1027e0 0x7f2cec0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1142665107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec102210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1142665107 >> 172.21.15.19:0/1142665107 conn(0x7f2cec071d90 msgr2=0x7f2cec072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1142665107 shutdown_connections 2024-02-22T15:58:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.576+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1142665107 wait complete. 2024-02-22T15:58:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cf24ce700 1 Processor -- start 2024-02-22T15:58:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cf24ce700 1 -- start start 2024-02-22T15:58:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cf24ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cf24ce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cec1027e0 0x7f2cec113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cf24ce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec114510 con 0x7f2cec101e40 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cf24ce700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec10d840 con 0x7f2cec1027e0 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57016/0 (socket says 172.21.15.19:57016) 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cebfff700 1 -- 172.21.15.19:0/1722550727 learned_addr learned my addr 172.21.15.19:0/1722550727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2ceb7fe700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cec1027e0 0x7f2cec113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cebfff700 1 -- 172.21.15.19:0/1722550727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cec1027e0 msgr2=0x7f2cec113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cebfff700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cec1027e0 0x7f2cec113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.577+0000 7f2cebfff700 1 -- 172.21.15.19:0/1722550727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc005040 con 0x7f2cec101e40 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.578+0000 7f2cebfff700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec113980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00b3b0 tx=0x7f2cdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.578+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cdc00e040 con 0x7f2cec101e40 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.578+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec10dac0 con 0x7f2cec101e40 2024-02-22T15:58:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.578+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cdc01c430 con 0x7f2cec101e40 2024-02-22T15:58:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.578+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cdc013640 con 0x7f2cec101e40 2024-02-22T15:58:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.578+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec10e000 con 0x7f2cec101e40 2024-02-22T15:58:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.579+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec0738b0 con 0x7f2cec101e40 2024-02-22T15:58:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.579+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2cdc01c5a0 con 0x7f2cec101e40 2024-02-22T15:58:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.580+0000 7f2ce8ff9700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd806cc20 0x7f2cd806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.580+0000 7f2ceb7fe700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd806cc20 0x7f2cd806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.580+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2cdc08ca80 con 0x7f2cec101e40 2024-02-22T15:58:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.580+0000 7f2ceb7fe700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd806cc20 0x7f2cd806f0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cec10f1b0 tx=0x7f2cd400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.584+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2cdc05a6a0 con 0x7f2cec101e40 2024-02-22T15:58:08.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.796+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cec10f000 con 0x7f2cd806cc20 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.806+0000 7f2ce8ff9700 1 -- 172.21.15.19:0/1722550727 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2cec10f000 con 0x7f2cd806cc20 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (5m) 58s ago 16m 26.8M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (2m) 58s ago 16m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (2m) 25s ago 15m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (5m) 58s ago 16m 85.8M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (10m) 58s ago 10m 113M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (10m) 58s ago 10m 67.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (10m) 25s ago 10m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:58:08.785 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (10m) 25s ago 10m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (7m) 58s ago 17m 499M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (6m) 25s ago 15m 449M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (3m) 58s ago 18m 47.5M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (3m) 25s ago 15m 38.9M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (6m) 58s ago 16m 18.4M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (6m) 25s ago 15m 17.0M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (2m) 58s ago 14m 87.3M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (100s) 58s ago 14m 93.0M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (64s) 58s ago 13m 11.4M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (29s) 25s ago 13m 11.4M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (12m) 25s ago 12m 232M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 99cccd96af1a 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (12m) 25s ago 12m 222M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:58:08.786 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (5m) 58s ago 15m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:58:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd806cc20 msgr2=0x7f2cd806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd806cc20 0x7f2cd806f0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cec10f1b0 tx=0x7f2cd400b040 comp rx=0 tx=0).stop 2024-02-22T15:58:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 msgr2=0x7f2cec113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec113980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00b3b0 tx=0x7f2cdc009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 shutdown_connections 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd806cc20 0x7f2cd806f0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec101e40 0x7f2cec113980 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 --2- 172.21.15.19:0/1722550727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cec1027e0 0x7f2cec113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 >> 172.21.15.19:0/1722550727 conn(0x7f2cec071d90 msgr2=0x7f2cec065780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 shutdown_connections 2024-02-22T15:58:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.808+0000 7f2cf24ce700 1 -- 172.21.15.19:0/1722550727 wait complete. 2024-02-22T15:58:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 -- 172.21.15.19:0/2449111655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34841038e0 msgr2=0x7f34840fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 --2- 172.21.15.19:0/2449111655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34841038e0 0x7f34840fcf50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3474004940 tx=0x7f3474009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 -- 172.21.15.19:0/2449111655 shutdown_connections 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 --2- 172.21.15.19:0/2449111655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34841038e0 0x7f34840fcf50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 --2- 172.21.15.19:0/2449111655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3484102f40 0x7f3484103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 -- 172.21.15.19:0/2449111655 >> 172.21.15.19:0/2449111655 conn(0x7f34840f8af0 msgr2=0x7f34840faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 -- 172.21.15.19:0/2449111655 shutdown_connections 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.911+0000 7f3489fd7700 1 -- 172.21.15.19:0/2449111655 wait complete. 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3489fd7700 1 Processor -- start 2024-02-22T15:58:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3489fd7700 1 -- start start 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3489fd7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 0x7f3484113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3489fd7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34841038e0 0x7f3484113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3489fd7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841144b0 con 0x7f3484102f40 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3489fd7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348410d7e0 con 0x7f34841038e0 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f34837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 0x7f3484113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f34837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 0x7f3484113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57032/0 (socket says 172.21.15.19:57032) 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f34837fe700 1 -- 172.21.15.19:0/1886883946 learned_addr learned my addr 172.21.15.19:0/1886883946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f3482ffd700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34841038e0 0x7f3484113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f34837fe700 1 -- 172.21.15.19:0/1886883946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34841038e0 msgr2=0x7f3484113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f34837fe700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34841038e0 0x7f3484113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.912+0000 7f34837fe700 1 -- 172.21.15.19:0/1886883946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474005040 con 0x7f3484102f40 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.913+0000 7f34837fe700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 0x7f3484113920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f346c00eb40 tx=0x7f346c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.913+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346c007cc0 con 0x7f3484102f40 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.913+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348410da60 con 0x7f3484102f40 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.913+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f346c00f040 con 0x7f3484102f40 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.913+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346c009570 con 0x7f3484102f40 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.913+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348410df70 con 0x7f3484102f40 2024-02-22T15:58:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.914+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3484105410 con 0x7f3484102f40 2024-02-22T15:58:08.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.914+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f346c0096d0 con 0x7f3484102f40 2024-02-22T15:58:08.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.915+0000 7f347bfff700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f347006cc20 0x7f347006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:08.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.915+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f346c08bc90 con 0x7f3484102f40 2024-02-22T15:58:08.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.915+0000 7f3482ffd700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f347006cc20 0x7f347006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:08.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.915+0000 7f3482ffd700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f347006cc20 0x7f347006f0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f348410eb10 tx=0x7f3474009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:08.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:08.919+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f346c059830 con 0x7f3484102f40 2024-02-22T15:58:08.940 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:08 smithi062 bash[125627]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4 2024-02-22T15:58:08.941 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:08 smithi062 ceph-mon[106861]: pgmap v130: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:58:08.941 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:08 smithi062 ceph-mon[106861]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:58:08.941 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:08 smithi062 ceph-mon[106861]: failed to encode map e71 with expected crc 2024-02-22T15:58:08.941 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:08 smithi062 ceph-mon[106861]: osdmap e71: 6 total, 5 up, 6 in 2024-02-22T15:58:09.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.230+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f348404bed0 con 0x7f3484102f40 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.231+0000 7f347bfff700 1 -- 172.21.15.19:0/1886883946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f346c0593c0 con 0x7f3484102f40 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:58:09.210 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 1, 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 5, 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 8 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:58:09.211 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f347006cc20 msgr2=0x7f347006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f347006cc20 0x7f347006f0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f348410eb10 tx=0x7f3474009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 msgr2=0x7f3484113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 0x7f3484113920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f346c00eb40 tx=0x7f346c00b040 comp rx=0 tx=0).stop 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 shutdown_connections 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f347006cc20 0x7f347006f0d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3484102f40 0x7f3484113920 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 --2- 172.21.15.19:0/1886883946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34841038e0 0x7f3484113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 >> 172.21.15.19:0/1886883946 conn(0x7f34840f8af0 msgr2=0x7f34840fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:09.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 shutdown_connections 2024-02-22T15:58:09.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.233+0000 7f3489fd7700 1 -- 172.21.15.19:0/1886883946 wait complete. 2024-02-22T15:58:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:08 smithi019 ceph-mon[173579]: pgmap v130: 65 pgs: 65 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:58:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:08 smithi019 ceph-mon[173579]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:58:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:08 smithi019 ceph-mon[173579]: failed to encode map e71 with expected crc 2024-02-22T15:58:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:08 smithi019 ceph-mon[173579]: osdmap e71: 6 total, 5 up, 6 in 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.335+0000 7fbe7346d700 1 -- 172.21.15.19:0/2369548892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 msgr2=0x7fbe6c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.335+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2369548892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c0fe3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe68004970 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- 172.21.15.19:0/2369548892 shutdown_connections 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2369548892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c0fe8f0 0x7fbe6c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2369548892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c0fe3b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- 172.21.15.19:0/2369548892 >> 172.21.15.19:0/2369548892 conn(0x7fbe6c071d90 msgr2=0x7fbe6c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- 172.21.15.19:0/2369548892 shutdown_connections 2024-02-22T15:58:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- 172.21.15.19:0/2369548892 wait complete. 2024-02-22T15:58:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 Processor -- start 2024-02-22T15:58:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- start start 2024-02-22T15:58:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c0fe8f0 0x7fbe6c116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c1167e0 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.336+0000 7fbe7346d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c10fa80 con 0x7fbe6c0fe8f0 2024-02-22T15:58:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:09.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57048/0 (socket says 172.21.15.19:57048) 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 -- 172.21.15.19:0/2670324867 learned_addr learned my addr 172.21.15.19:0/2670324867 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe70a08700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c0fe8f0 0x7fbe6c116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 -- 172.21.15.19:0/2670324867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c0fe8f0 msgr2=0x7fbe6c116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c0fe8f0 0x7fbe6c116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 -- 172.21.15.19:0/2670324867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68005040 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe71209700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c115bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c00 tx=0x7fbe68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6800e040 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe68013420 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c10fd00 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6801c630 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.337+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c110240 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.338+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c106b20 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.339+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbe680139e0 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.339+0000 7fbe61ffb700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe5806cc20 0x7fbe5806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.339+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe6808d130 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.339+0000 7fbe70a08700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe5806cc20 0x7fbe5806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.340+0000 7fbe70a08700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe5806cc20 0x7fbe5806f0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c1113f0 tx=0x7fbe5c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.344+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fbe6805adf0 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.588+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbe6c04bed0 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.589+0000 7fbe61ffb700 1 -- 172.21.15.19:0/2670324867 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fbe6805a980 con 0x7fbe6c0fdfe0 2024-02-22T15:58:09.568 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:58:09.569 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:58:09.570 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:58:09.571 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:09.572 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:09.572 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:09.572 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:09.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:58:09.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:58:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe5806cc20 msgr2=0x7fbe5806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe5806cc20 0x7fbe5806f0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c1113f0 tx=0x7fbe5c0091a0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 msgr2=0x7fbe6c115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c115bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c00 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 shutdown_connections 2024-02-22T15:58:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe5806cc20 0x7fbe5806f0d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6c0fdfe0 0x7fbe6c115bc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 --2- 172.21.15.19:0/2670324867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe6c0fe8f0 0x7fbe6c116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 >> 172.21.15.19:0/2670324867 conn(0x7fbe6c071d90 msgr2=0x7fbe6c0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 shutdown_connections 2024-02-22T15:58:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.591+0000 7fbe7346d700 1 -- 172.21.15.19:0/2670324867 wait complete. 2024-02-22T15:58:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 -- 172.21.15.19:0/605830931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a581038f0 msgr2=0x7f9a580fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/605830931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a581038f0 0x7f9a580fcf60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a48004940 tx=0x7f9a48009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 -- 172.21.15.19:0/605830931 shutdown_connections 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/605830931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a581038f0 0x7f9a580fcf60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/605830931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a58102f50 0x7f9a58103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 -- 172.21.15.19:0/605830931 >> 172.21.15.19:0/605830931 conn(0x7f9a580f8ae0 msgr2=0x7f9a580faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 -- 172.21.15.19:0/605830931 shutdown_connections 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.695+0000 7f9a5ce32700 1 -- 172.21.15.19:0/605830931 wait complete. 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5ce32700 1 Processor -- start 2024-02-22T15:58:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5ce32700 1 -- start start 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5ce32700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 0x7f9a58113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5ce32700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a581038f0 0x7f9a58113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5ce32700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a581144c0 con 0x7f9a58102f50 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5ce32700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5810d7f0 con 0x7f9a581038f0 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 0x7f9a58113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 0x7f9a58113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57060/0 (socket says 172.21.15.19:57060) 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 -- 172.21.15.19:0/1824095811 learned_addr learned my addr 172.21.15.19:0/1824095811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a55d9b700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a581038f0 0x7f9a58113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 -- 172.21.15.19:0/1824095811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a581038f0 msgr2=0x7f9a58113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a581038f0 0x7f9a58113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 -- 172.21.15.19:0/1824095811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48005040 con 0x7f9a58102f50 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a5659c700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 0x7f9a58113930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00eb40 tx=0x7f9a4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.696+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4c007cc0 con 0x7f9a58102f50 2024-02-22T15:58:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.697+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a4c00f040 con 0x7f9a58102f50 2024-02-22T15:58:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.697+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5810da70 con 0x7f9a58102f50 2024-02-22T15:58:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.697+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4c00c990 con 0x7f9a58102f50 2024-02-22T15:58:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.697+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5810df80 con 0x7f9a58102f50 2024-02-22T15:58:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.697+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a580fc6e0 con 0x7f9a58102f50 2024-02-22T15:58:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.698+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9a4c00cc00 con 0x7f9a58102f50 2024-02-22T15:58:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.698+0000 7f9a46ffd700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4006cc20 0x7f9a4006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.698+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9a4c08bbc0 con 0x7f9a58102f50 2024-02-22T15:58:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.698+0000 7f9a55d9b700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4006cc20 0x7f9a4006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.699+0000 7f9a55d9b700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4006cc20 0x7f9a4006f0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a48000c00 tx=0x7f9a48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:09.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.703+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9a4c059760 con 0x7f9a58102f50 2024-02-22T15:58:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.925+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a5810ef80 con 0x7f9a4006cc20 2024-02-22T15:58:09.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.927+0000 7f9a46ffd700 1 -- 172.21.15.19:0/1824095811 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9a5810ef80 con 0x7f9a4006cc20 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "10/21 daemons upgraded", 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading osd daemons", 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:58:09.906 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4006cc20 msgr2=0x7f9a4006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4006cc20 0x7f9a4006f0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a48000c00 tx=0x7f9a48009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 msgr2=0x7f9a58113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 0x7f9a58113930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00eb40 tx=0x7f9a4c00b040 comp rx=0 tx=0).stop 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 shutdown_connections 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4006cc20 0x7f9a4006f0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a58102f50 0x7f9a58113930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 --2- 172.21.15.19:0/1824095811 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a581038f0 0x7f9a58113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 >> 172.21.15.19:0/1824095811 conn(0x7f9a580f8ae0 msgr2=0x7f9a580fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:09.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 shutdown_connections 2024-02-22T15:58:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:09.929+0000 7f9a5ce32700 1 -- 172.21.15.19:0/1824095811 wait complete. 2024-02-22T15:58:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 -- 172.21.15.19:0/377557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa748102f50 msgr2=0x7fa748103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/377557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa748102f50 0x7fa748103320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa738004940 tx=0x7fa738009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 -- 172.21.15.19:0/377557601 shutdown_connections 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/377557601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7481038f0 0x7fa7480fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/377557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa748102f50 0x7fa748103320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 -- 172.21.15.19:0/377557601 >> 172.21.15.19:0/377557601 conn(0x7fa7480f8ae0 msgr2=0x7fa7480faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 -- 172.21.15.19:0/377557601 shutdown_connections 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.033+0000 7fa74fd6f700 1 -- 172.21.15.19:0/377557601 wait complete. 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74fd6f700 1 Processor -- start 2024-02-22T15:58:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74fd6f700 1 -- start start 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74fd6f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa748102f50 0x7fa7481139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74fd6f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 0x7fa748113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74fd6f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748114530 con 0x7fa7481038f0 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74fd6f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74810d860 con 0x7fa748102f50 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74d30a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 0x7fa748113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74d30a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 0x7fa748113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57088/0 (socket says 172.21.15.19:57088) 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74d30a700 1 -- 172.21.15.19:0/3186807394 learned_addr learned my addr 172.21.15.19:0/3186807394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74db0b700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa748102f50 0x7fa7481139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74d30a700 1 -- 172.21.15.19:0/3186807394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa748102f50 msgr2=0x7fa7481139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74d30a700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa748102f50 0x7fa7481139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74d30a700 1 -- 172.21.15.19:0/3186807394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738005040 con 0x7fa7481038f0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.034+0000 7fa74db0b700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa748102f50 0x7fa7481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.035+0000 7fa74d30a700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 0x7fa748113ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa74400eb40 tx=0x7fa74400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.035+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa744007cc0 con 0x7fa7481038f0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.035+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa74400f040 con 0x7fa7481038f0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.035+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74810dae0 con 0x7fa7481038f0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.035+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa744009570 con 0x7fa7481038f0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.035+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74810e020 con 0x7fa7481038f0 2024-02-22T15:58:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.036+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa748100260 con 0x7fa7481038f0 2024-02-22T15:58:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.037+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa7440096d0 con 0x7fa7481038f0 2024-02-22T15:58:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.037+0000 7fa73e7fc700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa73406cc30 0x7fa73406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.037+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa74408c360 con 0x7fa7481038f0 2024-02-22T15:58:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.037+0000 7fa74db0b700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa73406cc30 0x7fa73406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.037+0000 7fa74db0b700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa73406cc30 0x7fa73406f0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b0d0 tx=0x7fa738009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.041+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa744059ed0 con 0x7fa7481038f0 2024-02-22T15:58:10.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: failed to encode map e72 with expected crc 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: osdmap e72: 6 total, 5 up, 6 in 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: failed to encode map e72 with expected crc 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1886883946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:10.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:09 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2670324867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:58:10.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:10.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:10.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:10.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: failed to encode map e72 with expected crc 2024-02-22T15:58:10.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: osdmap e72: 6 total, 5 up, 6 in 2024-02-22T15:58:10.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: failed to encode map e72 with expected crc 2024-02-22T15:58:10.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1886883946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:10.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:09 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2670324867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:58:10.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.329+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa748109f10 con 0x7fa7481038f0 2024-02-22T15:58:10.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.329+0000 7fa73e7fc700 1 -- 172.21.15.19:0/3186807394 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+380 (secure 0 0 0) 0x7fa744059a60 con 0x7fa7481038f0 2024-02-22T15:58:10.308 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 5/318 objects degraded (1.572%), 2 pgs degraded 2024-02-22T15:58:10.309 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-22T15:58:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: osd.4 (root=default,host=smithi062) is down 2024-02-22T15:58:10.309 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 5/318 objects degraded (1.572%), 2 pgs degraded 2024-02-22T15:58:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-02-22T15:58:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-02-22T15:58:10.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa73406cc30 msgr2=0x7fa73406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:10.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa73406cc30 0x7fa73406f0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b0d0 tx=0x7fa738009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 msgr2=0x7fa748113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 0x7fa748113ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa74400eb40 tx=0x7fa74400b040 comp rx=0 tx=0).stop 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 shutdown_connections 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa73406cc30 0x7fa73406f0e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa748102f50 0x7fa7481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 --2- 172.21.15.19:0/3186807394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7481038f0 0x7fa748113ee0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.331+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 >> 172.21.15.19:0/3186807394 conn(0x7fa7480f8ae0 msgr2=0x7fa7480fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.332+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 shutdown_connections 2024-02-22T15:58:10.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:10.332+0000 7fa74fd6f700 1 -- 172.21.15.19:0/3186807394 wait complete. 2024-02-22T15:58:10.943 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:10 smithi062 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.4.service: Succeeded. 2024-02-22T15:58:10.943 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:10 smithi062 systemd[1]: Stopped Ceph osd.4 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:58:10.943 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:10 smithi062 systemd[1]: Starting Ceph osd.4 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:58:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:10 smithi062 ceph-mon[106861]: pgmap v133: 65 pgs: 2 active+undersized+degraded, 5 stale+active+clean, 3 active+undersized, 55 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 5/318 objects degraded (1.572%) 2024-02-22T15:58:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:10 smithi062 ceph-mon[106861]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:10 smithi062 ceph-mon[106861]: Health check failed: Degraded data redundancy: 5/318 objects degraded (1.572%), 2 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:11.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:10 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3186807394' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:58:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:10 smithi019 ceph-mon[173579]: pgmap v133: 65 pgs: 2 active+undersized+degraded, 5 stale+active+clean, 3 active+undersized, 55 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 5/318 objects degraded (1.572%) 2024-02-22T15:58:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:10 smithi019 ceph-mon[173579]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:10 smithi019 ceph-mon[173579]: Health check failed: Degraded data redundancy: 5/318 objects degraded (1.572%), 2 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:11.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:10 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3186807394' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-22T15:58:11.955 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:11 smithi062 bash[125898]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-22T15:58:13.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:12 smithi062 ceph-mon[106861]: pgmap v134: 65 pgs: 12 active+undersized+degraded, 2 stale+active+clean, 14 active+undersized, 37 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 40/318 objects degraded (12.579%) 2024-02-22T15:58:13.205 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:12 smithi062 ceph-osd[126196]: -- 172.21.15.62:0/1127737972 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dca4c824e0 con 0x55dca4c52c00 2024-02-22T15:58:13.205 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:12 smithi062 bash[126158]: 162bbdb5756dad539d6cdea6cb6e93dd76a1da2a71ae3ded8eebfda12e8ced12 2024-02-22T15:58:13.205 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:12 smithi062 systemd[1]: Started Ceph osd.4 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:58:13.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:12 smithi019 ceph-mon[173579]: pgmap v134: 65 pgs: 12 active+undersized+degraded, 2 stale+active+clean, 14 active+undersized, 37 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 40/318 objects degraded (12.579%) 2024-02-22T15:58:13.704 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:13 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T15:58:13.436+0000 7f5bac9df640 -1 Falling back to public interface 2024-02-22T15:58:14.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:14.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:14.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:14 smithi062 ceph-mon[106861]: pgmap v135: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-02-22T15:58:14.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:14.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:14.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:14.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:14 smithi019 ceph-mon[173579]: pgmap v135: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-02-22T15:58:17.121 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:16 smithi062 ceph-mon[106861]: pgmap v136: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-02-22T15:58:17.121 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.121 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.122 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.122 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:16 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:16 smithi019 ceph-mon[173579]: pgmap v136: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-02-22T15:58:17.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:17.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:16 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:18.204 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:17 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T15:58:17.942+0000 7f5bac9df640 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-22T15:58:18.704 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T15:58:18.472+0000 7f5bac9df640 -1 osd.4 70 log_to_monitors true 2024-02-22T15:58:19.158 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-mon[106861]: pgmap v137: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-02-22T15:58:19.159 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 53/318 objects degraded (16.667%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:19.159 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:19.159 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:19.159 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-mon[106861]: from='osd.4 [v2:172.21.15.62:6808/227974398,v1:172.21.15.62:6809/227974398]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:58:19.159 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:18 smithi062 ceph-mon[106861]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:58:19.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:18 smithi019 ceph-mon[173579]: pgmap v137: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/318 objects degraded (16.667%) 2024-02-22T15:58:19.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:18 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 53/318 objects degraded (16.667%), 16 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:18 smithi019 ceph-mon[173579]: from='osd.4 [v2:172.21.15.62:6808/227974398,v1:172.21.15.62:6809/227974398]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:58:19.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:18 smithi019 ceph-mon[173579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-22T15:58:19.454 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 15:58:19 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T15:58:19.179+0000 7f5b9c635700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 53/318 objects degraded (16.667%) 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: failed to encode map e73 with expected crc 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: osdmap e73: 6 total, 5 up, 6 in 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='osd.4 [v2:172.21.15.62:6808/227974398,v1:172.21.15.62:6809/227974398]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:20.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:20.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:20.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 254 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 53/318 objects degraded (16.667%) 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: failed to encode map e73 with expected crc 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: osdmap e73: 6 total, 5 up, 6 in 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='osd.4 [v2:172.21.15.62:6808/227974398,v1:172.21.15.62:6809/227974398]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:20.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:20.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:20.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:21.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-22T15:58:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:58:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: failed to encode map e74 with expected crc 2024-02-22T15:58:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: osd.4 [v2:172.21.15.62:6808/227974398,v1:172.21.15.62:6809/227974398] boot 2024-02-22T15:58:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: osdmap e74: 6 total, 6 up, 6 in 2024-02-22T15:58:21.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:21 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:58:21.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:21.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-22T15:58:21.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:58:21.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: failed to encode map e74 with expected crc 2024-02-22T15:58:21.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: osd.4 [v2:172.21.15.62:6808/227974398,v1:172.21.15.62:6809/227974398] boot 2024-02-22T15:58:21.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: osdmap e74: 6 total, 6 up, 6 in 2024-02-22T15:58:21.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:21 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-22T15:58:22.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:22 smithi062 ceph-mon[106861]: pgmap v141: 65 pgs: 18 peering, 7 active+undersized+degraded, 9 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 20/318 objects degraded (6.289%) 2024-02-22T15:58:22.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:22 smithi062 ceph-mon[106861]: failed to encode map e75 with expected crc 2024-02-22T15:58:22.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:22 smithi062 ceph-mon[106861]: osdmap e75: 6 total, 6 up, 6 in 2024-02-22T15:58:22.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:22 smithi062 ceph-mon[106861]: failed to encode map e75 with expected crc 2024-02-22T15:58:22.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:22 smithi062 ceph-mon[106861]: failed to encode map e75 with expected crc 2024-02-22T15:58:22.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:22 smithi019 ceph-mon[173579]: pgmap v141: 65 pgs: 18 peering, 7 active+undersized+degraded, 9 active+undersized, 31 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 20/318 objects degraded (6.289%) 2024-02-22T15:58:22.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:22 smithi019 ceph-mon[173579]: failed to encode map e75 with expected crc 2024-02-22T15:58:22.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:22 smithi019 ceph-mon[173579]: osdmap e75: 6 total, 6 up, 6 in 2024-02-22T15:58:22.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:22 smithi019 ceph-mon[173579]: failed to encode map e75 with expected crc 2024-02-22T15:58:22.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:22 smithi019 ceph-mon[173579]: failed to encode map e75 with expected crc 2024-02-22T15:58:23.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:23 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 20/318 objects degraded (6.289%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:23.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:23 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 20/318 objects degraded (6.289%), 7 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:25.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:24 smithi062 ceph-mon[106861]: pgmap v143: 65 pgs: 18 peering, 4 active+undersized+degraded, 6 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 9/318 objects degraded (2.830%) 2024-02-22T15:58:25.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:24 smithi019 ceph-mon[173579]: pgmap v143: 65 pgs: 18 peering, 4 active+undersized+degraded, 6 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 9/318 objects degraded (2.830%) 2024-02-22T15:58:26.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:25 smithi019 ceph-mon[173579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/318 objects degraded (2.830%), 4 pgs degraded) 2024-02-22T15:58:26.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:25 smithi019 ceph-mon[173579]: Cluster is now healthy 2024-02-22T15:58:26.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:25 smithi062 ceph-mon[106861]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/318 objects degraded (2.830%), 4 pgs degraded) 2024-02-22T15:58:26.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:25 smithi062 ceph-mon[106861]: Cluster is now healthy 2024-02-22T15:58:27.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:26 smithi019 ceph-mon[173579]: pgmap v144: 65 pgs: 18 peering, 47 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-22T15:58:27.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:26 smithi062 ceph-mon[106861]: pgmap v144: 65 pgs: 18 peering, 47 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-22T15:58:29.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:28 smithi019 ceph-mon[173579]: pgmap v145: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:58:29.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:28 smithi062 ceph-mon[106861]: pgmap v145: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:58:31.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:30 smithi019 ceph-mon[173579]: pgmap v146: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-22T15:58:31.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:30 smithi062 ceph-mon[106861]: pgmap v146: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-02-22T15:58:33.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:32 smithi019 ceph-mon[173579]: pgmap v147: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-02-22T15:58:33.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:32 smithi062 ceph-mon[106861]: pgmap v147: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-02-22T15:58:34.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:34 smithi062 ceph-mon[106861]: pgmap v148: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:58:34.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:34.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:34.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:34.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:34 smithi019 ceph-mon[173579]: pgmap v148: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T15:58:34.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:34.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:34.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:35.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:35 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:35.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:35 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:36.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:36 smithi062 ceph-mon[106861]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:36.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:36 smithi062 ceph-mon[106861]: Upgrade: osd.5 is safe to restart 2024-02-22T15:58:36.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:36 smithi062 ceph-mon[106861]: pgmap v149: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:58:36.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:36 smithi019 ceph-mon[173579]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-22T15:58:36.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:36 smithi019 ceph-mon[173579]: Upgrade: osd.5 is safe to restart 2024-02-22T15:58:36.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:36 smithi019 ceph-mon[173579]: pgmap v149: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-02-22T15:58:37.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:37 smithi019 ceph-mon[173579]: Upgrade: Updating osd.5 2024-02-22T15:58:37.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:37 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:37.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:37 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T15:58:37.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:37 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:37.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:37 smithi019 ceph-mon[173579]: Deploying daemon osd.5 on smithi062 2024-02-22T15:58:37.823 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:37 smithi062 ceph-mon[106861]: Upgrade: Updating osd.5 2024-02-22T15:58:37.824 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:37 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:37.824 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:37 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-22T15:58:37.824 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:37 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:37.824 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:37 smithi062 ceph-mon[106861]: Deploying daemon osd.5 on smithi062 2024-02-22T15:58:39.123 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:38 smithi062 ceph-mon[106861]: pgmap v150: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:58:39.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:38 smithi019 ceph-mon[173579]: pgmap v150: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-02-22T15:58:39.704 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:39 smithi062 systemd[1]: Stopping Ceph osd.5 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:58:39.705 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:39 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[71291]: 2024-02-22T15:58:39.665+0000 7f05c8a21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T15:58:39.705 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:39 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[71291]: 2024-02-22T15:58:39.665+0000 7f05c8a21700 -1 osd.5 75 *** Got signal Terminated *** 2024-02-22T15:58:39.705 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:39 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[71291]: 2024-02-22T15:58:39.665+0000 7f05c8a21700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T15:58:40.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:39 smithi062 ceph-mon[106861]: osd.5 marked itself down and dead 2024-02-22T15:58:40.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:39 smithi019 ceph-mon[173579]: osd.5 marked itself down and dead 2024-02-22T15:58:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 -- 172.21.15.19:0/2909361362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 msgr2=0x7f6ad8107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/2909361362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad8107050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6acc004970 tx=0x7f6acc009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 -- 172.21.15.19:0/2909361362 shutdown_connections 2024-02-22T15:58:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/2909361362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad8107050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/2909361362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad80fdfe0 0x7f6ad80fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 -- 172.21.15.19:0/2909361362 >> 172.21.15.19:0/2909361362 conn(0x7f6ad8071d90 msgr2=0x7f6ad8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 -- 172.21.15.19:0/2909361362 shutdown_connections 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.450+0000 7f6adf5dd700 1 -- 172.21.15.19:0/2909361362 wait complete. 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adf5dd700 1 Processor -- start 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adf5dd700 1 -- start start 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adf5dd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad80fdfe0 0x7f6ad80fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adf5dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad80fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adf5dd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80fc9e0 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adf5dd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80fcb50 con 0x7f6ad80fdfe0 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adcb78700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad80fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:40.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adcb78700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad80fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40992/0 (socket says 172.21.15.19:40992) 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adcb78700 1 -- 172.21.15.19:0/57023089 learned_addr learned my addr 172.21.15.19:0/57023089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6add379700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad80fdfe0 0x7f6ad80fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adcb78700 1 -- 172.21.15.19:0/57023089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad80fdfe0 msgr2=0x7f6ad80fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adcb78700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad80fdfe0 0x7f6ad80fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.451+0000 7f6adcb78700 1 -- 172.21.15.19:0/57023089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc005040 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.452+0000 7f6adcb78700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad80fc3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6acc000c00 tx=0x7f6acc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.452+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6acc00e040 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.452+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6acc013420 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.452+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad8119bb0 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.452+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6acc01c630 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.452+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad811a010 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.453+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad81046c0 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.454+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6acc0139e0 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.454+0000 7f6ac9ffb700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac406cc20 0x7f6ac406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.454+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6acc08d260 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.454+0000 7f6add379700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac406cc20 0x7f6ac406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:40.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.455+0000 7f6add379700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac406cc20 0x7f6ac406f0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4004f40 tx=0x7f6ad400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:40.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.459+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6acc057340 con 0x7f6ad80fe8f0 2024-02-22T15:58:40.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.681+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ad80fd5e0 con 0x7f6ac406cc20 2024-02-22T15:58:40.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.682+0000 7f6ac9ffb700 1 -- 172.21.15.19:0/57023089 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6ad80fd5e0 con 0x7f6ac406cc20 2024-02-22T15:58:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac406cc20 msgr2=0x7f6ac406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac406cc20 0x7f6ac406f0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4004f40 tx=0x7f6ad400b040 comp rx=0 tx=0).stop 2024-02-22T15:58:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 msgr2=0x7f6ad80fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad80fc3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6acc000c00 tx=0x7f6acc009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 shutdown_connections 2024-02-22T15:58:40.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac406cc20 0x7f6ac406f0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad80fdfe0 0x7f6ad80fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 --2- 172.21.15.19:0/57023089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad80fe8f0 0x7f6ad80fc3e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 >> 172.21.15.19:0/57023089 conn(0x7f6ad8071d90 msgr2=0x7f6ad8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:40.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 shutdown_connections 2024-02-22T15:58:40.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.684+0000 7f6adf5dd700 1 -- 172.21.15.19:0/57023089 wait complete. 2024-02-22T15:58:40.673 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:58:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 -- 172.21.15.19:0/2314883401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 msgr2=0x7f9c40103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 --2- 172.21.15.19:0/2314883401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40103320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c004940 tx=0x7f9c3c009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 -- 172.21.15.19:0/2314883401 shutdown_connections 2024-02-22T15:58:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 --2- 172.21.15.19:0/2314883401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c401038f0 0x7f9c400fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 --2- 172.21.15.19:0/2314883401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40103320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 -- 172.21.15.19:0/2314883401 >> 172.21.15.19:0/2314883401 conn(0x7f9c400f8ae0 msgr2=0x7f9c400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 -- 172.21.15.19:0/2314883401 shutdown_connections 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.788+0000 7f9c481e7700 1 -- 172.21.15.19:0/2314883401 wait complete. 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c481e7700 1 Processor -- start 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c481e7700 1 -- start start 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c481e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c481e7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c401038f0 0x7f9c401160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c481e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40116730 con 0x7f9c40102f50 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c481e7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4010fa60 con 0x7f9c401038f0 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45f83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45f83700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41000/0 (socket says 172.21.15.19:41000) 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45f83700 1 -- 172.21.15.19:0/1171084386 learned_addr learned my addr 172.21.15.19:0/1171084386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45782700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c401038f0 0x7f9c401160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45f83700 1 -- 172.21.15.19:0/1171084386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c401038f0 msgr2=0x7f9c401160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45f83700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c401038f0 0x7f9c401160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.789+0000 7f9c45f83700 1 -- 172.21.15.19:0/1171084386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c005040 con 0x7f9c40102f50 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.790+0000 7f9c45f83700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40115ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009180 tx=0x7f9c3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.790+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c00e040 con 0x7f9c40102f50 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.790+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c3c004290 con 0x7f9c40102f50 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.790+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4010fce0 con 0x7f9c40102f50 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.790+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c01c3f0 con 0x7f9c40102f50 2024-02-22T15:58:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.790+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c40110220 con 0x7f9c40102f50 2024-02-22T15:58:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.791+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c400fc6e0 con 0x7f9c40102f50 2024-02-22T15:58:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.791+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9c3c013420 con 0x7f9c40102f50 2024-02-22T15:58:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.792+0000 7f9c327fc700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c2c06cc20 0x7f9c2c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.792+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c3c08d2e0 con 0x7f9c40102f50 2024-02-22T15:58:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.792+0000 7f9c45782700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c2c06cc20 0x7f9c2c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.792+0000 7f9c45782700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c2c06cc20 0x7f9c2c06f0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c401113d0 tx=0x7f9c3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:40.796+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9c3c05af00 con 0x7f9c40102f50 2024-02-22T15:58:41.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.033+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c401111f0 con 0x7f9c2c06cc20 2024-02-22T15:58:41.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.034+0000 7f9c327fc700 1 -- 172.21.15.19:0/1171084386 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9c401111f0 con 0x7f9c2c06cc20 2024-02-22T15:58:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c2c06cc20 msgr2=0x7f9c2c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c2c06cc20 0x7f9c2c06f0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c401113d0 tx=0x7f9c3400b040 comp rx=0 tx=0).stop 2024-02-22T15:58:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 msgr2=0x7f9c40115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40115ba0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009180 tx=0x7f9c3c009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 shutdown_connections 2024-02-22T15:58:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c2c06cc20 0x7f9c2c06f0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c40102f50 0x7f9c40115ba0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 --2- 172.21.15.19:0/1171084386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c401038f0 0x7f9c401160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 >> 172.21.15.19:0/1171084386 conn(0x7f9c400f8ae0 msgr2=0x7f9c400fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:41.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 shutdown_connections 2024-02-22T15:58:41.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.037+0000 7f9c481e7700 1 -- 172.21.15.19:0/1171084386 wait complete. 2024-02-22T15:58:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.140+0000 7f06c801a700 1 -- 172.21.15.19:0/1824698077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0102f40 msgr2=0x7f06c0103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.140+0000 7f06c801a700 1 --2- 172.21.15.19:0/1824698077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0102f40 0x7f06c0103310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06bc004940 tx=0x7f06bc009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- 172.21.15.19:0/1824698077 shutdown_connections 2024-02-22T15:58:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 --2- 172.21.15.19:0/1824698077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c01038e0 0x7f06c00fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 --2- 172.21.15.19:0/1824698077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c0102f40 0x7f06c0103310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- 172.21.15.19:0/1824698077 >> 172.21.15.19:0/1824698077 conn(0x7f06c00f8af0 msgr2=0x7f06c00faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- 172.21.15.19:0/1824698077 shutdown_connections 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- 172.21.15.19:0/1824698077 wait complete. 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 Processor -- start 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- start start 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0102f40 0x7f06c0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 0x7f06c00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0070e40 con 0x7f06c01038e0 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.141+0000 7f06c801a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0070fb0 con 0x7f06c0102f40 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 0x7f06c00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 0x7f06c00707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41018/0 (socket says 172.21.15.19:41018) 2024-02-22T15:58:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 -- 172.21.15.19:0/2294766681 learned_addr learned my addr 172.21.15.19:0/2294766681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:41.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c5db6700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0102f40 0x7f06c0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 -- 172.21.15.19:0/2294766681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0102f40 msgr2=0x7f06c0070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0102f40 0x7f06c0070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 -- 172.21.15.19:0/2294766681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc005040 con 0x7f06c01038e0 2024-02-22T15:58:41.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.142+0000 7f06c55b5700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 0x7f06c00707f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06b0002a50 tx=0x7f06b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.143+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b0007cc0 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.143+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06b000f040 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.143+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0119ba0 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.143+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06b000da30 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.143+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c011a0e0 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.144+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c00fc6f0 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.144+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f06b000db90 con 0x7f06c01038e0 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.145+0000 7f06b67fc700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f06ac06cc30 0x7f06ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.145+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f06b008bbe0 con 0x7f06c01038e0 2024-02-22T15:58:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.145+0000 7f06c5db6700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f06ac06cc30 0x7f06ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.145+0000 7f06c5db6700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f06ac06cc30 0x7f06ac06f0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0043d0 tx=0x7f06bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:41.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.149+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f06b0055c40 con 0x7f06c01038e0 2024-02-22T15:58:41.204 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:40 smithi062 bash[131864]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5 2024-02-22T15:58:41.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:40 smithi062 ceph-mon[106861]: pgmap v151: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:58:41.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:40 smithi062 ceph-mon[106861]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:58:41.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:40 smithi062 ceph-mon[106861]: osdmap e76: 6 total, 5 up, 6 in 2024-02-22T15:58:41.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:40 smithi019 ceph-mon[173579]: pgmap v151: 65 pgs: 65 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-02-22T15:58:41.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:40 smithi019 ceph-mon[173579]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-22T15:58:41.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:40 smithi019 ceph-mon[173579]: osdmap e76: 6 total, 5 up, 6 in 2024-02-22T15:58:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.361+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f06c0071a70 con 0x7f06ac06cc30 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.372+0000 7f06b67fc700 1 -- 172.21.15.19:0/2294766681 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f06c0071a70 con 0x7f06ac06cc30 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (5m) 91s ago 16m 26.8M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (3m) 91s ago 17m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (3m) 25s ago 16m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (5m) 91s ago 16m 85.8M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (11m) 91s ago 11m 113M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (11m) 91s ago 11m 67.4M - 17.2.7-527-g6ea2a55e e2e4ad29904c a1c11c6c0290 2024-02-22T15:58:41.351 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (11m) 25s ago 11m 62.0M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (11m) 25s ago 11m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (7m) 91s ago 18m 499M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (7m) 25s ago 15m 449M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (4m) 91s ago 18m 47.5M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (3m) 25s ago 15m 39.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (6m) 91s ago 17m 18.4M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (6m) 25s ago 16m 17.2M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (2m) 91s ago 15m 87.3M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (2m) 91s ago 14m 93.0M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (96s) 91s ago 14m 11.4M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (61s) 25s ago 13m 96.9M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (28s) 25s ago 13m 13.2M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (12m) 25s ago 12m 224M 2592M 17.2.7-527-g6ea2a55e e2e4ad29904c 4d7c097069f3 2024-02-22T15:58:41.352 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (6m) 91s ago 16m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:58:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f06ac06cc30 msgr2=0x7f06ac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f06ac06cc30 0x7f06ac06f0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0043d0 tx=0x7f06bc009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 msgr2=0x7f06c00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 0x7f06c00707f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06b0002a50 tx=0x7f06b00046b0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 shutdown_connections 2024-02-22T15:58:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f06ac06cc30 0x7f06ac06f0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06c0102f40 0x7f06c0070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 --2- 172.21.15.19:0/2294766681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06c01038e0 0x7f06c00707f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 >> 172.21.15.19:0/2294766681 conn(0x7f06c00f8af0 msgr2=0x7f06c00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 shutdown_connections 2024-02-22T15:58:41.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.374+0000 7f06c801a700 1 -- 172.21.15.19:0/2294766681 wait complete. 2024-02-22T15:58:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 -- 172.21.15.19:0/883591819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 msgr2=0x7f6b3c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 --2- 172.21.15.19:0/883591819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c102210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6b38004940 tx=0x7f6b38009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 -- 172.21.15.19:0/883591819 shutdown_connections 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 --2- 172.21.15.19:0/883591819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b3c1027e0 0x7f6b3c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 --2- 172.21.15.19:0/883591819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c102210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 -- 172.21.15.19:0/883591819 >> 172.21.15.19:0/883591819 conn(0x7f6b3c071d90 msgr2=0x7f6b3c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 -- 172.21.15.19:0/883591819 shutdown_connections 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.476+0000 7f6b44737700 1 -- 172.21.15.19:0/883591819 wait complete. 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b44737700 1 Processor -- start 2024-02-22T15:58:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b44737700 1 -- start start 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b44737700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b44737700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b3c1027e0 0x7f6b3c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b44737700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c116710 con 0x7f6b3c101e40 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b44737700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c10fa40 con 0x7f6b3c1027e0 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b424d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b424d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41034/0 (socket says 172.21.15.19:41034) 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b424d3700 1 -- 172.21.15.19:0/1177215845 learned_addr learned my addr 172.21.15.19:0/1177215845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.477+0000 7f6b41cd2700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b3c1027e0 0x7f6b3c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b424d3700 1 -- 172.21.15.19:0/1177215845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b3c1027e0 msgr2=0x7f6b3c1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b424d3700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b3c1027e0 0x7f6b3c1160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b424d3700 1 -- 172.21.15.19:0/1177215845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38005040 con 0x7f6b3c101e40 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b424d3700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c115b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b38000c00 tx=0x7f6b38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b38021070 con 0x7f6b3c101e40 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b3801c400 con 0x7f6b3c101e40 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c10fcc0 con 0x7f6b3c101e40 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b3800e070 con 0x7f6b3c101e40 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.478+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c1101a0 con 0x7f6b3c101e40 2024-02-22T15:58:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.479+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c0738b0 con 0x7f6b3c101e40 2024-02-22T15:58:41.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.480+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6b3801c970 con 0x7f6b3c101e40 2024-02-22T15:58:41.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.480+0000 7f6b32ffd700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2806cc20 0x7f6b2806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.480+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6b3808d100 con 0x7f6b3c101e40 2024-02-22T15:58:41.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.480+0000 7f6b41cd2700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2806cc20 0x7f6b2806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.480+0000 7f6b41cd2700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2806cc20 0x7f6b2806f0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c1113b0 tx=0x7f6b2c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:41.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.484+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6b3805ac70 con 0x7f6b3c101e40 2024-02-22T15:58:41.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.771+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b3c111160 con 0x7f6b3c101e40 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.772+0000 7f6b32ffd700 1 -- 172.21.15.19:0/1177215845 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f6b3805a800 con 0x7f6b3c101e40 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 5 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 4, 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 9 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:58:41.752 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2806cc20 msgr2=0x7f6b2806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2806cc20 0x7f6b2806f0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c1113b0 tx=0x7f6b2c00b040 comp rx=0 tx=0).stop 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 msgr2=0x7f6b3c115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c115b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b38000c00 tx=0x7f6b38009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 shutdown_connections 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2806cc20 0x7f6b2806f0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b3c101e40 0x7f6b3c115b80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 --2- 172.21.15.19:0/1177215845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b3c1027e0 0x7f6b3c1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 >> 172.21.15.19:0/1177215845 conn(0x7f6b3c071d90 msgr2=0x7f6b3c065740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:41.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 shutdown_connections 2024-02-22T15:58:41.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.774+0000 7f6b44737700 1 -- 172.21.15.19:0/1177215845 wait complete. 2024-02-22T15:58:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- 172.21.15.19:0/2306122548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a81027f0 msgr2=0x7fb6a80fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/2306122548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a81027f0 0x7fb6a80fbe60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb698004940 tx=0x7fb698009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- 172.21.15.19:0/2306122548 shutdown_connections 2024-02-22T15:58:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/2306122548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a81027f0 0x7fb6a80fbe60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/2306122548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a8101e50 0x7fb6a8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- 172.21.15.19:0/2306122548 >> 172.21.15.19:0/2306122548 conn(0x7fb6a80f79e0 msgr2=0x7fb6a80f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- 172.21.15.19:0/2306122548 shutdown_connections 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- 172.21.15.19:0/2306122548 wait complete. 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 Processor -- start 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- start start 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 0x7fb6a8113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a81027f0 0x7fb6a8113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81144c0 con 0x7fb6a8101e50 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.878+0000 7fb6aff4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a810d7f0 con 0x7fb6a81027f0 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 0x7fb6a8113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 0x7fb6a8113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41048/0 (socket says 172.21.15.19:41048) 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 -- 172.21.15.19:0/1221254938 learned_addr learned my addr 172.21.15.19:0/1221254938 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6ad4e9700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a81027f0 0x7fb6a8113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 -- 172.21.15.19:0/1221254938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a81027f0 msgr2=0x7fb6a8113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a81027f0 0x7fb6a8113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 -- 172.21.15.19:0/1221254938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb698005040 con 0x7fb6a8101e50 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6adcea700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 0x7fb6a8113930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400eb40 tx=0x7fb6a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a4007cc0 con 0x7fb6a8101e50 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a810da70 con 0x7fb6a8101e50 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb6a400f040 con 0x7fb6a8101e50 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a400c990 con 0x7fb6a8101e50 2024-02-22T15:58:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.879+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a810df80 con 0x7fb6a8101e50 2024-02-22T15:58:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.880+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a8064b00 con 0x7fb6a8101e50 2024-02-22T15:58:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.881+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb6a400cc00 con 0x7fb6a8101e50 2024-02-22T15:58:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.881+0000 7fb69e7fc700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb69406cc30 0x7fb69406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.881+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb6a408bc60 con 0x7fb6a8101e50 2024-02-22T15:58:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.881+0000 7fb6ad4e9700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb69406cc30 0x7fb69406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.882+0000 7fb6ad4e9700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb69406cc30 0x7fb69406f0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6a810eb20 tx=0x7fb698009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:41.886+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb6a4059760 con 0x7fb6a8101e50 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.132+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb6a80fb5e0 con 0x7fb6a8101e50 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.133+0000 7fb69e7fc700 1 -- 172.21.15.19:0/1221254938 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fb6a40592f0 con 0x7fb6a8101e50 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stdout:e24 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:58:42.112 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 24 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:54:05.091029+0000 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:58:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 47 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=14472,1=24267} 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.19:6828/3100392983,v1:172.21.15.19:6829/3100392983] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:14484} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/588513461,v1:172.21.15.19:6827/588513461] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:42.114 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.62:6824/1888029196,v1:172.21.15.62:6825/1888029196] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:42.115 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:24275} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.62:6826/3228927024,v1:172.21.15.62:6827/3228927024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:58:42.115 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:58:42.115 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:58:42.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb69406cc30 msgr2=0x7fb69406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb69406cc30 0x7fb69406f0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6a810eb20 tx=0x7fb698009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 msgr2=0x7fb6a8113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 0x7fb6a8113930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400eb40 tx=0x7fb6a400b040 comp rx=0 tx=0).stop 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 shutdown_connections 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb69406cc30 0x7fb69406f0e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6a8101e50 0x7fb6a8113930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 --2- 172.21.15.19:0/1221254938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6a81027f0 0x7fb6a8113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 >> 172.21.15.19:0/1221254938 conn(0x7fb6a80f79e0 msgr2=0x7fb6a8066010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 shutdown_connections 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.135+0000 7fb6aff4e700 1 -- 172.21.15.19:0/1221254938 wait complete. 2024-02-22T15:58:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T15:58:42.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:41 smithi062 ceph-mon[106861]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:42.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:41 smithi062 ceph-mon[106861]: osdmap e77: 6 total, 5 up, 6 in 2024-02-22T15:58:42.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:41 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1177215845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 -- 172.21.15.19:0/3206812432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0101e40 msgr2=0x7f06b0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 --2- 172.21.15.19:0/3206812432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0101e40 0x7f06b0102210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06a0004940 tx=0x7f06a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 -- 172.21.15.19:0/3206812432 shutdown_connections 2024-02-22T15:58:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 --2- 172.21.15.19:0/3206812432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b01027e0 0x7f06b00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 --2- 172.21.15.19:0/3206812432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0101e40 0x7f06b0102210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 -- 172.21.15.19:0/3206812432 >> 172.21.15.19:0/3206812432 conn(0x7f06b0071d90 msgr2=0x7f06b0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 -- 172.21.15.19:0/3206812432 shutdown_connections 2024-02-22T15:58:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.239+0000 7f06b64ca700 1 -- 172.21.15.19:0/3206812432 wait complete. 2024-02-22T15:58:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06b64ca700 1 Processor -- start 2024-02-22T15:58:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06b64ca700 1 -- start start 2024-02-22T15:58:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06b64ca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b0101e40 0x7f06b01138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06b64ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 0x7f06b0113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06b64ca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0114480 con 0x7f06b01027e0 2024-02-22T15:58:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06b64ca700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b010d7b0 con 0x7f06b0101e40 2024-02-22T15:58:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 0x7f06b0113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 0x7f06b0113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41066/0 (socket says 172.21.15.19:41066) 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06af7fe700 1 -- 172.21.15.19:0/1360607232 learned_addr learned my addr 172.21.15.19:0/1360607232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.240+0000 7f06affff700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b0101e40 0x7f06b01138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06af7fe700 1 -- 172.21.15.19:0/1360607232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b0101e40 msgr2=0x7f06b01138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06af7fe700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b0101e40 0x7f06b01138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06af7fe700 1 -- 172.21.15.19:0/1360607232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0005040 con 0x7f06b01027e0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06af7fe700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 0x7f06b0113e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06a400eb40 tx=0x7f06a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a4007cc0 con 0x7f06b01027e0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06a400f040 con 0x7f06b01027e0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b010da30 con 0x7f06b01027e0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a4009570 con 0x7f06b01027e0 2024-02-22T15:58:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.241+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b010df70 con 0x7f06b01027e0 2024-02-22T15:58:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.242+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b00738b0 con 0x7f06b01027e0 2024-02-22T15:58:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.242+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f06a40096d0 con 0x7f06b01027e0 2024-02-22T15:58:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.243+0000 7f06acff9700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f069806cc20 0x7f069806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.243+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f06a408bb00 con 0x7f06b01027e0 2024-02-22T15:58:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.243+0000 7f06affff700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f069806cc20 0x7f069806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.243+0000 7f06affff700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f069806cc20 0x7f069806f0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06a000b0d0 tx=0x7f06a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.247+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f06a40596a0 con 0x7f06b01027e0 2024-02-22T15:58:42.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:41 smithi019 ceph-mon[173579]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:42.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:41 smithi019 ceph-mon[173579]: osdmap e77: 6 total, 5 up, 6 in 2024-02-22T15:58:42.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:41 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1177215845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:42.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.471+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06b004bed0 con 0x7f069806cc20 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.472+0000 7f06acff9700 1 -- 172.21.15.19:0/1360607232 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f06b004bed0 con 0x7f069806cc20 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:58:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "mon" 2024-02-22T15:58:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:58:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "11/21 daemons upgraded", 2024-02-22T15:58:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading osd daemons", 2024-02-22T15:58:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:58:42.452 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f069806cc20 msgr2=0x7f069806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f069806cc20 0x7f069806f0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06a000b0d0 tx=0x7f06a0009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 msgr2=0x7f06b0113e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 0x7f06b0113e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06a400eb40 tx=0x7f06a400b040 comp rx=0 tx=0).stop 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 shutdown_connections 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f069806cc20 0x7f069806f0d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b0101e40 0x7f06b01138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 --2- 172.21.15.19:0/1360607232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b01027e0 0x7f06b0113e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 >> 172.21.15.19:0/1360607232 conn(0x7f06b0071d90 msgr2=0x7f06b00656e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 shutdown_connections 2024-02-22T15:58:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.474+0000 7f06b64ca700 1 -- 172.21.15.19:0/1360607232 wait complete. 2024-02-22T15:58:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 -- 172.21.15.19:0/686192577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac601038f0 msgr2=0x7fac600fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 --2- 172.21.15.19:0/686192577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac601038f0 0x7fac600fcf60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac54004940 tx=0x7fac54009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 -- 172.21.15.19:0/686192577 shutdown_connections 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 --2- 172.21.15.19:0/686192577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac601038f0 0x7fac600fcf60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 --2- 172.21.15.19:0/686192577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac60102f50 0x7fac60103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 -- 172.21.15.19:0/686192577 >> 172.21.15.19:0/686192577 conn(0x7fac600f8ae0 msgr2=0x7fac600faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 -- 172.21.15.19:0/686192577 shutdown_connections 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.578+0000 7fac653b4700 1 -- 172.21.15.19:0/686192577 wait complete. 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac653b4700 1 Processor -- start 2024-02-22T15:58:42.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac653b4700 1 -- start start 2024-02-22T15:58:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac653b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 0x7fac60113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac653b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac601038f0 0x7fac60113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac653b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac601144e0 con 0x7fac60102f50 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac653b4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6010d810 con 0x7fac601038f0 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 0x7fac60113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 0x7fac60113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41084/0 (socket says 172.21.15.19:41084) 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5effd700 1 -- 172.21.15.19:0/974739697 learned_addr learned my addr 172.21.15.19:0/974739697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5e7fc700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac601038f0 0x7fac60113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5effd700 1 -- 172.21.15.19:0/974739697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac601038f0 msgr2=0x7fac60113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5effd700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac601038f0 0x7fac60113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.579+0000 7fac5effd700 1 -- 172.21.15.19:0/974739697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac54005040 con 0x7fac60102f50 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.580+0000 7fac5effd700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 0x7fac60113950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac5000ec30 tx=0x7fac5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.580+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac50007b30 con 0x7fac60102f50 2024-02-22T15:58:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.580+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac5000f040 con 0x7fac60102f50 2024-02-22T15:58:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.580+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6010daf0 con 0x7fac60102f50 2024-02-22T15:58:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.580+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac5000cbd0 con 0x7fac60102f50 2024-02-22T15:58:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.580+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6010e000 con 0x7fac60102f50 2024-02-22T15:58:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.581+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac600fc6e0 con 0x7fac60102f50 2024-02-22T15:58:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.581+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fac5000cd30 con 0x7fac60102f50 2024-02-22T15:58:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.582+0000 7fac477fe700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac4806cc30 0x7fac4806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:58:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.582+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fac5008c3a0 con 0x7fac60102f50 2024-02-22T15:58:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.582+0000 7fac5e7fc700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac4806cc30 0x7fac4806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:58:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.582+0000 7fac5e7fc700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac4806cc30 0x7fac4806f0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac6010eb40 tx=0x7fac54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:58:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.586+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fac50059f10 con 0x7fac60102f50 2024-02-22T15:58:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.878+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fac6010ef30 con 0x7fac60102f50 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.879+0000 7fac477fe700 1 -- 172.21.15.19:0/974739697 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+748 (secure 0 0 0) 0x7fac50007c90 con 0x7fac60102f50 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 7 pgs peering 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: osd.5 (root=default,host=smithi062) is down 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 7 pgs peering 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.2 is stuck peering for 7m, current state peering, last acting [1,0] 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.3 is stuck peering for 7m, current state peering, last acting [2,1] 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.16 is stuck peering for 7m, current state peering, last acting [3,2] 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 2.17 is stuck peering for 2m, current state peering, last acting [2,1] 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 3.5 is stuck peering for 6m, current state peering, last acting [2,3] 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 3.f is stuck peering for 6m, current state peering, last acting [3,0] 2024-02-22T15:58:42.858 INFO:teuthology.orchestra.run.smithi019.stdout: pg 3.16 is stuck peering for 6m, current state peering, last acting [3,1] 2024-02-22T15:58:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac4806cc30 msgr2=0x7fac4806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac4806cc30 0x7fac4806f0e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac6010eb40 tx=0x7fac54009e30 comp rx=0 tx=0).stop 2024-02-22T15:58:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 msgr2=0x7fac60113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:58:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 0x7fac60113950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac5000ec30 tx=0x7fac5000b040 comp rx=0 tx=0).stop 2024-02-22T15:58:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 shutdown_connections 2024-02-22T15:58:42.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac4806cc30 0x7fac4806f0e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac60102f50 0x7fac60113950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 --2- 172.21.15.19:0/974739697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac601038f0 0x7fac60113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:58:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 >> 172.21.15.19:0/974739697 conn(0x7fac600f8ae0 msgr2=0x7fac600fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:58:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 shutdown_connections 2024-02-22T15:58:42.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:58:42.881+0000 7fac653b4700 1 -- 172.21.15.19:0/974739697 wait complete. 2024-02-22T15:58:42.924 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:42 smithi062 systemd[1]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.5.service: Succeeded. 2024-02-22T15:58:42.924 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:42 smithi062 systemd[1]: Stopped Ceph osd.5 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:58:42.924 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:42 smithi062 systemd[1]: Starting Ceph osd.5 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T15:58:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:42 smithi062 ceph-mon[106861]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:42 smithi062 ceph-mon[106861]: pgmap v154: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:58:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:42 smithi062 ceph-mon[106861]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:42 smithi062 ceph-mon[106861]: Health check failed: Reduced data availability: 7 pgs peering (PG_AVAILABILITY) 2024-02-22T15:58:43.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:42 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1221254938' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:58:43.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:42 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/974739697' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:58:43.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:42 smithi019 ceph-mon[173579]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:43.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:42 smithi019 ceph-mon[173579]: pgmap v154: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:58:43.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:42 smithi019 ceph-mon[173579]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:43.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:42 smithi019 ceph-mon[173579]: Health check failed: Reduced data availability: 7 pgs peering (PG_AVAILABILITY) 2024-02-22T15:58:43.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:42 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1221254938' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:58:43.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:42 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/974739697' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:58:43.940 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-22T15:58:43.941 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-22T15:58:43.941 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-22T15:58:43.941 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-22T15:58:43.941 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-22T15:58:43.941 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-22T15:58:43.941 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:43 smithi062 bash[132139]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-22T15:58:44.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:43 smithi062 ceph-mon[106861]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:44.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:43 smithi019 ceph-mon[173579]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:58:45.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:44 smithi062 ceph-mon[106861]: pgmap v155: 65 pgs: 18 peering, 4 stale+active+clean, 3 active+undersized, 40 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:58:45.205 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:45 smithi062 ceph-osd[132461]: -- 172.21.15.62:0/1309873574 <== mon.1 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5592541b04e0 con 0x559254180c00 2024-02-22T15:58:45.205 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:45 smithi062 bash[132424]: bd8ad90a0c7ac399aeebc0ec4d40bc91ffd68e57852e6d96b28ed753ac587c41 2024-02-22T15:58:45.205 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:45 smithi062 systemd[1]: Started Ceph osd.5 for 8db8e6fa-d198-11ee-95bf-87774f69a715. 2024-02-22T15:58:45.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:44 smithi019 ceph-mon[173579]: pgmap v155: 65 pgs: 18 peering, 4 stale+active+clean, 3 active+undersized, 40 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:58:45.955 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:45 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T15:58:45.623+0000 7f33d1391640 -1 Falling back to public interface 2024-02-22T15:58:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:46 smithi062 ceph-mon[106861]: pgmap v156: 65 pgs: 3 active+undersized+degraded, 18 peering, 7 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 7/318 objects degraded (2.201%) 2024-02-22T15:58:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:46 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:46 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:46 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:46 smithi062 ceph-mon[106861]: Health check failed: Degraded data redundancy: 7/318 objects degraded (2.201%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:46.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:46 smithi019 ceph-mon[173579]: pgmap v156: 65 pgs: 3 active+undersized+degraded, 18 peering, 7 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 7/318 objects degraded (2.201%) 2024-02-22T15:58:46.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:46.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:46.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:46 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:46.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:46 smithi019 ceph-mon[173579]: Health check failed: Degraded data redundancy: 7/318 objects degraded (2.201%), 3 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:47.643 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:47 smithi062 ceph-mon[106861]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-02-22T15:58:47.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:47 smithi019 ceph-mon[173579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs peering) 2024-02-22T15:58:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: pgmap v157: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-02-22T15:58:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: pgmap v157: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-02-22T15:58:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:58:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:50.204 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:49 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T15:58:49.845+0000 7f33d1391640 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-22T15:58:50.564 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:50 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T15:58:50.260+0000 7f33d1391640 -1 osd.5 75 log_to_monitors true 2024-02-22T15:58:50.932 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:50 smithi062 ceph-mon[106861]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-02-22T15:58:50.932 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:50 smithi062 ceph-mon[106861]: from='osd.5 [v2:172.21.15.62:6816/823340437,v1:172.21.15.62:6817/823340437]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:58:50.932 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:50 smithi062 ceph-mon[106861]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:58:51.204 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 15:58:50 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T15:58:50.966+0000 7f33bffe5700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-22T15:58:51.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:50 smithi019 ceph-mon[173579]: pgmap v158: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/318 objects degraded (15.409%) 2024-02-22T15:58:51.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:50 smithi019 ceph-mon[173579]: from='osd.5 [v2:172.21.15.62:6816/823340437,v1:172.21.15.62:6817/823340437]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:58:51.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:50 smithi019 ceph-mon[173579]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-22T15:58:52.162 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:51 smithi062 ceph-mon[106861]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T15:58:52.162 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:51 smithi062 ceph-mon[106861]: osdmap e78: 6 total, 5 up, 6 in 2024-02-22T15:58:52.163 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:51 smithi062 ceph-mon[106861]: from='osd.5 [v2:172.21.15.62:6816/823340437,v1:172.21.15.62:6817/823340437]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:52.163 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:51 smithi062 ceph-mon[106861]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:52.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:51 smithi019 ceph-mon[173579]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-22T15:58:52.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:51 smithi019 ceph-mon[173579]: osdmap e78: 6 total, 5 up, 6 in 2024-02-22T15:58:52.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:51 smithi019 ceph-mon[173579]: from='osd.5 [v2:172.21.15.62:6816/823340437,v1:172.21.15.62:6817/823340437]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:52.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:51 smithi019 ceph-mon[173579]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-02-22T15:58:53.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 49/318 objects degraded (15.409%) 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: osd.5 [v2:172.21.15.62:6816/823340437,v1:172.21.15.62:6817/823340437] boot 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: osdmap e79: 6 total, 6 up, 6 in 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-22T15:58:53.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-02-22T15:58:53.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:52 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 49/318 objects degraded (15.409%), 14 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:53.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 49/318 objects degraded (15.409%) 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: osd.5 [v2:172.21.15.62:6816/823340437,v1:172.21.15.62:6817/823340437] boot 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: osdmap e79: 6 total, 6 up, 6 in 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:53.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-22T15:58:53.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-02-22T15:58:53.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:52 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 49/318 objects degraded (15.409%), 14 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:54.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all osd 2024-02-22T15:58:54.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: Upgrade: Setting require_osd_release to 18 reef 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: osdmap e80: 6 total, 6 up, 6 in 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: osdmap e81: 6 total, 6 up, 6 in 2024-02-22T15:58:54.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:53 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all osd 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: Upgrade: Setting require_osd_release to 18 reef 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: osdmap e80: 6 total, 6 up, 6 in 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-02-22T15:58:54.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: osdmap e81: 6 total, 6 up, 6 in 2024-02-22T15:58:54.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:53 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:54.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T15:58:54.153+0000 7f6435d06700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-22T15:58:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: pgmap v164: 65 pgs: 1 peering, 14 active+undersized+degraded, 13 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 49/318 objects degraded (15.409%) 2024-02-22T15:58:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-22T15:58:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-02-22T15:58:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2024-02-22T15:58:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:55.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: osdmap e82: 6 total, 6 up, 6 in 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.62:6826/2221505494,v1:172.21.15.62:6827/2221505494] up:boot 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6826/720124526,v1:172.21.15.19:6827/720124526] up:boot 2024-02-22T15:58:55.206 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-22T15:58:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-02-22T15:58:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:58:55.207 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:54 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:58:55.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: pgmap v164: 65 pgs: 1 peering, 14 active+undersized+degraded, 13 active+undersized, 37 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 49/318 objects degraded (15.409%) 2024-02-22T15:58:55.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-22T15:58:55.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-02-22T15:58:55.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: fsmap cephfs:2 {0=cephfs.smithi019.fesnwx=up:active,1=cephfs.smithi062.oyfagg=up:active} 2024-02-22T15:58:55.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: osdmap e82: 6 total, 6 up, 6 in 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.62:6826/2221505494,v1:172.21.15.62:6827/2221505494] up:boot 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6826/720124526,v1:172.21.15.19:6827/720124526] up:boot 2024-02-22T15:58:55.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-22T15:58:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-02-22T15:58:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:58:55.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:54 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:58:56.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-02-22T15:58:56.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.62:6824/4079594902,v1:172.21.15.62:6825/4079594902] up:boot 2024-02-22T15:58:56.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6828/3538613108,v1:172.21.15.19:6829/3538613108] up:boot 2024-02-22T15:58:56.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:58:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:58:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:58:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:58:56.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:55 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:56.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-02-22T15:58:56.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.62:6824/4079594902,v1:172.21.15.62:6825/4079594902] up:boot 2024-02-22T15:58:56.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6828/3538613108,v1:172.21.15.19:6829/3538613108] up:boot 2024-02-22T15:58:56.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:58:56.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:58:56.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:58:56.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:58:56.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.lkzblb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:58:56.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:55 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:58:57.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:56 smithi019 ceph-mon[173579]: pgmap v166: 65 pgs: 1 peering, 12 active+undersized+degraded, 12 active+undersized, 40 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 45/318 objects degraded (14.151%) 2024-02-22T15:58:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:56 smithi019 ceph-mon[173579]: Upgrade: Updating mds.cephfs.smithi019.lkzblb 2024-02-22T15:58:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:56 smithi019 ceph-mon[173579]: Deploying daemon mds.cephfs.smithi019.lkzblb on smithi019 2024-02-22T15:58:57.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:56 smithi062 ceph-mon[106861]: pgmap v166: 65 pgs: 1 peering, 12 active+undersized+degraded, 12 active+undersized, 40 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 45/318 objects degraded (14.151%) 2024-02-22T15:58:57.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:56 smithi062 ceph-mon[106861]: Upgrade: Updating mds.cephfs.smithi019.lkzblb 2024-02-22T15:58:57.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:56 smithi062 ceph-mon[106861]: Deploying daemon mds.cephfs.smithi019.lkzblb on smithi019 2024-02-22T15:58:58.061 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:57 smithi019 ceph-mon[173579]: Health check update: Degraded data redundancy: 45/318 objects degraded (14.151%), 12 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:57 smithi062 ceph-mon[106861]: Health check update: Degraded data redundancy: 45/318 objects degraded (14.151%), 12 pgs degraded (PG_DEGRADED) 2024-02-22T15:58:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:58 smithi019 ceph-mon[173579]: pgmap v167: 65 pgs: 1 peering, 64 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:58:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:58 smithi019 ceph-mon[173579]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/318 objects degraded (14.151%), 12 pgs degraded) 2024-02-22T15:58:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:58 smithi019 ceph-mon[173579]: osdmap e83: 6 total, 6 up, 6 in 2024-02-22T15:58:59.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:58 smithi062 ceph-mon[106861]: pgmap v167: 65 pgs: 1 peering, 64 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:58:59.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:58 smithi062 ceph-mon[106861]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/318 objects degraded (14.151%), 12 pgs degraded) 2024-02-22T15:58:59.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:58 smithi062 ceph-mon[106861]: osdmap e83: 6 total, 6 up, 6 in 2024-02-22T15:59:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:59 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:58:59 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:59:00.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:59 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:00.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:58:59 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:59:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:00 smithi019 ceph-mon[173579]: pgmap v169: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:01.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:00 smithi062 ceph-mon[106861]: pgmap v169: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:03.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:03 smithi019 ceph-mon[173579]: pgmap v170: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:03.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:03 smithi062 ceph-mon[106861]: pgmap v170: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: pgmap v171: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:04.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: pgmap v171: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:04.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:04.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:04.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:05.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:05 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:boot 2024-02-22T15:59:05.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:05 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:05.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:59:05.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:05 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:boot 2024-02-22T15:59:05.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:05 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:05.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T15:59:06.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:06 smithi019 ceph-mon[173579]: pgmap v172: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:06.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:06 smithi062 ceph-mon[106861]: pgmap v172: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:09.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:08 smithi062 ceph-mon[106861]: pgmap v173: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:09.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:09.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:08 smithi019 ceph-mon[173579]: pgmap v173: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:10.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:10 smithi019 ceph-mon[173579]: pgmap v174: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:10.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:10.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:10 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:10.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:10 smithi062 ceph-mon[106861]: pgmap v174: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:10.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:10.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:10 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:12.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:12 smithi062 ceph-mon[106861]: pgmap v175: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.989+0000 7f8410d06700 1 -- 172.21.15.19:0/3659037150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 msgr2=0x7f840c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.989+0000 7f8410d06700 1 --2- 172.21.15.19:0/3659037150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c102210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f83fc004940 tx=0x7f83fc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.990+0000 7f8410d06700 1 -- 172.21.15.19:0/3659037150 shutdown_connections 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.990+0000 7f8410d06700 1 --2- 172.21.15.19:0/3659037150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c1027e0 0x7f840c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.990+0000 7f8410d06700 1 --2- 172.21.15.19:0/3659037150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c102210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.990+0000 7f8410d06700 1 -- 172.21.15.19:0/3659037150 >> 172.21.15.19:0/3659037150 conn(0x7f840c071d90 msgr2=0x7f840c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.990+0000 7f8410d06700 1 -- 172.21.15.19:0/3659037150 shutdown_connections 2024-02-22T15:59:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.990+0000 7f8410d06700 1 -- 172.21.15.19:0/3659037150 wait complete. 2024-02-22T15:59:12.970 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:12 smithi019 ceph-mon[173579]: pgmap v175: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f8410d06700 1 Processor -- start 2024-02-22T15:59:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f8410d06700 1 -- start start 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f8410d06700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c1129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f8410d06700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c1027e0 0x7f840c112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f8410d06700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c113530 con 0x7f840c101e40 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f8410d06700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c10c860 con 0x7f840c1027e0 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f840a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c1129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f840a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c1129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58526/0 (socket says 172.21.15.19:58526) 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.991+0000 7f840a59c700 1 -- 172.21.15.19:0/2395977993 learned_addr learned my addr 172.21.15.19:0/2395977993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f8409d9b700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c1027e0 0x7f840c112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f840a59c700 1 -- 172.21.15.19:0/2395977993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c1027e0 msgr2=0x7f840c112ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f840a59c700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c1027e0 0x7f840c112ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f840a59c700 1 -- 172.21.15.19:0/2395977993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc005040 con 0x7f840c101e40 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f840a59c700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c1129a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00b3b0 tx=0x7f83fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc00e040 con 0x7f840c101e40 2024-02-22T15:59:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f83fc00bc50 con 0x7f840c101e40 2024-02-22T15:59:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c10cae0 con 0x7f840c101e40 2024-02-22T15:59:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc013420 con 0x7f840c101e40 2024-02-22T15:59:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.992+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c10d020 con 0x7f840c101e40 2024-02-22T15:59:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.993+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c0ff150 con 0x7f840c101e40 2024-02-22T15:59:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.994+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f83fc01c430 con 0x7f840c101e40 2024-02-22T15:59:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.994+0000 7f83faffd700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83f406cc20 0x7f83f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:12.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.994+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f83fc08d6d0 con 0x7f840c101e40 2024-02-22T15:59:12.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.994+0000 7f8409d9b700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83f406cc20 0x7f83f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:12.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.995+0000 7f8409d9b700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83f406cc20 0x7f83f406f0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f840c10e1d0 tx=0x7f840000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:12.998+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f83fc05b220 con 0x7f840c101e40 2024-02-22T15:59:13.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.218+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f840c04bed0 con 0x7f83f406cc20 2024-02-22T15:59:13.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.219+0000 7f83faffd700 1 -- 172.21.15.19:0/2395977993 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f840c04bed0 con 0x7f83f406cc20 2024-02-22T15:59:13.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83f406cc20 msgr2=0x7f83f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83f406cc20 0x7f83f406f0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f840c10e1d0 tx=0x7f840000b040 comp rx=0 tx=0).stop 2024-02-22T15:59:13.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 msgr2=0x7f840c1129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c1129a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00b3b0 tx=0x7f83fc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:13.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 shutdown_connections 2024-02-22T15:59:13.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83f406cc20 0x7f83f406f0d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c101e40 0x7f840c1129a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 --2- 172.21.15.19:0/2395977993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f840c1027e0 0x7f840c112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.221+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 >> 172.21.15.19:0/2395977993 conn(0x7f840c071d90 msgr2=0x7f840c0faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:13.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.222+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 shutdown_connections 2024-02-22T15:59:13.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.222+0000 7f8410d06700 1 -- 172.21.15.19:0/2395977993 wait complete. 2024-02-22T15:59:13.211 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:59:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 -- 172.21.15.19:0/751277810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 msgr2=0x7f9aec103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 --2- 172.21.15.19:0/751277810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec103320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9adc004940 tx=0x7f9adc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 -- 172.21.15.19:0/751277810 shutdown_connections 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 --2- 172.21.15.19:0/751277810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aec1038f0 0x7f9aec0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 --2- 172.21.15.19:0/751277810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec103320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 -- 172.21.15.19:0/751277810 >> 172.21.15.19:0/751277810 conn(0x7f9aec0f8ae0 msgr2=0x7f9aec0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 -- 172.21.15.19:0/751277810 shutdown_connections 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.326+0000 7f9af337e700 1 -- 172.21.15.19:0/751277810 wait complete. 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af337e700 1 Processor -- start 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af337e700 1 -- start start 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af337e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af337e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aec1038f0 0x7f9aec0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af337e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec070dd0 con 0x7f9aec102f50 2024-02-22T15:59:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af337e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec070f40 con 0x7f9aec1038f0 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af111a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af111a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58542/0 (socket says 172.21.15.19:58542) 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af111a700 1 -- 172.21.15.19:0/3994662688 learned_addr learned my addr 172.21.15.19:0/3994662688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af111a700 1 -- 172.21.15.19:0/3994662688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aec1038f0 msgr2=0x7f9aec0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af0919700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aec1038f0 0x7f9aec0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.327+0000 7f9af111a700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aec1038f0 0x7f9aec0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9af111a700 1 -- 172.21.15.19:0/3994662688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc005040 con 0x7f9aec102f50 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9af111a700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec070290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b3b0 tx=0x7f9adc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc00e040 con 0x7f9aec102f50 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9adc00bc50 con 0x7f9aec102f50 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec119bb0 con 0x7f9aec102f50 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc013420 con 0x7f9aec102f50 2024-02-22T15:59:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.328+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec11a010 con 0x7f9aec102f50 2024-02-22T15:59:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.329+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec100260 con 0x7f9aec102f50 2024-02-22T15:59:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.329+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9adc01c430 con 0x7f9aec102f50 2024-02-22T15:59:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.330+0000 7f9ae1ffb700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ad806cc20 0x7f9ad806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.330+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9adc08ce40 con 0x7f9aec102f50 2024-02-22T15:59:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.330+0000 7f9af0919700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ad806cc20 0x7f9ad806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.330+0000 7f9af0919700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ad806cc20 0x7f9ad806f0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9aec071c90 tx=0x7f9ae8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.335+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9adc056f00 con 0x7f9aec102f50 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.562+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9aec0719d0 con 0x7f9ad806cc20 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.564+0000 7f9ae1ffb700 1 -- 172.21.15.19:0/3994662688 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9aec0719d0 con 0x7f9ad806cc20 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ad806cc20 msgr2=0x7f9ad806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ad806cc20 0x7f9ad806f0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9aec071c90 tx=0x7f9ae8009040 comp rx=0 tx=0).stop 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 msgr2=0x7f9aec070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec070290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b3b0 tx=0x7f9adc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 shutdown_connections 2024-02-22T15:59:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ad806cc20 0x7f9ad806f0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aec102f50 0x7f9aec070290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 --2- 172.21.15.19:0/3994662688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9aec1038f0 0x7f9aec0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 >> 172.21.15.19:0/3994662688 conn(0x7f9aec0f8ae0 msgr2=0x7f9aec0fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 shutdown_connections 2024-02-22T15:59:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.566+0000 7f9af337e700 1 -- 172.21.15.19:0/3994662688 wait complete. 2024-02-22T15:59:13.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 -- 172.21.15.19:0/2718438315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fe8f0 msgr2=0x7f94c4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2718438315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fe8f0 0x7f94c4107050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f94b8004970 tx=0x7f94b8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:13.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 -- 172.21.15.19:0/2718438315 shutdown_connections 2024-02-22T15:59:13.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2718438315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fe8f0 0x7f94c4107050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2718438315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94c40fdfe0 0x7f94c40fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 -- 172.21.15.19:0/2718438315 >> 172.21.15.19:0/2718438315 conn(0x7f94c4071d90 msgr2=0x7f94c4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 -- 172.21.15.19:0/2718438315 shutdown_connections 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 -- 172.21.15.19:0/2718438315 wait complete. 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 Processor -- start 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.680+0000 7f94c88d9700 1 -- start start 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c88d9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 0x7f94c4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c88d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94c40fe8f0 0x7f94c4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c88d9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4114500 con 0x7f94c40fdfe0 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c88d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c410d830 con 0x7f94c40fe8f0 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 0x7f94c4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 0x7f94c4113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58566/0 (socket says 172.21.15.19:58566) 2024-02-22T15:59:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 -- 172.21.15.19:0/2876052472 learned_addr learned my addr 172.21.15.19:0/2876052472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c1d9b700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94c40fe8f0 0x7f94c4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 -- 172.21.15.19:0/2876052472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94c40fe8f0 msgr2=0x7f94c4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94c40fe8f0 0x7f94c4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 -- 172.21.15.19:0/2876052472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8005040 con 0x7f94c40fdfe0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94c259c700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 0x7f94c4113970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94b400eb40 tx=0x7f94b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.681+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b4007cc0 con 0x7f94c40fdfe0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.682+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c410dab0 con 0x7f94c40fdfe0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.682+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94b400f040 con 0x7f94c40fdfe0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.682+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c410dfc0 con 0x7f94c40fdfe0 2024-02-22T15:59:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.682+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b4009570 con 0x7f94c40fdfe0 2024-02-22T15:59:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.683+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a40052e0 con 0x7f94c40fdfe0 2024-02-22T15:59:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.683+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f94b40096d0 con 0x7f94c40fdfe0 2024-02-22T15:59:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.684+0000 7f94b2ffd700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f94ac06cc20 0x7f94ac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.684+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f94b408bc80 con 0x7f94c40fdfe0 2024-02-22T15:59:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.684+0000 7f94c1d9b700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f94ac06cc20 0x7f94ac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.684+0000 7f94c1d9b700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f94ac06cc20 0x7f94ac06f0d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94c410eb60 tx=0x7f94b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.688+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f94b4059770 con 0x7f94c40fdfe0 2024-02-22T15:59:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.913+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a4000bf0 con 0x7f94ac06cc20 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.923+0000 7f94b2ffd700 1 -- 172.21.15.19:0/2876052472 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f94a4000bf0 con 0x7f94ac06cc20 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (6m) 5s ago 17m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (3m) 5s ago 17m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (3m) 25s ago 16m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (6m) 5s ago 17m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (11m) 5s ago 11m 10.5M - 17.2.7-527-g6ea2a55e e2e4ad29904c 81486bb974f4 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (10s) 5s ago 11m 10.8M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (11m) 25s ago 11m 62.1M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:59:13.902 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (11m) 25s ago 11m 109M - 17.2.7-527-g6ea2a55e e2e4ad29904c db1a23ce6535 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (8m) 5s ago 19m 503M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (7m) 25s ago 16m 449M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (4m) 5s ago 19m 51.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (4m) 25s ago 16m 40.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (7m) 5s ago 17m 18.6M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (7m) 25s ago 16m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (3m) 5s ago 15m 92.0M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (2m) 5s ago 15m 95.8M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (2m) 5s ago 14m 83.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (94s) 25s ago 14m 98.5M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (61s) 25s ago 13m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (29s) 25s ago 13m 11.4M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T15:59:13.903 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (6m) 5s ago 17m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:59:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f94ac06cc20 msgr2=0x7f94ac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f94ac06cc20 0x7f94ac06f0d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94c410eb60 tx=0x7f94b8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 msgr2=0x7f94c4113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 0x7f94c4113970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94b400eb40 tx=0x7f94b400b040 comp rx=0 tx=0).stop 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 shutdown_connections 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f94ac06cc20 0x7f94ac06f0d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94c40fdfe0 0x7f94c4113970 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 --2- 172.21.15.19:0/2876052472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94c40fe8f0 0x7f94c4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 >> 172.21.15.19:0/2876052472 conn(0x7f94c4071d90 msgr2=0x7f94c4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 shutdown_connections 2024-02-22T15:59:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:13.925+0000 7f94c88d9700 1 -- 172.21.15.19:0/2876052472 wait complete. 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 -- 172.21.15.19:0/712191753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 msgr2=0x7f1704102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 --2- 172.21.15.19:0/712191753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704102210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1700004940 tx=0x7f1700009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 -- 172.21.15.19:0/712191753 shutdown_connections 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 --2- 172.21.15.19:0/712191753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17041027e0 0x7f17040fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 --2- 172.21.15.19:0/712191753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704102210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 -- 172.21.15.19:0/712191753 >> 172.21.15.19:0/712191753 conn(0x7f1704071d90 msgr2=0x7f1704072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 -- 172.21.15.19:0/712191753 shutdown_connections 2024-02-22T15:59:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.030+0000 7f170beff700 1 -- 172.21.15.19:0/712191753 wait complete. 2024-02-22T15:59:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f170beff700 1 Processor -- start 2024-02-22T15:59:14.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f170beff700 1 -- start start 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f170beff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f170beff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17041027e0 0x7f1704112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f170beff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1704113410 con 0x7f1704101e40 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f170beff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170410c740 con 0x7f17041027e0 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.031+0000 7f1709c9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f1709c9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704112880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58588/0 (socket says 172.21.15.19:58588) 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f1709c9b700 1 -- 172.21.15.19:0/2651294480 learned_addr learned my addr 172.21.15.19:0/2651294480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f170949a700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17041027e0 0x7f1704112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f1709c9b700 1 -- 172.21.15.19:0/2651294480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17041027e0 msgr2=0x7f1704112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f1709c9b700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17041027e0 0x7f1704112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f1709c9b700 1 -- 172.21.15.19:0/2651294480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700005040 con 0x7f1704101e40 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f1709c9b700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704112880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f170000b3b0 tx=0x7f1700009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f170000e040 con 0x7f1704101e40 2024-02-22T15:59:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170410c9c0 con 0x7f1704101e40 2024-02-22T15:59:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1700009af0 con 0x7f1704101e40 2024-02-22T15:59:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1700013770 con 0x7f1704101e40 2024-02-22T15:59:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.032+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170410cea0 con 0x7f1704101e40 2024-02-22T15:59:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.033+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17040738b0 con 0x7f1704101e40 2024-02-22T15:59:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.034+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f170001e420 con 0x7f1704101e40 2024-02-22T15:59:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.034+0000 7f16fa7fc700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16f006cc20 0x7f16f006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.034+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f170008ff60 con 0x7f1704101e40 2024-02-22T15:59:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.034+0000 7f170949a700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16f006cc20 0x7f16f006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.035+0000 7f170949a700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16f006cc20 0x7f16f006f0d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f170410e0b0 tx=0x7f16f4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:14.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.039+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f17000c3960 con 0x7f1704101e40 2024-02-22T15:59:14.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.333+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f170410de60 con 0x7f1704101e40 2024-02-22T15:59:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.334+0000 7f16fa7fc700 1 -- 172.21.15.19:0/2651294480 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+842 (secure 0 0 0) 0x7f170005da20 con 0x7f1704101e40 2024-02-22T15:59:14.313 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:59:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:59:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:59:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:59:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 3, 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 1 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 3, 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 11 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:59:14.315 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16f006cc20 msgr2=0x7f16f006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16f006cc20 0x7f16f006f0d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f170410e0b0 tx=0x7f16f4009ee0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 msgr2=0x7f1704112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704112880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f170000b3b0 tx=0x7f1700009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 shutdown_connections 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f16f006cc20 0x7f16f006f0d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1704101e40 0x7f1704112880 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 --2- 172.21.15.19:0/2651294480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17041027e0 0x7f1704112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 >> 172.21.15.19:0/2651294480 conn(0x7f1704071d90 msgr2=0x7f17040fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:14.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 shutdown_connections 2024-02-22T15:59:14.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.336+0000 7f170beff700 1 -- 172.21.15.19:0/2651294480 wait complete. 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.442+0000 7ff0eabb4700 1 -- 172.21.15.19:0/581418648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e4102f50 msgr2=0x7ff0e4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.442+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/581418648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e4102f50 0x7ff0e4103320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8004940 tx=0x7ff0d8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 -- 172.21.15.19:0/581418648 shutdown_connections 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/581418648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e41038f0 0x7ff0e40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/581418648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e4102f50 0x7ff0e4103320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 -- 172.21.15.19:0/581418648 >> 172.21.15.19:0/581418648 conn(0x7ff0e40f8ae0 msgr2=0x7ff0e40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 -- 172.21.15.19:0/581418648 shutdown_connections 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 -- 172.21.15.19:0/581418648 wait complete. 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 Processor -- start 2024-02-22T15:59:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.443+0000 7ff0eabb4700 1 -- start start 2024-02-22T15:59:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0eabb4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e4102f50 0x7ff0e4070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0eabb4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 0x7ff0e40707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0eabb4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4070dd0 con 0x7ff0e41038f0 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0eabb4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4070f40 con 0x7ff0e4102f50 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 0x7ff0e40707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 0x7ff0e40707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58610/0 (socket says 172.21.15.19:58610) 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 -- 172.21.15.19:0/2879839935 learned_addr learned my addr 172.21.15.19:0/2879839935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e8950700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e4102f50 0x7ff0e4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 -- 172.21.15.19:0/2879839935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e4102f50 msgr2=0x7ff0e4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e4102f50 0x7ff0e4070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 -- 172.21.15.19:0/2879839935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8005040 con 0x7ff0e41038f0 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e8950700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e4102f50 0x7ff0e4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e3fff700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 0x7ff0e40707d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0002a50 tx=0x7ff0d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d0007cc0 con 0x7ff0e41038f0 2024-02-22T15:59:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e4119be0 con 0x7ff0e41038f0 2024-02-22T15:59:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0d000f040 con 0x7ff0e41038f0 2024-02-22T15:59:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d00095d0 con 0x7ff0e41038f0 2024-02-22T15:59:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.444+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e411a0c0 con 0x7ff0e41038f0 2024-02-22T15:59:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.446+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4100260 con 0x7ff0e41038f0 2024-02-22T15:59:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.446+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff0d0009730 con 0x7ff0e41038f0 2024-02-22T15:59:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.446+0000 7ff0e17fa700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff0d406cc30 0x7ff0d406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.447+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff0d008bd50 con 0x7ff0e41038f0 2024-02-22T15:59:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.447+0000 7ff0e8950700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff0d406cc30 0x7ff0d406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.447+0000 7ff0e8950700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff0d406cc30 0x7ff0d406f0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8000c00 tx=0x7ff0d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.451+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff0d0055d00 con 0x7ff0e41038f0 2024-02-22T15:59:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.700+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff0e40719d0 con 0x7ff0e41038f0 2024-02-22T15:59:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.701+0000 7ff0e17fa700 1 -- 172.21.15.19:0/2879839935 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1837 (secure 0 0 0) 0x7ff0d0059320 con 0x7ff0e41038f0 2024-02-22T15:59:14.679 INFO:teuthology.orchestra.run.smithi019.stdout:e29 2024-02-22T15:59:14.679 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:59:14.680 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:59:14.681 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3538613108,v1:172.21.15.19:6829/3538613108] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/2221505494,v1:172.21.15.62:6827/2221505494] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:14.682 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44299} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/4079594902,v1:172.21.15.62:6825/4079594902] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff0d406cc30 msgr2=0x7ff0d406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff0d406cc30 0x7ff0d406f0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8000c00 tx=0x7ff0d8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 msgr2=0x7ff0e40707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 0x7ff0e40707d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0002a50 tx=0x7ff0d00046b0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 shutdown_connections 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff0d406cc30 0x7ff0d406f0e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0e4102f50 0x7ff0e4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 --2- 172.21.15.19:0/2879839935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41038f0 0x7ff0e40707d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 >> 172.21.15.19:0/2879839935 conn(0x7ff0e40f8ae0 msgr2=0x7ff0e40fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 shutdown_connections 2024-02-22T15:59:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.703+0000 7ff0eabb4700 1 -- 172.21.15.19:0/2879839935 wait complete. 2024-02-22T15:59:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-02-22T15:59:14.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: pgmap v176: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:14.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:14.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:14.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:14.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:14 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2651294480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: pgmap v176: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:14.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:14.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:14.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:14.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:14.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:14.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:14 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2651294480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:14.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 -- 172.21.15.19:0/1995799400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc1027e0 msgr2=0x7fadcc0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 --2- 172.21.15.19:0/1995799400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc1027e0 0x7fadcc0fbea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fadc0004940 tx=0x7fadc0009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 -- 172.21.15.19:0/1995799400 shutdown_connections 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 --2- 172.21.15.19:0/1995799400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc1027e0 0x7fadcc0fbea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 --2- 172.21.15.19:0/1995799400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadcc101e40 0x7fadcc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 -- 172.21.15.19:0/1995799400 >> 172.21.15.19:0/1995799400 conn(0x7fadcc071d90 msgr2=0x7fadcc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 -- 172.21.15.19:0/1995799400 shutdown_connections 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.809+0000 7fadd1d75700 1 -- 172.21.15.19:0/1995799400 wait complete. 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadd1d75700 1 Processor -- start 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadd1d75700 1 -- start start 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadd1d75700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 0x7fadcc113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadd1d75700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadcc1027e0 0x7fadcc113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadd1d75700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc1144f0 con 0x7fadcc101e40 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadd1d75700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc10d820 con 0x7fadcc1027e0 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadcb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 0x7fadcc113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadcb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 0x7fadcc113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58628/0 (socket says 172.21.15.19:58628) 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadcb7fe700 1 -- 172.21.15.19:0/2876713616 learned_addr learned my addr 172.21.15.19:0/2876713616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.810+0000 7fadcaffd700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadcc1027e0 0x7fadcc113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadcb7fe700 1 -- 172.21.15.19:0/2876713616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadcc1027e0 msgr2=0x7fadcc113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadcb7fe700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadcc1027e0 0x7fadcc113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadcb7fe700 1 -- 172.21.15.19:0/2876713616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0005040 con 0x7fadcc101e40 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadcb7fe700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 0x7fadcc113960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00eb70 tx=0x7fadbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadbc007cc0 con 0x7fadcc101e40 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fadbc00f040 con 0x7fadcc101e40 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc10daa0 con 0x7fadcc101e40 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadbc0095f0 con 0x7fadcc101e40 2024-02-22T15:59:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.811+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc10dfb0 con 0x7fadcc101e40 2024-02-22T15:59:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.812+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0738b0 con 0x7fadcc101e40 2024-02-22T15:59:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.813+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fadbc009750 con 0x7fadcc101e40 2024-02-22T15:59:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.813+0000 7fadb3fff700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fadb406cc20 0x7fadb406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.813+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fadbc08ba80 con 0x7fadcc101e40 2024-02-22T15:59:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.813+0000 7fadcaffd700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fadb406cc20 0x7fadb406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.813+0000 7fadcaffd700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fadb406cc20 0x7fadb406f0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadcc10eb50 tx=0x7fadc0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:14.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:14.818+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fadbc059540 con 0x7fadcc101e40 2024-02-22T15:59:15.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.039+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadcc10efb0 con 0x7fadb406cc20 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.040+0000 7fadb3fff700 1 -- 172.21.15.19:0/2876713616 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fadcc10efb0 con 0x7fadb406cc20 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "osd" 2024-02-22T15:59:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:59:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "13/21 daemons upgraded", 2024-02-22T15:59:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T15:59:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:59:15.020 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fadb406cc20 msgr2=0x7fadb406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fadb406cc20 0x7fadb406f0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadcc10eb50 tx=0x7fadc0009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 msgr2=0x7fadcc113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 0x7fadcc113960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00eb70 tx=0x7fadbc00b040 comp rx=0 tx=0).stop 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 shutdown_connections 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fadb406cc20 0x7fadb406f0d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadcc101e40 0x7fadcc113960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 --2- 172.21.15.19:0/2876713616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadcc1027e0 0x7fadcc113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 >> 172.21.15.19:0/2876713616 conn(0x7fadcc071d90 msgr2=0x7fadcc0657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 shutdown_connections 2024-02-22T15:59:15.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.042+0000 7fadd1d75700 1 -- 172.21.15.19:0/2876713616 wait complete. 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.146+0000 7f1312f68700 1 -- 172.21.15.19:0/3319951685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 msgr2=0x7f130c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.146+0000 7f1312f68700 1 --2- 172.21.15.19:0/3319951685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fbea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12f8004940 tx=0x7f12f8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 -- 172.21.15.19:0/3319951685 shutdown_connections 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 --2- 172.21.15.19:0/3319951685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fbea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 --2- 172.21.15.19:0/3319951685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130c101e40 0x7f130c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 -- 172.21.15.19:0/3319951685 >> 172.21.15.19:0/3319951685 conn(0x7f130c071d90 msgr2=0x7f130c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 -- 172.21.15.19:0/3319951685 shutdown_connections 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 -- 172.21.15.19:0/3319951685 wait complete. 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 Processor -- start 2024-02-22T15:59:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.147+0000 7f1312f68700 1 -- start start 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1312f68700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130c101e40 0x7f130c0fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1312f68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1312f68700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0fe600 con 0x7f130c1027e0 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1312f68700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0fe770 con 0x7f130c101e40 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58652/0 (socket says 172.21.15.19:58652) 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 -- 172.21.15.19:0/2063784479 learned_addr learned my addr 172.21.15.19:0/2063784479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1310d04700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130c101e40 0x7f130c0fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 -- 172.21.15.19:0/2063784479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130c101e40 msgr2=0x7f130c0fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130c101e40 0x7f130c0fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 -- 172.21.15.19:0/2063784479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8005040 con 0x7f130c1027e0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f130bfff700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fe000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b3b0 tx=0x7f12f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f800e040 con 0x7f130c1027e0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12f8013420 con 0x7f130c1027e0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c119bf0 con 0x7f130c1027e0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f801c630 con 0x7f130c1027e0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.148+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c11a050 con 0x7f130c1027e0 2024-02-22T15:59:15.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.150+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c0738b0 con 0x7f130c1027e0 2024-02-22T15:59:15.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.150+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f12f80139c0 con 0x7f130c1027e0 2024-02-22T15:59:15.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.150+0000 7f13097fa700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f12fc06cc20 0x7f12fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:15.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.151+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f12f808d360 con 0x7f130c1027e0 2024-02-22T15:59:15.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.151+0000 7f1310d04700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f12fc06cc20 0x7f12fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:15.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.151+0000 7f1310d04700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f12fc06cc20 0x7f12fc06f0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1300007fd0 tx=0x7f1300009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:15.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.155+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f12f8057390 con 0x7f130c1027e0 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.452+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f130c108de0 con 0x7f130c1027e0 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.452+0000 7f13097fa700 1 -- 172.21.15.19:0/2063784479 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f12f805a9b0 con 0x7f130c1027e0 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T15:59:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T15:59:15.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.454+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f12fc06cc20 msgr2=0x7f12fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:15.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.454+0000 7f1312f68700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f12fc06cc20 0x7f12fc06f0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1300007fd0 tx=0x7f1300009040 comp rx=0 tx=0).stop 2024-02-22T15:59:15.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.454+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 msgr2=0x7f130c0fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:15.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.454+0000 7f1312f68700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fe000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b3b0 tx=0x7f12f8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:15.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 shutdown_connections 2024-02-22T15:59:15.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f12fc06cc20 0x7f12fc06f0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f130c101e40 0x7f130c0fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 --2- 172.21.15.19:0/2063784479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f130c1027e0 0x7f130c0fe000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 >> 172.21.15.19:0/2063784479 conn(0x7f130c071d90 msgr2=0x7f130c0657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 shutdown_connections 2024-02-22T15:59:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:15.455+0000 7f1312f68700 1 -- 172.21.15.19:0/2063784479 wait complete. 2024-02-22T15:59:15.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:15 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2879839935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:59:15.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:15 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:15.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:15 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:59:15.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:15 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:15.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:15 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2879839935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:59:15.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:15 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:15.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:15 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.fesnwx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:59:15.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:15 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:16.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:16 smithi062 ceph-mon[106861]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:16.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:16 smithi062 ceph-mon[106861]: pgmap v177: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:16.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:16 smithi062 ceph-mon[106861]: Upgrade: Updating mds.cephfs.smithi019.fesnwx 2024-02-22T15:59:16.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:16 smithi062 ceph-mon[106861]: Deploying daemon mds.cephfs.smithi019.fesnwx on smithi019 2024-02-22T15:59:16.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:16 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2063784479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:59:16.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:16 smithi019 ceph-mon[173579]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:16.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:16 smithi019 ceph-mon[173579]: pgmap v177: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:16.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:16 smithi019 ceph-mon[173579]: Upgrade: Updating mds.cephfs.smithi019.fesnwx 2024-02-22T15:59:16.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:16 smithi019 ceph-mon[173579]: Deploying daemon mds.cephfs.smithi019.fesnwx on smithi019 2024-02-22T15:59:16.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:16 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2063784479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:59:18.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:18 smithi062 ceph-mon[106861]: pgmap v178: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:18.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:18 smithi062 ceph-mon[106861]: osdmap e84: 6 total, 6 up, 6 in 2024-02-22T15:59:18.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:18 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:18.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:18.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:18 smithi019 ceph-mon[173579]: pgmap v178: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:18.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:18 smithi019 ceph-mon[173579]: osdmap e84: 6 total, 6 up, 6 in 2024-02-22T15:59:18.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:18 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:18.743 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:20.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:20 smithi062 ceph-mon[106861]: pgmap v180: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:20.705 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:20 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:59:20.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:20 smithi019 ceph-mon[173579]: pgmap v180: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:20.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:20 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:59:22.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:22 smithi019 ceph-mon[173579]: pgmap v181: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:22.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:22 smithi062 ceph-mon[106861]: pgmap v181: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:24.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:24.455 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:24.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:24.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: pgmap v182: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:24.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:boot 2024-02-22T15:59:24.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:24.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:24 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:59:24.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:24.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:24.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:24.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: pgmap v182: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:24.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:boot 2024-02-22T15:59:24.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:24.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:24 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T15:59:26.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:26 smithi062 ceph-mon[106861]: pgmap v183: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:26.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:26 smithi019 ceph-mon[173579]: pgmap v183: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:28.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:28 smithi062 ceph-mon[106861]: pgmap v184: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:28.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:28 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:28 smithi019 ceph-mon[173579]: pgmap v184: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:28.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:28.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:28 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:30.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:30 smithi062 ceph-mon[106861]: pgmap v185: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:30.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:30 smithi019 ceph-mon[173579]: pgmap v185: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: pgmap v186: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.954 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.955 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:32 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: pgmap v186: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:32.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:32.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:32.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:32.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:59:32.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:32.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:32.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:32.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:32 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:34.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:34.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:34.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:59:34.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:34.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:34.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:34.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.oyfagg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:59:34.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:35.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:34 smithi062 ceph-mon[106861]: pgmap v187: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:35.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:34 smithi062 ceph-mon[106861]: Upgrade: Updating mds.cephfs.smithi062.oyfagg 2024-02-22T15:59:35.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:34 smithi062 ceph-mon[106861]: Deploying daemon mds.cephfs.smithi062.oyfagg on smithi062 2024-02-22T15:59:35.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:34 smithi019 ceph-mon[173579]: pgmap v187: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:35.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:34 smithi019 ceph-mon[173579]: Upgrade: Updating mds.cephfs.smithi062.oyfagg 2024-02-22T15:59:35.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:34 smithi019 ceph-mon[173579]: Deploying daemon mds.cephfs.smithi062.oyfagg on smithi062 2024-02-22T15:59:36.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:36 smithi062 ceph-mon[106861]: pgmap v188: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:36.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:36 smithi062 ceph-mon[106861]: osdmap e85: 6 total, 6 up, 6 in 2024-02-22T15:59:36.704 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:36 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:36.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:36 smithi019 ceph-mon[173579]: pgmap v188: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:36.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:36 smithi019 ceph-mon[173579]: osdmap e85: 6 total, 6 up, 6 in 2024-02-22T15:59:36.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:36 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:39.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:38 smithi062 ceph-mon[106861]: pgmap v190: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:39.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:38 smithi019 ceph-mon[173579]: pgmap v190: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:41.132 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: pgmap v191: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:41.132 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:41.132 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:41.132 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:41.133 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] up:boot 2024-02-22T15:59:41.133 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:41.133 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:40 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:59:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: pgmap v191: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] up:boot 2024-02-22T15:59:41.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:41.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:40 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T15:59:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:42 smithi062 ceph-mon[106861]: pgmap v192: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:42 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:43.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:42 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:43.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:42 smithi019 ceph-mon[173579]: pgmap v192: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:43.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:42 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:43.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:42 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:44.600 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:44 smithi062 ceph-mon[106861]: pgmap v193: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:44.600 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:44 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:44.600 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:44 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:44.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:44 smithi019 ceph-mon[173579]: pgmap v193: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:44.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:44.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:44 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 -- 172.21.15.19:0/4203258813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 msgr2=0x7f9cec102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 --2- 172.21.15.19:0/4203258813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec102210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc004940 tx=0x7f9cdc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 -- 172.21.15.19:0/4203258813 shutdown_connections 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 --2- 172.21.15.19:0/4203258813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 0x7f9cec0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 --2- 172.21.15.19:0/4203258813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec102210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 -- 172.21.15.19:0/4203258813 >> 172.21.15.19:0/4203258813 conn(0x7f9cec071d90 msgr2=0x7f9cec072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.573+0000 7f9cf242f700 1 -- 172.21.15.19:0/4203258813 shutdown_connections 2024-02-22T15:59:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.574+0000 7f9cf242f700 1 -- 172.21.15.19:0/4203258813 wait complete. 2024-02-22T15:59:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.574+0000 7f9cf242f700 1 Processor -- start 2024-02-22T15:59:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.574+0000 7f9cf242f700 1 -- start start 2024-02-22T15:59:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cf242f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cf242f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 0x7f9cec113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cf242f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec1144a0 con 0x7f9cec101e40 2024-02-22T15:59:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cf242f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec10d7d0 con 0x7f9cec1027e0 2024-02-22T15:59:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50366/0 (socket says 172.21.15.19:50366) 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 -- 172.21.15.19:0/712745722 learned_addr learned my addr 172.21.15.19:0/712745722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9ceb7fe700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 0x7f9cec113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 -- 172.21.15.19:0/712745722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 msgr2=0x7f9cec113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 0x7f9cec113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 -- 172.21.15.19:0/712745722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc005040 con 0x7f9cec101e40 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9ceb7fe700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 0x7f9cec113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9cebfff700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec113910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc008d80 tx=0x7f9cdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.575+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cdc00e030 con 0x7f9cec101e40 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.576+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec10da50 con 0x7f9cec101e40 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.576+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9cdc004290 con 0x7f9cec101e40 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.576+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cdc01cda0 con 0x7f9cec101e40 2024-02-22T15:59:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.576+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec10df90 con 0x7f9cec101e40 2024-02-22T15:59:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.577+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec0738b0 con 0x7f9cec101e40 2024-02-22T15:59:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.577+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9cdc01c600 con 0x7f9cec101e40 2024-02-22T15:59:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.577+0000 7f9ce8ff9700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9cd406cc30 0x7f9cd406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.578+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9cdc08d900 con 0x7f9cec101e40 2024-02-22T15:59:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.578+0000 7f9ceb7fe700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9cd406cc30 0x7f9cd406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.578+0000 7f9ceb7fe700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9cd406cc30 0x7f9cd406f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9cec10f140 tx=0x7f9ce000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.582+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9cdc05b3c0 con 0x7f9cec101e40 2024-02-22T15:59:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.802+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cec10ef90 con 0x7f9cd406cc30 2024-02-22T15:59:45.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.803+0000 7f9ce8ff9700 1 -- 172.21.15.19:0/712745722 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9cec10ef90 con 0x7f9cd406cc30 2024-02-22T15:59:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9cd406cc30 msgr2=0x7f9cd406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9cd406cc30 0x7f9cd406f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9cec10f140 tx=0x7f9ce000b040 comp rx=0 tx=0).stop 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 msgr2=0x7f9cec113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec113910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc008d80 tx=0x7f9cdc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 shutdown_connections 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9cd406cc30 0x7f9cd406f0e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cec101e40 0x7f9cec113910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 --2- 172.21.15.19:0/712745722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9cec1027e0 0x7f9cec113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 >> 172.21.15.19:0/712745722 conn(0x7f9cec071d90 msgr2=0x7f9cec065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 shutdown_connections 2024-02-22T15:59:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.805+0000 7f9cf242f700 1 -- 172.21.15.19:0/712745722 wait complete. 2024-02-22T15:59:45.795 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T15:59:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.910+0000 7f62b9364700 1 -- 172.21.15.19:0/3050307536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 msgr2=0x7f62b41033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.910+0000 7f62b9364700 1 --2- 172.21.15.19:0/3050307536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b41033b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62a8004940 tx=0x7f62a8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 -- 172.21.15.19:0/3050307536 shutdown_connections 2024-02-22T15:59:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 --2- 172.21.15.19:0/3050307536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62b40fb560 0x7f62b40fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 --2- 172.21.15.19:0/3050307536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b41033b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 -- 172.21.15.19:0/3050307536 >> 172.21.15.19:0/3050307536 conn(0x7f62b4071500 msgr2=0x7f62b4071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:45.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 -- 172.21.15.19:0/3050307536 shutdown_connections 2024-02-22T15:59:45.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 -- 172.21.15.19:0/3050307536 wait complete. 2024-02-22T15:59:45.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 Processor -- start 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.911+0000 7f62b9364700 1 -- start start 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b9364700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62b40fb560 0x7f62b40fc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b9364700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b9364700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40fd280 con 0x7f62b4102fe0 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b9364700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40fd3f0 con 0x7f62b40fb560 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62b40fb560 0x7f62b40fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50390/0 (socket says 172.21.15.19:50390) 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 -- 172.21.15.19:0/1318424393 learned_addr learned my addr 172.21.15.19:0/1318424393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 -- 172.21.15.19:0/1318424393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62b40fb560 msgr2=0x7f62b40fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62b40fb560 0x7f62b40fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 -- 172.21.15.19:0/1318424393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8005040 con 0x7f62b4102fe0 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b27fc700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b40fcc80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f62a400eb70 tx=0x7f62a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:45.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a4007cc0 con 0x7f62b4102fe0 2024-02-22T15:59:45.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62a400f040 con 0x7f62b4102fe0 2024-02-22T15:59:45.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b4119c10 con 0x7f62b4102fe0 2024-02-22T15:59:45.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a400c9f0 con 0x7f62b4102fe0 2024-02-22T15:59:45.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.912+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b411a070 con 0x7f62b4102fe0 2024-02-22T15:59:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.914+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b4100290 con 0x7f62b4102fe0 2024-02-22T15:59:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.914+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f62a400cb50 con 0x7f62b4102fe0 2024-02-22T15:59:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.914+0000 7f629b7fe700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f629c06cc30 0x7f629c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.914+0000 7f62b2ffd700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f629c06cc30 0x7f629c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.914+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f62a408bc40 con 0x7f62b4102fe0 2024-02-22T15:59:45.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.915+0000 7f62b2ffd700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f629c06cc30 0x7f629c06f0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009fd0 tx=0x7f62a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:45.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:45.919+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f62a4055bf0 con 0x7f62b4102fe0 2024-02-22T15:59:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.142+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b40fde80 con 0x7f629c06cc30 2024-02-22T15:59:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.143+0000 7f629b7fe700 1 -- 172.21.15.19:0/1318424393 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f62b40fde80 con 0x7f629c06cc30 2024-02-22T15:59:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f629c06cc30 msgr2=0x7f629c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f629c06cc30 0x7f629c06f0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009fd0 tx=0x7f62a8009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 msgr2=0x7f62b40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b40fcc80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f62a400eb70 tx=0x7f62a400b040 comp rx=0 tx=0).stop 2024-02-22T15:59:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 shutdown_connections 2024-02-22T15:59:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f629c06cc30 0x7f629c06f0e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62b40fb560 0x7f62b40fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 --2- 172.21.15.19:0/1318424393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b4102fe0 0x7f62b40fcc80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.145+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 >> 172.21.15.19:0/1318424393 conn(0x7f62b4071500 msgr2=0x7f62b40fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.146+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 shutdown_connections 2024-02-22T15:59:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.146+0000 7f62b9364700 1 -- 172.21.15.19:0/1318424393 wait complete. 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 -- 172.21.15.19:0/1592979207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fe8f0 msgr2=0x7f8a20107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 --2- 172.21.15.19:0/1592979207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fe8f0 0x7f8a20107050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a08004970 tx=0x7f8a08009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 -- 172.21.15.19:0/1592979207 shutdown_connections 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 --2- 172.21.15.19:0/1592979207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fe8f0 0x7f8a20107050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 --2- 172.21.15.19:0/1592979207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 -- 172.21.15.19:0/1592979207 >> 172.21.15.19:0/1592979207 conn(0x7f8a20071d90 msgr2=0x7f8a20072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 -- 172.21.15.19:0/1592979207 shutdown_connections 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 -- 172.21.15.19:0/1592979207 wait complete. 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.250+0000 7f8a265dc700 1 Processor -- start 2024-02-22T15:59:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a265dc700 1 -- start start 2024-02-22T15:59:46.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a265dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a265dc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a200fe8f0 0x7f8a200fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a265dc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200fc9e0 con 0x7f8a200fdfe0 2024-02-22T15:59:46.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a265dc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200fcb50 con 0x7f8a200fe8f0 2024-02-22T15:59:46.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50418/0 (socket says 172.21.15.19:50418) 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1ffff700 1 -- 172.21.15.19:0/3238359063 learned_addr learned my addr 172.21.15.19:0/3238359063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1f7fe700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a200fe8f0 0x7f8a200fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1ffff700 1 -- 172.21.15.19:0/3238359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a200fe8f0 msgr2=0x7f8a200fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1ffff700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a200fe8f0 0x7f8a200fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.251+0000 7f8a1ffff700 1 -- 172.21.15.19:0/3238359063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a08005040 con 0x7f8a200fdfe0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.252+0000 7f8a1ffff700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fbea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a10002a50 tx=0x7f8a100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.252+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a10007cc0 con 0x7f8a200fdfe0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.252+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a1000f040 con 0x7f8a200fdfe0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.252+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a20119bb0 con 0x7f8a200fdfe0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.252+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a1000d9c0 con 0x7f8a200fdfe0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.252+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2011a060 con 0x7f8a200fdfe0 2024-02-22T15:59:46.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.253+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a201046c0 con 0x7f8a200fdfe0 2024-02-22T15:59:46.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.253+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8a1000db20 con 0x7f8a200fdfe0 2024-02-22T15:59:46.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.254+0000 7f8a1cff9700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8a0c06cc30 0x7f8a0c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.254+0000 7f8a1f7fe700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8a0c06cc30 0x7f8a0c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.254+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8a1008bd80 con 0x7f8a200fdfe0 2024-02-22T15:59:46.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.254+0000 7f8a1f7fe700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8a0c06cc30 0x7f8a0c06f0e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009fd0 tx=0x7f8a08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.258+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8a10055d30 con 0x7f8a200fdfe0 2024-02-22T15:59:46.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.471+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a200fd5e0 con 0x7f8a0c06cc30 2024-02-22T15:59:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:46 smithi062 ceph-mon[106861]: pgmap v194: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:46.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:46 smithi062 ceph-mon[106861]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:46.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.481+0000 7f8a1cff9700 1 -- 172.21.15.19:0/3238359063 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f8a200fd5e0 con 0x7f8a0c06cc30 2024-02-22T15:59:46.460 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (6m) 18s ago 18m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (4m) 18s ago 18m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (4m) 3s ago 17m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (6m) 18s ago 17m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (23s) 18s ago 12m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (43s) 18s ago 12m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (12m) 3s ago 12m 10.6M - 17.2.7-527-g6ea2a55e e2e4ad29904c cfb365dd1a41 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (7s) 3s ago 12m 12.2M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (8m) 18s ago 19m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (8m) 3s ago 17m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (5m) 18s ago 19m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (4m) 3s ago 16m 44.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (7m) 18s ago 18m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T15:59:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (7m) 3s ago 17m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (3m) 18s ago 16m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (3m) 18s ago 15m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (2m) 18s ago 15m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (2m) 3s ago 14m 99.7M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (94s) 3s ago 14m 92.0M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (61s) 3s ago 13m 85.9M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T15:59:46.462 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (7m) 18s ago 17m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8a0c06cc30 msgr2=0x7f8a0c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8a0c06cc30 0x7f8a0c06f0e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009fd0 tx=0x7f8a08009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 msgr2=0x7f8a200fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fbea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a10002a50 tx=0x7f8a100046b0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 shutdown_connections 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8a0c06cc30 0x7f8a0c06f0e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a200fdfe0 0x7f8a200fbea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 --2- 172.21.15.19:0/3238359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a200fe8f0 0x7f8a200fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 >> 172.21.15.19:0/3238359063 conn(0x7f8a20071d90 msgr2=0x7f8a20101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 shutdown_connections 2024-02-22T15:59:46.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.483+0000 7f8a265dc700 1 -- 172.21.15.19:0/3238359063 wait complete. 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 -- 172.21.15.19:0/2142012629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 msgr2=0x7f35b0107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 --2- 172.21.15.19:0/2142012629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0107050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3598004970 tx=0x7f3598009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 -- 172.21.15.19:0/2142012629 shutdown_connections 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 --2- 172.21.15.19:0/2142012629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0107050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 --2- 172.21.15.19:0/2142012629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b00fdfe0 0x7f35b00fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 -- 172.21.15.19:0/2142012629 >> 172.21.15.19:0/2142012629 conn(0x7f35b0071d90 msgr2=0x7f35b0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 -- 172.21.15.19:0/2142012629 shutdown_connections 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.586+0000 7f35b5b74700 1 -- 172.21.15.19:0/2142012629 wait complete. 2024-02-22T15:59:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35b5b74700 1 Processor -- start 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35b5b74700 1 -- start start 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35b5b74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b00fdfe0 0x7f35b0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35b5b74700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35b5b74700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0114500 con 0x7f35b00fe8f0 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35b5b74700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b010d830 con 0x7f35b00fdfe0 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50440/0 (socket says 172.21.15.19:50440) 2024-02-22T15:59:46.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35aeffd700 1 -- 172.21.15.19:0/317276196 learned_addr learned my addr 172.21.15.19:0/317276196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.587+0000 7f35af7fe700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b00fdfe0 0x7f35b0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35aeffd700 1 -- 172.21.15.19:0/317276196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b00fdfe0 msgr2=0x7f35b0113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35aeffd700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b00fdfe0 0x7f35b0113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35aeffd700 1 -- 172.21.15.19:0/317276196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598005040 con 0x7f35b00fe8f0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35aeffd700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0113eb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3598009380 tx=0x7f3598009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f359800e040 con 0x7f35b00fe8f0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b010dab0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3598004290 con 0x7f35b00fe8f0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f359801c3f0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.588+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b010dff0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.589+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b01046c0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.590+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f359800bc10 con 0x7f35b00fe8f0 2024-02-22T15:59:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.590+0000 7f35a7fff700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f359c06cc20 0x7f359c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.590+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f359808d3a0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.590+0000 7f35af7fe700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f359c06cc20 0x7f359c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.591+0000 7f35af7fe700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f359c06cc20 0x7f359c06f0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f35a0007a40 tx=0x7f35a0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.594+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f359805ae60 con 0x7f35b00fe8f0 2024-02-22T15:59:46.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:46 smithi019 ceph-mon[173579]: pgmap v194: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:46.742 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:46 smithi019 ceph-mon[173579]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.887+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f35b0109ef0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.888+0000 7f35a7fff700 1 -- 172.21.15.19:0/317276196 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+842 (secure 0 0 0) 0x7f359805a9f0 con 0x7f35b00fe8f0 2024-02-22T15:59:46.867 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:59:46.867 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 1, 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 3 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T15:59:46.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 17.2.7-527-g6ea2a55e (6ea2a55e2bea6e09fb2c0628bd94434d58fe78f9) quincy (stable)": 1, 2024-02-22T15:59:46.869 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 13 2024-02-22T15:59:46.869 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T15:59:46.869 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f359c06cc20 msgr2=0x7f359c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f359c06cc20 0x7f359c06f0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f35a0007a40 tx=0x7f35a0009da0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 msgr2=0x7f35b0113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0113eb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3598009380 tx=0x7f3598009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 shutdown_connections 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f359c06cc20 0x7f359c06f0d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b00fdfe0 0x7f35b0113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 --2- 172.21.15.19:0/317276196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b00fe8f0 0x7f35b0113eb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 >> 172.21.15.19:0/317276196 conn(0x7f35b0071d90 msgr2=0x7f35b0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 shutdown_connections 2024-02-22T15:59:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.890+0000 7f35b5b74700 1 -- 172.21.15.19:0/317276196 wait complete. 2024-02-22T15:59:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 -- 172.21.15.19:0/3624444484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9658101e40 msgr2=0x7f9658102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 --2- 172.21.15.19:0/3624444484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9658101e40 0x7f9658102210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9654004940 tx=0x7f9654009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 -- 172.21.15.19:0/3624444484 shutdown_connections 2024-02-22T15:59:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 --2- 172.21.15.19:0/3624444484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96581027e0 0x7f96580fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 --2- 172.21.15.19:0/3624444484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9658101e40 0x7f9658102210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 -- 172.21.15.19:0/3624444484 >> 172.21.15.19:0/3624444484 conn(0x7f9658071d90 msgr2=0x7f9658072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 -- 172.21.15.19:0/3624444484 shutdown_connections 2024-02-22T15:59:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.994+0000 7f965f2e5700 1 -- 172.21.15.19:0/3624444484 wait complete. 2024-02-22T15:59:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965f2e5700 1 Processor -- start 2024-02-22T15:59:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965f2e5700 1 -- start start 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965f2e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9658101e40 0x7f965810ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965f2e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 0x7f965810ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965f2e5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965810f4d0 con 0x7f96581027e0 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965f2e5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965810f640 con 0x7f9658101e40 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965c880700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 0x7f965810ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965c880700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 0x7f965810ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50452/0 (socket says 172.21.15.19:50452) 2024-02-22T15:59:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965c880700 1 -- 172.21.15.19:0/141708966 learned_addr learned my addr 172.21.15.19:0/141708966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.995+0000 7f965d081700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9658101e40 0x7f965810ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965d081700 1 -- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 msgr2=0x7f965810ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965d081700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 0x7f965810ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965d081700 1 -- 172.21.15.19:0/141708966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9654005040 con 0x7f9658101e40 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965c880700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 0x7f965810ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965d081700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9658101e40 0x7f965810ea00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f965400b0d0 tx=0x7f9654009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9654021070 con 0x7f9658101e40 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9658117970 con 0x7f9658101e40 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96540093b0 con 0x7f9658101e40 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965400e070 con 0x7f9658101e40 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.996+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9658117dd0 con 0x7f9658101e40 2024-02-22T15:59:46.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.997+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96580ff150 con 0x7f9658101e40 2024-02-22T15:59:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.998+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f965401c600 con 0x7f9658101e40 2024-02-22T15:59:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.998+0000 7f9649ffb700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f964406cc20 0x7f964406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.998+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f965408d100 con 0x7f9658101e40 2024-02-22T15:59:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.998+0000 7f965c880700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f964406cc20 0x7f964406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:46.999+0000 7f965c880700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f964406cc20 0x7f964406f0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96580661a0 tx=0x7f964c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:46.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.003+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f96540570d0 con 0x7f9658101e40 2024-02-22T15:59:47.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.252+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9658065ee0 con 0x7f9658101e40 2024-02-22T15:59:47.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.253+0000 7f9649ffb700 1 -- 172.21.15.19:0/141708966 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f9654004490 con 0x7f9658101e40 2024-02-22T15:59:47.233 INFO:teuthology.orchestra.run.smithi019.stdout:e33 2024-02-22T15:59:47.233 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T15:59:47.233 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T15:59:47.234 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T15:59:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T15:59:47.236 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:47.237 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T15:59:47.238 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T15:59:47.238 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:47.238 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:47.238 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/2221505494,v1:172.21.15.62:6827/2221505494] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:47.238 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f964406cc20 msgr2=0x7f964406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f964406cc20 0x7f964406f0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96580661a0 tx=0x7f964c00b040 comp rx=0 tx=0).stop 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9658101e40 msgr2=0x7f965810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9658101e40 0x7f965810ea00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f965400b0d0 tx=0x7f9654009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 shutdown_connections 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f964406cc20 0x7f964406f0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9658101e40 0x7f965810ea00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 --2- 172.21.15.19:0/141708966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96581027e0 0x7f965810ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 >> 172.21.15.19:0/141708966 conn(0x7f9658071d90 msgr2=0x7f96580fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 shutdown_connections 2024-02-22T15:59:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.255+0000 7f965f2e5700 1 -- 172.21.15.19:0/141708966 wait complete. 2024-02-22T15:59:47.240 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 33 2024-02-22T15:59:47.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 -- 172.21.15.19:0/1562731244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d8101e40 msgr2=0x7f15d8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 --2- 172.21.15.19:0/1562731244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d8101e40 0x7f15d8102210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15cc004940 tx=0x7f15cc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:47.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 -- 172.21.15.19:0/1562731244 shutdown_connections 2024-02-22T15:59:47.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 --2- 172.21.15.19:0/1562731244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d81027e0 0x7f15d80fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 --2- 172.21.15.19:0/1562731244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d8101e40 0x7f15d8102210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 -- 172.21.15.19:0/1562731244 >> 172.21.15.19:0/1562731244 conn(0x7f15d8071d90 msgr2=0x7f15d8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 -- 172.21.15.19:0/1562731244 shutdown_connections 2024-02-22T15:59:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.360+0000 7f15def9b700 1 -- 172.21.15.19:0/1562731244 wait complete. 2024-02-22T15:59:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15def9b700 1 Processor -- start 2024-02-22T15:59:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15def9b700 1 -- start start 2024-02-22T15:59:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15def9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d8101e40 0x7f15d8112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15def9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 0x7f15d8112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15def9b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8113410 con 0x7f15d81027e0 2024-02-22T15:59:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15def9b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d810c740 con 0x7f15d8101e40 2024-02-22T15:59:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 0x7f15d8112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 0x7f15d8112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50460/0 (socket says 172.21.15.19:50460) 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15d7fff700 1 -- 172.21.15.19:0/3469591499 learned_addr learned my addr 172.21.15.19:0/3469591499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.361+0000 7f15dcd37700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d8101e40 0x7f15d8112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d7fff700 1 -- 172.21.15.19:0/3469591499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d8101e40 msgr2=0x7f15d8112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d7fff700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d8101e40 0x7f15d8112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d7fff700 1 -- 172.21.15.19:0/3469591499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc005040 con 0x7f15d81027e0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15dcd37700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d8101e40 0x7f15d8112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d7fff700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 0x7f15d8112dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15c4010d50 tx=0x7f15c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c400ed60 con 0x7f15d81027e0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d810c9c0 con 0x7f15d81027e0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15c40090d0 con 0x7f15d81027e0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c4012670 con 0x7f15d81027e0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.362+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d810cf00 con 0x7f15d81027e0 2024-02-22T15:59:47.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.363+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d80738b0 con 0x7f15d81027e0 2024-02-22T15:59:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.364+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f15c4018030 con 0x7f15d81027e0 2024-02-22T15:59:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.365+0000 7f15d57fa700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15c806cc20 0x7f15c806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:47.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.365+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f15c4016070 con 0x7f15d81027e0 2024-02-22T15:59:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.365+0000 7f15dcd37700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15c806cc20 0x7f15c806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:47.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.365+0000 7f15dcd37700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15c806cc20 0x7f15c806f0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15cc000c00 tx=0x7f15cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.369+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f15c405c2d0 con 0x7f15d81027e0 2024-02-22T15:59:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.587+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15d8112550 con 0x7f15c806cc20 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.588+0000 7f15d57fa700 1 -- 172.21.15.19:0/3469591499 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f15d8112550 con 0x7f15c806cc20 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T15:59:47.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr", 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "osd" 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "15/21 daemons upgraded", 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T15:59:47.569 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15c806cc20 msgr2=0x7f15c806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15c806cc20 0x7f15c806f0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15cc000c00 tx=0x7f15cc009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 msgr2=0x7f15d8112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 0x7f15d8112dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15c4010d50 tx=0x7f15c400b040 comp rx=0 tx=0).stop 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 shutdown_connections 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15c806cc20 0x7f15c806f0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d8101e40 0x7f15d8112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 --2- 172.21.15.19:0/3469591499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81027e0 0x7f15d8112dc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.590+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 >> 172.21.15.19:0/3469591499 conn(0x7f15d8071d90 msgr2=0x7f15d80fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.591+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 shutdown_connections 2024-02-22T15:59:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.591+0000 7f15def9b700 1 -- 172.21.15.19:0/3469591499 wait complete. 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 -- 172.21.15.19:0/383231778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 msgr2=0x7fd7b80fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/383231778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b80fe3b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4004970 tx=0x7fd7b4009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 -- 172.21.15.19:0/383231778 shutdown_connections 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/383231778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 0x7fd7b8107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/383231778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b80fe3b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 -- 172.21.15.19:0/383231778 >> 172.21.15.19:0/383231778 conn(0x7fd7b8071d90 msgr2=0x7fd7b8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 -- 172.21.15.19:0/383231778 shutdown_connections 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.698+0000 7fd7c03b5700 1 -- 172.21.15.19:0/383231778 wait complete. 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7c03b5700 1 Processor -- start 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7c03b5700 1 -- start start 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7c03b5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b8115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7c03b5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 0x7fd7b81160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7c03b5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81166f0 con 0x7fd7b80fdfe0 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7c03b5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b810fa20 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7be151700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b8115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7be151700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b8115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37702/0 (socket says 172.21.15.19:37702) 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7be151700 1 -- 172.21.15.19:0/2565968958 learned_addr learned my addr 172.21.15.19:0/2565968958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7bd950700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 0x7fd7b81160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7bd950700 1 -- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 msgr2=0x7fd7b8115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7bd950700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b8115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7bd950700 1 -- 172.21.15.19:0/2565968958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4005040 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.699+0000 7fd7be151700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b8115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.700+0000 7fd7bd950700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 0x7fd7b81160a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009fd0 tx=0x7fd7ac00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.700+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7ac004e70 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.700+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7ac00f040 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.700+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b810fca0 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.700+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7ac00c930 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.700+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b81101e0 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.701+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b8106ac0 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.701+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd7ac007cc0 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.702+0000 7fd7aaffd700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7a406cc30 0x7fd7a406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.702+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd7ac08bc00 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.702+0000 7fd7be151700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7a406cc30 0x7fd7a406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T15:59:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.702+0000 7fd7be151700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7a406cc30 0x7fd7a406f0e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4000c00 tx=0x7fd7b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T15:59:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.707+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd7ac059620 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:47.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:47.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:47.875 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/317276196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.876 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:47 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/141708966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.991+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd7b8109f10 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.992+0000 7fd7aaffd700 1 -- 172.21.15.19:0/2565968958 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd7ac0591b0 con 0x7fd7b80fe8f0 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T15:59:47.971 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7a406cc30 msgr2=0x7fd7a406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7a406cc30 0x7fd7a406f0e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4000c00 tx=0x7fd7b4009e30 comp rx=0 tx=0).stop 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 msgr2=0x7fd7b81160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 0x7fd7b81160a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009fd0 tx=0x7fd7ac00b040 comp rx=0 tx=0).stop 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 shutdown_connections 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd7a406cc30 0x7fd7a406f0e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7b80fdfe0 0x7fd7b8115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 --2- 172.21.15.19:0/2565968958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b80fe8f0 0x7fd7b81160a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T15:59:47.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 >> 172.21.15.19:0/2565968958 conn(0x7fd7b8071d90 msgr2=0x7fd7b8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T15:59:47.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 shutdown_connections 2024-02-22T15:59:47.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T15:59:47.994+0000 7fd7c03b5700 1 -- 172.21.15.19:0/2565968958 wait complete. 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.992 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/317276196' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T15:59:47.993 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:47 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/141708966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: pgmap v195: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2565968958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:59:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: pgmap v195: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T15:59:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2565968958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T15:59:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T15:59:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:49.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi062.eyhruo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-22T15:59:49.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T15:59:50.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:49 smithi062 ceph-mon[106861]: Upgrade: Updating mds.cephfs.smithi062.eyhruo 2024-02-22T15:59:50.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:49 smithi062 ceph-mon[106861]: Deploying daemon mds.cephfs.smithi062.eyhruo on smithi062 2024-02-22T15:59:50.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:49 smithi019 ceph-mon[173579]: Upgrade: Updating mds.cephfs.smithi062.eyhruo 2024-02-22T15:59:50.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:49 smithi019 ceph-mon[173579]: Deploying daemon mds.cephfs.smithi062.eyhruo on smithi062 2024-02-22T15:59:51.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:50 smithi062 ceph-mon[106861]: pgmap v196: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:51.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:50 smithi062 ceph-mon[106861]: osdmap e86: 6 total, 6 up, 6 in 2024-02-22T15:59:51.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:50 smithi019 ceph-mon[173579]: pgmap v196: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:51.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:50 smithi019 ceph-mon[173579]: osdmap e86: 6 total, 6 up, 6 in 2024-02-22T15:59:52.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:51 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:52.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:51 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-22T15:59:53.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:52 smithi062 ceph-mon[106861]: pgmap v198: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:53.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:52 smithi019 ceph-mon[173579]: pgmap v198: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:55.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:54 smithi062 ceph-mon[106861]: pgmap v199: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:55.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:54 smithi019 ceph-mon[173579]: pgmap v199: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:57.123 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: pgmap v200: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:57.124 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:57.124 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:57.124 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:57.124 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] up:boot 2024-02-22T15:59:57.124 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:57.125 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:59:57.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: pgmap v200: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:57.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:57.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:57.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T15:59:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] up:boot 2024-02-22T15:59:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T15:59:57.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T15:59:59.150 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:58 smithi062 ceph-mon[106861]: pgmap v201: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:59.151 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.151 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.151 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.151 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 15:59:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:58 smithi019 ceph-mon[173579]: pgmap v201: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T15:59:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T15:59:59.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 15:59:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:00:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T15:59:59.999+0000 7f643850b700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:00:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T15:59:59.999+0000 7f643850b700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:00:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T15:59:59.999+0000 7f643850b700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-02-22T16:00:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T15:59:59.999+0000 7f643850b700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:00:00.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T15:59:59.999+0000 7f643850b700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-02-22T16:00:01.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:00 smithi062 ceph-mon[106861]: pgmap v202: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:01.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:00 smithi062 ceph-mon[106861]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:00:01.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:00 smithi062 ceph-mon[106861]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:00:01.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:00 smithi062 ceph-mon[106861]: fs cephfs is degraded 2024-02-22T16:00:01.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:00 smithi062 ceph-mon[106861]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:00:01.205 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:00 smithi062 ceph-mon[106861]: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:00:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-mon[173579]: pgmap v202: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-mon[173579]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:00:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-mon[173579]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:00:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-mon[173579]: fs cephfs is degraded 2024-02-22T16:00:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-mon[173579]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:00:01.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:00 smithi019 ceph-mon[173579]: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:00:03.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:02 smithi062 ceph-mon[106861]: pgmap v203: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:03.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:02 smithi019 ceph-mon[173579]: pgmap v203: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:04.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:04.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:05.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:04 smithi062 ceph-mon[106861]: pgmap v204: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:05.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:04 smithi019 ceph-mon[173579]: pgmap v204: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:07.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:06 smithi062 ceph-mon[106861]: pgmap v205: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:06 smithi019 ceph-mon[173579]: pgmap v205: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:09.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:08 smithi062 ceph-mon[106861]: pgmap v206: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:08 smithi019 ceph-mon[173579]: pgmap v206: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:11.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:10 smithi062 ceph-mon[106861]: pgmap v207: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:11.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:10 smithi019 ceph-mon[173579]: pgmap v207: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:13.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:12 smithi062 ceph-mon[106861]: pgmap v208: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:13.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:12 smithi019 ceph-mon[173579]: pgmap v208: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:15.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:14 smithi062 ceph-mon[106861]: pgmap v209: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:15.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:14 smithi019 ceph-mon[173579]: pgmap v209: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:17.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:16 smithi062 ceph-mon[106861]: pgmap v210: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:17.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:16 smithi019 ceph-mon[173579]: pgmap v210: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 -- 172.21.15.19:0/2577825327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93681038f0 msgr2=0x7f93680fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 --2- 172.21.15.19:0/2577825327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93681038f0 0x7f93680fcf60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f935c004940 tx=0x7f935c009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 -- 172.21.15.19:0/2577825327 shutdown_connections 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 --2- 172.21.15.19:0/2577825327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93681038f0 0x7f93680fcf60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 --2- 172.21.15.19:0/2577825327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9368102f50 0x7f9368103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 -- 172.21.15.19:0/2577825327 >> 172.21.15.19:0/2577825327 conn(0x7f93680f8ae0 msgr2=0x7f93680faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 -- 172.21.15.19:0/2577825327 shutdown_connections 2024-02-22T16:00:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 -- 172.21.15.19:0/2577825327 wait complete. 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 Processor -- start 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.114+0000 7f936fc53700 1 -- start start 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936fc53700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 0x7f9368113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936fc53700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93681038f0 0x7f9368113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936fc53700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93681144c0 con 0x7f9368102f50 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936fc53700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936810d7f0 con 0x7f93681038f0 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 0x7f9368113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 0x7f9368113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58754/0 (socket says 172.21.15.19:58754) 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 -- 172.21.15.19:0/3180669457 learned_addr learned my addr 172.21.15.19:0/3180669457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d1ee700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93681038f0 0x7f9368113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 -- 172.21.15.19:0/3180669457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93681038f0 msgr2=0x7f9368113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93681038f0 0x7f9368113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 -- 172.21.15.19:0/3180669457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c005040 con 0x7f9368102f50 2024-02-22T16:00:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f936d9ef700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 0x7f9368113930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f936400eb40 tx=0x7f936400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.115+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9364007cc0 con 0x7f9368102f50 2024-02-22T16:00:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.116+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936810da70 con 0x7f9368102f50 2024-02-22T16:00:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.116+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f936400f040 con 0x7f9368102f50 2024-02-22T16:00:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.116+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f936400c990 con 0x7f9368102f50 2024-02-22T16:00:18.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.116+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936810df80 con 0x7f9368102f50 2024-02-22T16:00:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.117+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93680fc6e0 con 0x7f9368102f50 2024-02-22T16:00:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.117+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f936400caf0 con 0x7f9368102f50 2024-02-22T16:00:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.118+0000 7f935a7fc700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f935406cc20 0x7f935406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.118+0000 7f936d1ee700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f935406cc20 0x7f935406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.118+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f936408bba0 con 0x7f9368102f50 2024-02-22T16:00:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.118+0000 7f936d1ee700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f935406cc20 0x7f935406f0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f936810eb20 tx=0x7f935c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.122+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9364059710 con 0x7f9368102f50 2024-02-22T16:00:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.348+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f936810ef80 con 0x7f935406cc20 2024-02-22T16:00:18.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.349+0000 7f935a7fc700 1 -- 172.21.15.19:0/3180669457 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f936810ef80 con 0x7f935406cc20 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f935406cc20 msgr2=0x7f935406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f935406cc20 0x7f935406f0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f936810eb20 tx=0x7f935c009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 msgr2=0x7f9368113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 0x7f9368113930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f936400eb40 tx=0x7f936400b040 comp rx=0 tx=0).stop 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 shutdown_connections 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f935406cc20 0x7f935406f0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9368102f50 0x7f9368113930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 --2- 172.21.15.19:0/3180669457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93681038f0 0x7f9368113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 >> 172.21.15.19:0/3180669457 conn(0x7f93680f8ae0 msgr2=0x7f93680fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.351+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 shutdown_connections 2024-02-22T16:00:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.352+0000 7f936fc53700 1 -- 172.21.15.19:0/3180669457 wait complete. 2024-02-22T16:00:18.340 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 -- 172.21.15.19:0/2785770825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 msgr2=0x7f8c40103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 --2- 172.21.15.19:0/2785770825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c40103320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8c30004940 tx=0x7f8c30009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 -- 172.21.15.19:0/2785770825 shutdown_connections 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 --2- 172.21.15.19:0/2785770825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c401038f0 0x7f8c400fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 --2- 172.21.15.19:0/2785770825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c40103320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 -- 172.21.15.19:0/2785770825 >> 172.21.15.19:0/2785770825 conn(0x7f8c400f8ae0 msgr2=0x7f8c400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 -- 172.21.15.19:0/2785770825 shutdown_connections 2024-02-22T16:00:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.457+0000 7f8c47442700 1 -- 172.21.15.19:0/2785770825 wait complete. 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c47442700 1 Processor -- start 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c47442700 1 -- start start 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c47442700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c401139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c47442700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c401038f0 0x7f8c40113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c47442700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40114530 con 0x7f8c40102f50 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c47442700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4010d860 con 0x7f8c401038f0 2024-02-22T16:00:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c451de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c401139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c451de700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c401139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58776/0 (socket says 172.21.15.19:58776) 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.458+0000 7f8c451de700 1 -- 172.21.15.19:0/1017885742 learned_addr learned my addr 172.21.15.19:0/1017885742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c449dd700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c401038f0 0x7f8c40113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c451de700 1 -- 172.21.15.19:0/1017885742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c401038f0 msgr2=0x7f8c40113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c451de700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c401038f0 0x7f8c40113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c451de700 1 -- 172.21.15.19:0/1017885742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30005040 con 0x7f8c40102f50 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c451de700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c401139a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8c30008d80 tx=0x7f8c30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3000e030 con 0x7f8c40102f50 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4010dae0 con 0x7f8c40102f50 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c30004290 con 0x7f8c40102f50 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3001cda0 con 0x7f8c40102f50 2024-02-22T16:00:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.459+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4010e020 con 0x7f8c40102f50 2024-02-22T16:00:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.460+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c40100260 con 0x7f8c40102f50 2024-02-22T16:00:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.461+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8c3001c600 con 0x7f8c40102f50 2024-02-22T16:00:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.462+0000 7f8c35ffb700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8c2c06cc30 0x7f8c2c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.462+0000 7f8c449dd700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8c2c06cc30 0x7f8c2c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.462+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8c3008de80 con 0x7f8c40102f50 2024-02-22T16:00:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.462+0000 7f8c449dd700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8c2c06cc30 0x7f8c2c06f0e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8c4010f1d0 tx=0x7f8c3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.466+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8c3005b920 con 0x7f8c40102f50 2024-02-22T16:00:18.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.687+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c4010ef80 con 0x7f8c2c06cc30 2024-02-22T16:00:18.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.689+0000 7f8c35ffb700 1 -- 172.21.15.19:0/1017885742 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8c4010ef80 con 0x7f8c2c06cc30 2024-02-22T16:00:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8c2c06cc30 msgr2=0x7f8c2c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8c2c06cc30 0x7f8c2c06f0e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8c4010f1d0 tx=0x7f8c3c00b040 comp rx=0 tx=0).stop 2024-02-22T16:00:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 msgr2=0x7f8c401139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c401139a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8c30008d80 tx=0x7f8c30009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 shutdown_connections 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8c2c06cc30 0x7f8c2c06f0e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c40102f50 0x7f8c401139a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 --2- 172.21.15.19:0/1017885742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c401038f0 0x7f8c40113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 >> 172.21.15.19:0/1017885742 conn(0x7f8c400f8ae0 msgr2=0x7f8c400fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 shutdown_connections 2024-02-22T16:00:18.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.691+0000 7f8c47442700 1 -- 172.21.15.19:0/1017885742 wait complete. 2024-02-22T16:00:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 -- 172.21.15.19:0/1697378741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1038f0 msgr2=0x7f7efc0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 --2- 172.21.15.19:0/1697378741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1038f0 0x7f7efc0fcf60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8004940 tx=0x7f7ee8009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 -- 172.21.15.19:0/1697378741 shutdown_connections 2024-02-22T16:00:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 --2- 172.21.15.19:0/1697378741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1038f0 0x7f7efc0fcf60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 --2- 172.21.15.19:0/1697378741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7efc102f50 0x7f7efc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 -- 172.21.15.19:0/1697378741 >> 172.21.15.19:0/1697378741 conn(0x7f7efc0f8ae0 msgr2=0x7f7efc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 -- 172.21.15.19:0/1697378741 shutdown_connections 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 -- 172.21.15.19:0/1697378741 wait complete. 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.795+0000 7f7f0305b700 1 Processor -- start 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f0305b700 1 -- start start 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f0305b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 0x7f7efc113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f0305b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7efc1038f0 0x7f7efc113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f0305b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1144c0 con 0x7f7efc102f50 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f0305b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc10d7f0 con 0x7f7efc1038f0 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 0x7f7efc113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 0x7f7efc113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58790/0 (socket says 172.21.15.19:58790) 2024-02-22T16:00:18.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 -- 172.21.15.19:0/362165202 learned_addr learned my addr 172.21.15.19:0/362165202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:18.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7efbfff700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7efc1038f0 0x7f7efc113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 -- 172.21.15.19:0/362165202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7efc1038f0 msgr2=0x7f7efc113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7efc1038f0 0x7f7efc113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 -- 172.21.15.19:0/362165202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8005040 con 0x7f7efc102f50 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.796+0000 7f7f00df7700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 0x7f7efc113930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0002a80 tx=0x7f7ef00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.797+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef0007cc0 con 0x7f7efc102f50 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.797+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ef000f040 con 0x7f7efc102f50 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.797+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc10dad0 con 0x7f7efc102f50 2024-02-22T16:00:18.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.797+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ef000da60 con 0x7f7efc102f50 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.797+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc10dfe0 con 0x7f7efc102f50 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.798+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc0fc6e0 con 0x7f7efc102f50 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.799+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ef000dbc0 con 0x7f7efc102f50 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.799+0000 7f7ef97fa700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eec06cc20 0x7f7eec06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.799+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7ef008c2b0 con 0x7f7efc102f50 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.799+0000 7f7efbfff700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eec06cc20 0x7f7eec06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:18.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.800+0000 7f7efbfff700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eec06cc20 0x7f7eec06f0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7efc10eb20 tx=0x7f7ee8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:18.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:18.803+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7ef0059d70 con 0x7f7efc102f50 2024-02-22T16:00:18.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.015+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7efc10efb0 con 0x7f7eec06cc20 2024-02-22T16:00:19.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.025+0000 7f7ef97fa700 1 -- 172.21.15.19:0/362165202 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f7efc10efb0 con 0x7f7eec06cc20 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (7m) 51s ago 18m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (4m) 51s ago 18m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (4m) 21s ago 17m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (7m) 51s ago 18m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (56s) 51s ago 12m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (75s) 51s ago 12m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (23s) 21s ago 12m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (39s) 21s ago 12m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (9m) 51s ago 20m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:00:19.004 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (8m) 21s ago 17m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (5m) 51s ago 20m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (5m) 21s ago 17m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (8m) 51s ago 18m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (8m) 21s ago 17m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (4m) 51s ago 16m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (3m) 51s ago 16m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (3m) 51s ago 15m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (2m) 21s ago 15m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (2m) 21s ago 14m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (94s) 21s ago 14m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:00:19.005 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (7m) 51s ago 18m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.027+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eec06cc20 msgr2=0x7f7eec06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.027+0000 7f7f0305b700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eec06cc20 0x7f7eec06f0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7efc10eb20 tx=0x7f7ee8009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.027+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 msgr2=0x7f7efc113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.027+0000 7f7f0305b700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 0x7f7efc113930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0002a80 tx=0x7f7ef00046b0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 shutdown_connections 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eec06cc20 0x7f7eec06f0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc102f50 0x7f7efc113930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 --2- 172.21.15.19:0/362165202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7efc1038f0 0x7f7efc113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 >> 172.21.15.19:0/362165202 conn(0x7f7efc0f8ae0 msgr2=0x7f7efc0fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:19.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 shutdown_connections 2024-02-22T16:00:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.028+0000 7f7f0305b700 1 -- 172.21.15.19:0/362165202 wait complete. 2024-02-22T16:00:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 -- 172.21.15.19:0/1569738004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 msgr2=0x7efce00fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 --2- 172.21.15.19:0/1569738004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce00fcf50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efcdc004940 tx=0x7efcdc009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 -- 172.21.15.19:0/1569738004 shutdown_connections 2024-02-22T16:00:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 --2- 172.21.15.19:0/1569738004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce00fcf50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 --2- 172.21.15.19:0/1569738004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce0102f40 0x7efce0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 -- 172.21.15.19:0/1569738004 >> 172.21.15.19:0/1569738004 conn(0x7efce00f8af0 msgr2=0x7efce00faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 -- 172.21.15.19:0/1569738004 shutdown_connections 2024-02-22T16:00:19.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.130+0000 7efce7827700 1 -- 172.21.15.19:0/1569738004 wait complete. 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce7827700 1 Processor -- start 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce7827700 1 -- start start 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce7827700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce0102f40 0x7efce0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce7827700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce7827700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01145d0 con 0x7efce01038e0 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce7827700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce010d900 con 0x7efce0102f40 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce0113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58800/0 (socket says 172.21.15.19:58800) 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 -- 172.21.15.19:0/3689710071 learned_addr learned my addr 172.21.15.19:0/3689710071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce55c3700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce0102f40 0x7efce0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 -- 172.21.15.19:0/3689710071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce0102f40 msgr2=0x7efce0113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce0102f40 0x7efce0113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 -- 172.21.15.19:0/3689710071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc005040 con 0x7efce01038e0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.131+0000 7efce4dc2700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce0113f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009180 tx=0x7efcdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.132+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcdc00e040 con 0x7efce01038e0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.132+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcdc004290 con 0x7efce01038e0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.132+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce010db80 con 0x7efce01038e0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.132+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcdc01c3f0 con 0x7efce01038e0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.132+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce010e0c0 con 0x7efce01038e0 2024-02-22T16:00:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.133+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce0105410 con 0x7efce01038e0 2024-02-22T16:00:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.133+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efcdc013770 con 0x7efce01038e0 2024-02-22T16:00:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.134+0000 7efcd1ffb700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efccc06cc30 0x7efccc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.134+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efcdc08d360 con 0x7efce01038e0 2024-02-22T16:00:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.134+0000 7efce55c3700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efccc06cc30 0x7efccc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.134+0000 7efce55c3700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efccc06cc30 0x7efccc06f0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcd4004f40 tx=0x7efcd400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:19.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.138+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7efcdc05aed0 con 0x7efce01038e0 2024-02-22T16:00:19.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:18 smithi019 ceph-mon[173579]: pgmap v211: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:19.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:19.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.424+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efce0109ee0 con 0x7efce01038e0 2024-02-22T16:00:19.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.425+0000 7efcd1ffb700 1 -- 172.21.15.19:0/3689710071 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7efcdc05aa60 con 0x7efce01038e0 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:19.404 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:00:19.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:00:19.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:00:19.405 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:00:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efccc06cc30 msgr2=0x7efccc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efccc06cc30 0x7efccc06f0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efcd4004f40 tx=0x7efcd400b040 comp rx=0 tx=0).stop 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 msgr2=0x7efce0113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce0113f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009180 tx=0x7efcdc009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 shutdown_connections 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efccc06cc30 0x7efccc06f0e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce0102f40 0x7efce0113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 --2- 172.21.15.19:0/3689710071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efce01038e0 0x7efce0113f80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 >> 172.21.15.19:0/3689710071 conn(0x7efce00f8af0 msgr2=0x7efce00fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 shutdown_connections 2024-02-22T16:00:19.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.427+0000 7efce7827700 1 -- 172.21.15.19:0/3689710071 wait complete. 2024-02-22T16:00:19.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:18 smithi062 ceph-mon[106861]: pgmap v211: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:19.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 -- 172.21.15.19:0/2097654420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 msgr2=0x7f03c40fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 --2- 172.21.15.19:0/2097654420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c40fe3b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03b8004970 tx=0x7f03b8009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 -- 172.21.15.19:0/2097654420 shutdown_connections 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 --2- 172.21.15.19:0/2097654420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c40fe8f0 0x7f03c4107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 --2- 172.21.15.19:0/2097654420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c40fe3b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 -- 172.21.15.19:0/2097654420 >> 172.21.15.19:0/2097654420 conn(0x7f03c4071d90 msgr2=0x7f03c4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 -- 172.21.15.19:0/2097654420 shutdown_connections 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 -- 172.21.15.19:0/2097654420 wait complete. 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 Processor -- start 2024-02-22T16:00:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.531+0000 7f03cafc6700 1 -- start start 2024-02-22T16:00:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03cafc6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03cafc6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c40fe8f0 0x7f03c4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03cafc6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c4114500 con 0x7f03c40fdfe0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03cafc6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c410d830 con 0x7f03c40fe8f0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c4113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58808/0 (socket says 172.21.15.19:58808) 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 -- 172.21.15.19:0/1838372469 learned_addr learned my addr 172.21.15.19:0/1838372469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c3fff700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c40fe8f0 0x7f03c4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 -- 172.21.15.19:0/1838372469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c40fe8f0 msgr2=0x7f03c4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c40fe8f0 0x7f03c4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 -- 172.21.15.19:0/1838372469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8005040 con 0x7f03c40fdfe0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c8d62700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c4113970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03b8000c00 tx=0x7f03b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.532+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b800e040 con 0x7f03c40fdfe0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.533+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c410dab0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.533+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03b8009b10 con 0x7f03c40fdfe0 2024-02-22T16:00:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.533+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b8013e60 con 0x7f03c40fdfe0 2024-02-22T16:00:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.533+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c410dff0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.533+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c41046c0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.534+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f03b80136b0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.535+0000 7f03c17fa700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03b406cbb0 0x7f03b406f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.535+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f03b808d2f0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.535+0000 7f03c3fff700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03b406cbb0 0x7f03b406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.535+0000 7f03c3fff700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03b406cbb0 0x7f03b406f060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f03c410f1a0 tx=0x7f03b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.539+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f03b805adb0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.791+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f03c404bed0 con 0x7f03c40fdfe0 2024-02-22T16:00:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.791+0000 7f03c17fa700 1 -- 172.21.15.19:0/1838372469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f03b805a940 con 0x7f03c40fdfe0 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:00:19.770 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:00:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:19.772 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03b406cbb0 msgr2=0x7f03b406f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03b406cbb0 0x7f03b406f060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f03c410f1a0 tx=0x7f03b000b040 comp rx=0 tx=0).stop 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 msgr2=0x7f03c4113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c4113970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03b8000c00 tx=0x7f03b8009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 shutdown_connections 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03b406cbb0 0x7f03b406f060 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c40fdfe0 0x7f03c4113970 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 --2- 172.21.15.19:0/1838372469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c40fe8f0 0x7f03c4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 >> 172.21.15.19:0/1838372469 conn(0x7f03c4071d90 msgr2=0x7f03c4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 shutdown_connections 2024-02-22T16:00:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.793+0000 7f03cafc6700 1 -- 172.21.15.19:0/1838372469 wait complete. 2024-02-22T16:00:19.774 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:00:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 -- 172.21.15.19:0/1968825343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 msgr2=0x7f1ab00fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/1968825343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fbea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4004940 tx=0x7f1aa4009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 -- 172.21.15.19:0/1968825343 shutdown_connections 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/1968825343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fbea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/1968825343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab0101e40 0x7f1ab0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 -- 172.21.15.19:0/1968825343 >> 172.21.15.19:0/1968825343 conn(0x7f1ab0071d90 msgr2=0x7f1ab0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 -- 172.21.15.19:0/1968825343 shutdown_connections 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.906+0000 7f1ab87b9700 1 -- 172.21.15.19:0/1968825343 wait complete. 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab87b9700 1 Processor -- start 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab87b9700 1 -- start start 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab87b9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab0101e40 0x7f1ab00fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab87b9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab87b9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00fe670 con 0x7f1ab01027e0 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab87b9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00fe7e0 con 0x7f1ab0101e40 2024-02-22T16:00:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab5d54700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab5d54700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fe020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58816/0 (socket says 172.21.15.19:58816) 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab5d54700 1 -- 172.21.15.19:0/808411290 learned_addr learned my addr 172.21.15.19:0/808411290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab6555700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab0101e40 0x7f1ab00fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab5d54700 1 -- 172.21.15.19:0/808411290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab0101e40 msgr2=0x7f1ab00fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab5d54700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab0101e40 0x7f1ab00fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.907+0000 7f1ab5d54700 1 -- 172.21.15.19:0/808411290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4005040 con 0x7f1ab01027e0 2024-02-22T16:00:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.908+0000 7f1ab5d54700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fe020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000c00 tx=0x7f1aa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.908+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa400e040 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.908+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1aa4004290 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.908+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab0119be0 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.908+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa401c3f0 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.908+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab011a0c0 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.909+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab00738b0 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.909+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1aa400bb80 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.910+0000 7f1aa2ffd700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a9c06cc30 0x7f1a9c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.910+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1aa408d140 con 0x7f1ab01027e0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.910+0000 7f1ab6555700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a9c06cc30 0x7f1a9c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.910+0000 7f1ab6555700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a9c06cc30 0x7f1a9c06f0e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0041d0 tx=0x7f1aac0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:19.914+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1aa4057210 con 0x7f1ab01027e0 2024-02-22T16:00:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.136+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ab00ff2a0 con 0x7f1a9c06cc30 2024-02-22T16:00:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.137+0000 7f1aa2ffd700 1 -- 172.21.15.19:0/808411290 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1ab00ff2a0 con 0x7f1a9c06cc30 2024-02-22T16:00:20.115 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:00:20.115 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:00:20.115 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:00:20.115 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:00:20.116 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:00:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a9c06cc30 msgr2=0x7f1a9c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a9c06cc30 0x7f1a9c06f0e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0041d0 tx=0x7f1aac0043c0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 msgr2=0x7f1ab00fe020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fe020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000c00 tx=0x7f1aa4009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 shutdown_connections 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a9c06cc30 0x7f1a9c06f0e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ab0101e40 0x7f1ab00fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 --2- 172.21.15.19:0/808411290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab01027e0 0x7f1ab00fe020 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 >> 172.21.15.19:0/808411290 conn(0x7f1ab0071d90 msgr2=0x7f1ab0065800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 shutdown_connections 2024-02-22T16:00:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.139+0000 7f1ab87b9700 1 -- 172.21.15.19:0/808411290 wait complete. 2024-02-22T16:00:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.243+0000 7f5522cad700 1 -- 172.21.15.19:0/1457572471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c101e40 msgr2=0x7f551c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.243+0000 7f5522cad700 1 --2- 172.21.15.19:0/1457572471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c101e40 0x7f551c102210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5510004940 tx=0x7f5510009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 -- 172.21.15.19:0/1457572471 shutdown_connections 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 --2- 172.21.15.19:0/1457572471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f551c1027e0 0x7f551c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 --2- 172.21.15.19:0/1457572471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c101e40 0x7f551c102210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 -- 172.21.15.19:0/1457572471 >> 172.21.15.19:0/1457572471 conn(0x7f551c071d90 msgr2=0x7f551c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 -- 172.21.15.19:0/1457572471 shutdown_connections 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 -- 172.21.15.19:0/1457572471 wait complete. 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 Processor -- start 2024-02-22T16:00:20.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.244+0000 7f5522cad700 1 -- start start 2024-02-22T16:00:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5522cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f551c101e40 0x7f551c10ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5522cad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 0x7f551c10ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5522cad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c10f4f0 con 0x7f551c1027e0 2024-02-22T16:00:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5522cad700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c10f630 con 0x7f551c101e40 2024-02-22T16:00:20.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 0x7f551c10ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 0x7f551c10ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58838/0 (socket says 172.21.15.19:58838) 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 -- 172.21.15.19:0/2707777567 learned_addr learned my addr 172.21.15.19:0/2707777567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5520a49700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f551c101e40 0x7f551c10ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 -- 172.21.15.19:0/2707777567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f551c101e40 msgr2=0x7f551c10ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f551c101e40 0x7f551c10ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 -- 172.21.15.19:0/2707777567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510005040 con 0x7f551c1027e0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f551bfff700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 0x7f551c10ef60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f550800c800 tx=0x7f5508004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55080106f0 con 0x7f551c1027e0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c117990 con 0x7f551c1027e0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5508010d00 con 0x7f551c1027e0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55080113a0 con 0x7f551c1027e0 2024-02-22T16:00:20.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.245+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c117ea0 con 0x7f551c1027e0 2024-02-22T16:00:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.246+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c0ff150 con 0x7f551c1027e0 2024-02-22T16:00:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.247+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5508012400 con 0x7f551c1027e0 2024-02-22T16:00:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.247+0000 7f55197fa700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f550c06cc20 0x7f550c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.247+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f550808bfc0 con 0x7f551c1027e0 2024-02-22T16:00:20.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.247+0000 7f5520a49700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f550c06cc20 0x7f550c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:20.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.248+0000 7f5520a49700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f550c06cc20 0x7f550c06f0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5510004f90 tx=0x7f5510009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:20.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.252+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5508055ff0 con 0x7f551c1027e0 2024-02-22T16:00:20.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:19 smithi019 ceph-mon[173579]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:20.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:19 smithi019 ceph-mon[173579]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:20.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:19 smithi019 ceph-mon[173579]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:20.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:19 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3689710071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:00:20.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:19 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1838372469' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:00:20.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:19 smithi062 ceph-mon[106861]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:20.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:19 smithi062 ceph-mon[106861]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:20.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:19 smithi062 ceph-mon[106861]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:20.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:19 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3689710071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:00:20.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:19 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1838372469' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:00:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.534+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f551c065ec0 con 0x7f551c1027e0 2024-02-22T16:00:20.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.534+0000 7f55197fa700 1 -- 172.21.15.19:0/2707777567 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5508059610 con 0x7f551c1027e0 2024-02-22T16:00:20.513 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:00:20.513 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:00:20.513 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:00:20.513 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:00:20.513 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:00:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f550c06cc20 msgr2=0x7f550c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f550c06cc20 0x7f550c06f0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5510004f90 tx=0x7f5510009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 msgr2=0x7f551c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 0x7f551c10ef60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f550800c800 tx=0x7f5508004470 comp rx=0 tx=0).stop 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 shutdown_connections 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f550c06cc20 0x7f550c06f0d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f551c101e40 0x7f551c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 --2- 172.21.15.19:0/2707777567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f551c1027e0 0x7f551c10ef60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 >> 172.21.15.19:0/2707777567 conn(0x7f551c071d90 msgr2=0x7f551c0fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 shutdown_connections 2024-02-22T16:00:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:20.536+0000 7f5522cad700 1 -- 172.21.15.19:0/2707777567 wait complete. 2024-02-22T16:00:21.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:20 smithi019 ceph-mon[173579]: pgmap v212: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:21.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:20 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2707777567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:00:21.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:20 smithi062 ceph-mon[106861]: pgmap v212: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:21.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:20 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2707777567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:00:22.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:21 smithi019 ceph-mon[173579]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:22.459 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:21 smithi062 ceph-mon[106861]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:23.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:22 smithi019 ceph-mon[173579]: pgmap v213: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:23.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:22 smithi062 ceph-mon[106861]: pgmap v213: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:24 smithi019 ceph-mon[173579]: pgmap v214: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:25.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:24 smithi062 ceph-mon[106861]: pgmap v214: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:27.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:26 smithi019 ceph-mon[173579]: pgmap v215: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:27.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:26 smithi062 ceph-mon[106861]: pgmap v215: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:29.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:29 smithi019 ceph-mon[173579]: pgmap v216: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:29.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:29 smithi062 ceph-mon[106861]: pgmap v216: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:31 smithi019 ceph-mon[173579]: pgmap v217: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:31.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:31 smithi062 ceph-mon[106861]: pgmap v217: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:33.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:33 smithi019 ceph-mon[173579]: pgmap v218: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:33.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:33 smithi062 ceph-mon[106861]: pgmap v218: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:34.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:34.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:35.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:35 smithi062 ceph-mon[106861]: pgmap v219: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:35.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:35 smithi019 ceph-mon[173579]: pgmap v219: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:37.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:37 smithi062 ceph-mon[106861]: pgmap v220: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:37.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:37 smithi019 ceph-mon[173579]: pgmap v220: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:39.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:39 smithi062 ceph-mon[106861]: pgmap v221: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:39.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:39 smithi019 ceph-mon[173579]: pgmap v221: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:41.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:41 smithi019 ceph-mon[173579]: pgmap v222: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:41.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:41 smithi062 ceph-mon[106861]: pgmap v222: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:43.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:43 smithi062 ceph-mon[106861]: pgmap v223: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:43.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:43 smithi019 ceph-mon[173579]: pgmap v223: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:45.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:45 smithi062 ceph-mon[106861]: pgmap v224: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:45.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:45 smithi019 ceph-mon[173579]: pgmap v224: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:47.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:47 smithi062 ceph-mon[106861]: pgmap v225: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:47.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:47 smithi019 ceph-mon[173579]: pgmap v225: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:49.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:49 smithi062 ceph-mon[106861]: pgmap v226: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:49.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:49.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:49 smithi019 ceph-mon[173579]: pgmap v226: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:49.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:00:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 -- 172.21.15.19:0/146948835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 msgr2=0x7f20bc0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 --2- 172.21.15.19:0/146948835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc0fbea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20a4004940 tx=0x7f20a4009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 -- 172.21.15.19:0/146948835 shutdown_connections 2024-02-22T16:00:50.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 --2- 172.21.15.19:0/146948835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc0fbea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 --2- 172.21.15.19:0/146948835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20bc101e40 0x7f20bc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 -- 172.21.15.19:0/146948835 >> 172.21.15.19:0/146948835 conn(0x7f20bc071d90 msgr2=0x7f20bc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 -- 172.21.15.19:0/146948835 shutdown_connections 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 -- 172.21.15.19:0/146948835 wait complete. 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 Processor -- start 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.656+0000 7f20c10d5700 1 -- start start 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20c10d5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20bc101e40 0x7f20bc115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20c10d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc1160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20c10d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc1166f0 con 0x7f20bc1027e0 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20c10d5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc10fa20 con 0x7f20bc101e40 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc1160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc1160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54682/0 (socket says 172.21.15.19:54682) 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 -- 172.21.15.19:0/2181654150 learned_addr learned my addr 172.21.15.19:0/2181654150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20bad9d700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20bc101e40 0x7f20bc115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 -- 172.21.15.19:0/2181654150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20bc101e40 msgr2=0x7f20bc115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20bc101e40 0x7f20bc115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 -- 172.21.15.19:0/2181654150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a4005040 con 0x7f20bc1027e0 2024-02-22T16:00:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b3fff700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc1160a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20a4000c00 tx=0x7f20a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.657+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a4021070 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.658+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f20a40093b0 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.658+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc10fca0 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.658+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a400e070 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.658+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc110180 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.658+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20bc0738b0 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.659+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f20a401c670 con 0x7f20bc1027e0 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.659+0000 7f20b37fe700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20a806cc20 0x7f20a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:50.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.660+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f20a408d0f0 con 0x7f20bc1027e0 2024-02-22T16:00:50.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.660+0000 7f20bad9d700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20a806cc20 0x7f20a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:50.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.660+0000 7f20bad9d700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20a806cc20 0x7f20a806f0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009b80 tx=0x7f20ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:50.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.664+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f20a405ac50 con 0x7f20bc1027e0 2024-02-22T16:00:50.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.885+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20bc04bed0 con 0x7f20a806cc20 2024-02-22T16:00:50.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.886+0000 7f20b37fe700 1 -- 172.21.15.19:0/2181654150 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f20bc04bed0 con 0x7f20a806cc20 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20a806cc20 msgr2=0x7f20a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20a806cc20 0x7f20a806f0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009b80 tx=0x7f20ac00b040 comp rx=0 tx=0).stop 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 msgr2=0x7f20bc1160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc1160a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20a4000c00 tx=0x7f20a4009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 shutdown_connections 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20a806cc20 0x7f20a806f0d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20bc101e40 0x7f20bc115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 --2- 172.21.15.19:0/2181654150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20bc1027e0 0x7f20bc1160a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 >> 172.21.15.19:0/2181654150 conn(0x7f20bc071d90 msgr2=0x7f20bc065720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 shutdown_connections 2024-02-22T16:00:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.889+0000 7f20c10d5700 1 -- 172.21.15.19:0/2181654150 wait complete. 2024-02-22T16:00:50.877 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:00:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 -- 172.21.15.19:0/2202741838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 msgr2=0x7ffbbc0fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2202741838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc0fcf50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffbac004940 tx=0x7ffbac009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 -- 172.21.15.19:0/2202741838 shutdown_connections 2024-02-22T16:00:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2202741838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc0fcf50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2202741838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbbc102f40 0x7ffbbc103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 -- 172.21.15.19:0/2202741838 >> 172.21.15.19:0/2202741838 conn(0x7ffbbc0f8af0 msgr2=0x7ffbbc0faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:50.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 -- 172.21.15.19:0/2202741838 shutdown_connections 2024-02-22T16:00:50.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 -- 172.21.15.19:0/2202741838 wait complete. 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.994+0000 7ffbc188a700 1 Processor -- start 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbc188a700 1 -- start start 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbc188a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbbc102f40 0x7ffbbc113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbc188a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbc188a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc1145d0 con 0x7ffbbc1038e0 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbc188a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc10d900 con 0x7ffbbc102f40 2024-02-22T16:00:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54698/0 (socket says 172.21.15.19:54698) 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 -- 172.21.15.19:0/2220698527 learned_addr learned my addr 172.21.15.19:0/2220698527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbbaffd700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbbc102f40 0x7ffbbc113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 -- 172.21.15.19:0/2220698527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbbc102f40 msgr2=0x7ffbbc113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbbc102f40 0x7ffbbc113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 -- 172.21.15.19:0/2220698527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac005040 con 0x7ffbbc1038e0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.995+0000 7ffbba7fc700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc113f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009380 tx=0x7ffbac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.996+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbac00e040 con 0x7ffbbc1038e0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.996+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc10db80 con 0x7ffbbc1038e0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.996+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffbac00bc50 con 0x7ffbbc1038e0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.996+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbac01c3f0 con 0x7ffbbc1038e0 2024-02-22T16:00:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.996+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc10e0c0 con 0x7ffbbc1038e0 2024-02-22T16:00:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.997+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc105410 con 0x7ffbbc1038e0 2024-02-22T16:00:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.997+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ffbac013420 con 0x7ffbbc1038e0 2024-02-22T16:00:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.998+0000 7ffbb37fe700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffba806cc20 0x7ffba806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.998+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ffbac08d470 con 0x7ffbbc1038e0 2024-02-22T16:00:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.998+0000 7ffbbaffd700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffba806cc20 0x7ffba806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:50.998+0000 7ffbbaffd700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffba806cc20 0x7ffba806f0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffba4007fd0 tx=0x7ffba4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.002+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ffbac05af30 con 0x7ffbbc1038e0 2024-02-22T16:00:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.225+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbbc10f090 con 0x7ffba806cc20 2024-02-22T16:00:51.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.226+0000 7ffbb37fe700 1 -- 172.21.15.19:0/2220698527 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffbbc10f090 con 0x7ffba806cc20 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.228+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffba806cc20 msgr2=0x7ffba806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.228+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffba806cc20 0x7ffba806f0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffba4007fd0 tx=0x7ffba4009040 comp rx=0 tx=0).stop 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.228+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 msgr2=0x7ffbbc113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.228+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc113f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009380 tx=0x7ffbac009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.228+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 shutdown_connections 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.229+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffba806cc20 0x7ffba806f0d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.229+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbbc102f40 0x7ffbbc113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.229+0000 7ffbc188a700 1 --2- 172.21.15.19:0/2220698527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbbc1038e0 0x7ffbbc113f80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.229+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 >> 172.21.15.19:0/2220698527 conn(0x7ffbbc0f8af0 msgr2=0x7ffbbc0fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.229+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 shutdown_connections 2024-02-22T16:00:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.229+0000 7ffbc188a700 1 -- 172.21.15.19:0/2220698527 wait complete. 2024-02-22T16:00:51.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:51 smithi019 ceph-mon[173579]: pgmap v227: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.332+0000 7ffa0fa78700 1 -- 172.21.15.19:0/4161758352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa08101e40 msgr2=0x7ffa08102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.332+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/4161758352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa08101e40 0x7ffa08102210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa04004940 tx=0x7ffa04009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.332+0000 7ffa0fa78700 1 -- 172.21.15.19:0/4161758352 shutdown_connections 2024-02-22T16:00:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.332+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/4161758352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa081027e0 0x7ffa080fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/4161758352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa08101e40 0x7ffa08102210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 -- 172.21.15.19:0/4161758352 >> 172.21.15.19:0/4161758352 conn(0x7ffa080f79d0 msgr2=0x7ffa080f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 -- 172.21.15.19:0/4161758352 shutdown_connections 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 -- 172.21.15.19:0/4161758352 wait complete. 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 Processor -- start 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 -- start start 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa08101e40 0x7ffa08102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 0x7ffa081031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081037e0 con 0x7ffa081027e0 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.333+0000 7ffa0fa78700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08103950 con 0x7ffa08101e40 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 0x7ffa081031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 0x7ffa081031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54716/0 (socket says 172.21.15.19:54716) 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 -- 172.21.15.19:0/320344358 learned_addr learned my addr 172.21.15.19:0/320344358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d814700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa08101e40 0x7ffa08102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 -- 172.21.15.19:0/320344358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa08101e40 msgr2=0x7ffa08102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa08101e40 0x7ffa08102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 -- 172.21.15.19:0/320344358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04005040 con 0x7ffa081027e0 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0d013700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 0x7ffa081031e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc002a50 tx=0x7ff9fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9fc007cc0 con 0x7ffa081027e0 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9fc00f040 con 0x7ffa081027e0 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa08119c10 con 0x7ffa081027e0 2024-02-22T16:00:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9fc00d920 con 0x7ffa081027e0 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.334+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0811a0f0 con 0x7ffa081027e0 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.335+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa080fb5d0 con 0x7ffa081027e0 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.336+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff9fc009680 con 0x7ffa081027e0 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.336+0000 7ff9fa7fc700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff9f406cc20 0x7ff9f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.336+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff9fc08be70 con 0x7ffa081027e0 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.336+0000 7ffa0d814700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff9f406cc20 0x7ff9f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.337+0000 7ffa0d814700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff9f406cc20 0x7ff9f406f0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffa04004f90 tx=0x7ffa04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.341+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff9fc055d80 con 0x7ffa081027e0 2024-02-22T16:00:51.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:51 smithi062 ceph-mon[106861]: pgmap v227: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.553+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa08066740 con 0x7ff9f406cc20 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.563+0000 7ff9fa7fc700 1 -- 172.21.15.19:0/320344358 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ffa08066740 con 0x7ff9f406cc20 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (8m) 83s ago 19m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (5m) 83s ago 19m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (5m) 53s ago 18m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (7m) 83s ago 18m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:00:51.541 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (89s) 83s ago 13m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (108s) 83s ago 13m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (56s) 53s ago 13m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (72s) 53s ago 13m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (9m) 83s ago 20m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (9m) 53s ago 18m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (6m) 83s ago 20m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (6m) 53s ago 18m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (8m) 83s ago 19m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (8m) 53s ago 18m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (4m) 83s ago 17m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (4m) 83s ago 16m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (3m) 83s ago 16m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (3m) 53s ago 15m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (2m) 53s ago 15m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:00:51.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (2m) 53s ago 14m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (8m) 83s ago 18m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff9f406cc20 msgr2=0x7ff9f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff9f406cc20 0x7ff9f406f0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffa04004f90 tx=0x7ffa04009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 msgr2=0x7ffa081031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 0x7ffa081031e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc002a50 tx=0x7ff9fc004600 comp rx=0 tx=0).stop 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 shutdown_connections 2024-02-22T16:00:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff9f406cc20 0x7ff9f406f0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa08101e40 0x7ffa08102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 --2- 172.21.15.19:0/320344358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa081027e0 0x7ffa081031e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 >> 172.21.15.19:0/320344358 conn(0x7ffa080f79d0 msgr2=0x7ffa08066030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 shutdown_connections 2024-02-22T16:00:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.565+0000 7ffa0fa78700 1 -- 172.21.15.19:0/320344358 wait complete. 2024-02-22T16:00:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/3238742404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 msgr2=0x7f6a28102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/3238742404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28102210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a18004940 tx=0x7f6a18009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/3238742404 shutdown_connections 2024-02-22T16:00:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/3238742404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a281027e0 0x7f6a280fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/3238742404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28102210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/3238742404 >> 172.21.15.19:0/3238742404 conn(0x7f6a28071d90 msgr2=0x7f6a28072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/3238742404 shutdown_connections 2024-02-22T16:00:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/3238742404 wait complete. 2024-02-22T16:00:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 Processor -- start 2024-02-22T16:00:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.668+0000 7f6a2ff2d700 1 -- start start 2024-02-22T16:00:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2ff2d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2ff2d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a281027e0 0x7f6a28113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2ff2d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28114510 con 0x7f6a28101e40 2024-02-22T16:00:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2ff2d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2810d840 con 0x7f6a281027e0 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54734/0 (socket says 172.21.15.19:54734) 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 -- 172.21.15.19:0/2892782109 learned_addr learned my addr 172.21.15.19:0/2892782109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2d4c8700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a281027e0 0x7f6a28113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 -- 172.21.15.19:0/2892782109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a281027e0 msgr2=0x7f6a28113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a281027e0 0x7f6a28113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 -- 172.21.15.19:0/2892782109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a18005040 con 0x7f6a28101e40 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a2dcc9700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28113980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b3b0 tx=0x7f6a18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a1800e040 con 0x7f6a28101e40 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a1801c430 con 0x7f6a28101e40 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.669+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a18013640 con 0x7f6a28101e40 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.670+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2810dac0 con 0x7f6a28101e40 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.670+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2810e000 con 0x7f6a28101e40 2024-02-22T16:00:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.670+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a280738b0 con 0x7f6a28101e40 2024-02-22T16:00:51.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.671+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6a1801c5a0 con 0x7f6a28101e40 2024-02-22T16:00:51.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.672+0000 7f6a1e7fc700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6a1406cc20 0x7f6a1406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:51.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.672+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6a1808d320 con 0x7f6a28101e40 2024-02-22T16:00:51.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.672+0000 7f6a2d4c8700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6a1406cc20 0x7f6a1406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:51.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.672+0000 7f6a2d4c8700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6a1406cc20 0x7f6a1406f0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a2810f1b0 tx=0x7f6a2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:51.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.676+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6a1805af30 con 0x7f6a28101e40 2024-02-22T16:00:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.963+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a28108e00 con 0x7f6a28101e40 2024-02-22T16:00:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.964+0000 7f6a1e7fc700 1 -- 172.21.15.19:0/2892782109 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f6a1805aac0 con 0x7f6a28101e40 2024-02-22T16:00:51.942 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:00:51.942 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:00:51.942 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:00:51.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:00:51.943 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:00:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6a1406cc20 msgr2=0x7f6a1406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6a1406cc20 0x7f6a1406f0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a2810f1b0 tx=0x7f6a2400b040 comp rx=0 tx=0).stop 2024-02-22T16:00:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 msgr2=0x7f6a28113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28113980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800b3b0 tx=0x7f6a18009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 shutdown_connections 2024-02-22T16:00:51.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6a1406cc20 0x7f6a1406f0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a28101e40 0x7f6a28113980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 --2- 172.21.15.19:0/2892782109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a281027e0 0x7f6a28113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:51.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 >> 172.21.15.19:0/2892782109 conn(0x7f6a28071d90 msgr2=0x7f6a280657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:51.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 shutdown_connections 2024-02-22T16:00:51.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:51.966+0000 7f6a2ff2d700 1 -- 172.21.15.19:0/2892782109 wait complete. 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 -- 172.21.15.19:0/737669991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474102800 msgr2=0x7fd4740fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 --2- 172.21.15.19:0/737669991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474102800 0x7fd4740fbe70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd468004940 tx=0x7fd468009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 -- 172.21.15.19:0/737669991 shutdown_connections 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 --2- 172.21.15.19:0/737669991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474102800 0x7fd4740fbe70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 --2- 172.21.15.19:0/737669991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd474101e60 0x7fd474102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 -- 172.21.15.19:0/737669991 >> 172.21.15.19:0/737669991 conn(0x7fd4740f7a10 msgr2=0x7fd4740f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:52.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 -- 172.21.15.19:0/737669991 shutdown_connections 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.070+0000 7fd479693700 1 -- 172.21.15.19:0/737669991 wait complete. 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd479693700 1 Processor -- start 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd479693700 1 -- start start 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd479693700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 0x7fd4740fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd479693700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd474102800 0x7fd4740fe020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd479693700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740fe620 con 0x7fd474101e60 2024-02-22T16:00:52.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd479693700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740fe790 con 0x7fd474102800 2024-02-22T16:00:52.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd472ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 0x7fd4740fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd472ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 0x7fd4740fdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54752/0 (socket says 172.21.15.19:54752) 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.071+0000 7fd472ffd700 1 -- 172.21.15.19:0/1480755110 learned_addr learned my addr 172.21.15.19:0/1480755110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd4727fc700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd474102800 0x7fd4740fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd472ffd700 1 -- 172.21.15.19:0/1480755110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd474102800 msgr2=0x7fd4740fe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd472ffd700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd474102800 0x7fd4740fe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd472ffd700 1 -- 172.21.15.19:0/1480755110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468005040 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd472ffd700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 0x7fd4740fdae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd464002a80 tx=0x7fd4640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd464007cc0 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd474119ba0 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd46400f040 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46400da60 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.072+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47411a050 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.073+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4740fb610 con 0x7fd474101e60 2024-02-22T16:00:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.074+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd46400dbc0 con 0x7fd474101e60 2024-02-22T16:00:52.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.074+0000 7fd45b7fe700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd45c06cc20 0x7fd45c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.074+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd46408bd00 con 0x7fd474101e60 2024-02-22T16:00:52.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.074+0000 7fd4727fc700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd45c06cc20 0x7fd45c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.075+0000 7fd4727fc700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd45c06cc20 0x7fd45c06f0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ffbf0 tx=0x7fd468009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:52.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.079+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd46408f030 con 0x7fd474101e60 2024-02-22T16:00:52.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.321+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd4740ff320 con 0x7fd474101e60 2024-02-22T16:00:52.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.321+0000 7fd45b7fe700 1 -- 172.21.15.19:0/1480755110 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7fd464055cb0 con 0x7fd474101e60 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:00:52.300 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:00:52.301 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:52.302 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:00:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.323+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd45c06cc20 msgr2=0x7fd45c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.323+0000 7fd479693700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd45c06cc20 0x7fd45c06f0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ffbf0 tx=0x7fd468009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.323+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 msgr2=0x7fd4740fdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.323+0000 7fd479693700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 0x7fd4740fdae0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd464002a80 tx=0x7fd4640046b0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 shutdown_connections 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd45c06cc20 0x7fd45c06f0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474101e60 0x7fd4740fdae0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 --2- 172.21.15.19:0/1480755110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd474102800 0x7fd4740fe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 >> 172.21.15.19:0/1480755110 conn(0x7fd4740f7a10 msgr2=0x7fd474066070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 shutdown_connections 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.324+0000 7fd479693700 1 -- 172.21.15.19:0/1480755110 wait complete. 2024-02-22T16:00:52.304 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:00:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 -- 172.21.15.19:0/1734247517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fc640 msgr2=0x7fcedc1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 --2- 172.21.15.19:0/1734247517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fc640 0x7fcedc1093a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcecc004940 tx=0x7fcecc009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 -- 172.21.15.19:0/1734247517 shutdown_connections 2024-02-22T16:00:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 --2- 172.21.15.19:0/1734247517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fc640 0x7fcedc1093a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 --2- 172.21.15.19:0/1734247517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcedc0fbd30 0x7fcedc0fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 -- 172.21.15.19:0/1734247517 >> 172.21.15.19:0/1734247517 conn(0x7fcedc0f79e0 msgr2=0x7fcedc0f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 -- 172.21.15.19:0/1734247517 shutdown_connections 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.429+0000 7fcee0be7700 1 -- 172.21.15.19:0/1734247517 wait complete. 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fcee0be7700 1 Processor -- start 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fcee0be7700 1 -- start start 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fcee0be7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 0x7fcedc115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fcee0be7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcedc0fc640 0x7fcedc116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fcee0be7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc116780 con 0x7fcedc0fbd30 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fcee0be7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc10fab0 con 0x7fcedc0fc640 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 0x7fcedc115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 0x7fcedc115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54762/0 (socket says 172.21.15.19:54762) 2024-02-22T16:00:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 -- 172.21.15.19:0/2727834351 learned_addr learned my addr 172.21.15.19:0/2727834351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fced9d9b700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcedc0fc640 0x7fcedc116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 -- 172.21.15.19:0/2727834351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcedc0fc640 msgr2=0x7fcedc116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcedc0fc640 0x7fcedc116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 -- 172.21.15.19:0/2727834351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc005040 con 0x7fcedc0fbd30 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.430+0000 7fceda59c700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 0x7fcedc115bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fced000eb40 tx=0x7fced000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.431+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced0007cc0 con 0x7fcedc0fbd30 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.431+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fced000f040 con 0x7fcedc0fbd30 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.431+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc10fd30 con 0x7fcedc0fbd30 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.431+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced000c990 con 0x7fcedc0fbd30 2024-02-22T16:00:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.431+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc110270 con 0x7fcedc0fbd30 2024-02-22T16:00:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.432+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc106ad0 con 0x7fcedc0fbd30 2024-02-22T16:00:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.432+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fced000cc00 con 0x7fcedc0fbd30 2024-02-22T16:00:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.433+0000 7fcecaffd700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcec406cc20 0x7fcec406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.433+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fced008bc70 con 0x7fcedc0fbd30 2024-02-22T16:00:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.433+0000 7fced9d9b700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcec406cc20 0x7fcec406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.433+0000 7fced9d9b700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcec406cc20 0x7fcec406f0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcedc111420 tx=0x7fcecc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.437+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fced0059760 con 0x7fcedc0fbd30 2024-02-22T16:00:52.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:52 smithi062 ceph-mon[106861]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:52.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:52 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2892782109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:00:52.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:52 smithi019 ceph-mon[173579]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:52.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:52 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2892782109' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:00:52.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.656+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcedc04bf40 con 0x7fcec406cc20 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.657+0000 7fcecaffd700 1 -- 172.21.15.19:0/2727834351 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcedc04bf40 con 0x7fcec406cc20 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:00:52.635 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:00:52.636 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:00:52.636 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:00:52.636 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:00:52.636 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:00:52.636 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:00:52.636 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcec406cc20 msgr2=0x7fcec406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcec406cc20 0x7fcec406f0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcedc111420 tx=0x7fcecc009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 msgr2=0x7fcedc115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 0x7fcedc115bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fced000eb40 tx=0x7fced000b040 comp rx=0 tx=0).stop 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 shutdown_connections 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcec406cc20 0x7fcec406f0d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcedc0fbd30 0x7fcedc115bf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 --2- 172.21.15.19:0/2727834351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcedc0fc640 0x7fcedc116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 >> 172.21.15.19:0/2727834351 conn(0x7fcedc0f79e0 msgr2=0x7fcedc0f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 shutdown_connections 2024-02-22T16:00:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.659+0000 7fcee0be7700 1 -- 172.21.15.19:0/2727834351 wait complete. 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 -- 172.21.15.19:0/1730444042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0540101e40 msgr2=0x7f0540102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 --2- 172.21.15.19:0/1730444042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0540101e40 0x7f0540102210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0534004940 tx=0x7f0534009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 -- 172.21.15.19:0/1730444042 shutdown_connections 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 --2- 172.21.15.19:0/1730444042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05401027e0 0x7f05400fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 --2- 172.21.15.19:0/1730444042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0540101e40 0x7f0540102210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 -- 172.21.15.19:0/1730444042 >> 172.21.15.19:0/1730444042 conn(0x7f0540071d90 msgr2=0x7f0540072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 -- 172.21.15.19:0/1730444042 shutdown_connections 2024-02-22T16:00:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.762+0000 7f0547144700 1 -- 172.21.15.19:0/1730444042 wait complete. 2024-02-22T16:00:52.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.763+0000 7f0547144700 1 Processor -- start 2024-02-22T16:00:52.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.763+0000 7f0547144700 1 -- start start 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.763+0000 7f0547144700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0540101e40 0x7f05401139c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.763+0000 7f0547144700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 0x7f0540113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.763+0000 7f0547144700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0540114550 con 0x7f05401027e0 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.763+0000 7f0547144700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054010d880 con 0x7f0540101e40 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 0x7f0540113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 0x7f0540113f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54782/0 (socket says 172.21.15.19:54782) 2024-02-22T16:00:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 -- 172.21.15.19:0/1350916774 learned_addr learned my addr 172.21.15.19:0/1350916774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f0544ee0700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0540101e40 0x7f05401139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 -- 172.21.15.19:0/1350916774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0540101e40 msgr2=0x7f05401139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0540101e40 0x7f05401139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 -- 172.21.15.19:0/1350916774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534005040 con 0x7f05401027e0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053ffff700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 0x7f0540113f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f052c002a80 tx=0x7f052c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052c004e80 con 0x7f05401027e0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f052c00f040 con 0x7f05401027e0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054010db60 con 0x7f05401027e0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052c00da00 con 0x7f05401027e0 2024-02-22T16:00:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.764+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054010e0a0 con 0x7f05401027e0 2024-02-22T16:00:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.766+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05400738b0 con 0x7f05401027e0 2024-02-22T16:00:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.766+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f052c007cc0 con 0x7f05401027e0 2024-02-22T16:00:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.766+0000 7f053d7fa700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f053006cc20 0x7f053006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:00:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.767+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f052c08b6d0 con 0x7f05401027e0 2024-02-22T16:00:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.767+0000 7f0544ee0700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f053006cc20 0x7f053006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:00:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.767+0000 7f0544ee0700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f053006cc20 0x7f053006f0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0534004f90 tx=0x7f0534009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:00:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:52.771+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f052c059110 con 0x7f05401027e0 2024-02-22T16:00:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.058+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f054010efa0 con 0x7f05401027e0 2024-02-22T16:00:53.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.058+0000 7f053d7fa700 1 -- 172.21.15.19:0/1350916774 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f052c058ca0 con 0x7f05401027e0 2024-02-22T16:00:53.036 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:00:53.037 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:00:53.037 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:00:53.037 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:00:53.037 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:00:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f053006cc20 msgr2=0x7f053006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f053006cc20 0x7f053006f0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0534004f90 tx=0x7f0534009e30 comp rx=0 tx=0).stop 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 msgr2=0x7f0540113f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 0x7f0540113f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f052c002a80 tx=0x7f052c004630 comp rx=0 tx=0).stop 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 shutdown_connections 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f053006cc20 0x7f053006f0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0540101e40 0x7f05401139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 --2- 172.21.15.19:0/1350916774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05401027e0 0x7f0540113f00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 >> 172.21.15.19:0/1350916774 conn(0x7f0540071d90 msgr2=0x7f05400657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 shutdown_connections 2024-02-22T16:00:53.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:00:53.060+0000 7f0547144700 1 -- 172.21.15.19:0/1350916774 wait complete. 2024-02-22T16:00:53.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:53 smithi062 ceph-mon[106861]: pgmap v228: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:53.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:53 smithi062 ceph-mon[106861]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:53.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:53 smithi062 ceph-mon[106861]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:53.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:53 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1480755110' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:00:53.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:53 smithi062 ceph-mon[106861]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:53.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:53 smithi019 ceph-mon[173579]: pgmap v228: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:53.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:53 smithi019 ceph-mon[173579]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:53.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:53 smithi019 ceph-mon[173579]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:53.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:53 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1480755110' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:00:53.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:53 smithi019 ceph-mon[173579]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:00:54.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:54 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1350916774' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:00:54.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:54 smithi062 ceph-mon[106861]: pgmap v229: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:54.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:54 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1350916774' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:00:54.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:54 smithi019 ceph-mon[173579]: pgmap v229: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:56.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:56 smithi019 ceph-mon[173579]: pgmap v230: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:56.703 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:56 smithi062 ceph-mon[106861]: pgmap v230: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:59.203 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:00:58 smithi062 ceph-mon[106861]: pgmap v231: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:00:59.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:00:58 smithi019 ceph-mon[173579]: pgmap v231: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:01.203 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:00 smithi062 ceph-mon[106861]: pgmap v232: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:01.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:00 smithi019 ceph-mon[173579]: pgmap v232: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:03.203 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:02 smithi062 ceph-mon[106861]: pgmap v233: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:03.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:02 smithi019 ceph-mon[173579]: pgmap v233: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:04.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:04.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:05.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:04 smithi019 ceph-mon[173579]: pgmap v234: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:05.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:04 smithi062 ceph-mon[106861]: pgmap v234: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:06 smithi019 ceph-mon[173579]: pgmap v235: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:07.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:06 smithi062 ceph-mon[106861]: pgmap v235: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:09.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:08 smithi019 ceph-mon[173579]: pgmap v236: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:09.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:08 smithi062 ceph-mon[106861]: pgmap v236: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:11.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:10 smithi019 ceph-mon[173579]: pgmap v237: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:11.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:10 smithi062 ceph-mon[106861]: pgmap v237: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:13.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:12 smithi019 ceph-mon[173579]: pgmap v238: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:13.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:12 smithi062 ceph-mon[106861]: pgmap v238: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:15.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:14 smithi019 ceph-mon[173579]: pgmap v239: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:15.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:14 smithi062 ceph-mon[106861]: pgmap v239: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:17.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:16 smithi019 ceph-mon[173579]: pgmap v240: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:17.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:16 smithi062 ceph-mon[106861]: pgmap v240: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:19.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:19 smithi019 ceph-mon[173579]: pgmap v241: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:19.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:19 smithi062 ceph-mon[106861]: pgmap v241: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:21.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:21 smithi019 ceph-mon[173579]: pgmap v242: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:21.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:21 smithi062 ceph-mon[106861]: pgmap v242: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 -- 172.21.15.19:0/2904134652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 msgr2=0x7feadc106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 --2- 172.21.15.19:0/2904134652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc106fa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feac8004940 tx=0x7feac8009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 -- 172.21.15.19:0/2904134652 shutdown_connections 2024-02-22T16:01:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 --2- 172.21.15.19:0/2904134652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc106fa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 --2- 172.21.15.19:0/2904134652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc0fdfc0 0x7feadc0fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 -- 172.21.15.19:0/2904134652 >> 172.21.15.19:0/2904134652 conn(0x7feadc0f7a10 msgr2=0x7feadc0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 -- 172.21.15.19:0/2904134652 shutdown_connections 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 -- 172.21.15.19:0/2904134652 wait complete. 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.179+0000 7feae2b8f700 1 Processor -- start 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feae2b8f700 1 -- start start 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feae2b8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc0fdfc0 0x7feadc0fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feae2b8f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc0fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feae2b8f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0fc930 con 0x7feadc0fe960 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feae2b8f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0fcaa0 con 0x7feadc0fdfc0 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feadbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc0fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feadbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc0fc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32908/0 (socket says 172.21.15.19:32908) 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feadbfff700 1 -- 172.21.15.19:0/907071361 learned_addr learned my addr 172.21.15.19:0/907071361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:23.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feae092b700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc0fdfc0 0x7feadc0fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feadbfff700 1 -- 172.21.15.19:0/907071361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc0fdfc0 msgr2=0x7feadc0fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feadbfff700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc0fdfc0 0x7feadc0fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.180+0000 7feadbfff700 1 -- 172.21.15.19:0/907071361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac8005040 con 0x7feadc0fe960 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.181+0000 7feadbfff700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc0fc330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feac8009390 tx=0x7feac8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.181+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feac800e040 con 0x7feadc0fe960 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.181+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feac8004290 con 0x7feadc0fe960 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.181+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc119b20 con 0x7feadc0fe960 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.181+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feac801c630 con 0x7feadc0fe960 2024-02-22T16:01:23.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.181+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc11a000 con 0x7feadc0fe960 2024-02-22T16:01:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.182+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc1045b0 con 0x7feadc0fe960 2024-02-22T16:01:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.183+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feac800bc50 con 0x7feadc0fe960 2024-02-22T16:01:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.183+0000 7fead97fa700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feacc06cc30 0x7feacc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.183+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7feac808d4d0 con 0x7feadc0fe960 2024-02-22T16:01:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.183+0000 7feae092b700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feacc06cc30 0x7feacc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.184+0000 7feae092b700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feacc06cc30 0x7feacc06f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fead0009fd0 tx=0x7fead000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:23.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.188+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7feac80575a0 con 0x7feadc0fe960 2024-02-22T16:01:23.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:23 smithi019 ceph-mon[173579]: pgmap v243: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.408+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feadc04bf70 con 0x7feacc06cc30 2024-02-22T16:01:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.409+0000 7fead97fa700 1 -- 172.21.15.19:0/907071361 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fead000f040 con 0x7feacc06cc30 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feacc06cc30 msgr2=0x7feacc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feacc06cc30 0x7feacc06f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fead0009fd0 tx=0x7fead000b040 comp rx=0 tx=0).stop 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 msgr2=0x7feadc0fc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc0fc330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feac8009390 tx=0x7feac8009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 shutdown_connections 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feacc06cc30 0x7feacc06f0e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feadc0fdfc0 0x7feadc0fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 --2- 172.21.15.19:0/907071361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feadc0fe960 0x7feadc0fc330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 >> 172.21.15.19:0/907071361 conn(0x7feadc0f7a10 msgr2=0x7feadc101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 shutdown_connections 2024-02-22T16:01:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.411+0000 7feae2b8f700 1 -- 172.21.15.19:0/907071361 wait complete. 2024-02-22T16:01:23.400 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:01:23.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:23 smithi062 ceph-mon[106861]: pgmap v243: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:23.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.515+0000 7fa1f18d2700 1 -- 172.21.15.19:0/58412369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 msgr2=0x7fa1ec102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.515+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/58412369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec102230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc004940 tx=0x7fa1dc009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 -- 172.21.15.19:0/58412369 shutdown_connections 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/58412369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec102800 0x7fa1ec0fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/58412369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec102230 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 -- 172.21.15.19:0/58412369 >> 172.21.15.19:0/58412369 conn(0x7fa1ec0f7a10 msgr2=0x7fa1ec0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 -- 172.21.15.19:0/58412369 shutdown_connections 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 -- 172.21.15.19:0/58412369 wait complete. 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 Processor -- start 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.516+0000 7fa1f18d2700 1 -- start start 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1f18d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec1128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1f18d2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec102800 0x7fa1ec112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1f18d2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec113470 con 0x7fa1ec101e60 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1f18d2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec10c7a0 con 0x7fa1ec102800 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec1128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec1128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32916/0 (socket says 172.21.15.19:32916) 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 -- 172.21.15.19:0/210393855 learned_addr learned my addr 172.21.15.19:0/210393855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1ea7fc700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec102800 0x7fa1ec112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 -- 172.21.15.19:0/210393855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec102800 msgr2=0x7fa1ec112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec102800 0x7fa1ec112e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 -- 172.21.15.19:0/210393855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc005040 con 0x7fa1ec101e60 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1eaffd700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec1128e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00b3b0 tx=0x7fa1dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1dc00e040 con 0x7fa1ec101e60 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1dc00bc50 con 0x7fa1ec101e60 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1dc013420 con 0x7fa1ec101e60 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec10ca20 con 0x7fa1ec101e60 2024-02-22T16:01:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.517+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec10cf10 con 0x7fa1ec101e60 2024-02-22T16:01:23.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.518+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec0ff170 con 0x7fa1ec101e60 2024-02-22T16:01:23.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.519+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa1dc01c430 con 0x7fa1ec101e60 2024-02-22T16:01:23.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.519+0000 7fa1d37fe700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1d406cc20 0x7fa1d406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.520+0000 7fa1ea7fc700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1d406cc20 0x7fa1d406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.520+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa1dc08ce70 con 0x7fa1ec101e60 2024-02-22T16:01:23.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.520+0000 7fa1ea7fc700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1d406cc20 0x7fa1d406f0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec10e110 tx=0x7fa1e0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:23.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.524+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa1dc05a9e0 con 0x7fa1ec101e60 2024-02-22T16:01:23.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.749+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1ec04bed0 con 0x7fa1d406cc20 2024-02-22T16:01:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.750+0000 7fa1d37fe700 1 -- 172.21.15.19:0/210393855 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa1ec04bed0 con 0x7fa1d406cc20 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1d406cc20 msgr2=0x7fa1d406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1d406cc20 0x7fa1d406f0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec10e110 tx=0x7fa1e0009ee0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 msgr2=0x7fa1ec1128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec1128e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00b3b0 tx=0x7fa1dc009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 shutdown_connections 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa1d406cc20 0x7fa1d406f0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1ec101e60 0x7fa1ec1128e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 --2- 172.21.15.19:0/210393855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1ec102800 0x7fa1ec112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 >> 172.21.15.19:0/210393855 conn(0x7fa1ec0f7a10 msgr2=0x7fa1ec0f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:23.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 shutdown_connections 2024-02-22T16:01:23.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.752+0000 7fa1f18d2700 1 -- 172.21.15.19:0/210393855 wait complete. 2024-02-22T16:01:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 -- 172.21.15.19:0/2213461338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad4101e40 msgr2=0x7f6ad4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2213461338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad4101e40 0x7f6ad4102210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004940 tx=0x7f6ac4009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 -- 172.21.15.19:0/2213461338 shutdown_connections 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2213461338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41027e0 0x7f6ad40fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2213461338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad4101e40 0x7f6ad4102210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 -- 172.21.15.19:0/2213461338 >> 172.21.15.19:0/2213461338 conn(0x7f6ad4071d90 msgr2=0x7f6ad4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 -- 172.21.15.19:0/2213461338 shutdown_connections 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.856+0000 7f6adbd42700 1 -- 172.21.15.19:0/2213461338 wait complete. 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6adbd42700 1 Processor -- start 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6adbd42700 1 -- start start 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6adbd42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4101e40 0x7f6ad4102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6adbd42700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 0x7f6ad41031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6adbd42700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad41037e0 con 0x7f6ad41027e0 2024-02-22T16:01:23.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6adbd42700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4103950 con 0x7f6ad4101e40 2024-02-22T16:01:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 0x7f6ad41031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 0x7f6ad41031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32920/0 (socket says 172.21.15.19:32920) 2024-02-22T16:01:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 -- 172.21.15.19:0/2574668350 learned_addr learned my addr 172.21.15.19:0/2574668350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad9ade700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4101e40 0x7f6ad4102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 -- 172.21.15.19:0/2574668350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4101e40 msgr2=0x7f6ad4102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4101e40 0x7f6ad4102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 -- 172.21.15.19:0/2574668350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4005040 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.857+0000 7f6ad92dd700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 0x7f6ad41031e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000c800 tx=0x7f6ad0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.858+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad00106f0 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.858+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ad0010d00 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.858+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad4119bc0 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.858+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad00113a0 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.858+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad411a0a0 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.859+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad40738b0 con 0x7f6ad41027e0 2024-02-22T16:01:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.859+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6ad0012400 con 0x7f6ad41027e0 2024-02-22T16:01:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.860+0000 7f6aca7fc700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac006cc20 0x7f6ac006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.860+0000 7f6ad9ade700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac006cc20 0x7f6ac006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.860+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6ad008bf00 con 0x7f6ad41027e0 2024-02-22T16:01:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.860+0000 7f6ad9ade700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac006cc20 0x7f6ac006f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004f90 tx=0x7f6ac4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:23.864+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6ad0055fd0 con 0x7f6ad41027e0 2024-02-22T16:01:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.077+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ad4065ef0 con 0x7f6ac006cc20 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.086+0000 7f6aca7fc700 1 -- 172.21.15.19:0/2574668350 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ad4065ef0 con 0x7f6ac006cc20 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (8m) 116s ago 19m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (5m) 116s ago 19m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (5m) 86s ago 18m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (8m) 116s ago 19m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (2m) 116s ago 13m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:01:24.066 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (2m) 116s ago 13m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (88s) 86s ago 13m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (104s) 86s ago 13m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (10m) 116s ago 21m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (9m) 86s ago 18m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (7m) 116s ago 21m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (6m) 86s ago 18m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (9m) 116s ago 19m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (9m) 86s ago 18m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (5m) 116s ago 17m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (4m) 116s ago 17m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (4m) 116s ago 16m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (3m) 86s ago 16m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (3m) 86s ago 15m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:01:24.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (2m) 86s ago 15m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:01:24.068 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (8m) 116s ago 19m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:01:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.088+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac006cc20 msgr2=0x7f6ac006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.088+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac006cc20 0x7f6ac006f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004f90 tx=0x7f6ac4009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.088+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 msgr2=0x7f6ad41031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.088+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 0x7f6ad41031e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000c800 tx=0x7f6ad0004470 comp rx=0 tx=0).stop 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 shutdown_connections 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ac006cc20 0x7f6ac006f0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4101e40 0x7f6ad4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 --2- 172.21.15.19:0/2574668350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ad41027e0 0x7f6ad41031e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 >> 172.21.15.19:0/2574668350 conn(0x7f6ad4071d90 msgr2=0x7f6ad40657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 shutdown_connections 2024-02-22T16:01:24.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.089+0000 7f6adbd42700 1 -- 172.21.15.19:0/2574668350 wait complete. 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 -- 172.21.15.19:0/2645264542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 msgr2=0x7f1f6c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/2645264542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1021b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c004940 tx=0x7f1f5c009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 -- 172.21.15.19:0/2645264542 shutdown_connections 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/2645264542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f6c102780 0x7f1f6c0fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/2645264542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c1021b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 -- 172.21.15.19:0/2645264542 >> 172.21.15.19:0/2645264542 conn(0x7f1f6c071d50 msgr2=0x7f1f6c072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 -- 172.21.15.19:0/2645264542 shutdown_connections 2024-02-22T16:01:24.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 -- 172.21.15.19:0/2645264542 wait complete. 2024-02-22T16:01:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 Processor -- start 2024-02-22T16:01:24.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 -- start start 2024-02-22T16:01:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.191+0000 7f1f70a8e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f6c102780 0x7f1f6c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f70a8e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c103780 con 0x7f1f6c101de0 2024-02-22T16:01:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f70a8e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1038f0 con 0x7f1f6c102780 2024-02-22T16:01:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c102c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32952/0 (socket says 172.21.15.19:32952) 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 -- 172.21.15.19:0/557851289 learned_addr learned my addr 172.21.15.19:0/557851289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f69d9b700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f6c102780 0x7f1f6c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 -- 172.21.15.19:0/557851289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f6c102780 msgr2=0x7f1f6c103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f6c102780 0x7f1f6c103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 -- 172.21.15.19:0/557851289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c005040 con 0x7f1f6c101de0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f6a59c700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c102c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009180 tx=0x7f1f5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f5c00e040 con 0x7f1f6c101de0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f5c004290 con 0x7f1f6c101de0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c119ab0 con 0x7f1f6c101de0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f5c01c3f0 con 0x7f1f6c101de0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.192+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c119f90 con 0x7f1f6c101de0 2024-02-22T16:01:24.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.193+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c075ca0 con 0x7f1f6c101de0 2024-02-22T16:01:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.194+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1f5c013420 con 0x7f1f6c101de0 2024-02-22T16:01:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.194+0000 7f1f5affd700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f5406cc20 0x7f1f5406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.195+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1f5c08d270 con 0x7f1f6c101de0 2024-02-22T16:01:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.195+0000 7f1f69d9b700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f5406cc20 0x7f1f5406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.195+0000 7f1f69d9b700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f5406cc20 0x7f1f5406f0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c104640 tx=0x7f1f60009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:24.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.199+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1f5c0573f0 con 0x7f1f6c101de0 2024-02-22T16:01:24.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.487+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f6c108da0 con 0x7f1f6c101de0 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.488+0000 7f1f5affd700 1 -- 172.21.15.19:0/557851289 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f1f5c05aa10 con 0x7f1f6c101de0 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:01:24.467 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:01:24.468 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:01:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f5406cc20 msgr2=0x7f1f5406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f5406cc20 0x7f1f5406f0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c104640 tx=0x7f1f60009040 comp rx=0 tx=0).stop 2024-02-22T16:01:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 msgr2=0x7f1f6c102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c102c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009180 tx=0x7f1f5c009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:24.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 shutdown_connections 2024-02-22T16:01:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f5406cc20 0x7f1f5406f0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f6c101de0 0x7f1f6c102c40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 --2- 172.21.15.19:0/557851289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f6c102780 0x7f1f6c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 >> 172.21.15.19:0/557851289 conn(0x7f1f6c071d50 msgr2=0x7f1f6c065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 shutdown_connections 2024-02-22T16:01:24.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.490+0000 7f1f70a8e700 1 -- 172.21.15.19:0/557851289 wait complete. 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 -- 172.21.15.19:0/3101873103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7924101e40 msgr2=0x7f7924102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 --2- 172.21.15.19:0/3101873103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7924101e40 0x7f7924102210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7918004940 tx=0x7f7918009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 -- 172.21.15.19:0/3101873103 shutdown_connections 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 --2- 172.21.15.19:0/3101873103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79241027e0 0x7f79240fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 --2- 172.21.15.19:0/3101873103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7924101e40 0x7f7924102210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 -- 172.21.15.19:0/3101873103 >> 172.21.15.19:0/3101873103 conn(0x7f7924071d90 msgr2=0x7f7924072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 -- 172.21.15.19:0/3101873103 shutdown_connections 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.593+0000 7f792b081700 1 -- 172.21.15.19:0/3101873103 wait complete. 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.594+0000 7f792b081700 1 Processor -- start 2024-02-22T16:01:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.594+0000 7f792b081700 1 -- start start 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f792b081700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7924101e40 0x7f7924112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f792b081700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 0x7f7924112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f792b081700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924113410 con 0x7f79241027e0 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f792b081700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792410c740 con 0x7f7924101e40 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 0x7f7924112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 0x7f7924112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32966/0 (socket says 172.21.15.19:32966) 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 -- 172.21.15.19:0/1813269201 learned_addr learned my addr 172.21.15.19:0/1813269201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7928e1d700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7924101e40 0x7f7924112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 -- 172.21.15.19:0/1813269201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7924101e40 msgr2=0x7f7924112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7924101e40 0x7f7924112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 -- 172.21.15.19:0/1813269201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918005040 con 0x7f79241027e0 2024-02-22T16:01:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f7923fff700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 0x7f7924112dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f791000eb40 tx=0x7f791000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7910007cc0 con 0x7f79241027e0 2024-02-22T16:01:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792410c9c0 con 0x7f79241027e0 2024-02-22T16:01:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f791000f040 con 0x7f79241027e0 2024-02-22T16:01:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f791000c8f0 con 0x7f79241027e0 2024-02-22T16:01:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.595+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792410cf00 con 0x7f79241027e0 2024-02-22T16:01:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.596+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79240ff150 con 0x7f79241027e0 2024-02-22T16:01:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.597+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f791000cb60 con 0x7f79241027e0 2024-02-22T16:01:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.597+0000 7f79217fa700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f791406cc30 0x7f791406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.598+0000 7f7928e1d700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f791406cc30 0x7f791406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.598+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f791008baf0 con 0x7f79241027e0 2024-02-22T16:01:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.598+0000 7f7928e1d700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f791406cc30 0x7f791406f0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7918009fd0 tx=0x7f7918009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:24.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.602+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f79100595e0 con 0x7f79241027e0 2024-02-22T16:01:24.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.850+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7924108e00 con 0x7f79241027e0 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.851+0000 7f79217fa700 1 -- 172.21.15.19:0/1813269201 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f7910059170 con 0x7f79241027e0 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:01:24.830 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:01:24.831 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:24.832 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:01:24.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:24.833 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f791406cc30 msgr2=0x7f791406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f791406cc30 0x7f791406f0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7918009fd0 tx=0x7f7918009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 msgr2=0x7f7924112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 0x7f7924112dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f791000eb40 tx=0x7f791000b040 comp rx=0 tx=0).stop 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 shutdown_connections 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f791406cc30 0x7f791406f0e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7924101e40 0x7f7924112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 --2- 172.21.15.19:0/1813269201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79241027e0 0x7f7924112dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 >> 172.21.15.19:0/1813269201 conn(0x7f7924071d90 msgr2=0x7f79240fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 shutdown_connections 2024-02-22T16:01:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.857+0000 7f792b081700 1 -- 172.21.15.19:0/1813269201 wait complete. 2024-02-22T16:01:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 -- 172.21.15.19:0/3779221245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 msgr2=0x7fe0b4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/3779221245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4107050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe09c004970 tx=0x7fe09c009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 -- 172.21.15.19:0/3779221245 shutdown_connections 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/3779221245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4107050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/3779221245 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b40fdfe0 0x7fe0b40fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 -- 172.21.15.19:0/3779221245 >> 172.21.15.19:0/3779221245 conn(0x7fe0b4071d90 msgr2=0x7fe0b4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 -- 172.21.15.19:0/3779221245 shutdown_connections 2024-02-22T16:01:24.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 -- 172.21.15.19:0/3779221245 wait complete. 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.963+0000 7fe0ba49d700 1 Processor -- start 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0ba49d700 1 -- start start 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0ba49d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b40fdfe0 0x7fe0b4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0ba49d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0ba49d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b4114500 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0ba49d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b410d830 con 0x7fe0b40fdfe0 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32984/0 (socket says 172.21.15.19:32984) 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 -- 172.21.15.19:0/2449118718 learned_addr learned my addr 172.21.15.19:0/2449118718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b3fff700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b40fdfe0 0x7fe0b4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 -- 172.21.15.19:0/2449118718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b40fdfe0 msgr2=0x7fe0b4113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b40fdfe0 0x7fe0b4113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 -- 172.21.15.19:0/2449118718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c005040 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b37fe700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4113eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe09c005010 tx=0x7fe09c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.964+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe09c00e040 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.965+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe09c004290 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.965+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b410dab0 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.965+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe09c01c3f0 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.965+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b410dff0 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.966+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b41046c0 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.966+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe09c0136b0 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.967+0000 7fe0b0ff9700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe0a006cc30 0x7fe0a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.967+0000 7fe0b3fff700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe0a006cc30 0x7fe0a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.967+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe09c08d2f0 con 0x7fe0b40fe8f0 2024-02-22T16:01:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.967+0000 7fe0b3fff700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe0a006cc30 0x7fe0a006f0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009b80 tx=0x7fe0a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:24.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:24.971+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe09c05adb0 con 0x7fe0b40fe8f0 2024-02-22T16:01:25.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.195+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0b4109ef0 con 0x7fe0a006cc30 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.196+0000 7fe0b0ff9700 1 -- 172.21.15.19:0/2449118718 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe0b4109ef0 con 0x7fe0a006cc30 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:01:25.175 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:01:25.176 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:01:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe0a006cc30 msgr2=0x7fe0a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe0a006cc30 0x7fe0a006f0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009b80 tx=0x7fe0a400b040 comp rx=0 tx=0).stop 2024-02-22T16:01:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 msgr2=0x7fe0b4113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4113eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe09c005010 tx=0x7fe09c009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 shutdown_connections 2024-02-22T16:01:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe0a006cc30 0x7fe0a006f0e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0b40fdfe0 0x7fe0b4113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 --2- 172.21.15.19:0/2449118718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0b40fe8f0 0x7fe0b4113eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 >> 172.21.15.19:0/2449118718 conn(0x7fe0b4071d90 msgr2=0x7fe0b4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 shutdown_connections 2024-02-22T16:01:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.198+0000 7fe0ba49d700 1 -- 172.21.15.19:0/2449118718 wait complete. 2024-02-22T16:01:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:25 smithi019 ceph-mon[173579]: pgmap v244: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:25 smithi019 ceph-mon[173579]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:25 smithi019 ceph-mon[173579]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:25 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/557851289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:01:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:25 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1813269201' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 -- 172.21.15.19:0/543766643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 msgr2=0x7f6ca0107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 --2- 172.21.15.19:0/543766643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca0107050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c90004970 tx=0x7f6c90009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 -- 172.21.15.19:0/543766643 shutdown_connections 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 --2- 172.21.15.19:0/543766643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca0107050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 --2- 172.21.15.19:0/543766643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ca00fdfe0 0x7f6ca00fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 -- 172.21.15.19:0/543766643 >> 172.21.15.19:0/543766643 conn(0x7f6ca0071d90 msgr2=0x7f6ca0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 -- 172.21.15.19:0/543766643 shutdown_connections 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.303+0000 7f6ca8263700 1 -- 172.21.15.19:0/543766643 wait complete. 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca8263700 1 Processor -- start 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca8263700 1 -- start start 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca8263700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ca00fdfe0 0x7f6ca00fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca8263700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca00fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca8263700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00fc9e0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca8263700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00fcb50 con 0x7f6ca00fdfe0 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca00fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca00fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33006/0 (socket says 172.21.15.19:33006) 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 -- 172.21.15.19:0/4137929105 learned_addr learned my addr 172.21.15.19:0/4137929105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca5fff700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ca00fdfe0 0x7f6ca00fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 -- 172.21.15.19:0/4137929105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ca00fdfe0 msgr2=0x7f6ca00fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ca00fdfe0 0x7f6ca00fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 -- 172.21.15.19:0/4137929105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90005040 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.304+0000 7f6ca57fe700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca00fc3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000c00 tx=0x7f6c90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.305+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9000e040 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.305+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca0119bb0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.305+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c90009b10 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.305+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c90013e60 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.305+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca011a010 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.306+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca01046c0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.306+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6c900136b0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.307+0000 7f6c967fc700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c8c06cbb0 0x7f6c8c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.307+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6c9008d290 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.307+0000 7f6ca5fff700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c8c06cbb0 0x7f6c8c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.307+0000 7f6ca5fff700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c8c06cbb0 0x7f6c8c06f060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009b80 tx=0x7f6c9c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.311+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6c900572c0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:25 smithi062 ceph-mon[106861]: pgmap v244: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:25.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:25 smithi062 ceph-mon[106861]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:25.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:25 smithi062 ceph-mon[106861]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:25.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:25 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/557851289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:01:25.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:25 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1813269201' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:01:25.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.595+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6ca00738b0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.596+0000 7f6c967fc700 1 -- 172.21.15.19:0/4137929105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6c9005a8e0 con 0x7f6ca00fe8f0 2024-02-22T16:01:25.575 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:01:25.575 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:01:25.575 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:01:25.575 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:01:25.575 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:01:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c8c06cbb0 msgr2=0x7f6c8c06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c8c06cbb0 0x7f6c8c06f060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009b80 tx=0x7f6c9c00b040 comp rx=0 tx=0).stop 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 msgr2=0x7f6ca00fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca00fc3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000c00 tx=0x7f6c90009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 shutdown_connections 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c8c06cbb0 0x7f6c8c06f060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ca00fdfe0 0x7f6ca00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 --2- 172.21.15.19:0/4137929105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ca00fe8f0 0x7f6ca00fc3e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 >> 172.21.15.19:0/4137929105 conn(0x7f6ca0071d90 msgr2=0x7f6ca0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 shutdown_connections 2024-02-22T16:01:25.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:25.598+0000 7f6ca8263700 1 -- 172.21.15.19:0/4137929105 wait complete. 2024-02-22T16:01:26.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:26 smithi062 ceph-mon[106861]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:26.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:26 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4137929105' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:01:26.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:26 smithi019 ceph-mon[173579]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:26.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:26 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4137929105' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:01:27.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:27 smithi062 ceph-mon[106861]: pgmap v245: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:27.456 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:27 smithi062 ceph-mon[106861]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:27.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:27 smithi019 ceph-mon[173579]: pgmap v245: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:27.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:27 smithi019 ceph-mon[173579]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:29.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:29 smithi062 ceph-mon[106861]: pgmap v246: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:29.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:29 smithi019 ceph-mon[173579]: pgmap v246: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:31 smithi019 ceph-mon[173579]: pgmap v247: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:31.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:31 smithi062 ceph-mon[106861]: pgmap v247: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:33.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:33 smithi062 ceph-mon[106861]: pgmap v248: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:33.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:33 smithi019 ceph-mon[173579]: pgmap v248: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:34.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:34.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:35 smithi062 ceph-mon[106861]: pgmap v249: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:35.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:35 smithi019 ceph-mon[173579]: pgmap v249: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:37.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:37 smithi062 ceph-mon[106861]: pgmap v250: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:37.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:37 smithi019 ceph-mon[173579]: pgmap v250: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:39.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:39 smithi062 ceph-mon[106861]: pgmap v251: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:39.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:39 smithi019 ceph-mon[173579]: pgmap v251: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:41.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:41 smithi019 ceph-mon[173579]: pgmap v252: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:41.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:41 smithi062 ceph-mon[106861]: pgmap v252: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:43.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:43 smithi062 ceph-mon[106861]: pgmap v253: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:43.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:43 smithi019 ceph-mon[173579]: pgmap v253: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:44.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:44 smithi062 ceph-mon[106861]: pgmap v254: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:44.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:44 smithi019 ceph-mon[173579]: pgmap v254: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:46.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:46 smithi019 ceph-mon[173579]: pgmap v255: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:46.703 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:46 smithi062 ceph-mon[106861]: pgmap v255: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:49.203 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:48 smithi062 ceph-mon[106861]: pgmap v256: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:49.204 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:49.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:48 smithi019 ceph-mon[173579]: pgmap v256: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:49.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:01:51.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:50 smithi019 ceph-mon[173579]: pgmap v257: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:51.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:50 smithi062 ceph-mon[106861]: pgmap v257: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:53.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:52 smithi019 ceph-mon[173579]: pgmap v258: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:53.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:52 smithi062 ceph-mon[106861]: pgmap v258: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:55.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:54 smithi019 ceph-mon[173579]: pgmap v259: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:55.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:54 smithi062 ceph-mon[106861]: pgmap v259: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 -- 172.21.15.19:0/487950893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fb570 msgr2=0x7fa7ac0fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 --2- 172.21.15.19:0/487950893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fb570 0x7fa7ac0fb9e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004940 tx=0x7fa7a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 -- 172.21.15.19:0/487950893 shutdown_connections 2024-02-22T16:01:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 --2- 172.21.15.19:0/487950893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fb570 0x7fa7ac0fb9e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 --2- 172.21.15.19:0/487950893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac0fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 -- 172.21.15.19:0/487950893 >> 172.21.15.19:0/487950893 conn(0x7fa7ac071560 msgr2=0x7fa7ac071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 -- 172.21.15.19:0/487950893 shutdown_connections 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.718+0000 7fa7b3328700 1 -- 172.21.15.19:0/487950893 wait complete. 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b3328700 1 Processor -- start 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b3328700 1 -- start start 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b3328700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7ac0fb570 0x7fa7ac115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b3328700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b3328700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac116710 con 0x7fa7ac0fde50 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b3328700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac10fa40 con 0x7fa7ac0fb570 2024-02-22T16:01:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b08c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b08c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac1160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49506/0 (socket says 172.21.15.19:49506) 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b08c3700 1 -- 172.21.15.19:0/353402722 learned_addr learned my addr 172.21.15.19:0/353402722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b10c4700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7ac0fb570 0x7fa7ac115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b08c3700 1 -- 172.21.15.19:0/353402722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7ac0fb570 msgr2=0x7fa7ac115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b08c3700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7ac0fb570 0x7fa7ac115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.719+0000 7fa7b08c3700 1 -- 172.21.15.19:0/353402722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8005040 con 0x7fa7ac0fde50 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.720+0000 7fa7b08c3700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac1160c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.720+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a8021070 con 0x7fa7ac0fde50 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.720+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7a801c400 con 0x7fa7ac0fde50 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.720+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac10fcc0 con 0x7fa7ac0fde50 2024-02-22T16:01:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.720+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a800e070 con 0x7fa7ac0fde50 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.720+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1101a0 con 0x7fa7ac0fde50 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.721+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac107420 con 0x7fa7ac0fde50 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.722+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa7a801c970 con 0x7fa7ac0fde50 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.722+0000 7fa7a1ffb700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa79806cc20 0x7fa79806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.723+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa7a808d8b0 con 0x7fa7ac0fde50 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.723+0000 7fa7b10c4700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa79806cc20 0x7fa79806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.723+0000 7fa7b10c4700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa79806cc20 0x7fa79806f0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009b80 tx=0x7fa79c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.727+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa7a805b410 con 0x7fa7ac0fde50 2024-02-22T16:01:55.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.949+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7ac10c200 con 0x7fa79806cc20 2024-02-22T16:01:55.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.950+0000 7fa7a1ffb700 1 -- 172.21.15.19:0/353402722 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa7ac10c200 con 0x7fa79806cc20 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa79806cc20 msgr2=0x7fa79806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa79806cc20 0x7fa79806f0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009b80 tx=0x7fa79c00b040 comp rx=0 tx=0).stop 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 msgr2=0x7fa7ac1160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac1160c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 shutdown_connections 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa79806cc20 0x7fa79806f0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7ac0fb570 0x7fa7ac115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 --2- 172.21.15.19:0/353402722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7ac0fde50 0x7fa7ac1160c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 >> 172.21.15.19:0/353402722 conn(0x7fa7ac071560 msgr2=0x7fa7ac0f9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 shutdown_connections 2024-02-22T16:01:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:55.952+0000 7fa7b3328700 1 -- 172.21.15.19:0/353402722 wait complete. 2024-02-22T16:01:55.940 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:01:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 -- 172.21.15.19:0/2934991539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9741038f0 msgr2=0x7fb9740fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2934991539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9741038f0 0x7fb9740fcf60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb970004940 tx=0x7fb970009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 -- 172.21.15.19:0/2934991539 shutdown_connections 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2934991539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9741038f0 0x7fb9740fcf60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2934991539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb974102f50 0x7fb974103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 -- 172.21.15.19:0/2934991539 >> 172.21.15.19:0/2934991539 conn(0x7fb9740f8ae0 msgr2=0x7fb9740faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 -- 172.21.15.19:0/2934991539 shutdown_connections 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.058+0000 7fb97bc60700 1 -- 172.21.15.19:0/2934991539 wait complete. 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb97bc60700 1 Processor -- start 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb97bc60700 1 -- start start 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb97bc60700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 0x7fb974070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb97bc60700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9741038f0 0x7fb9740707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb97bc60700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974070e40 con 0x7fb974102f50 2024-02-22T16:01:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb97bc60700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974070fb0 con 0x7fb9741038f0 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 0x7fb974070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 0x7fb974070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49526/0 (socket says 172.21.15.19:49526) 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 -- 172.21.15.19:0/2430332150 learned_addr learned my addr 172.21.15.19:0/2430332150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9791fb700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9741038f0 0x7fb9740707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 -- 172.21.15.19:0/2430332150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9741038f0 msgr2=0x7fb9740707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9741038f0 0x7fb9740707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 -- 172.21.15.19:0/2430332150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970005040 con 0x7fb974102f50 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9791fb700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9741038f0 0x7fb9740707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.059+0000 7fb9799fc700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 0x7fb974070290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb964002a50 tx=0x7fb9640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.060+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb964007cc0 con 0x7fb974102f50 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.060+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb96400f040 con 0x7fb974102f50 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.060+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb974119be0 con 0x7fb974102f50 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.060+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb964009540 con 0x7fb974102f50 2024-02-22T16:01:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.060+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97411a120 con 0x7fb974102f50 2024-02-22T16:01:56.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.061+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb974105420 con 0x7fb974102f50 2024-02-22T16:01:56.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.062+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb9640096a0 con 0x7fb974102f50 2024-02-22T16:01:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.062+0000 7fb96a7fc700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb96006cc20 0x7fb96006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.063+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb96408c4b0 con 0x7fb974102f50 2024-02-22T16:01:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.063+0000 7fb9791fb700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb96006cc20 0x7fb96006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.063+0000 7fb9791fb700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb96006cc20 0x7fb96006f0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb97000b3b0 tx=0x7fb970009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.066+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb9640564c0 con 0x7fb974102f50 2024-02-22T16:01:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.287+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9740719d0 con 0x7fb96006cc20 2024-02-22T16:01:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.288+0000 7fb96a7fc700 1 -- 172.21.15.19:0/2430332150 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb9740719d0 con 0x7fb96006cc20 2024-02-22T16:01:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb96006cc20 msgr2=0x7fb96006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb96006cc20 0x7fb96006f0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb97000b3b0 tx=0x7fb970009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 msgr2=0x7fb974070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 0x7fb974070290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb964002a50 tx=0x7fb9640046b0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 shutdown_connections 2024-02-22T16:01:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb96006cc20 0x7fb96006f0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974102f50 0x7fb974070290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 --2- 172.21.15.19:0/2430332150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9741038f0 0x7fb9740707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.290+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 >> 172.21.15.19:0/2430332150 conn(0x7fb9740f8ae0 msgr2=0x7fb9740fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.291+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 shutdown_connections 2024-02-22T16:01:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.291+0000 7fb97bc60700 1 -- 172.21.15.19:0/2430332150 wait complete. 2024-02-22T16:01:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/1521393090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db8101e40 msgr2=0x7f8db8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/1521393090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db8101e40 0x7f8db8102210 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8db4004940 tx=0x7f8db4009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/1521393090 shutdown_connections 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/1521393090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db81027e0 0x7f8db80fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/1521393090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db8101e40 0x7f8db8102210 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/1521393090 >> 172.21.15.19:0/1521393090 conn(0x7f8db8071d90 msgr2=0x7f8db8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/1521393090 shutdown_connections 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/1521393090 wait complete. 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 Processor -- start 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.394+0000 7f8dbf4f5700 1 -- start start 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbf4f5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8101e40 0x7f8db8113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbf4f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 0x7f8db8113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbf4f5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db81144a0 con 0x7f8db81027e0 2024-02-22T16:01:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbf4f5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db810d7d0 con 0x7f8db8101e40 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 0x7f8db8113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 0x7f8db8113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49540/0 (socket says 172.21.15.19:49540) 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 -- 172.21.15.19:0/3099971116 learned_addr learned my addr 172.21.15.19:0/3099971116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbd291700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8101e40 0x7f8db8113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 -- 172.21.15.19:0/3099971116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8101e40 msgr2=0x7f8db8113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8101e40 0x7f8db8113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 -- 172.21.15.19:0/3099971116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4005040 con 0x7f8db81027e0 2024-02-22T16:01:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbca90700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 0x7f8db8113e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8da800eb40 tx=0x7f8da800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da8007cc0 con 0x7f8db81027e0 2024-02-22T16:01:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8da800f040 con 0x7f8db81027e0 2024-02-22T16:01:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db810da50 con 0x7f8db81027e0 2024-02-22T16:01:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.395+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da800ca30 con 0x7f8db81027e0 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.396+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db810df90 con 0x7f8db81027e0 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.397+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db80738b0 con 0x7f8db81027e0 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.397+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8da800cb90 con 0x7f8db81027e0 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.397+0000 7f8dadffb700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8da406cc20 0x7f8da406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.398+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8da808bc00 con 0x7f8db81027e0 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.398+0000 7f8dbd291700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8da406cc20 0x7f8da406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.398+0000 7f8dbd291700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8da406cc20 0x7f8da406f0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8db4000c00 tx=0x7f8db4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.402+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8da8091020 con 0x7f8db81027e0 2024-02-22T16:01:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.616+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8db804bed0 con 0x7f8da406cc20 2024-02-22T16:01:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.626+0000 7f8dadffb700 1 -- 172.21.15.19:0/3099971116 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f8db804bed0 con 0x7f8da406cc20 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (9m) 2m ago 20m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (6m) 2m ago 20m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (6m) 118s ago 19m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (8m) 2m ago 19m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (2m) 2m ago 14m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (2m) 2m ago 14m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (2m) 118s ago 14m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (2m) 118s ago 14m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (10m) 2m ago 21m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (10m) 118s ago 19m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (7m) 2m ago 21m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (7m) 118s ago 19m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:01:56.604 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (10m) 2m ago 20m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (9m) 118s ago 19m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (6m) 2m ago 18m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (5m) 2m ago 17m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (4m) 2m ago 17m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (4m) 118s ago 16m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (3m) 118s ago 16m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (3m) 118s ago 16m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:01:56.605 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (9m) 2m ago 19m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8da406cc20 msgr2=0x7f8da406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8da406cc20 0x7f8da406f0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8db4000c00 tx=0x7f8db4009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 msgr2=0x7f8db8113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 0x7f8db8113e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8da800eb40 tx=0x7f8da800b040 comp rx=0 tx=0).stop 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 shutdown_connections 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8da406cc20 0x7f8da406f0d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8db8101e40 0x7f8db8113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 --2- 172.21.15.19:0/3099971116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8db81027e0 0x7f8db8113e50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 >> 172.21.15.19:0/3099971116 conn(0x7f8db8071d90 msgr2=0x7f8db8105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 shutdown_connections 2024-02-22T16:01:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.628+0000 7f8dbf4f5700 1 -- 172.21.15.19:0/3099971116 wait complete. 2024-02-22T16:01:56.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/2850375819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f48070be0 msgr2=0x7f1f481092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/2850375819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f48070be0 0x7f1f481092c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004970 tx=0x7f1f38009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/2850375819 shutdown_connections 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/2850375819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f48070be0 0x7f1f481092c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/2850375819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f480702d0 0x7f1f480706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/2850375819 >> 172.21.15.19:0/2850375819 conn(0x7f1f480f8ae0 msgr2=0x7f1f480faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/2850375819 shutdown_connections 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.730+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/2850375819 wait complete. 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4eb6c700 1 Processor -- start 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4eb6c700 1 -- start start 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4eb6c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 0x7f1f48115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4eb6c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f48070be0 0x7f1f481160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4eb6c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48116700 con 0x7f1f480702d0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4eb6c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4810fa30 con 0x7f1f48070be0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4c908700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 0x7f1f48115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4c908700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 0x7f1f48115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49550/0 (socket says 172.21.15.19:49550) 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4c908700 1 -- 172.21.15.19:0/3824170528 learned_addr learned my addr 172.21.15.19:0/3824170528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f47fff700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f48070be0 0x7f1f481160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4c908700 1 -- 172.21.15.19:0/3824170528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f48070be0 msgr2=0x7f1f481160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4c908700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f48070be0 0x7f1f481160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.731+0000 7f1f4c908700 1 -- 172.21.15.19:0/3824170528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38005040 con 0x7f1f480702d0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.732+0000 7f1f4c908700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 0x7f1f48115b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f40001d80 tx=0x7f1f40009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.732+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f40007d10 con 0x7f1f480702d0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.732+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4810fd10 con 0x7f1f480702d0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.732+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f40001e80 con 0x7f1f480702d0 2024-02-22T16:01:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.732+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f40014400 con 0x7f1f480702d0 2024-02-22T16:01:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.732+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f48110220 con 0x7f1f480702d0 2024-02-22T16:01:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.733+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f48106980 con 0x7f1f480702d0 2024-02-22T16:01:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.733+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1f4000e960 con 0x7f1f480702d0 2024-02-22T16:01:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.734+0000 7f1f457fa700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f3406cc20 0x7f1f3406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.734+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1f4008bdf0 con 0x7f1f480702d0 2024-02-22T16:01:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.734+0000 7f1f47fff700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f3406cc20 0x7f1f3406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.734+0000 7f1f47fff700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f3406cc20 0x7f1f3406f0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004f50 tx=0x7f1f38009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:56.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:56.738+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1f400598e0 con 0x7f1f480702d0 2024-02-22T16:01:57.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.031+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f4804bed0 con 0x7f1f480702d0 2024-02-22T16:01:57.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.032+0000 7f1f457fa700 1 -- 172.21.15.19:0/3824170528 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f1f40059470 con 0x7f1f480702d0 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:01:57.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:01:57.011 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:01:57.011 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:01:57.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:01:57.011 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:01:57.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f3406cc20 msgr2=0x7f1f3406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f3406cc20 0x7f1f3406f0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1f38004f50 tx=0x7f1f38009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 msgr2=0x7f1f48115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 0x7f1f48115b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1f40001d80 tx=0x7f1f40009ec0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 shutdown_connections 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1f3406cc20 0x7f1f3406f0d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f480702d0 0x7f1f48115b70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 --2- 172.21.15.19:0/3824170528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f48070be0 0x7f1f481160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 >> 172.21.15.19:0/3824170528 conn(0x7f1f480f8ae0 msgr2=0x7f1f48103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 shutdown_connections 2024-02-22T16:01:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.034+0000 7f1f4eb6c700 1 -- 172.21.15.19:0/3824170528 wait complete. 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 -- 172.21.15.19:0/184939907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4102600 msgr2=0x7f85a40fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 --2- 172.21.15.19:0/184939907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4102600 0x7f85a40fbc70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8598004940 tx=0x7f8598009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 -- 172.21.15.19:0/184939907 shutdown_connections 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 --2- 172.21.15.19:0/184939907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4102600 0x7f85a40fbc70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 --2- 172.21.15.19:0/184939907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4101c60 0x7f85a4102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 -- 172.21.15.19:0/184939907 >> 172.21.15.19:0/184939907 conn(0x7f85a4073ab0 msgr2=0x7f85a4073eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 -- 172.21.15.19:0/184939907 shutdown_connections 2024-02-22T16:01:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 -- 172.21.15.19:0/184939907 wait complete. 2024-02-22T16:01:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.138+0000 7f85a959d700 1 Processor -- start 2024-02-22T16:01:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a959d700 1 -- start start 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a959d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 0x7f85a4115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a959d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4102600 0x7f85a4115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a959d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4116500 con 0x7f85a4101c60 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a959d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a410f830 con 0x7f85a4102600 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 0x7f85a4115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 0x7f85a4115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49566/0 (socket says 172.21.15.19:49566) 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 -- 172.21.15.19:0/3230339043 learned_addr learned my addr 172.21.15.19:0/3230339043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a37fe700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4102600 0x7f85a4115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 -- 172.21.15.19:0/3230339043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4102600 msgr2=0x7f85a4115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4102600 0x7f85a4115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 -- 172.21.15.19:0/3230339043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598005040 con 0x7f85a4101c60 2024-02-22T16:01:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.139+0000 7f85a3fff700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 0x7f85a4115970 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8594002a50 tx=0x7f85940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.140+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8594007cc0 con 0x7f85a4101c60 2024-02-22T16:01:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.140+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a410fb10 con 0x7f85a4101c60 2024-02-22T16:01:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.140+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f859400f040 con 0x7f85a4101c60 2024-02-22T16:01:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.140+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f859400d9c0 con 0x7f85a4101c60 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.140+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a41100a0 con 0x7f85a4101c60 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.141+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a405f5c0 con 0x7f85a4101c60 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.141+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f859400db20 con 0x7f85a4101c60 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.142+0000 7f85a0ff9700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f858c06cc20 0x7f858c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.142+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f859408be20 con 0x7f85a4101c60 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.142+0000 7f85a37fe700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f858c06cc20 0x7f858c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.142+0000 7f85a37fe700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f858c06cc20 0x7f858c06f0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f859800b3b0 tx=0x7f8598009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:57.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.146+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f85940597c0 con 0x7f85a4101c60 2024-02-22T16:01:57.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:56 smithi019 ceph-mon[173579]: pgmap v260: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:57.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:56 smithi019 ceph-mon[173579]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.394+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f85a4110f50 con 0x7f85a4101c60 2024-02-22T16:01:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.394+0000 7f85a0ff9700 1 -- 172.21.15.19:0/3230339043 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f8594059350 con 0x7f85a4101c60 2024-02-22T16:01:57.372 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:01:57.372 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:01:57.373 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:01:57.374 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:01:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f858c06cc20 msgr2=0x7f858c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f858c06cc20 0x7f858c06f0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f859800b3b0 tx=0x7f8598009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 msgr2=0x7f85a4115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 0x7f85a4115970 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8594002a50 tx=0x7f85940046b0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 shutdown_connections 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f858c06cc20 0x7f858c06f0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85a4101c60 0x7f85a4115970 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 --2- 172.21.15.19:0/3230339043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85a4102600 0x7f85a4115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 >> 172.21.15.19:0/3230339043 conn(0x7f85a4073ab0 msgr2=0x7f85a4065740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 shutdown_connections 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.397+0000 7f85a959d700 1 -- 172.21.15.19:0/3230339043 wait complete. 2024-02-22T16:01:57.378 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:01:57.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:56 smithi062 ceph-mon[106861]: pgmap v260: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:57.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:56 smithi062 ceph-mon[106861]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.501+0000 7f893d9fa700 1 -- 172.21.15.19:0/3962206127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fdfc0 msgr2=0x7f89380fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.501+0000 7f893d9fa700 1 --2- 172.21.15.19:0/3962206127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fdfc0 0x7f89380fe390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8928004940 tx=0x7f8928009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.501+0000 7f893d9fa700 1 -- 172.21.15.19:0/3962206127 shutdown_connections 2024-02-22T16:01:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.501+0000 7f893d9fa700 1 --2- 172.21.15.19:0/3962206127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89380fe960 0x7f8938106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.502+0000 7f893d9fa700 1 --2- 172.21.15.19:0/3962206127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fdfc0 0x7f89380fe390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.502+0000 7f893d9fa700 1 -- 172.21.15.19:0/3962206127 >> 172.21.15.19:0/3962206127 conn(0x7f89380f7a10 msgr2=0x7f89380f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.502+0000 7f893d9fa700 1 -- 172.21.15.19:0/3962206127 shutdown_connections 2024-02-22T16:01:57.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.502+0000 7f893d9fa700 1 -- 172.21.15.19:0/3962206127 wait complete. 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.502+0000 7f893d9fa700 1 Processor -- start 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f893d9fa700 1 -- start start 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f893d9fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89380fdfc0 0x7f89380fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f893d9fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 0x7f89380fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f893d9fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380fc930 con 0x7f89380fe960 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f893d9fa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380fcaa0 con 0x7f89380fdfc0 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 0x7f89380fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 0x7f89380fc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59948/0 (socket says 172.21.15.19:59948) 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 -- 172.21.15.19:0/1993195057 learned_addr learned my addr 172.21.15.19:0/1993195057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f8936ffd700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89380fdfc0 0x7f89380fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 -- 172.21.15.19:0/1993195057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89380fdfc0 msgr2=0x7f89380fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89380fdfc0 0x7f89380fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 -- 172.21.15.19:0/1993195057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928005040 con 0x7f89380fe960 2024-02-22T16:01:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.503+0000 7f89367fc700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 0x7f89380fc330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c830 tx=0x7f892c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.504+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892c0106f0 con 0x7f89380fe960 2024-02-22T16:01:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.504+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8938119c40 con 0x7f89380fe960 2024-02-22T16:01:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.504+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f892c010d00 con 0x7f89380fe960 2024-02-22T16:01:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.504+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892c012e90 con 0x7f89380fe960 2024-02-22T16:01:57.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.504+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893811a120 con 0x7f89380fe960 2024-02-22T16:01:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.505+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89381045b0 con 0x7f89380fe960 2024-02-22T16:01:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.505+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f892c010850 con 0x7f89380fe960 2024-02-22T16:01:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.506+0000 7f891f7fe700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f892006cc20 0x7f892006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.506+0000 7f8936ffd700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f892006cc20 0x7f892006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.506+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f892c08c000 con 0x7f89380fe960 2024-02-22T16:01:57.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.506+0000 7f8936ffd700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f892006cc20 0x7f892006f0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8928004890 tx=0x7f8928009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:57.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.510+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f892c056030 con 0x7f89380fe960 2024-02-22T16:01:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.731+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89380fd530 con 0x7f892006cc20 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.732+0000 7f891f7fe700 1 -- 172.21.15.19:0/1993195057 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f89380fd530 con 0x7f892006cc20 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:01:57.710 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:01:57.711 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:01:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.734+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f892006cc20 msgr2=0x7f892006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.734+0000 7f893d9fa700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f892006cc20 0x7f892006f0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8928004890 tx=0x7f8928009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.734+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 msgr2=0x7f89380fc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.734+0000 7f893d9fa700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 0x7f89380fc330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c830 tx=0x7f892c004470 comp rx=0 tx=0).stop 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 shutdown_connections 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f892006cc20 0x7f892006f0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89380fdfc0 0x7f89380fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 --2- 172.21.15.19:0/1993195057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89380fe960 0x7f89380fc330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 >> 172.21.15.19:0/1993195057 conn(0x7f89380f7a10 msgr2=0x7f8938101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 shutdown_connections 2024-02-22T16:01:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.735+0000 7f893d9fa700 1 -- 172.21.15.19:0/1993195057 wait complete. 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 -- 172.21.15.19:0/534625853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c1027e0 msgr2=0x7feb4c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 --2- 172.21.15.19:0/534625853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c1027e0 0x7feb4c0fbea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb3c004940 tx=0x7feb3c009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 -- 172.21.15.19:0/534625853 shutdown_connections 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 --2- 172.21.15.19:0/534625853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c1027e0 0x7feb4c0fbea0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 --2- 172.21.15.19:0/534625853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c101e40 0x7feb4c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 -- 172.21.15.19:0/534625853 >> 172.21.15.19:0/534625853 conn(0x7feb4c071d90 msgr2=0x7feb4c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 -- 172.21.15.19:0/534625853 shutdown_connections 2024-02-22T16:01:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 -- 172.21.15.19:0/534625853 wait complete. 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.838+0000 7feb53bce700 1 Processor -- start 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb53bce700 1 -- start start 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb53bce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 0x7feb4c1128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb53bce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c1027e0 0x7feb4c112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb53bce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c113460 con 0x7feb4c101e40 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb53bce700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c10c790 con 0x7feb4c1027e0 2024-02-22T16:01:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 0x7feb4c1128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 0x7feb4c1128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59956/0 (socket says 172.21.15.19:59956) 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 -- 172.21.15.19:0/1236477367 learned_addr learned my addr 172.21.15.19:0/1236477367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb51169700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c1027e0 0x7feb4c112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 -- 172.21.15.19:0/1236477367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c1027e0 msgr2=0x7feb4c112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c1027e0 0x7feb4c112e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 -- 172.21.15.19:0/1236477367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c005040 con 0x7feb4c101e40 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.839+0000 7feb5196a700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 0x7feb4c1128d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb4800ead0 tx=0x7feb48004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.840+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4800cb90 con 0x7feb4c101e40 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.840+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb4800ccf0 con 0x7feb4c101e40 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.840+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c10ca10 con 0x7feb4c101e40 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.840+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb48010810 con 0x7feb4c101e40 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.840+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c10cf20 con 0x7feb4c101e40 2024-02-22T16:01:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.841+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c0738b0 con 0x7feb4c101e40 2024-02-22T16:01:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.841+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feb48010990 con 0x7feb4c101e40 2024-02-22T16:01:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.841+0000 7feb427fc700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feb3806cc30 0x7feb3806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:01:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.841+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7feb4808e8b0 con 0x7feb4c101e40 2024-02-22T16:01:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.842+0000 7feb51169700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feb3806cc30 0x7feb3806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:01:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.842+0000 7feb51169700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feb3806cc30 0x7feb3806f0e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feb4c10dac0 tx=0x7feb3c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:01:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:57.846+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7feb4805c370 con 0x7feb4c101e40 2024-02-22T16:01:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.139+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb4c10deb0 con 0x7feb4c101e40 2024-02-22T16:01:58.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.139+0000 7feb427fc700 1 -- 172.21.15.19:0/1236477367 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7feb4800ce60 con 0x7feb4c101e40 2024-02-22T16:01:58.117 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:01:58.118 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:01:58.118 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:01:58.118 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:01:58.118 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feb3806cc30 msgr2=0x7feb3806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feb3806cc30 0x7feb3806f0e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feb4c10dac0 tx=0x7feb3c009e30 comp rx=0 tx=0).stop 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 msgr2=0x7feb4c1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 0x7feb4c1128d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb4800ead0 tx=0x7feb48004500 comp rx=0 tx=0).stop 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 shutdown_connections 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feb3806cc30 0x7feb3806f0e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb4c101e40 0x7feb4c1128d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 --2- 172.21.15.19:0/1236477367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb4c1027e0 0x7feb4c112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:01:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 >> 172.21.15.19:0/1236477367 conn(0x7feb4c071d90 msgr2=0x7feb4c0faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:01:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 shutdown_connections 2024-02-22T16:01:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:01:58.142+0000 7feb53bce700 1 -- 172.21.15.19:0/1236477367 wait complete. 2024-02-22T16:01:58.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:57 smithi019 ceph-mon[173579]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:58.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:57 smithi019 ceph-mon[173579]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:58.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:57 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3824170528' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:01:58.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:57 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3230339043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:01:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:57 smithi062 ceph-mon[106861]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:57 smithi062 ceph-mon[106861]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:57 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3824170528' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:01:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:57 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3230339043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:01:59.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:58 smithi019 ceph-mon[173579]: pgmap v261: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:59.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:58 smithi019 ceph-mon[173579]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:59.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:01:58 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1236477367' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:01:59.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:58 smithi062 ceph-mon[106861]: pgmap v261: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:01:59.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:58 smithi062 ceph-mon[106861]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:01:59.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:01:58 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1236477367' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:02:01.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:00 smithi019 ceph-mon[173579]: pgmap v262: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:01.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:00 smithi062 ceph-mon[106861]: pgmap v262: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:03.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:03 smithi019 ceph-mon[173579]: pgmap v263: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:03.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:03 smithi062 ceph-mon[106861]: pgmap v263: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:04.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:04.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:05.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:05 smithi062 ceph-mon[106861]: pgmap v264: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:05.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:05 smithi019 ceph-mon[173579]: pgmap v264: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:07.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:07 smithi062 ceph-mon[106861]: pgmap v265: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:07.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:07 smithi019 ceph-mon[173579]: pgmap v265: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:09.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:09 smithi062 ceph-mon[106861]: pgmap v266: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:09.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:09 smithi019 ceph-mon[173579]: pgmap v266: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:11.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:11 smithi019 ceph-mon[173579]: pgmap v267: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:11.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:11 smithi062 ceph-mon[106861]: pgmap v267: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:13.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:13 smithi062 ceph-mon[106861]: pgmap v268: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:13.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:13 smithi019 ceph-mon[173579]: pgmap v268: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:15.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:15 smithi062 ceph-mon[106861]: pgmap v269: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:15.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:15 smithi019 ceph-mon[173579]: pgmap v269: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:17.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:17 smithi062 ceph-mon[106861]: pgmap v270: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:17.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:17 smithi019 ceph-mon[173579]: pgmap v270: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:19 smithi062 ceph-mon[106861]: pgmap v271: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:19.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:19 smithi019 ceph-mon[173579]: pgmap v271: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:19.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:21.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:21 smithi019 ceph-mon[173579]: pgmap v272: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:21.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:21 smithi062 ceph-mon[106861]: pgmap v272: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:23.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:23 smithi062 ceph-mon[106861]: pgmap v273: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:23.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:23 smithi019 ceph-mon[173579]: pgmap v273: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:25.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:25 smithi062 ceph-mon[106861]: pgmap v274: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:25.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:25 smithi019 ceph-mon[173579]: pgmap v274: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:27.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:27 smithi062 ceph-mon[106861]: pgmap v275: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:27.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:27 smithi019 ceph-mon[173579]: pgmap v275: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 -- 172.21.15.19:0/892752624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b41027e0 msgr2=0x7f29b40fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/892752624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b41027e0 0x7f29b40fbea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f29a8004940 tx=0x7f29a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 -- 172.21.15.19:0/892752624 shutdown_connections 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/892752624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b41027e0 0x7f29b40fbea0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/892752624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b4101e40 0x7f29b4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 -- 172.21.15.19:0/892752624 >> 172.21.15.19:0/892752624 conn(0x7f29b4071d90 msgr2=0x7f29b4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.259+0000 7f29b9a7a700 1 -- 172.21.15.19:0/892752624 shutdown_connections 2024-02-22T16:02:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.260+0000 7f29b9a7a700 1 -- 172.21.15.19:0/892752624 wait complete. 2024-02-22T16:02:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.260+0000 7f29b9a7a700 1 Processor -- start 2024-02-22T16:02:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.260+0000 7f29b9a7a700 1 -- start start 2024-02-22T16:02:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b9a7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 0x7f29b40fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b9a7a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b41027e0 0x7f29b40fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b9a7a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40fe590 con 0x7f29b4101e40 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b9a7a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40fe700 con 0x7f29b41027e0 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 0x7f29b40fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 0x7f29b40fdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38446/0 (socket says 172.21.15.19:38446) 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 -- 172.21.15.19:0/3884248387 learned_addr learned my addr 172.21.15.19:0/3884248387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b2ffd700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b41027e0 0x7f29b40fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 -- 172.21.15.19:0/3884248387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b41027e0 msgr2=0x7f29b40fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b41027e0 0x7f29b40fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 -- 172.21.15.19:0/3884248387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8005040 con 0x7f29b4101e40 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b37fe700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 0x7f29b40fdac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f29a4002a50 tx=0x7f29a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29a4007cc0 con 0x7f29b4101e40 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29a400f040 con 0x7f29b4101e40 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.261+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b4119c80 con 0x7f29b4101e40 2024-02-22T16:02:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.262+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29a400d9c0 con 0x7f29b4101e40 2024-02-22T16:02:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.262+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b411a0e0 con 0x7f29b4101e40 2024-02-22T16:02:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.263+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b40738b0 con 0x7f29b4101e40 2024-02-22T16:02:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.263+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f29a400db20 con 0x7f29b4101e40 2024-02-22T16:02:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.264+0000 7f299bfff700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f299c06cc20 0x7f299c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.264+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f29a408bdc0 con 0x7f29b4101e40 2024-02-22T16:02:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.264+0000 7f29b2ffd700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f299c06cc20 0x7f299c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.264+0000 7f29b2ffd700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f299c06cc20 0x7f299c06f0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29a8004890 tx=0x7f29a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.268+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f29a4055d70 con 0x7f29b4101e40 2024-02-22T16:02:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.490+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29b40ff270 con 0x7f299c06cc20 2024-02-22T16:02:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.491+0000 7f299bfff700 1 -- 172.21.15.19:0/3884248387 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f29b40ff270 con 0x7f299c06cc20 2024-02-22T16:02:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f299c06cc20 msgr2=0x7f299c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f299c06cc20 0x7f299c06f0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29a8004890 tx=0x7f29a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 msgr2=0x7f29b40fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 0x7f29b40fdac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f29a4002a50 tx=0x7f29a40046b0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 shutdown_connections 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f299c06cc20 0x7f299c06f0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b4101e40 0x7f29b40fdac0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 --2- 172.21.15.19:0/3884248387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b41027e0 0x7f29b40fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 >> 172.21.15.19:0/3884248387 conn(0x7f29b4071d90 msgr2=0x7f29b4065820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 shutdown_connections 2024-02-22T16:02:28.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.493+0000 7f29b9a7a700 1 -- 172.21.15.19:0/3884248387 wait complete. 2024-02-22T16:02:28.481 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 -- 172.21.15.19:0/1131337275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b34101e40 msgr2=0x7f4b34102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1131337275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b34101e40 0x7f4b34102210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b24004940 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 -- 172.21.15.19:0/1131337275 shutdown_connections 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1131337275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b341027e0 0x7f4b340fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1131337275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b34101e40 0x7f4b34102210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 -- 172.21.15.19:0/1131337275 >> 172.21.15.19:0/1131337275 conn(0x7f4b34071d90 msgr2=0x7f4b34072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 -- 172.21.15.19:0/1131337275 shutdown_connections 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.597+0000 7f4b3922c700 1 -- 172.21.15.19:0/1131337275 wait complete. 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3922c700 1 Processor -- start 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3922c700 1 -- start start 2024-02-22T16:02:28.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3922c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b34101e40 0x7f4b34113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3922c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 0x7f4b34113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3922c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b341144a0 con 0x7f4b341027e0 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3922c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3410d7d0 con 0x7f4b34101e40 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 0x7f4b34113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 0x7f4b34113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38470/0 (socket says 172.21.15.19:38470) 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.598+0000 7f4b3259c700 1 -- 172.21.15.19:0/1404441371 learned_addr learned my addr 172.21.15.19:0/1404441371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b32d9d700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b34101e40 0x7f4b34113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b3259c700 1 -- 172.21.15.19:0/1404441371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b34101e40 msgr2=0x7f4b34113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b3259c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b34101e40 0x7f4b34113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b3259c700 1 -- 172.21.15.19:0/1404441371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24005040 con 0x7f4b341027e0 2024-02-22T16:02:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b3259c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 0x7f4b34113e50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b28002a50 tx=0x7f4b280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b28007cc0 con 0x7f4b341027e0 2024-02-22T16:02:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b2800f040 con 0x7f4b341027e0 2024-02-22T16:02:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3410dab0 con 0x7f4b341027e0 2024-02-22T16:02:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2800da30 con 0x7f4b341027e0 2024-02-22T16:02:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.599+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3410dff0 con 0x7f4b341027e0 2024-02-22T16:02:28.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.600+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b340738b0 con 0x7f4b341027e0 2024-02-22T16:02:28.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.601+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4b2800db90 con 0x7f4b341027e0 2024-02-22T16:02:28.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.601+0000 7f4b1b7fe700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4b1c06cc20 0x7f4b1c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.601+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4b2808bca0 con 0x7f4b341027e0 2024-02-22T16:02:28.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.601+0000 7f4b32d9d700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4b1c06cc20 0x7f4b1c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.601+0000 7f4b32d9d700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4b1c06cc20 0x7f4b1c06f0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b24004f90 tx=0x7f4b24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:28.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.606+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4b28090070 con 0x7f4b341027e0 2024-02-22T16:02:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.828+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b3404bed0 con 0x7f4b1c06cc20 2024-02-22T16:02:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.829+0000 7f4b1b7fe700 1 -- 172.21.15.19:0/1404441371 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4b3404bed0 con 0x7f4b1c06cc20 2024-02-22T16:02:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4b1c06cc20 msgr2=0x7f4b1c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4b1c06cc20 0x7f4b1c06f0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b24004f90 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 msgr2=0x7f4b34113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 0x7f4b34113e50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b28002a50 tx=0x7f4b280046b0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 shutdown_connections 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4b1c06cc20 0x7f4b1c06f0d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b34101e40 0x7f4b34113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 --2- 172.21.15.19:0/1404441371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b341027e0 0x7f4b34113e50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 >> 172.21.15.19:0/1404441371 conn(0x7f4b34071d90 msgr2=0x7f4b34065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 shutdown_connections 2024-02-22T16:02:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.831+0000 7f4b3922c700 1 -- 172.21.15.19:0/1404441371 wait complete. 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 -- 172.21.15.19:0/2009995519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8101ee0 msgr2=0x7f69d81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 --2- 172.21.15.19:0/2009995519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8101ee0 0x7f69d81022b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69c8004940 tx=0x7f69c8009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 -- 172.21.15.19:0/2009995519 shutdown_connections 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 --2- 172.21.15.19:0/2009995519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d8102880 0x7f69d80fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 --2- 172.21.15.19:0/2009995519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8101ee0 0x7f69d81022b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 -- 172.21.15.19:0/2009995519 >> 172.21.15.19:0/2009995519 conn(0x7f69d8071560 msgr2=0x7f69d8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 -- 172.21.15.19:0/2009995519 shutdown_connections 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.934+0000 7f69de211700 1 -- 172.21.15.19:0/2009995519 wait complete. 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69de211700 1 Processor -- start 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69de211700 1 -- start start 2024-02-22T16:02:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69de211700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d8101ee0 0x7f69d8115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69de211700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 0x7f69d8116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69de211700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d81166e0 con 0x7f69d8102880 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69de211700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d810fa10 con 0x7f69d8101ee0 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 0x7f69d8116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69d6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 0x7f69d8116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38480/0 (socket says 172.21.15.19:38480) 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.935+0000 7f69d6ffd700 1 -- 172.21.15.19:0/2164727260 learned_addr learned my addr 172.21.15.19:0/2164727260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69d77fe700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d8101ee0 0x7f69d8115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69d6ffd700 1 -- 172.21.15.19:0/2164727260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d8101ee0 msgr2=0x7f69d8115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69d6ffd700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d8101ee0 0x7f69d8115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69d6ffd700 1 -- 172.21.15.19:0/2164727260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c8005040 con 0x7f69d8102880 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69d6ffd700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 0x7f69d8116090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00eb40 tx=0x7f69cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69cc007cc0 con 0x7f69d8102880 2024-02-22T16:02:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d810fc90 con 0x7f69d8102880 2024-02-22T16:02:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69cc00f040 con 0x7f69d8102880 2024-02-22T16:02:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69cc00c8f0 con 0x7f69d8102880 2024-02-22T16:02:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.936+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d81101d0 con 0x7f69d8102880 2024-02-22T16:02:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.937+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d8111130 con 0x7f69d8102880 2024-02-22T16:02:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.938+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f69cc00cb60 con 0x7f69d8102880 2024-02-22T16:02:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.938+0000 7f69bffff700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69c006cc20 0x7f69c006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.938+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f69cc08baf0 con 0x7f69d8102880 2024-02-22T16:02:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.938+0000 7f69d77fe700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69c006cc20 0x7f69c006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.939+0000 7f69d77fe700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69c006cc20 0x7f69c006f0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f69c8000c00 tx=0x7f69c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:28.943+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f69cc0595e0 con 0x7f69d8102880 2024-02-22T16:02:29.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.158+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69d80730d0 con 0x7f69c006cc20 2024-02-22T16:02:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.168+0000 7f69bffff700 1 -- 172.21.15.19:0/2164727260 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f69d80730d0 con 0x7f69c006cc20 2024-02-22T16:02:29.145 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:02:29.145 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (9m) 3m ago 20m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:02:29.145 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (7m) 3m ago 21m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:02:29.145 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (6m) 2m ago 19m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:02:29.145 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (9m) 3m ago 20m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (3m) 3m ago 14m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (3m) 3m ago 15m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (2m) 2m ago 14m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (2m) 2m ago 14m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (11m) 3m ago 22m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (10m) 2m ago 19m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (8m) 3m ago 22m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (7m) 2m ago 19m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (10m) 3m ago 20m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (10m) 2m ago 19m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (6m) 3m ago 18m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (6m) 3m ago 18m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (5m) 3m ago 17m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (4m) 2m ago 17m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:02:29.146 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (4m) 2m ago 17m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:02:29.147 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (3m) 2m ago 16m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:02:29.147 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (10m) 3m ago 20m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:02:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69c006cc20 msgr2=0x7f69c006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69c006cc20 0x7f69c006f0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f69c8000c00 tx=0x7f69c8009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 msgr2=0x7f69d8116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 0x7f69d8116090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00eb40 tx=0x7f69cc00b040 comp rx=0 tx=0).stop 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 shutdown_connections 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69c006cc20 0x7f69c006f0d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d8101ee0 0x7f69d8115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 --2- 172.21.15.19:0/2164727260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69d8102880 0x7f69d8116090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 >> 172.21.15.19:0/2164727260 conn(0x7f69d8071560 msgr2=0x7f69d80fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 shutdown_connections 2024-02-22T16:02:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.170+0000 7f69de211700 1 -- 172.21.15.19:0/2164727260 wait complete. 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 -- 172.21.15.19:0/1597861185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c103140 msgr2=0x7f985c103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 --2- 172.21.15.19:0/1597861185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c103140 0x7f985c103510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9850004940 tx=0x7f9850009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 -- 172.21.15.19:0/1597861185 shutdown_connections 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 --2- 172.21.15.19:0/1597861185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f985c0fb570 0x7f985c0fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 --2- 172.21.15.19:0/1597861185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c103140 0x7f985c103510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 -- 172.21.15.19:0/1597861185 >> 172.21.15.19:0/1597861185 conn(0x7f985c071560 msgr2=0x7f985c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 -- 172.21.15.19:0/1597861185 shutdown_connections 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 -- 172.21.15.19:0/1597861185 wait complete. 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 Processor -- start 2024-02-22T16:02:29.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.274+0000 7f9861407700 1 -- start start 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f9861407700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 0x7f985c113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f9861407700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f985c103140 0x7f985c113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f9861407700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c1145d0 con 0x7f985c0fb570 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f9861407700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c10d900 con 0x7f985c103140 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 0x7f985c113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 0x7f985c113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38492/0 (socket says 172.21.15.19:38492) 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 -- 172.21.15.19:0/484526738 learned_addr learned my addr 172.21.15.19:0/484526738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985a7fc700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f985c103140 0x7f985c113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 -- 172.21.15.19:0/484526738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f985c103140 msgr2=0x7f985c113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f985c103140 0x7f985c113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 -- 172.21.15.19:0/484526738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9850005040 con 0x7f985c0fb570 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f985affd700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 0x7f985c113a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f985000b3b0 tx=0x7f9850009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985000e040 con 0x7f985c0fb570 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c10db80 con 0x7f985c0fb570 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9850004290 con 0x7f985c0fb570 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985001c3f0 con 0x7f985c0fb570 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.275+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c10e0c0 con 0x7f985c0fb570 2024-02-22T16:02:29.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.276+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c100310 con 0x7f985c0fb570 2024-02-22T16:02:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.277+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9850013790 con 0x7f985c0fb570 2024-02-22T16:02:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.277+0000 7f98437fe700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f984406cc20 0x7f984406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.278+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f985008d370 con 0x7f985c0fb570 2024-02-22T16:02:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.278+0000 7f985a7fc700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f984406cc20 0x7f984406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.278+0000 7f985a7fc700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f984406cc20 0x7f984406f0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f985c10f270 tx=0x7f984c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:29.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.282+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f985005aee0 con 0x7f985c0fb570 2024-02-22T16:02:29.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:29 smithi062 ceph-mon[106861]: pgmap v276: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:29.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:29 smithi019 ceph-mon[173579]: pgmap v276: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.569+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f985c10a100 con 0x7f985c0fb570 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.569+0000 7f98437fe700 1 -- 172.21.15.19:0/484526738 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f9850013420 con 0x7f985c0fb570 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:02:29.547 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:02:29.548 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:02:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f984406cc20 msgr2=0x7f984406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f984406cc20 0x7f984406f0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f985c10f270 tx=0x7f984c00b040 comp rx=0 tx=0).stop 2024-02-22T16:02:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 msgr2=0x7f985c113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 0x7f985c113a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f985000b3b0 tx=0x7f9850009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 shutdown_connections 2024-02-22T16:02:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f984406cc20 0x7f984406f0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985c0fb570 0x7f985c113a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 --2- 172.21.15.19:0/484526738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f985c103140 0x7f985c113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 >> 172.21.15.19:0/484526738 conn(0x7f985c071560 msgr2=0x7f985c0fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 shutdown_connections 2024-02-22T16:02:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.571+0000 7f9861407700 1 -- 172.21.15.19:0/484526738 wait complete. 2024-02-22T16:02:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 -- 172.21.15.19:0/288733456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 msgr2=0x7f7e80102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 --2- 172.21.15.19:0/288733456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80102210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c004940 tx=0x7f7e7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 -- 172.21.15.19:0/288733456 shutdown_connections 2024-02-22T16:02:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 --2- 172.21.15.19:0/288733456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e801027e0 0x7f7e800fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 --2- 172.21.15.19:0/288733456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80102210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 -- 172.21.15.19:0/288733456 >> 172.21.15.19:0/288733456 conn(0x7f7e800f79d0 msgr2=0x7f7e800f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 -- 172.21.15.19:0/288733456 shutdown_connections 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.675+0000 7f7e88481700 1 -- 172.21.15.19:0/288733456 wait complete. 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e88481700 1 Processor -- start 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e88481700 1 -- start start 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e88481700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e88481700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e801027e0 0x7f7e80113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e88481700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e801145b0 con 0x7f7e80101e40 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e88481700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8010d8e0 con 0x7f7e801027e0 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e8621d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e8621d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38500/0 (socket says 172.21.15.19:38500) 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e8621d700 1 -- 172.21.15.19:0/2682030017 learned_addr learned my addr 172.21.15.19:0/2682030017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.676+0000 7f7e85a1c700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e801027e0 0x7f7e80113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e8621d700 1 -- 172.21.15.19:0/2682030017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e801027e0 msgr2=0x7f7e80113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e8621d700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e801027e0 0x7f7e80113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e8621d700 1 -- 172.21.15.19:0/2682030017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c005040 con 0x7f7e80101e40 2024-02-22T16:02:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e8621d700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80113a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00b3b0 tx=0x7f7e7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e7c00e040 con 0x7f7e80101e40 2024-02-22T16:02:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8010db60 con 0x7f7e80101e40 2024-02-22T16:02:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e7c004290 con 0x7f7e80101e40 2024-02-22T16:02:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e7c01c3f0 con 0x7f7e80101e40 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.677+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8010e0a0 con 0x7f7e80101e40 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.678+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e800fb5d0 con 0x7f7e80101e40 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.679+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7e7c013790 con 0x7f7e80101e40 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.679+0000 7f7e72ffd700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e6c06cc20 0x7f7e6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.679+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7e7c08d370 con 0x7f7e80101e40 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.679+0000 7f7e85a1c700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e6c06cc20 0x7f7e6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.680+0000 7f7e85a1c700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e6c06cc20 0x7f7e6c06f0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e8010f250 tx=0x7f7e7400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.684+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7e7c05aee0 con 0x7f7e80101e40 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.932+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7e8010f000 con 0x7f7e80101e40 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.932+0000 7f7e72ffd700 1 -- 172.21.15.19:0/2682030017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f7e7c013420 con 0x7f7e80101e40 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:02:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:02:29.911 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:02:29.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:02:29.913 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:02:29.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:02:29.913 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:02:29.913 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:02:29.913 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:02:29.913 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.934+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e6c06cc20 msgr2=0x7f7e6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.934+0000 7f7e88481700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e6c06cc20 0x7f7e6c06f0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e8010f250 tx=0x7f7e7400b040 comp rx=0 tx=0).stop 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.934+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 msgr2=0x7f7e80113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.934+0000 7f7e88481700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80113a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00b3b0 tx=0x7f7e7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.934+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 shutdown_connections 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.935+0000 7f7e88481700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e6c06cc20 0x7f7e6c06f0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.935+0000 7f7e88481700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e80101e40 0x7f7e80113a20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.935+0000 7f7e88481700 1 --2- 172.21.15.19:0/2682030017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e801027e0 0x7f7e80113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.935+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 >> 172.21.15.19:0/2682030017 conn(0x7f7e800f79d0 msgr2=0x7f7e80066010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.935+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 shutdown_connections 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:29.935+0000 7f7e88481700 1 -- 172.21.15.19:0/2682030017 wait complete. 2024-02-22T16:02:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 -- 172.21.15.19:0/4123090569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 msgr2=0x7f9d841022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 --2- 172.21.15.19:0/4123090569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d841022b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d74004940 tx=0x7f9d74009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 -- 172.21.15.19:0/4123090569 shutdown_connections 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 --2- 172.21.15.19:0/4123090569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d84102880 0x7f9d840fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 --2- 172.21.15.19:0/4123090569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d841022b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 -- 172.21.15.19:0/4123090569 >> 172.21.15.19:0/4123090569 conn(0x7f9d84071560 msgr2=0x7f9d84071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 -- 172.21.15.19:0/4123090569 shutdown_connections 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 -- 172.21.15.19:0/4123090569 wait complete. 2024-02-22T16:02:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.039+0000 7f9d8a571700 1 Processor -- start 2024-02-22T16:02:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d8a571700 1 -- start start 2024-02-22T16:02:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d8a571700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d8410e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d8a571700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d84102880 0x7f9d8410eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d8a571700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8410f470 con 0x7f9d84101ee0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d8a571700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8410f5b0 con 0x7f9d84102880 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d8410e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d8410e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38518/0 (socket says 172.21.15.19:38518) 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 -- 172.21.15.19:0/3831456543 learned_addr learned my addr 172.21.15.19:0/3831456543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d837fe700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d84102880 0x7f9d8410eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 -- 172.21.15.19:0/3831456543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d84102880 msgr2=0x7f9d8410eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d84102880 0x7f9d8410eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 -- 172.21.15.19:0/3831456543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74005040 con 0x7f9d84101ee0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.040+0000 7f9d83fff700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d8410e9a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d74000c00 tx=0x7f9d74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.041+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d74021070 con 0x7f9d84101ee0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.041+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d7401c400 con 0x7f9d84101ee0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.041+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d841178b0 con 0x7f9d84101ee0 2024-02-22T16:02:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.041+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7400e070 con 0x7f9d84101ee0 2024-02-22T16:02:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.041+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d84117d60 con 0x7f9d84101ee0 2024-02-22T16:02:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.042+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d84073080 con 0x7f9d84101ee0 2024-02-22T16:02:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.042+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9d7401c970 con 0x7f9d84101ee0 2024-02-22T16:02:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.043+0000 7f9d80ff9700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9d6c06cc20 0x7f9d6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.043+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9d7408d0a0 con 0x7f9d84101ee0 2024-02-22T16:02:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.043+0000 7f9d837fe700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9d6c06cc20 0x7f9d6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.043+0000 7f9d837fe700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9d6c06cc20 0x7f9d6c06f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d840fc030 tx=0x7f9d7800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.047+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9d74057170 con 0x7f9d84101ee0 2024-02-22T16:02:30.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.269+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d840fac20 con 0x7f9d6c06cc20 2024-02-22T16:02:30.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.270+0000 7f9d80ff9700 1 -- 172.21.15.19:0/3831456543 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9d840fac20 con 0x7f9d6c06cc20 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:02:30.248 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:02:30.249 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:02:30.249 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:02:30.249 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9d6c06cc20 msgr2=0x7f9d6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9d6c06cc20 0x7f9d6c06f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d840fc030 tx=0x7f9d7800b040 comp rx=0 tx=0).stop 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 msgr2=0x7f9d8410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d8410e9a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d74000c00 tx=0x7f9d74009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 shutdown_connections 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9d6c06cc20 0x7f9d6c06f0d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d84101ee0 0x7f9d8410e9a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 --2- 172.21.15.19:0/3831456543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d84102880 0x7f9d8410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 >> 172.21.15.19:0/3831456543 conn(0x7f9d84071560 msgr2=0x7f9d840fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 shutdown_connections 2024-02-22T16:02:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.272+0000 7f9d8a571700 1 -- 172.21.15.19:0/3831456543 wait complete. 2024-02-22T16:02:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 -- 172.21.15.19:0/3582813934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 msgr2=0x7f19e0103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 --2- 172.21.15.19:0/3582813934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0103310 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f19d4004940 tx=0x7f19d4009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 -- 172.21.15.19:0/3582813934 shutdown_connections 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 --2- 172.21.15.19:0/3582813934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e01038e0 0x7f19e00fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 --2- 172.21.15.19:0/3582813934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0103310 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 -- 172.21.15.19:0/3582813934 >> 172.21.15.19:0/3582813934 conn(0x7f19e00f8af0 msgr2=0x7f19e00faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 -- 172.21.15.19:0/3582813934 shutdown_connections 2024-02-22T16:02:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.376+0000 7f19e5136700 1 -- 172.21.15.19:0/3582813934 wait complete. 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19e5136700 1 Processor -- start 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19e5136700 1 -- start start 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19e5136700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19e5136700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e01038e0 0x7f19e00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19e5136700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0070dd0 con 0x7f19e0102f40 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19e5136700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0070f40 con 0x7f19e01038e0 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38524/0 (socket says 172.21.15.19:38524) 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 -- 172.21.15.19:0/452422252 learned_addr learned my addr 172.21.15.19:0/452422252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:02:30.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19de59c700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e01038e0 0x7f19e00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 -- 172.21.15.19:0/452422252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e01038e0 msgr2=0x7f19e00707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e01038e0 0x7f19e00707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 -- 172.21.15.19:0/452422252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4005040 con 0x7f19e0102f40 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.377+0000 7f19ded9d700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0070290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f19d4000c00 tx=0x7f19d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.378+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d4021070 con 0x7f19e0102f40 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.378+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e0119b80 con 0x7f19e0102f40 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.378+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19d401c400 con 0x7f19e0102f40 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.378+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d400e070 con 0x7f19e0102f40 2024-02-22T16:02:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.378+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0119f80 con 0x7f19e0102f40 2024-02-22T16:02:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.379+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e00fc6f0 con 0x7f19e0102f40 2024-02-22T16:02:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.379+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f19d401c970 con 0x7f19e0102f40 2024-02-22T16:02:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.380+0000 7f19c77fe700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19c806cc20 0x7f19c806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:02:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.380+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19d408d1b0 con 0x7f19e0102f40 2024-02-22T16:02:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.380+0000 7f19de59c700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19c806cc20 0x7f19c806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:02:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.380+0000 7f19de59c700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19c806cc20 0x7f19c806f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f19e0071c90 tx=0x7f19d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:02:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.384+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f19d40571e0 con 0x7f19e0102f40 2024-02-22T16:02:30.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:30 smithi062 ceph-mon[106861]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:30.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:30 smithi062 ceph-mon[106861]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:30.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:30 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/484526738' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:02:30.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:30 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2682030017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:02:30.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:30 smithi019 ceph-mon[173579]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:30.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:30 smithi019 ceph-mon[173579]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:30.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:30 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/484526738' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:02:30.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:30 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2682030017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:02:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.670+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f19e011a1a0 con 0x7f19e0102f40 2024-02-22T16:02:30.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.670+0000 7f19c77fe700 1 -- 172.21.15.19:0/452422252 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f19d405a800 con 0x7f19e0102f40 2024-02-22T16:02:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:02:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:02:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:02:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:02:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19c806cc20 msgr2=0x7f19c806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19c806cc20 0x7f19c806f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f19e0071c90 tx=0x7f19d000b040 comp rx=0 tx=0).stop 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 msgr2=0x7f19e0070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0070290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f19d4000c00 tx=0x7f19d4009e30 comp rx=0 tx=0).stop 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 shutdown_connections 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19c806cc20 0x7f19c806f0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e0102f40 0x7f19e0070290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 --2- 172.21.15.19:0/452422252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19e01038e0 0x7f19e00707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:02:30.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.672+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 >> 172.21.15.19:0/452422252 conn(0x7f19e00f8af0 msgr2=0x7f19e00fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:02:30.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.673+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 shutdown_connections 2024-02-22T16:02:30.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:02:30.673+0000 7f19e5136700 1 -- 172.21.15.19:0/452422252 wait complete. 2024-02-22T16:02:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:31 smithi019 ceph-mon[173579]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:31 smithi019 ceph-mon[173579]: pgmap v277: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:31 smithi019 ceph-mon[173579]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:31 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/452422252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:02:31.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:31 smithi062 ceph-mon[106861]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:31.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:31 smithi062 ceph-mon[106861]: pgmap v277: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:31.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:31 smithi062 ceph-mon[106861]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:02:31.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:31 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/452422252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:02:32.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:32 smithi062 ceph-mon[106861]: pgmap v278: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:32.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:32 smithi019 ceph-mon[173579]: pgmap v278: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:34.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:34.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:35.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:34 smithi019 ceph-mon[173579]: pgmap v279: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:34 smithi062 ceph-mon[106861]: pgmap v279: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:37.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:36 smithi019 ceph-mon[173579]: pgmap v280: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:37.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:36 smithi062 ceph-mon[106861]: pgmap v280: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:39.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:38 smithi019 ceph-mon[173579]: pgmap v281: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:39.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:38 smithi062 ceph-mon[106861]: pgmap v281: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:41.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:41 smithi019 ceph-mon[173579]: pgmap v282: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:41.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:41 smithi062 ceph-mon[106861]: pgmap v282: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:43.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:43 smithi019 ceph-mon[173579]: pgmap v283: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:43.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:43 smithi062 ceph-mon[106861]: pgmap v283: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:45.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:45 smithi019 ceph-mon[173579]: pgmap v284: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:45.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:45 smithi062 ceph-mon[106861]: pgmap v284: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:47.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:47 smithi019 ceph-mon[173579]: pgmap v285: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:47.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:47 smithi062 ceph-mon[106861]: pgmap v285: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:49.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:49 smithi019 ceph-mon[173579]: pgmap v286: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:49.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:49.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:49 smithi062 ceph-mon[106861]: pgmap v286: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:49.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:02:51.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:51 smithi019 ceph-mon[173579]: pgmap v287: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:51.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:51 smithi062 ceph-mon[106861]: pgmap v287: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:53.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:53 smithi062 ceph-mon[106861]: pgmap v288: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:53.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:53 smithi019 ceph-mon[173579]: pgmap v288: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:55.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:55 smithi062 ceph-mon[106861]: pgmap v289: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:55.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:55 smithi019 ceph-mon[173579]: pgmap v289: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:57.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:57 smithi062 ceph-mon[106861]: pgmap v290: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:57.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:57 smithi019 ceph-mon[173579]: pgmap v290: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:59.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:02:59 smithi062 ceph-mon[106861]: pgmap v291: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:02:59.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:02:59 smithi019 ceph-mon[173579]: pgmap v291: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.789+0000 7fac44008700 1 -- 172.21.15.19:0/3437024778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c0fb570 msgr2=0x7fac3c0fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.789+0000 7fac44008700 1 --2- 172.21.15.19:0/3437024778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c0fb570 0x7fac3c0fb9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fac2c004940 tx=0x7fac2c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.789+0000 7fac44008700 1 -- 172.21.15.19:0/3437024778 shutdown_connections 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.789+0000 7fac44008700 1 --2- 172.21.15.19:0/3437024778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c0fb570 0x7fac3c0fb9c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.789+0000 7fac44008700 1 --2- 172.21.15.19:0/3437024778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c103140 0x7fac3c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.789+0000 7fac44008700 1 -- 172.21.15.19:0/3437024778 >> 172.21.15.19:0/3437024778 conn(0x7fac3c071560 msgr2=0x7fac3c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 -- 172.21.15.19:0/3437024778 shutdown_connections 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 -- 172.21.15.19:0/3437024778 wait complete. 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 Processor -- start 2024-02-22T16:03:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 -- start start 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c0fb570 0x7fac3c113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 0x7fac3c114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c114660 con 0x7fac3c103140 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac44008700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c10d990 con 0x7fac3c0fb570 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.790+0000 7fac415a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 0x7fac3c114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac415a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 0x7fac3c114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54222/0 (socket says 172.21.15.19:54222) 2024-02-22T16:03:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac415a3700 1 -- 172.21.15.19:0/349746808 learned_addr learned my addr 172.21.15.19:0/349746808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac41da4700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c0fb570 0x7fac3c113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac415a3700 1 -- 172.21.15.19:0/349746808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c0fb570 msgr2=0x7fac3c113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac415a3700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c0fb570 0x7fac3c113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac415a3700 1 -- 172.21.15.19:0/349746808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c005040 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac41da4700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c0fb570 0x7fac3c113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac415a3700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 0x7fac3c114010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac2c000c00 tx=0x7fac2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac2c021070 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac2c01c400 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c10dc10 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac2c00e070 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.791+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c10e0f0 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.792+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3c1054d0 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.793+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fac2c01c9e0 con 0x7fac3c103140 2024-02-22T16:03:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.793+0000 7fac327fc700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac2806cc20 0x7fac2806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:00.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.793+0000 7fac41da4700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac2806cc20 0x7fac2806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:00.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.794+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fac2c08d170 con 0x7fac3c103140 2024-02-22T16:03:00.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.794+0000 7fac41da4700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac2806cc20 0x7fac2806f0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fac38004600 tx=0x7fac3800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:00.798+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fac2c05acd0 con 0x7fac3c103140 2024-02-22T16:03:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.022+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac3c1137a0 con 0x7fac2806cc20 2024-02-22T16:03:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.023+0000 7fac327fc700 1 -- 172.21.15.19:0/349746808 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fac3c1137a0 con 0x7fac2806cc20 2024-02-22T16:03:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac2806cc20 msgr2=0x7fac2806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac2806cc20 0x7fac2806f0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fac38004600 tx=0x7fac3800b040 comp rx=0 tx=0).stop 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 msgr2=0x7fac3c114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 0x7fac3c114010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fac2c000c00 tx=0x7fac2c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 shutdown_connections 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fac2806cc20 0x7fac2806f0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac3c0fb570 0x7fac3c113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 --2- 172.21.15.19:0/349746808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac3c103140 0x7fac3c114010 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 >> 172.21.15.19:0/349746808 conn(0x7fac3c071560 msgr2=0x7fac3c0f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 shutdown_connections 2024-02-22T16:03:01.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.025+0000 7fac44008700 1 -- 172.21.15.19:0/349746808 wait complete. 2024-02-22T16:03:01.015 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:03:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 -- 172.21.15.19:0/718223275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fdfe0 msgr2=0x7f2d1c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/718223275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fdfe0 0x7f2d1c0fe3b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d10004970 tx=0x7f2d10009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 -- 172.21.15.19:0/718223275 shutdown_connections 2024-02-22T16:03:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/718223275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/718223275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fdfe0 0x7f2d1c0fe3b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 -- 172.21.15.19:0/718223275 >> 172.21.15.19:0/718223275 conn(0x7f2d1c071d90 msgr2=0x7f2d1c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 -- 172.21.15.19:0/718223275 shutdown_connections 2024-02-22T16:03:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 -- 172.21.15.19:0/718223275 wait complete. 2024-02-22T16:03:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 Processor -- start 2024-02-22T16:03:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.131+0000 7f2d22b8d700 1 -- start start 2024-02-22T16:03:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d22b8d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d1c0fdfe0 0x7f2d1c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d22b8d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d22b8d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c1144c0 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d22b8d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c10d7f0 con 0x7f2d1c0fdfe0 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54238/0 (socket says 172.21.15.19:54238) 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 -- 172.21.15.19:0/1587416329 learned_addr learned my addr 172.21.15.19:0/1587416329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d20929700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d1c0fdfe0 0x7f2d1c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 -- 172.21.15.19:0/1587416329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d1c0fdfe0 msgr2=0x7f2d1c113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d1c0fdfe0 0x7f2d1c113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 -- 172.21.15.19:0/1587416329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d10005040 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.132+0000 7f2d1bfff700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c113e70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d08002a50 tx=0x7f2d080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.133+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d08007cc0 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.133+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d0800f040 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.133+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c10dad0 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.133+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0800d9c0 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.133+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c10e010 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.133+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1c1046c0 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.134+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2d0800db20 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.135+0000 7f2d197fa700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2d0c06cc20 0x7f2d0c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.135+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2d0808bba0 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.135+0000 7f2d20929700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2d0c06cc20 0x7f2d0c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.135+0000 7f2d20929700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2d0c06cc20 0x7f2d0c06f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d10004f50 tx=0x7f2d10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.139+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2d08059640 con 0x7f2d1c0fe8f0 2024-02-22T16:03:01.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:01 smithi019 ceph-mon[173579]: pgmap v292: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:01.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.360+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d1c109ef0 con 0x7f2d0c06cc20 2024-02-22T16:03:01.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.361+0000 7f2d197fa700 1 -- 172.21.15.19:0/1587416329 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d1c109ef0 con 0x7f2d0c06cc20 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.363+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2d0c06cc20 msgr2=0x7f2d0c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.363+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2d0c06cc20 0x7f2d0c06f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d10004f50 tx=0x7f2d10009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.363+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 msgr2=0x7f2d1c113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.363+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c113e70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d08002a50 tx=0x7f2d080046b0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 shutdown_connections 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2d0c06cc20 0x7f2d0c06f0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d1c0fdfe0 0x7f2d1c113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 --2- 172.21.15.19:0/1587416329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c0fe8f0 0x7f2d1c113e70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 >> 172.21.15.19:0/1587416329 conn(0x7f2d1c071d90 msgr2=0x7f2d1c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:01.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 shutdown_connections 2024-02-22T16:03:01.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.364+0000 7f2d22b8d700 1 -- 172.21.15.19:0/1587416329 wait complete. 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 -- 172.21.15.19:0/1260134239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fe8f0 msgr2=0x7fc1cc107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/1260134239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc107050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4004970 tx=0x7fc1b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 -- 172.21.15.19:0/1260134239 shutdown_connections 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/1260134239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc107050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/1260134239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 -- 172.21.15.19:0/1260134239 >> 172.21.15.19:0/1260134239 conn(0x7fc1cc071d90 msgr2=0x7fc1cc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 -- 172.21.15.19:0/1260134239 shutdown_connections 2024-02-22T16:03:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.467+0000 7fc1d0a14700 1 -- 172.21.15.19:0/1260134239 wait complete. 2024-02-22T16:03:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1d0a14700 1 Processor -- start 2024-02-22T16:03:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1d0a14700 1 -- start start 2024-02-22T16:03:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1d0a14700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1d0a14700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1d0a14700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc114500 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1d0a14700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc10d830 con 0x7fc1cc0fe8f0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1ca59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54266/0 (socket says 172.21.15.19:54266) 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.468+0000 7fc1ca59c700 1 -- 172.21.15.19:0/180201301 learned_addr learned my addr 172.21.15.19:0/180201301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1c9d9b700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1ca59c700 1 -- 172.21.15.19:0/180201301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fe8f0 msgr2=0x7fc1cc113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1ca59c700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1ca59c700 1 -- 172.21.15.19:0/180201301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4005040 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1c9d9b700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1ca59c700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc113970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00eb70 tx=0x7fc1bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1bc007cc0 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1bc00f040 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc10dab0 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1bc0095f0 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.469+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc10dff0 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.470+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc1046c0 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.470+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc1bc009750 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.471+0000 7fc1c2ffd700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc1b806cc20 0x7fc1b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.471+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc1bc08bce0 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.471+0000 7fc1c9d9b700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc1b806cc20 0x7fc1b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.471+0000 7fc1c9d9b700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc1b806cc20 0x7fc1b806f0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4005010 tx=0x7fc1b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:01.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:01 smithi062 ceph-mon[106861]: pgmap v292: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:01.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.475+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc1bc059720 con 0x7fc1cc0fdfe0 2024-02-22T16:03:01.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.686+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1cc04bed0 con 0x7fc1b806cc20 2024-02-22T16:03:01.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.696+0000 7fc1c2ffd700 1 -- 172.21.15.19:0/180201301 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fc1cc04bed0 con 0x7fc1b806cc20 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (10m) 3m ago 21m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (7m) 3m ago 21m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (7m) 3m ago 20m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (9m) 3m ago 21m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (3m) 3m ago 15m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (3m) 3m ago 15m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (3m) 3m ago 15m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (3m) 3m ago 15m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (12m) 3m ago 22m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (11m) 3m ago 20m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:03:01.675 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (8m) 3m ago 22m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (8m) 3m ago 20m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (11m) 3m ago 21m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (10m) 3m ago 20m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 3m ago 19m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (6m) 3m ago 18m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (5m) 3m ago 18m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (5m) 3m ago 18m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (4m) 3m ago 17m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (4m) 3m ago 17m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:03:01.676 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (10m) 3m ago 20m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc1b806cc20 msgr2=0x7fc1b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc1b806cc20 0x7fc1b806f0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4005010 tx=0x7fc1b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 msgr2=0x7fc1cc113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc113970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00eb70 tx=0x7fc1bc00b040 comp rx=0 tx=0).stop 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 shutdown_connections 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc1b806cc20 0x7fc1b806f0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0fdfe0 0x7fc1cc113970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 --2- 172.21.15.19:0/180201301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1cc0fe8f0 0x7fc1cc113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 >> 172.21.15.19:0/180201301 conn(0x7fc1cc071d90 msgr2=0x7fc1cc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:01.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.698+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 shutdown_connections 2024-02-22T16:03:01.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.699+0000 7fc1d0a14700 1 -- 172.21.15.19:0/180201301 wait complete. 2024-02-22T16:03:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 -- 172.21.15.19:0/1507590893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 msgr2=0x7f20380fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 --2- 172.21.15.19:0/1507590893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f20380fbea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2034004940 tx=0x7f2034009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 -- 172.21.15.19:0/1507590893 shutdown_connections 2024-02-22T16:03:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 --2- 172.21.15.19:0/1507590893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f20380fbea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 --2- 172.21.15.19:0/1507590893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101e40 0x7f2038102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 -- 172.21.15.19:0/1507590893 >> 172.21.15.19:0/1507590893 conn(0x7f2038071d90 msgr2=0x7f2038072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 -- 172.21.15.19:0/1507590893 shutdown_connections 2024-02-22T16:03:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 -- 172.21.15.19:0/1507590893 wait complete. 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 Processor -- start 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.801+0000 7f2040406700 1 -- start start 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f2040406700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101e40 0x7f2038115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f2040406700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f2038116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f2040406700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20381166c0 con 0x7f20381027e0 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f2040406700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203810f9f0 con 0x7f2038101e40 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f2038116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f2038116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54278/0 (socket says 172.21.15.19:54278) 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 -- 172.21.15.19:0/1905912451 learned_addr learned my addr 172.21.15.19:0/1905912451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203e1a2700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101e40 0x7f2038115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 -- 172.21.15.19:0/1905912451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101e40 msgr2=0x7f2038115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101e40 0x7f2038115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 -- 172.21.15.19:0/1905912451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034005040 con 0x7f20381027e0 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.802+0000 7f203d9a1700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f2038116070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2034009380 tx=0x7f2034009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.803+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203400e040 con 0x7f20381027e0 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.803+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203810fc70 con 0x7f20381027e0 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.803+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2034013420 con 0x7f20381027e0 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.803+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203401c630 con 0x7f20381027e0 2024-02-22T16:03:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.803+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20381101b0 con 0x7f20381027e0 2024-02-22T16:03:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.804+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20380738b0 con 0x7f20381027e0 2024-02-22T16:03:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.804+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2034013590 con 0x7f20381027e0 2024-02-22T16:03:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.805+0000 7f202effd700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f202406cc20 0x7f202406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.805+0000 7f203e1a2700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f202406cc20 0x7f202406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.805+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f203408d3b0 con 0x7f20381027e0 2024-02-22T16:03:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.805+0000 7f203e1a2700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f202406cc20 0x7f202406f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2028009fd0 tx=0x7f202800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:01.809+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f203405add0 con 0x7f20381027e0 2024-02-22T16:03:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.099+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f203804bed0 con 0x7f20381027e0 2024-02-22T16:03:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.099+0000 7f202effd700 1 -- 172.21.15.19:0/1905912451 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f203405a960 con 0x7f20381027e0 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:03:02.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f202406cc20 msgr2=0x7f202406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f202406cc20 0x7f202406f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2028009fd0 tx=0x7f202800b040 comp rx=0 tx=0).stop 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 msgr2=0x7f2038116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f2038116070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2034009380 tx=0x7f2034009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 shutdown_connections 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f202406cc20 0x7f202406f0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2038101e40 0x7f2038115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 --2- 172.21.15.19:0/1905912451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20381027e0 0x7f2038116070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 >> 172.21.15.19:0/1905912451 conn(0x7f2038071d90 msgr2=0x7f2038065800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 shutdown_connections 2024-02-22T16:03:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.101+0000 7f2040406700 1 -- 172.21.15.19:0/1905912451 wait complete. 2024-02-22T16:03:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 -- 172.21.15.19:0/127217829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 msgr2=0x7f9e7c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 --2- 172.21.15.19:0/127217829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c103320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9e70004940 tx=0x7f9e70009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:02.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 -- 172.21.15.19:0/127217829 shutdown_connections 2024-02-22T16:03:02.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 --2- 172.21.15.19:0/127217829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 0x7f9e7c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 --2- 172.21.15.19:0/127217829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c103320 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 -- 172.21.15.19:0/127217829 >> 172.21.15.19:0/127217829 conn(0x7f9e7c0f8ae0 msgr2=0x7f9e7c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:02.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 -- 172.21.15.19:0/127217829 shutdown_connections 2024-02-22T16:03:02.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.205+0000 7f9e8183b700 1 -- 172.21.15.19:0/127217829 wait complete. 2024-02-22T16:03:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e8183b700 1 Processor -- start 2024-02-22T16:03:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e8183b700 1 -- start start 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e8183b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e8183b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 0x7f9e7c113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e8183b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c114490 con 0x7f9e7c102f50 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e8183b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c10d7c0 con 0x7f9e7c1038f0 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54290/0 (socket says 172.21.15.19:54290) 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 -- 172.21.15.19:0/2489549880 learned_addr learned my addr 172.21.15.19:0/2489549880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7a7fc700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 0x7f9e7c113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 -- 172.21.15.19:0/2489549880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 msgr2=0x7f9e7c113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 0x7f9e7c113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 -- 172.21.15.19:0/2489549880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e70005040 con 0x7f9e7c102f50 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7a7fc700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 0x7f9e7c113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.206+0000 7f9e7affd700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c113900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9e70000c00 tx=0x7f9e70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.207+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e70021070 con 0x7f9e7c102f50 2024-02-22T16:03:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.207+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e7001c400 con 0x7f9e7c102f50 2024-02-22T16:03:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.207+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c10da40 con 0x7f9e7c102f50 2024-02-22T16:03:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.207+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e7000e070 con 0x7f9e7c102f50 2024-02-22T16:03:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.207+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c10df20 con 0x7f9e7c102f50 2024-02-22T16:03:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.208+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c100260 con 0x7f9e7c102f50 2024-02-22T16:03:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.208+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9e7001c9e0 con 0x7f9e7c102f50 2024-02-22T16:03:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.208+0000 7f9e637fe700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e6406cc20 0x7f9e6406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.209+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9e7008d2a0 con 0x7f9e7c102f50 2024-02-22T16:03:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.209+0000 7f9e7a7fc700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e6406cc20 0x7f9e6406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.209+0000 7f9e7a7fc700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e6406cc20 0x7f9e6406f0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c004600 tx=0x7f9e6c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.213+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9e7005ad60 con 0x7f9e7c102f50 2024-02-22T16:03:02.203 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:02 smithi062 ceph-mon[106861]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:02.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.460+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9e7c04bed0 con 0x7f9e7c102f50 2024-02-22T16:03:02.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.461+0000 7f9e637fe700 1 -- 172.21.15.19:0/2489549880 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f9e7005a8f0 con 0x7f9e7c102f50 2024-02-22T16:03:02.439 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:03:02.439 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:03:02.439 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:03:02.439 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:03:02.440 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:03:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:02.442 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e6406cc20 msgr2=0x7f9e6406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e6406cc20 0x7f9e6406f0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c004600 tx=0x7f9e6c00b040 comp rx=0 tx=0).stop 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 msgr2=0x7f9e7c113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c113900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9e70000c00 tx=0x7f9e70009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 shutdown_connections 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e6406cc20 0x7f9e6406f0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e7c102f50 0x7f9e7c113900 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 --2- 172.21.15.19:0/2489549880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e7c1038f0 0x7f9e7c113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 >> 172.21.15.19:0/2489549880 conn(0x7f9e7c0f8ae0 msgr2=0x7f9e7c0fa490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:02.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 shutdown_connections 2024-02-22T16:03:02.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.463+0000 7f9e8183b700 1 -- 172.21.15.19:0/2489549880 wait complete. 2024-02-22T16:03:02.444 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:03:02.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:02 smithi019 ceph-mon[173579]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:02.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.567+0000 7f044f313700 1 -- 172.21.15.19:0/3194366175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 msgr2=0x7f04480fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.567+0000 7f044f313700 1 --2- 172.21.15.19:0/3194366175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f04480fcf60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f043c004940 tx=0x7f043c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:02.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.567+0000 7f044f313700 1 -- 172.21.15.19:0/3194366175 shutdown_connections 2024-02-22T16:03:02.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.567+0000 7f044f313700 1 --2- 172.21.15.19:0/3194366175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f04480fcf60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.567+0000 7f044f313700 1 --2- 172.21.15.19:0/3194366175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448102f50 0x7f0448103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.567+0000 7f044f313700 1 -- 172.21.15.19:0/3194366175 >> 172.21.15.19:0/3194366175 conn(0x7f04480f8ae0 msgr2=0x7f04480faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 -- 172.21.15.19:0/3194366175 shutdown_connections 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 -- 172.21.15.19:0/3194366175 wait complete. 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 Processor -- start 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 -- start start 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448102f50 0x7f04481139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f0448113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448114530 con 0x7f04481038f0 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044f313700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044810d860 con 0x7f0448102f50 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044c8ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f0448113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044c8ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f0448113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54314/0 (socket says 172.21.15.19:54314) 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.568+0000 7f044c8ae700 1 -- 172.21.15.19:0/3394213100 learned_addr learned my addr 172.21.15.19:0/3394213100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:02.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044d0af700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448102f50 0x7f04481139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044c8ae700 1 -- 172.21.15.19:0/3394213100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448102f50 msgr2=0x7f04481139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044c8ae700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448102f50 0x7f04481139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044c8ae700 1 -- 172.21.15.19:0/3394213100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c005040 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044c8ae700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f0448113ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f043c00b3b0 tx=0x7f043c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f043c00e040 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f043c013420 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044810dae0 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f043c01c610 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.569+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044810e020 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.570+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0448105420 con 0x7f04481038f0 2024-02-22T16:03:02.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.571+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f043c013a20 con 0x7f04481038f0 2024-02-22T16:03:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.571+0000 7f0439ffb700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f043406cc20 0x7f043406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.571+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f043c08d340 con 0x7f04481038f0 2024-02-22T16:03:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.571+0000 7f044d0af700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f043406cc20 0x7f043406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.572+0000 7f044d0af700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f043406cc20 0x7f043406f0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0444004990 tx=0x7f0444009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:02.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.576+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f043c05ada0 con 0x7f04481038f0 2024-02-22T16:03:02.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.798+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0448109ef0 con 0x7f043406cc20 2024-02-22T16:03:02.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.799+0000 7f0439ffb700 1 -- 172.21.15.19:0/3394213100 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0448109ef0 con 0x7f043406cc20 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:03:02.778 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:03:02.779 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:03:02.779 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:03:02.779 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f043406cc20 msgr2=0x7f043406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f043406cc20 0x7f043406f0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0444004990 tx=0x7f0444009ee0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 msgr2=0x7f0448113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f0448113ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f043c00b3b0 tx=0x7f043c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 shutdown_connections 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f043406cc20 0x7f043406f0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0448102f50 0x7f04481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 --2- 172.21.15.19:0/3394213100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04481038f0 0x7f0448113ee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.801+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 >> 172.21.15.19:0/3394213100 conn(0x7f04480f8ae0 msgr2=0x7f04480fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.802+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 shutdown_connections 2024-02-22T16:03:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.802+0000 7f044f313700 1 -- 172.21.15.19:0/3394213100 wait complete. 2024-02-22T16:03:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 -- 172.21.15.19:0/2644461150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 msgr2=0x7f93f0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 --2- 172.21.15.19:0/2644461150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f0102210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93e0004940 tx=0x7f93e0009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 -- 172.21.15.19:0/2644461150 shutdown_connections 2024-02-22T16:03:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 --2- 172.21.15.19:0/2644461150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01027e0 0x7f93f00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 --2- 172.21.15.19:0/2644461150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f0102210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 -- 172.21.15.19:0/2644461150 >> 172.21.15.19:0/2644461150 conn(0x7f93f0071d90 msgr2=0x7f93f0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 -- 172.21.15.19:0/2644461150 shutdown_connections 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.905+0000 7f93f4e79700 1 -- 172.21.15.19:0/2644461150 wait complete. 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93f4e79700 1 Processor -- start 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93f4e79700 1 -- start start 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93f4e79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f010ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93f4e79700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01027e0 0x7f93f010ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93f4e79700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f010f4d0 con 0x7f93f0101e40 2024-02-22T16:03:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93f4e79700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f010f640 con 0x7f93f01027e0 2024-02-22T16:03:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f010ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f010ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54330/0 (socket says 172.21.15.19:54330) 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93ee59c700 1 -- 172.21.15.19:0/3435839373 learned_addr learned my addr 172.21.15.19:0/3435839373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93edd9b700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01027e0 0x7f93f010ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93ee59c700 1 -- 172.21.15.19:0/3435839373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01027e0 msgr2=0x7f93f010ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93ee59c700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01027e0 0x7f93f010ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.906+0000 7f93ee59c700 1 -- 172.21.15.19:0/3435839373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0005040 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.907+0000 7f93ee59c700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f010ea00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.907+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e0021070 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.907+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93e001c400 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.907+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f0117970 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.907+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e000e070 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.907+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f0117dd0 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.908+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f00ff150 con 0x7f93f0101e40 2024-02-22T16:03:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.908+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f93e001c970 con 0x7f93f0101e40 2024-02-22T16:03:02.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.909+0000 7f93e6ffd700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f93dc06cc20 0x7f93dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:02.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.909+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f93e008d1e0 con 0x7f93f0101e40 2024-02-22T16:03:02.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.909+0000 7f93edd9b700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f93dc06cc20 0x7f93dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:02.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.909+0000 7f93edd9b700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f93dc06cc20 0x7f93dc06f0d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93f00661a0 tx=0x7f93d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:02.913+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f93e0057210 con 0x7f93f0101e40 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.201+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f93f00738b0 con 0x7f93f0101e40 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.201+0000 7f93e6ffd700 1 -- 172.21.15.19:0/3435839373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f93e005a830 con 0x7f93f0101e40 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:03:03.180 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f93dc06cc20 msgr2=0x7f93dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f93dc06cc20 0x7f93dc06f0d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93f00661a0 tx=0x7f93d800b040 comp rx=0 tx=0).stop 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 msgr2=0x7f93f010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f010ea00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 shutdown_connections 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f93dc06cc20 0x7f93dc06f0d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f0101e40 0x7f93f010ea00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.203+0000 7f93f4e79700 1 --2- 172.21.15.19:0/3435839373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93f01027e0 0x7f93f010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.204+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 >> 172.21.15.19:0/3435839373 conn(0x7f93f0071d90 msgr2=0x7f93f00fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:03.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.204+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 shutdown_connections 2024-02-22T16:03:03.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:03.204+0000 7f93f4e79700 1 -- 172.21.15.19:0/3435839373 wait complete. 2024-02-22T16:03:03.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:03 smithi062 ceph-mon[106861]: pgmap v293: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:03.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:03 smithi062 ceph-mon[106861]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:03.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:03 smithi062 ceph-mon[106861]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:03.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:03 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1905912451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:03:03.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:03 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2489549880' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:03:03.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:03 smithi019 ceph-mon[173579]: pgmap v293: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:03.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:03 smithi019 ceph-mon[173579]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:03.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:03 smithi019 ceph-mon[173579]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:03.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:03 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1905912451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:03:03.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:03 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2489549880' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:03:04.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:04 smithi062 ceph-mon[106861]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:04.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:04.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:04 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3435839373' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:03:04.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:04 smithi019 ceph-mon[173579]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:04.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:04.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:04 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3435839373' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:03:05.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:05 smithi062 ceph-mon[106861]: pgmap v294: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:05.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:05 smithi019 ceph-mon[173579]: pgmap v294: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:07.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:07 smithi062 ceph-mon[106861]: pgmap v295: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:07.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:07 smithi019 ceph-mon[173579]: pgmap v295: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:09.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:09 smithi062 ceph-mon[106861]: pgmap v296: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:09.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:09 smithi019 ceph-mon[173579]: pgmap v296: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:10.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:10 smithi062 ceph-mon[106861]: pgmap v297: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:10.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:10 smithi019 ceph-mon[173579]: pgmap v297: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:12.703 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:12 smithi062 ceph-mon[106861]: pgmap v298: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:12.741 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:12 smithi019 ceph-mon[173579]: pgmap v298: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:15.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:14 smithi019 ceph-mon[173579]: pgmap v299: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:15.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:14 smithi062 ceph-mon[106861]: pgmap v299: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:17.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:16 smithi019 ceph-mon[173579]: pgmap v300: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:17.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:16 smithi062 ceph-mon[106861]: pgmap v300: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:19.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:19 smithi019 ceph-mon[173579]: pgmap v301: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:19.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:19 smithi062 ceph-mon[106861]: pgmap v301: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:21.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:21 smithi019 ceph-mon[173579]: pgmap v302: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:21.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:21 smithi062 ceph-mon[106861]: pgmap v302: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:23.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:23 smithi019 ceph-mon[173579]: pgmap v303: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:23.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:23 smithi062 ceph-mon[106861]: pgmap v303: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:25.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:25 smithi019 ceph-mon[173579]: pgmap v304: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:25.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:25 smithi062 ceph-mon[106861]: pgmap v304: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:27.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:27 smithi062 ceph-mon[106861]: pgmap v305: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:27.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:27 smithi019 ceph-mon[173579]: pgmap v305: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:29.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:29 smithi062 ceph-mon[106861]: pgmap v306: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:29.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:29 smithi019 ceph-mon[173579]: pgmap v306: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:31 smithi019 ceph-mon[173579]: pgmap v307: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:31.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:31 smithi062 ceph-mon[106861]: pgmap v307: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 -- 172.21.15.19:0/2768158910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 msgr2=0x7f90480fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 --2- 172.21.15.19:0/2768158910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f90480fe3b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9038004970 tx=0x7f9038009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 -- 172.21.15.19:0/2768158910 shutdown_connections 2024-02-22T16:03:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 --2- 172.21.15.19:0/2768158910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 0x7f9048107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 --2- 172.21.15.19:0/2768158910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f90480fe3b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 -- 172.21.15.19:0/2768158910 >> 172.21.15.19:0/2768158910 conn(0x7f9048071d90 msgr2=0x7f9048072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:33.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 -- 172.21.15.19:0/2768158910 shutdown_connections 2024-02-22T16:03:33.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.320+0000 7f904d048700 1 -- 172.21.15.19:0/2768158910 wait complete. 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.321+0000 7f904d048700 1 Processor -- start 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.321+0000 7f904d048700 1 -- start start 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.321+0000 7f904d048700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f9048113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.321+0000 7f904d048700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 0x7f9048113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.321+0000 7f904d048700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90481144c0 con 0x7f90480fdfe0 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.321+0000 7f904d048700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904810d7f0 con 0x7f90480fe8f0 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f9048113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f9048113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35330/0 (socket says 172.21.15.19:35330) 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 -- 172.21.15.19:0/3840484024 learned_addr learned my addr 172.21.15.19:0/3840484024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f904659c700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 0x7f9048113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 -- 172.21.15.19:0/3840484024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 msgr2=0x7f9048113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 0x7f9048113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 -- 172.21.15.19:0/3840484024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038005040 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f904659c700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 0x7f9048113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f9046d9d700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f9048113930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9038005010 tx=0x7f9038009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903800e040 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9038004290 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f903801c3f0 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904810da70 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.322+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904810dfb0 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.323+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90481046c0 con 0x7f90480fdfe0 2024-02-22T16:03:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.324+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9038013420 con 0x7f90480fdfe0 2024-02-22T16:03:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.324+0000 7f902f7fe700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f903006cc30 0x7f903006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.324+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f903808d2c0 con 0x7f90480fdfe0 2024-02-22T16:03:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.324+0000 7f904659c700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f903006cc30 0x7f903006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.325+0000 7f904659c700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f903006cc30 0x7f903006f0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f904810f160 tx=0x7f903c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.329+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f903805ae20 con 0x7f90480fdfe0 2024-02-22T16:03:33.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:33 smithi062 ceph-mon[106861]: pgmap v308: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:33.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:33 smithi019 ceph-mon[173579]: pgmap v308: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:33.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.550+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f904804bed0 con 0x7f903006cc30 2024-02-22T16:03:33.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.551+0000 7f902f7fe700 1 -- 172.21.15.19:0/3840484024 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f904804bed0 con 0x7f903006cc30 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f903006cc30 msgr2=0x7f903006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f903006cc30 0x7f903006f0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f904810f160 tx=0x7f903c00b040 comp rx=0 tx=0).stop 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 msgr2=0x7f9048113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f9048113930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9038005010 tx=0x7f9038009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 shutdown_connections 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f903006cc30 0x7f903006f0e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90480fdfe0 0x7f9048113930 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 --2- 172.21.15.19:0/3840484024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90480fe8f0 0x7f9048113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 >> 172.21.15.19:0/3840484024 conn(0x7f9048071d90 msgr2=0x7f9048101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 shutdown_connections 2024-02-22T16:03:33.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.553+0000 7f904d048700 1 -- 172.21.15.19:0/3840484024 wait complete. 2024-02-22T16:03:33.541 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:03:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 -- 172.21.15.19:0/1702690118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c1027e0 msgr2=0x7fce0c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 --2- 172.21.15.19:0/1702690118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c1027e0 0x7fce0c0fbea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc004940 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 -- 172.21.15.19:0/1702690118 shutdown_connections 2024-02-22T16:03:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 --2- 172.21.15.19:0/1702690118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c1027e0 0x7fce0c0fbea0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 --2- 172.21.15.19:0/1702690118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c101e40 0x7fce0c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 -- 172.21.15.19:0/1702690118 >> 172.21.15.19:0/1702690118 conn(0x7fce0c071d90 msgr2=0x7fce0c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 -- 172.21.15.19:0/1702690118 shutdown_connections 2024-02-22T16:03:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.658+0000 7fce11728700 1 -- 172.21.15.19:0/1702690118 wait complete. 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce11728700 1 Processor -- start 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce11728700 1 -- start start 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce11728700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 0x7fce0c10ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce11728700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c1027e0 0x7fce0c10ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce11728700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c10f520 con 0x7fce0c101e40 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce11728700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c10f690 con 0x7fce0c1027e0 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce0affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 0x7fce0c10ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce0affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 0x7fce0c10ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35340/0 (socket says 172.21.15.19:35340) 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.659+0000 7fce0affd700 1 -- 172.21.15.19:0/4172288382 learned_addr learned my addr 172.21.15.19:0/4172288382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce0a7fc700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c1027e0 0x7fce0c10ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce0affd700 1 -- 172.21.15.19:0/4172288382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c1027e0 msgr2=0x7fce0c10ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce0affd700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c1027e0 0x7fce0c10ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce0affd700 1 -- 172.21.15.19:0/4172288382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc005040 con 0x7fce0c101e40 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce0a7fc700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c1027e0 0x7fce0c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce0affd700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 0x7fce0c10ea50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4002a50 tx=0x7fcdf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf4007cc0 con 0x7fce0c101e40 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcdf400f040 con 0x7fce0c101e40 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c117a20 con 0x7fce0c101e40 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf4009540 con 0x7fce0c101e40 2024-02-22T16:03:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.660+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c117ee0 con 0x7fce0c101e40 2024-02-22T16:03:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.661+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c104310 con 0x7fce0c101e40 2024-02-22T16:03:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.661+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcdf40096a0 con 0x7fce0c101e40 2024-02-22T16:03:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.662+0000 7fce037fe700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcdf806cc20 0x7fcdf806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.662+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcdf408bda0 con 0x7fce0c101e40 2024-02-22T16:03:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.662+0000 7fce0a7fc700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcdf806cc20 0x7fcdf806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.662+0000 7fce0a7fc700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcdf806cc20 0x7fcdf806f0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b3b0 tx=0x7fcdfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:33.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.667+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fcdf4055dd0 con 0x7fce0c101e40 2024-02-22T16:03:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.888+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce0c065ee0 con 0x7fcdf806cc20 2024-02-22T16:03:33.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.889+0000 7fce037fe700 1 -- 172.21.15.19:0/4172288382 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fce0c065ee0 con 0x7fcdf806cc20 2024-02-22T16:03:33.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcdf806cc20 msgr2=0x7fcdf806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcdf806cc20 0x7fcdf806f0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b3b0 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:33.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 msgr2=0x7fce0c10ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 0x7fce0c10ea50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4002a50 tx=0x7fcdf40046b0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 shutdown_connections 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcdf806cc20 0x7fcdf806f0d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce0c101e40 0x7fce0c10ea50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 --2- 172.21.15.19:0/4172288382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce0c1027e0 0x7fce0c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 >> 172.21.15.19:0/4172288382 conn(0x7fce0c071d90 msgr2=0x7fce0c0faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 shutdown_connections 2024-02-22T16:03:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.891+0000 7fce11728700 1 -- 172.21.15.19:0/4172288382 wait complete. 2024-02-22T16:03:33.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.994+0000 7f246c508700 1 -- 172.21.15.19:0/2688959115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24641027f0 msgr2=0x7f24640fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.994+0000 7f246c508700 1 --2- 172.21.15.19:0/2688959115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24641027f0 0x7f24640fbe60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2454004940 tx=0x7f2454009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 -- 172.21.15.19:0/2688959115 shutdown_connections 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 --2- 172.21.15.19:0/2688959115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24641027f0 0x7f24640fbe60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 --2- 172.21.15.19:0/2688959115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2464101e50 0x7f2464102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 -- 172.21.15.19:0/2688959115 >> 172.21.15.19:0/2688959115 conn(0x7f24640f79e0 msgr2=0x7f24640f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 -- 172.21.15.19:0/2688959115 shutdown_connections 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 -- 172.21.15.19:0/2688959115 wait complete. 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 Processor -- start 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 -- start start 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 0x7f24640fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24641027f0 0x7f24640fe010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.995+0000 7f246c508700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640fe610 con 0x7f2464101e50 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246c508700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640fe780 con 0x7f24641027f0 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 0x7f24640fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 0x7f24640fdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35346/0 (socket says 172.21.15.19:35346) 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 -- 172.21.15.19:0/1900843141 learned_addr learned my addr 172.21.15.19:0/1900843141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f2469aa3700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24641027f0 0x7f24640fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 -- 172.21.15.19:0/1900843141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24641027f0 msgr2=0x7f24640fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24641027f0 0x7f24640fe010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 -- 172.21.15.19:0/1900843141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2454005040 con 0x7f2464101e50 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246a2a4700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 0x7f24640fdad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2460002a50 tx=0x7f24600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2460007cc0 con 0x7f2464101e50 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2464119be0 con 0x7f2464101e50 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f246000f040 con 0x7f2464101e50 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2460009540 con 0x7f2464101e50 2024-02-22T16:03:33.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.996+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246411a0c0 con 0x7f2464101e50 2024-02-22T16:03:33.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.997+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24640fb5e0 con 0x7f2464101e50 2024-02-22T16:03:33.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.998+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f24600096a0 con 0x7f2464101e50 2024-02-22T16:03:33.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.998+0000 7f245affd700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f245006cc30 0x7f245006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:33.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.998+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f246008be70 con 0x7f2464101e50 2024-02-22T16:03:33.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.998+0000 7f2469aa3700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f245006cc30 0x7f245006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:33.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:33.999+0000 7f2469aa3700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f245006cc30 0x7f245006f0e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f245400b3b0 tx=0x7f2454009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.003+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2460055e20 con 0x7f2464101e50 2024-02-22T16:03:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.218+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f24640666b0 con 0x7f245006cc30 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.228+0000 7f245affd700 1 -- 172.21.15.19:0/1900843141 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f24640666b0 con 0x7f245006cc30 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (10m) 4m ago 21m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (8m) 4m ago 22m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (8m) 3m ago 20m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (10m) 4m ago 21m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (4m) 4m ago 16m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:03:34.205 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (4m) 4m ago 16m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (3m) 3m ago 15m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (3m) 3m ago 16m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (12m) 4m ago 23m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (11m) 3m ago 20m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (9m) 4m ago 23m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (8m) 3m ago 20m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (11m) 4m ago 22m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (11m) 3m ago 20m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 4m ago 19m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (7m) 4m ago 19m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (6m) 4m ago 18m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (5m) 3m ago 18m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (5m) 3m ago 18m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (4m) 3m ago 17m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:03:34.206 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (11m) 4m ago 21m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f245006cc30 msgr2=0x7f245006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f245006cc30 0x7f245006f0e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f245400b3b0 tx=0x7f2454009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 msgr2=0x7f24640fdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 0x7f24640fdad0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2460002a50 tx=0x7f24600046b0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 shutdown_connections 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f245006cc30 0x7f245006f0e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2464101e50 0x7f24640fdad0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 --2- 172.21.15.19:0/1900843141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24641027f0 0x7f24640fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 >> 172.21.15.19:0/1900843141 conn(0x7f24640f79e0 msgr2=0x7f2464065fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 shutdown_connections 2024-02-22T16:03:34.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.230+0000 7f246c508700 1 -- 172.21.15.19:0/1900843141 wait complete. 2024-02-22T16:03:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 -- 172.21.15.19:0/2717135796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 msgr2=0x7f9c840fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/2717135796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c840fbea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9c70004940 tx=0x7f9c70009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 -- 172.21.15.19:0/2717135796 shutdown_connections 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/2717135796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c840fbea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/2717135796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c84101e40 0x7f9c84102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 -- 172.21.15.19:0/2717135796 >> 172.21.15.19:0/2717135796 conn(0x7f9c84071d90 msgr2=0x7f9c84072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 -- 172.21.15.19:0/2717135796 shutdown_connections 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 -- 172.21.15.19:0/2717135796 wait complete. 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 Processor -- start 2024-02-22T16:03:34.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.333+0000 7f9c8acb5700 1 -- start start 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c8acb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c84101e40 0x7f9c8410ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c8acb5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c8410efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c8acb5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410f540 con 0x7f9c841027e0 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c8acb5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410f680 con 0x7f9c84101e40 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c8410efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c8410efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35372/0 (socket says 172.21.15.19:35372) 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 -- 172.21.15.19:0/236939786 learned_addr learned my addr 172.21.15.19:0/236939786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c88a51700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c84101e40 0x7f9c8410ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 -- 172.21.15.19:0/236939786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c84101e40 msgr2=0x7f9c8410ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c84101e40 0x7f9c8410ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 -- 172.21.15.19:0/236939786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70005040 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c83fff700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c8410efb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b3b0 tx=0x7f9c70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7000e040 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c70013420 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.334+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c84117980 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.335+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7001c610 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.335+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c84117e90 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.335+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c84104310 con 0x7f9c841027e0 2024-02-22T16:03:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.336+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9c70013590 con 0x7f9c841027e0 2024-02-22T16:03:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.336+0000 7f9c817fa700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c7406cc20 0x7f9c7406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.336+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9c7008d2a0 con 0x7f9c841027e0 2024-02-22T16:03:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.336+0000 7f9c88a51700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c7406cc20 0x7f9c7406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.337+0000 7f9c88a51700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c7406cc20 0x7f9c7406f0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c78004f40 tx=0x7f9c7800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:34.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.341+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9c70057380 con 0x7f9c841027e0 2024-02-22T16:03:34.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:34.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.626+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c840738b0 con 0x7f9c841027e0 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.627+0000 7f9c817fa700 1 -- 172.21.15.19:0/236939786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f9c70013840 con 0x7f9c841027e0 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:34.604 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:03:34.605 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c7406cc20 msgr2=0x7f9c7406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c7406cc20 0x7f9c7406f0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c78004f40 tx=0x7f9c7800b040 comp rx=0 tx=0).stop 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 msgr2=0x7f9c8410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c8410efb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b3b0 tx=0x7f9c70009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 shutdown_connections 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9c7406cc20 0x7f9c7406f0d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c84101e40 0x7f9c8410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 --2- 172.21.15.19:0/236939786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c841027e0 0x7f9c8410efb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 >> 172.21.15.19:0/236939786 conn(0x7f9c84071d90 msgr2=0x7f9c840faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 shutdown_connections 2024-02-22T16:03:34.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.629+0000 7f9c8acb5700 1 -- 172.21.15.19:0/236939786 wait complete. 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 -- 172.21.15.19:0/882747322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 msgr2=0x7f5b8c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 --2- 172.21.15.19:0/882747322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c107050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004970 tx=0x7f5b7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 -- 172.21.15.19:0/882747322 shutdown_connections 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 --2- 172.21.15.19:0/882747322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c107050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 --2- 172.21.15.19:0/882747322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0fdfe0 0x7f5b8c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 -- 172.21.15.19:0/882747322 >> 172.21.15.19:0/882747322 conn(0x7f5b8c071d90 msgr2=0x7f5b8c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 -- 172.21.15.19:0/882747322 shutdown_connections 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.732+0000 7f5b93454700 1 -- 172.21.15.19:0/882747322 wait complete. 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b93454700 1 Processor -- start 2024-02-22T16:03:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b93454700 1 -- start start 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b93454700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0fdfe0 0x7f5b8c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b93454700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b93454700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c114500 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b93454700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c10d830 con 0x7f5b8c0fdfe0 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35386/0 (socket says 172.21.15.19:35386) 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 -- 172.21.15.19:0/4082382352 learned_addr learned my addr 172.21.15.19:0/4082382352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b911f0700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0fdfe0 0x7f5b8c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 -- 172.21.15.19:0/4082382352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0fdfe0 msgr2=0x7f5b8c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0fdfe0 0x7f5b8c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 -- 172.21.15.19:0/4082382352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c005040 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.733+0000 7f5b909ef700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c113eb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005010 tx=0x7f5b7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.734+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7c00e040 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.734+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b7c004290 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.734+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8c10dab0 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.734+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7c01c3f0 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.734+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8c10dff0 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.735+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8c1046c0 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.735+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b7c0136b0 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.736+0000 7f5b81ffb700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7806cc30 0x7f5b7806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.736+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5b7c08d180 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.736+0000 7f5b911f0700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7806cc30 0x7f5b7806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.736+0000 7f5b911f0700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7806cc30 0x7f5b7806f0e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009b80 tx=0x7f5b8800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.740+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5b7c05ace0 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.986+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b8c109f10 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.986+0000 7f5b81ffb700 1 -- 172.21.15.19:0/4082382352 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f5b7c05a870 con 0x7f5b8c0fe8f0 2024-02-22T16:03:34.963 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:03:34.964 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:03:34.965 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:34.966 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7806cc30 msgr2=0x7f5b7806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7806cc30 0x7f5b7806f0e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009b80 tx=0x7f5b8800b040 comp rx=0 tx=0).stop 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 msgr2=0x7f5b8c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c113eb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005010 tx=0x7f5b7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 shutdown_connections 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7806cc30 0x7f5b7806f0e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b8c0fdfe0 0x7f5b8c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 --2- 172.21.15.19:0/4082382352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b8c0fe8f0 0x7f5b8c113eb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:34.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 >> 172.21.15.19:0/4082382352 conn(0x7f5b8c071d90 msgr2=0x7f5b8c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 shutdown_connections 2024-02-22T16:03:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:34.988+0000 7f5b93454700 1 -- 172.21.15.19:0/4082382352 wait complete. 2024-02-22T16:03:34.968 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.093+0000 7f7f41a5c700 1 -- 172.21.15.19:0/1552407524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 msgr2=0x7f7f3c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.093+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/1552407524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c103320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c004940 tx=0x7f7f2c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.093+0000 7f7f41a5c700 1 -- 172.21.15.19:0/1552407524 shutdown_connections 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.093+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/1552407524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f3c1038f0 0x7f7f3c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/1552407524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c103320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 -- 172.21.15.19:0/1552407524 >> 172.21.15.19:0/1552407524 conn(0x7f7f3c0f8ae0 msgr2=0x7f7f3c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 -- 172.21.15.19:0/1552407524 shutdown_connections 2024-02-22T16:03:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 -- 172.21.15.19:0/1552407524 wait complete. 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 Processor -- start 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 -- start start 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f3c1038f0 0x7f7f3c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c114500 con 0x7f7f3c102f50 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.094+0000 7f7f41a5c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c10d830 con 0x7f7f3c1038f0 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35402/0 (socket says 172.21.15.19:35402) 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 -- 172.21.15.19:0/3804351616 learned_addr learned my addr 172.21.15.19:0/3804351616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3a7fc700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f3c1038f0 0x7f7f3c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 -- 172.21.15.19:0/3804351616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f3c1038f0 msgr2=0x7f7f3c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f3c1038f0 0x7f7f3c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 -- 172.21.15.19:0/3804351616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c005040 con 0x7f7f3c102f50 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f3affd700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c113970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00b3b0 tx=0x7f7f2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f2c00e040 con 0x7f7f3c102f50 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f2c01c430 con 0x7f7f3c102f50 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c10dab0 con 0x7f7f3c102f50 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f2c013640 con 0x7f7f3c102f50 2024-02-22T16:03:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.095+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c10dff0 con 0x7f7f3c102f50 2024-02-22T16:03:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.096+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c100260 con 0x7f7f3c102f50 2024-02-22T16:03:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.097+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7f2c01c5a0 con 0x7f7f3c102f50 2024-02-22T16:03:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.097+0000 7f7f237fe700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7f2406cc20 0x7f7f2406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.097+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7f2c08cb30 con 0x7f7f3c102f50 2024-02-22T16:03:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.097+0000 7f7f3a7fc700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7f2406cc20 0x7f7f2406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.098+0000 7f7f3a7fc700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7f2406cc20 0x7f7f2406f0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c10f1a0 tx=0x7f7f3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.102+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7f2c05a6a0 con 0x7f7f3c102f50 2024-02-22T16:03:35.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.325+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f3c04bf70 con 0x7f7f2406cc20 2024-02-22T16:03:35.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.326+0000 7f7f237fe700 1 -- 172.21.15.19:0/3804351616 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f3000f040 con 0x7f7f2406cc20 2024-02-22T16:03:35.303 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:03:35.303 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:03:35.303 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:03:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:03:35.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7f2406cc20 msgr2=0x7f7f2406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7f2406cc20 0x7f7f2406f0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c10f1a0 tx=0x7f7f3000b040 comp rx=0 tx=0).stop 2024-02-22T16:03:35.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 msgr2=0x7f7f3c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c113970 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00b3b0 tx=0x7f7f2c009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 shutdown_connections 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7f2406cc20 0x7f7f2406f0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f3c102f50 0x7f7f3c113970 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 --2- 172.21.15.19:0/3804351616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f3c1038f0 0x7f7f3c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 >> 172.21.15.19:0/3804351616 conn(0x7f7f3c0f8ae0 msgr2=0x7f7f3c0fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 shutdown_connections 2024-02-22T16:03:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.328+0000 7f7f41a5c700 1 -- 172.21.15.19:0/3804351616 wait complete. 2024-02-22T16:03:35.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 -- 172.21.15.19:0/2514954137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fdfc0 msgr2=0x7fb4b80fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 --2- 172.21.15.19:0/2514954137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fdfc0 0x7fb4b80fe390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac004940 tx=0x7fb4ac009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 -- 172.21.15.19:0/2514954137 shutdown_connections 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 --2- 172.21.15.19:0/2514954137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b80fe960 0x7fb4b8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 --2- 172.21.15.19:0/2514954137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fdfc0 0x7fb4b80fe390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 -- 172.21.15.19:0/2514954137 >> 172.21.15.19:0/2514954137 conn(0x7fb4b80f7a10 msgr2=0x7fb4b80f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 -- 172.21.15.19:0/2514954137 shutdown_connections 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 -- 172.21.15.19:0/2514954137 wait complete. 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 Processor -- start 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.432+0000 7fb4bd728700 1 -- start start 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4bd728700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b80fdfc0 0x7fb4b8115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4bd728700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 0x7fb4b8116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4bd728700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8116770 con 0x7fb4b80fe960 2024-02-22T16:03:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4bd728700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b810faa0 con 0x7fb4b80fdfc0 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 0x7fb4b8116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 0x7fb4b8116120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35420/0 (socket says 172.21.15.19:35420) 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 -- 172.21.15.19:0/3660127078 learned_addr learned my addr 172.21.15.19:0/3660127078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b6ffd700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b80fdfc0 0x7fb4b8115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 -- 172.21.15.19:0/3660127078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b80fdfc0 msgr2=0x7fb4b8115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b80fdfc0 0x7fb4b8115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 -- 172.21.15.19:0/3660127078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac005040 con 0x7fb4b80fe960 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4b67fc700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 0x7fb4b8116120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8002a50 tx=0x7fb4a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4a8007cc0 con 0x7fb4b80fe960 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4a800f040 con 0x7fb4b80fe960 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b810fd80 con 0x7fb4b80fe960 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4a8009540 con 0x7fb4b80fe960 2024-02-22T16:03:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.433+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b81102c0 con 0x7fb4b80fe960 2024-02-22T16:03:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.434+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b81069b0 con 0x7fb4b80fe960 2024-02-22T16:03:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.436+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb4a80096c0 con 0x7fb4b80fe960 2024-02-22T16:03:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.436+0000 7fb49f7fe700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb4a006cc30 0x7fb4a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:03:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.436+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb4a808c4e0 con 0x7fb4b80fe960 2024-02-22T16:03:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.436+0000 7fb4b6ffd700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb4a006cc30 0x7fb4a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:03:35.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.437+0000 7fb4b6ffd700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb4a006cc30 0x7fb4a006f0e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000c00 tx=0x7fb4ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:03:35.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.440+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb4a8059fa0 con 0x7fb4b80fe960 2024-02-22T16:03:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:35 smithi062 ceph-mon[106861]: pgmap v309: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:35 smithi062 ceph-mon[106861]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:35 smithi062 ceph-mon[106861]: from='client.34628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:35 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/236939786' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:03:35.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:35 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4082382352' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:03:35.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:35 smithi019 ceph-mon[173579]: pgmap v309: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:35.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:35 smithi019 ceph-mon[173579]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:35.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:35 smithi019 ceph-mon[173579]: from='client.34628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:35.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:35 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/236939786' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:03:35.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:35 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4082382352' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:03:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.726+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb4b81111c0 con 0x7fb4b80fe960 2024-02-22T16:03:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.726+0000 7fb49f7fe700 1 -- 172.21.15.19:0/3660127078 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb4a8009970 con 0x7fb4b80fe960 2024-02-22T16:03:35.703 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:03:35.703 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:03:35.703 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:03:35.704 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:03:35.704 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:03:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb4a006cc30 msgr2=0x7fb4a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb4a006cc30 0x7fb4a006f0e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000c00 tx=0x7fb4ac009e30 comp rx=0 tx=0).stop 2024-02-22T16:03:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 msgr2=0x7fb4b8116120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:03:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 0x7fb4b8116120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8002a50 tx=0x7fb4a80046b0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 shutdown_connections 2024-02-22T16:03:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb4a006cc30 0x7fb4a006f0e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4b80fdfc0 0x7fb4b8115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 --2- 172.21.15.19:0/3660127078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4b80fe960 0x7fb4b8116120 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:03:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 >> 172.21.15.19:0/3660127078 conn(0x7fb4b80f7a10 msgr2=0x7fb4b8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:03:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 shutdown_connections 2024-02-22T16:03:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:03:35.728+0000 7fb4bd728700 1 -- 172.21.15.19:0/3660127078 wait complete. 2024-02-22T16:03:36.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:36 smithi062 ceph-mon[106861]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:36.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:36 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3660127078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:03:36.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:36 smithi019 ceph-mon[173579]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:36.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:36 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3660127078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:03:37.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:37 smithi062 ceph-mon[106861]: pgmap v310: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:37.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:37 smithi062 ceph-mon[106861]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:37.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:37 smithi019 ceph-mon[173579]: pgmap v310: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:37.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:37 smithi019 ceph-mon[173579]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:03:39.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:39 smithi062 ceph-mon[106861]: pgmap v311: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:39.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:39 smithi019 ceph-mon[173579]: pgmap v311: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:41.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:41 smithi019 ceph-mon[173579]: pgmap v312: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:41.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:41 smithi062 ceph-mon[106861]: pgmap v312: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:43.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:43 smithi062 ceph-mon[106861]: pgmap v313: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:43.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:43 smithi019 ceph-mon[173579]: pgmap v313: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:45.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:45 smithi062 ceph-mon[106861]: pgmap v314: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:45.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:45 smithi019 ceph-mon[173579]: pgmap v314: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:47.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:47 smithi062 ceph-mon[106861]: pgmap v315: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:47.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:47 smithi019 ceph-mon[173579]: pgmap v315: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:49.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:49 smithi062 ceph-mon[106861]: pgmap v316: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:49.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:49.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:49 smithi019 ceph-mon[173579]: pgmap v316: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:49.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:03:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:51 smithi019 ceph-mon[173579]: pgmap v317: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:51.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:51 smithi062 ceph-mon[106861]: pgmap v317: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:53.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:53 smithi062 ceph-mon[106861]: pgmap v318: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:53.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:53 smithi019 ceph-mon[173579]: pgmap v318: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:55.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:55 smithi062 ceph-mon[106861]: pgmap v319: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:55.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:55 smithi019 ceph-mon[173579]: pgmap v319: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:56.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:56 smithi062 ceph-mon[106861]: pgmap v320: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:56.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:56 smithi019 ceph-mon[173579]: pgmap v320: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:59.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:03:59 smithi019 ceph-mon[173579]: pgmap v321: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:03:59.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:03:59 smithi062 ceph-mon[106861]: pgmap v321: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:01.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:01 smithi019 ceph-mon[173579]: pgmap v322: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:01.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:01 smithi062 ceph-mon[106861]: pgmap v322: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:03.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:03 smithi019 ceph-mon[173579]: pgmap v323: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:03.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:03 smithi062 ceph-mon[106861]: pgmap v323: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:04.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:04.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:05.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:05 smithi062 ceph-mon[106861]: pgmap v324: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:05.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:05 smithi019 ceph-mon[173579]: pgmap v324: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:05.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 -- 172.21.15.19:0/355472288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2078102f50 msgr2=0x7f2078103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:05.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 --2- 172.21.15.19:0/355472288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2078102f50 0x7f2078103320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2074004940 tx=0x7f2074009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 -- 172.21.15.19:0/355472288 shutdown_connections 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 --2- 172.21.15.19:0/355472288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20781038f0 0x7f20780fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 --2- 172.21.15.19:0/355472288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2078102f50 0x7f2078103320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 -- 172.21.15.19:0/355472288 >> 172.21.15.19:0/355472288 conn(0x7f20780f8ae0 msgr2=0x7f20780faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 -- 172.21.15.19:0/355472288 shutdown_connections 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.846+0000 7f208015d700 1 -- 172.21.15.19:0/355472288 wait complete. 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f208015d700 1 Processor -- start 2024-02-22T16:04:05.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f208015d700 1 -- start start 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f208015d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2078102f50 0x7f2078070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f208015d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 0x7f20780707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f208015d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078070dd0 con 0x7f20781038f0 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f208015d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078070f40 con 0x7f2078102f50 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207d6f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 0x7f20780707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207d6f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 0x7f20780707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45746/0 (socket says 172.21.15.19:45746) 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207d6f8700 1 -- 172.21.15.19:0/369396640 learned_addr learned my addr 172.21.15.19:0/369396640 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207def9700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2078102f50 0x7f2078070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207d6f8700 1 -- 172.21.15.19:0/369396640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2078102f50 msgr2=0x7f2078070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:05.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207d6f8700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2078102f50 0x7f2078070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.847+0000 7f207d6f8700 1 -- 172.21.15.19:0/369396640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074005040 con 0x7f20781038f0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.848+0000 7f207d6f8700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 0x7f20780707d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2068002a50 tx=0x7f2068004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.848+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2068007cc0 con 0x7f20781038f0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.848+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f206800f040 con 0x7f20781038f0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.848+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2078119bb0 con 0x7f20781038f0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.848+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f206800d920 con 0x7f20781038f0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.848+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207811a090 con 0x7f20781038f0 2024-02-22T16:04:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.849+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078100260 con 0x7f20781038f0 2024-02-22T16:04:05.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.849+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2068009680 con 0x7f20781038f0 2024-02-22T16:04:05.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.850+0000 7f206e7fc700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f206406cc30 0x7f206406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:05.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.850+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f206808be60 con 0x7f20781038f0 2024-02-22T16:04:05.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.850+0000 7f207def9700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f206406cc30 0x7f206406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:05.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.851+0000 7f207def9700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f206406cc30 0x7f206406f0e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f207400b3b0 tx=0x7f2074009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:05.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:05.855+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2068055d70 con 0x7f20781038f0 2024-02-22T16:04:06.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.078+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20780719d0 con 0x7f206406cc30 2024-02-22T16:04:06.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.079+0000 7f206e7fc700 1 -- 172.21.15.19:0/369396640 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f20780719d0 con 0x7f206406cc30 2024-02-22T16:04:06.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.081+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f206406cc30 msgr2=0x7f206406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.081+0000 7f208015d700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f206406cc30 0x7f206406f0e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f207400b3b0 tx=0x7f2074009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.081+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 msgr2=0x7f20780707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.081+0000 7f208015d700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 0x7f20780707d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2068002a50 tx=0x7f2068004600 comp rx=0 tx=0).stop 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 shutdown_connections 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f206406cc30 0x7f206406f0e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2078102f50 0x7f2078070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 --2- 172.21.15.19:0/369396640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20781038f0 0x7f20780707d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 >> 172.21.15.19:0/369396640 conn(0x7f20780f8ae0 msgr2=0x7f20780fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 shutdown_connections 2024-02-22T16:04:06.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.082+0000 7f208015d700 1 -- 172.21.15.19:0/369396640 wait complete. 2024-02-22T16:04:06.069 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:04:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 -- 172.21.15.19:0/1151348302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc101e40 msgr2=0x7fbacc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 --2- 172.21.15.19:0/1151348302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc101e40 0x7fbacc102210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbac0004940 tx=0x7fbac0009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 -- 172.21.15.19:0/1151348302 shutdown_connections 2024-02-22T16:04:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 --2- 172.21.15.19:0/1151348302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbacc1027e0 0x7fbacc0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 --2- 172.21.15.19:0/1151348302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc101e40 0x7fbacc102210 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 -- 172.21.15.19:0/1151348302 >> 172.21.15.19:0/1151348302 conn(0x7fbacc071d90 msgr2=0x7fbacc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 -- 172.21.15.19:0/1151348302 shutdown_connections 2024-02-22T16:04:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.186+0000 7fbad378e700 1 -- 172.21.15.19:0/1151348302 wait complete. 2024-02-22T16:04:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad378e700 1 Processor -- start 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad378e700 1 -- start start 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad378e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbacc101e40 0x7fbacc10ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad378e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 0x7fbacc10ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad378e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc10f4f0 con 0x7fbacc1027e0 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad378e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc10f630 con 0x7fbacc101e40 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad0d29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 0x7fbacc10ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad0d29700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 0x7fbacc10ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45752/0 (socket says 172.21.15.19:45752) 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad0d29700 1 -- 172.21.15.19:0/3059301886 learned_addr learned my addr 172.21.15.19:0/3059301886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad152a700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbacc101e40 0x7fbacc10ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad0d29700 1 -- 172.21.15.19:0/3059301886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbacc101e40 msgr2=0x7fbacc10ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad0d29700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbacc101e40 0x7fbacc10ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.187+0000 7fbad0d29700 1 -- 172.21.15.19:0/3059301886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0005040 con 0x7fbacc1027e0 2024-02-22T16:04:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.188+0000 7fbad0d29700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 0x7fbacc10ef60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c890 tx=0x7fbac80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.188+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbac800a7d0 con 0x7fbacc1027e0 2024-02-22T16:04:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.188+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbac800ade0 con 0x7fbacc1027e0 2024-02-22T16:04:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.188+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc117990 con 0x7fbacc1027e0 2024-02-22T16:04:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.188+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbac80113a0 con 0x7fbacc1027e0 2024-02-22T16:04:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.188+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc117ea0 con 0x7fbacc1027e0 2024-02-22T16:04:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.189+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc0ff150 con 0x7fbacc1027e0 2024-02-22T16:04:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.189+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbac8012400 con 0x7fbacc1027e0 2024-02-22T16:04:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.190+0000 7fbabdffb700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbab806cc20 0x7fbab806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.190+0000 7fbad152a700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbab806cc20 0x7fbab806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.190+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbac808c160 con 0x7fbacc1027e0 2024-02-22T16:04:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.190+0000 7fbad152a700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbab806cc20 0x7fbab806f0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbac0004f90 tx=0x7fbac0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.195+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fbac8056190 con 0x7fbacc1027e0 2024-02-22T16:04:06.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.414+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbacc117b20 con 0x7fbab806cc20 2024-02-22T16:04:06.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.415+0000 7fbabdffb700 1 -- 172.21.15.19:0/3059301886 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbacc117b20 con 0x7fbab806cc20 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbab806cc20 msgr2=0x7fbab806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbab806cc20 0x7fbab806f0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbac0004f90 tx=0x7fbac0009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 msgr2=0x7fbacc10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 0x7fbacc10ef60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c890 tx=0x7fbac80044c0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 shutdown_connections 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbab806cc20 0x7fbab806f0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbacc101e40 0x7fbacc10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 --2- 172.21.15.19:0/3059301886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1027e0 0x7fbacc10ef60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 >> 172.21.15.19:0/3059301886 conn(0x7fbacc071d90 msgr2=0x7fbacc0fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:06.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 shutdown_connections 2024-02-22T16:04:06.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.417+0000 7fbad378e700 1 -- 172.21.15.19:0/3059301886 wait complete. 2024-02-22T16:04:06.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 -- 172.21.15.19:0/1637309848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 msgr2=0x7f5040102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 --2- 172.21.15.19:0/1637309848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f5040102210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f503c004940 tx=0x7f503c009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 -- 172.21.15.19:0/1637309848 shutdown_connections 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 --2- 172.21.15.19:0/1637309848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50401027e0 0x7f50400fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 --2- 172.21.15.19:0/1637309848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f5040102210 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 -- 172.21.15.19:0/1637309848 >> 172.21.15.19:0/1637309848 conn(0x7f5040071d90 msgr2=0x7f5040072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 -- 172.21.15.19:0/1637309848 shutdown_connections 2024-02-22T16:04:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.520+0000 7f504740d700 1 -- 172.21.15.19:0/1637309848 wait complete. 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f504740d700 1 Processor -- start 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f504740d700 1 -- start start 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f504740d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f504010c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f504740d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50401027e0 0x7f504010cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f504740d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504010d0d0 con 0x7f5040101e40 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f504740d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504010d240 con 0x7f50401027e0 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f50451a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f504010c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f50451a9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f504010c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45770/0 (socket says 172.21.15.19:45770) 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f50451a9700 1 -- 172.21.15.19:0/4218355639 learned_addr learned my addr 172.21.15.19:0/4218355639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.521+0000 7f50449a8700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50401027e0 0x7f504010cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f50451a9700 1 -- 172.21.15.19:0/4218355639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50401027e0 msgr2=0x7f504010cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f50451a9700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50401027e0 0x7f504010cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f50451a9700 1 -- 172.21.15.19:0/4218355639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c005040 con 0x7f5040101e40 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f50451a9700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f504010c650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f503c009fd0 tx=0x7f503c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f503c00e040 con 0x7f5040101e40 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f503c013420 con 0x7f5040101e40 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504010d4c0 con 0x7f5040101e40 2024-02-22T16:04:06.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f503c01c630 con 0x7f5040101e40 2024-02-22T16:04:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.522+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504011a070 con 0x7f5040101e40 2024-02-22T16:04:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.523+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50400ff150 con 0x7f5040101e40 2024-02-22T16:04:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.524+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f503c013590 con 0x7f5040101e40 2024-02-22T16:04:06.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.524+0000 7f5031ffb700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f502c06cc30 0x7f502c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.524+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f503c08d000 con 0x7f5040101e40 2024-02-22T16:04:06.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.524+0000 7f50449a8700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f502c06cc30 0x7f502c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.525+0000 7f50449a8700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f502c06cc30 0x7f502c06f0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5040066180 tx=0x7f503400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.529+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f503c05ab60 con 0x7f5040101e40 2024-02-22T16:04:06.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.742+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f504010d4c0 con 0x7f502c06cc30 2024-02-22T16:04:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.752+0000 7f5031ffb700 1 -- 172.21.15.19:0/4218355639 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f504010d4c0 con 0x7f502c06cc30 2024-02-22T16:04:06.729 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:04:06.729 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (11m) 4m ago 22m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:04:06.729 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (8m) 4m ago 22m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:04:06.729 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (8m) 4m ago 21m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:04:06.729 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (10m) 4m ago 22m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (4m) 4m ago 16m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (5m) 4m ago 16m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (4m) 4m ago 16m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (4m) 4m ago 16m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (13m) 4m ago 23m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (12m) 4m ago 21m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (9m) 4m ago 24m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (9m) 4m ago 21m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (12m) 4m ago 22m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (12m) 4m ago 21m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 4m ago 20m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (7m) 4m ago 20m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (7m) 4m ago 19m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:04:06.730 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (6m) 4m ago 19m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:04:06.731 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (5m) 4m ago 18m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:04:06.731 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (5m) 4m ago 18m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:04:06.731 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (11m) 4m ago 21m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:04:06.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f502c06cc30 msgr2=0x7f502c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f502c06cc30 0x7f502c06f0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5040066180 tx=0x7f503400b040 comp rx=0 tx=0).stop 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 msgr2=0x7f504010c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f504010c650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f503c009fd0 tx=0x7f503c009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 shutdown_connections 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f502c06cc30 0x7f502c06f0e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5040101e40 0x7f504010c650 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 --2- 172.21.15.19:0/4218355639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50401027e0 0x7f504010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 >> 172.21.15.19:0/4218355639 conn(0x7f5040071d90 msgr2=0x7f50400fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 shutdown_connections 2024-02-22T16:04:06.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.754+0000 7f504740d700 1 -- 172.21.15.19:0/4218355639 wait complete. 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3467902759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 msgr2=0x7fdc840fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3467902759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc840fcf60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc78004940 tx=0x7fdc78009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3467902759 shutdown_connections 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3467902759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc840fcf60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3467902759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc84102f50 0x7fdc84103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3467902759 >> 172.21.15.19:0/3467902759 conn(0x7fdc840f8ae0 msgr2=0x7fdc840faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3467902759 shutdown_connections 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.857+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3467902759 wait complete. 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc88c5b700 1 Processor -- start 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc88c5b700 1 -- start start 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc88c5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc84102f50 0x7fdc84113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc88c5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc84113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc88c5b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc841144c0 con 0x7fdc841038f0 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc88c5b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8410d7f0 con 0x7fdc84102f50 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc81d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc84113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc81d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc84113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45780/0 (socket says 172.21.15.19:45780) 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc81d9b700 1 -- 172.21.15.19:0/3291702400 learned_addr learned my addr 172.21.15.19:0/3291702400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.858+0000 7fdc8259c700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc84102f50 0x7fdc84113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc81d9b700 1 -- 172.21.15.19:0/3291702400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc84102f50 msgr2=0x7fdc84113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc81d9b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc84102f50 0x7fdc84113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc81d9b700 1 -- 172.21.15.19:0/3291702400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78005040 con 0x7fdc841038f0 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc81d9b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc84113e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009fd0 tx=0x7fdc78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7800e040 con 0x7fdc841038f0 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8410da70 con 0x7fdc841038f0 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc78013420 con 0x7fdc841038f0 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7801c630 con 0x7fdc841038f0 2024-02-22T16:04:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.859+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8410dfb0 con 0x7fdc841038f0 2024-02-22T16:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.860+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc84105420 con 0x7fdc841038f0 2024-02-22T16:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.861+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdc78013a00 con 0x7fdc841038f0 2024-02-22T16:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.861+0000 7fdc72ffd700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdc6c06cc20 0x7fdc6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.861+0000 7fdc8259c700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdc6c06cc20 0x7fdc6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.861+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdc7808d290 con 0x7fdc841038f0 2024-02-22T16:04:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.862+0000 7fdc8259c700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdc6c06cc20 0x7fdc6c06f0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc74004f40 tx=0x7fdc7400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:06.866+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fdc7805ad50 con 0x7fdc841038f0 2024-02-22T16:04:07.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.150+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdc8410ef10 con 0x7fdc841038f0 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.151+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3291702400 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fdc7805a8e0 con 0x7fdc841038f0 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:07.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:04:07.129 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdc6c06cc20 msgr2=0x7fdc6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdc6c06cc20 0x7fdc6c06f0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc74004f40 tx=0x7fdc7400b040 comp rx=0 tx=0).stop 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 msgr2=0x7fdc84113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc84113e70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009fd0 tx=0x7fdc78009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 shutdown_connections 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdc6c06cc20 0x7fdc6c06f0d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc84102f50 0x7fdc84113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 --2- 172.21.15.19:0/3291702400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc841038f0 0x7fdc84113e70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 >> 172.21.15.19:0/3291702400 conn(0x7fdc840f8ae0 msgr2=0x7fdc840fa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:07.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 shutdown_connections 2024-02-22T16:04:07.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.153+0000 7fdc88c5b700 1 -- 172.21.15.19:0/3291702400 wait complete. 2024-02-22T16:04:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 -- 172.21.15.19:0/12931721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b341027e0 msgr2=0x7f6b340fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/12931721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b341027e0 0x7f6b340fbea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6b28004940 tx=0x7f6b28009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 -- 172.21.15.19:0/12931721 shutdown_connections 2024-02-22T16:04:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/12931721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b341027e0 0x7f6b340fbea0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/12931721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b34101e40 0x7f6b34102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 -- 172.21.15.19:0/12931721 >> 172.21.15.19:0/12931721 conn(0x7f6b34071d90 msgr2=0x7f6b34072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 -- 172.21.15.19:0/12931721 shutdown_connections 2024-02-22T16:04:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.256+0000 7f6b3bd07700 1 -- 172.21.15.19:0/12931721 wait complete. 2024-02-22T16:04:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b3bd07700 1 Processor -- start 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b3bd07700 1 -- start start 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b3bd07700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 0x7f6b341128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b3bd07700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b341027e0 0x7f6b34112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b3bd07700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34113460 con 0x7f6b34101e40 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b3bd07700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3410c790 con 0x7f6b341027e0 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b39aa3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 0x7f6b341128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b39aa3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 0x7f6b341128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45798/0 (socket says 172.21.15.19:45798) 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b39aa3700 1 -- 172.21.15.19:0/1458376220 learned_addr learned my addr 172.21.15.19:0/1458376220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b392a2700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b341027e0 0x7f6b34112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b39aa3700 1 -- 172.21.15.19:0/1458376220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b341027e0 msgr2=0x7f6b34112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b39aa3700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b341027e0 0x7f6b34112e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b39aa3700 1 -- 172.21.15.19:0/1458376220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28005040 con 0x7f6b34101e40 2024-02-22T16:04:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.257+0000 7f6b392a2700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b341027e0 0x7f6b34112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.258+0000 7f6b39aa3700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 0x7f6b341128d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b30002a50 tx=0x7f6b300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.258+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b30007cc0 con 0x7f6b34101e40 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.258+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b3000f040 con 0x7f6b34101e40 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.258+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3410ca70 con 0x7f6b34101e40 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.258+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b30009540 con 0x7f6b34101e40 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.258+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3410cfb0 con 0x7f6b34101e40 2024-02-22T16:04:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.259+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b34104310 con 0x7f6b34101e40 2024-02-22T16:04:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.259+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6b300096a0 con 0x7f6b34101e40 2024-02-22T16:04:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.260+0000 7f6b267fc700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2006cc20 0x7f6b2006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.260+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6b3008be80 con 0x7f6b34101e40 2024-02-22T16:04:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.260+0000 7f6b392a2700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2006cc20 0x7f6b2006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.260+0000 7f6b392a2700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2006cc20 0x7f6b2006f0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800b3b0 tx=0x7f6b28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:07.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.264+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6b300598c0 con 0x7f6b34101e40 2024-02-22T16:04:07.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:07 smithi062 ceph-mon[106861]: pgmap v325: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:07.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.508+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b3404bed0 con 0x7f6b34101e40 2024-02-22T16:04:07.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.508+0000 7f6b267fc700 1 -- 172.21.15.19:0/1458376220 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f6b30059450 con 0x7f6b34101e40 2024-02-22T16:04:07.485 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:04:07.486 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:04:07.487 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:07.488 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2006cc20 msgr2=0x7f6b2006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2006cc20 0x7f6b2006f0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800b3b0 tx=0x7f6b28009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 msgr2=0x7f6b341128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 0x7f6b341128d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b30002a50 tx=0x7f6b300046b0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 shutdown_connections 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6b2006cc20 0x7f6b2006f0d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34101e40 0x7f6b341128d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 --2- 172.21.15.19:0/1458376220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b341027e0 0x7f6b34112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 >> 172.21.15.19:0/1458376220 conn(0x7f6b34071d90 msgr2=0x7f6b340faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.510+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 shutdown_connections 2024-02-22T16:04:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.511+0000 7f6b3bd07700 1 -- 172.21.15.19:0/1458376220 wait complete. 2024-02-22T16:04:07.490 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:04:07.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:07 smithi019 ceph-mon[173579]: pgmap v325: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.614+0000 7f5cb4d35700 1 -- 172.21.15.19:0/1876801805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 msgr2=0x7f5cb0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.614+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/1876801805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb0102210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0004940 tx=0x7f5ca0009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- 172.21.15.19:0/1876801805 shutdown_connections 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/1876801805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb01027e0 0x7f5cb00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/1876801805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb0102210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- 172.21.15.19:0/1876801805 >> 172.21.15.19:0/1876801805 conn(0x7f5cb0071d90 msgr2=0x7f5cb0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- 172.21.15.19:0/1876801805 shutdown_connections 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- 172.21.15.19:0/1876801805 wait complete. 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 Processor -- start 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- start start 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb01027e0 0x7f5cb010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb010f630 con 0x7f5cb0101e40 2024-02-22T16:04:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.615+0000 7f5cb4d35700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb010f7a0 con 0x7f5cb01027e0 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb010eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44788/0 (socket says 172.21.15.19:44788) 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 -- 172.21.15.19:0/3486266401 learned_addr learned my addr 172.21.15.19:0/3486266401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cadd9b700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb01027e0 0x7f5cb010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 -- 172.21.15.19:0/3486266401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb01027e0 msgr2=0x7f5cb010f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb01027e0 0x7f5cb010f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 -- 172.21.15.19:0/3486266401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0005040 con 0x7f5cb0101e40 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cae59c700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb010eb40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b3b0 tx=0x7f5ca0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca000e040 con 0x7f5cb0101e40 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ca000bc50 con 0x7f5cb0101e40 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb0117ab0 con 0x7f5cb0101e40 2024-02-22T16:04:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca0013420 con 0x7f5cb0101e40 2024-02-22T16:04:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.616+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb0117fc0 con 0x7f5cb0101e40 2024-02-22T16:04:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.617+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb00ff150 con 0x7f5cb0101e40 2024-02-22T16:04:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.618+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5ca001c430 con 0x7f5cb0101e40 2024-02-22T16:04:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.618+0000 7f5c9effd700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5c9806cc20 0x7f5c9806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.618+0000 7f5cadd9b700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5c9806cc20 0x7f5c9806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.618+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5ca008cd50 con 0x7f5cb0101e40 2024-02-22T16:04:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.619+0000 7f5cadd9b700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5c9806cc20 0x7f5c9806f0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0066180 tx=0x7f5ca4007ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.623+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5ca0056ed0 con 0x7f5cb0101e40 2024-02-22T16:04:07.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.844+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cb004bf40 con 0x7f5c9806cc20 2024-02-22T16:04:07.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.846+0000 7f5c9effd700 1 -- 172.21.15.19:0/3486266401 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5cb004bf40 con 0x7f5c9806cc20 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:04:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:04:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:04:07.824 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.847+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5c9806cc20 msgr2=0x7f5c9806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.847+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5c9806cc20 0x7f5c9806f0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0066180 tx=0x7f5ca4007ab0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.847+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 msgr2=0x7f5cb010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.847+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb010eb40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b3b0 tx=0x7f5ca0009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 shutdown_connections 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5c9806cc20 0x7f5c9806f0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cb0101e40 0x7f5cb010eb40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 --2- 172.21.15.19:0/3486266401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cb01027e0 0x7f5cb010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 >> 172.21.15.19:0/3486266401 conn(0x7f5cb0071d90 msgr2=0x7f5cb00faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 shutdown_connections 2024-02-22T16:04:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.848+0000 7f5cb4d35700 1 -- 172.21.15.19:0/3486266401 wait complete. 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 -- 172.21.15.19:0/2598234951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a01038e0 msgr2=0x7f20a00fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2598234951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a01038e0 0x7f20a00fcf50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f209c004940 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 -- 172.21.15.19:0/2598234951 shutdown_connections 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2598234951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a01038e0 0x7f20a00fcf50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2598234951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a0102f40 0x7f20a0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 -- 172.21.15.19:0/2598234951 >> 172.21.15.19:0/2598234951 conn(0x7f20a00f8af0 msgr2=0x7f20a00faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 -- 172.21.15.19:0/2598234951 shutdown_connections 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.951+0000 7f20a83ae700 1 -- 172.21.15.19:0/2598234951 wait complete. 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a83ae700 1 Processor -- start 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a83ae700 1 -- start start 2024-02-22T16:04:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a83ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 0x7f20a0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a83ae700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a01038e0 0x7f20a0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a83ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01145d0 con 0x7f20a0102f40 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a83ae700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a010d900 con 0x7f20a01038e0 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 0x7f20a0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 0x7f20a0113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44804/0 (socket says 172.21.15.19:44804) 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 -- 172.21.15.19:0/2098290871 learned_addr learned my addr 172.21.15.19:0/2098290871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a5949700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a01038e0 0x7f20a0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 -- 172.21.15.19:0/2098290871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a01038e0 msgr2=0x7f20a0113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a01038e0 0x7f20a0113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 -- 172.21.15.19:0/2098290871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c005040 con 0x7f20a0102f40 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.952+0000 7f20a614a700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 0x7f20a0113a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2090002a50 tx=0x7f20900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.953+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2090007cc0 con 0x7f20a0102f40 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.953+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f209000f040 con 0x7f20a0102f40 2024-02-22T16:04:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.953+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a010dbe0 con 0x7f20a0102f40 2024-02-22T16:04:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.953+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2090009540 con 0x7f20a0102f40 2024-02-22T16:04:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.953+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a010e120 con 0x7f20a0102f40 2024-02-22T16:04:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.954+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a0105410 con 0x7f20a0102f40 2024-02-22T16:04:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.954+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f20900096a0 con 0x7f20a0102f40 2024-02-22T16:04:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.955+0000 7f2096ffd700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f208c06cc20 0x7f208c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.955+0000 7f20a5949700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f208c06cc20 0x7f208c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.955+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f209008bd80 con 0x7f20a0102f40 2024-02-22T16:04:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.955+0000 7f20a5949700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f208c06cc20 0x7f208c06f0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20a010f270 tx=0x7f209c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:07.960+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2090059840 con 0x7f20a0102f40 2024-02-22T16:04:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.246+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f20a004bed0 con 0x7f20a0102f40 2024-02-22T16:04:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.246+0000 7f2096ffd700 1 -- 172.21.15.19:0/2098290871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f209008d100 con 0x7f20a0102f40 2024-02-22T16:04:08.223 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:04:08.223 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:04:08.224 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:04:08.224 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:04:08.224 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:04:08.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f208c06cc20 msgr2=0x7f208c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:08.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f208c06cc20 0x7f208c06f0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20a010f270 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:08.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 msgr2=0x7f20a0113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 0x7f20a0113a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2090002a50 tx=0x7f20900046b0 comp rx=0 tx=0).stop 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 shutdown_connections 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f208c06cc20 0x7f208c06f0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20a0102f40 0x7f20a0113a40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 --2- 172.21.15.19:0/2098290871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20a01038e0 0x7f20a0113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 >> 172.21.15.19:0/2098290871 conn(0x7f20a00f8af0 msgr2=0x7f20a00fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 shutdown_connections 2024-02-22T16:04:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:08.248+0000 7f20a83ae700 1 -- 172.21.15.19:0/2098290871 wait complete. 2024-02-22T16:04:08.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:08 smithi062 ceph-mon[106861]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:08.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:08 smithi062 ceph-mon[106861]: from='client.34656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:08.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:08 smithi062 ceph-mon[106861]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:08.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:08 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3291702400' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:04:08.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:08 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1458376220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:04:08.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:08 smithi019 ceph-mon[173579]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:08.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:08 smithi019 ceph-mon[173579]: from='client.34656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:08.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:08 smithi019 ceph-mon[173579]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:08.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:08 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3291702400' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:04:08.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:08 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1458376220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:04:09.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:09 smithi062 ceph-mon[106861]: pgmap v326: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:09.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:09 smithi062 ceph-mon[106861]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:09.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:09 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2098290871' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:04:09.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:09 smithi019 ceph-mon[173579]: pgmap v326: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:09.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:09 smithi019 ceph-mon[173579]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:09.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:09 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2098290871' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:04:11.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:11 smithi019 ceph-mon[173579]: pgmap v327: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:11.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:11 smithi062 ceph-mon[106861]: pgmap v327: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:13.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:13 smithi062 ceph-mon[106861]: pgmap v328: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:13 smithi019 ceph-mon[173579]: pgmap v328: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:15.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:15 smithi062 ceph-mon[106861]: pgmap v329: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:15.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:15 smithi019 ceph-mon[173579]: pgmap v329: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:17.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:17 smithi062 ceph-mon[106861]: pgmap v330: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:17.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:17 smithi019 ceph-mon[173579]: pgmap v330: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:19 smithi062 ceph-mon[106861]: pgmap v331: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:19.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:19 smithi019 ceph-mon[173579]: pgmap v331: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:21.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:21 smithi019 ceph-mon[173579]: pgmap v332: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:21.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:21 smithi062 ceph-mon[106861]: pgmap v332: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:23.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:23 smithi062 ceph-mon[106861]: pgmap v333: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:23 smithi019 ceph-mon[173579]: pgmap v333: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:25.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:25 smithi062 ceph-mon[106861]: pgmap v334: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:25.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:25 smithi019 ceph-mon[173579]: pgmap v334: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:27.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:27 smithi062 ceph-mon[106861]: pgmap v335: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:27.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:27 smithi019 ceph-mon[173579]: pgmap v335: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:29.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:29 smithi062 ceph-mon[106861]: pgmap v336: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:29.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:29 smithi019 ceph-mon[173579]: pgmap v336: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:31.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:31 smithi019 ceph-mon[173579]: pgmap v337: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:31.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:31 smithi062 ceph-mon[106861]: pgmap v337: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:33.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:33 smithi062 ceph-mon[106861]: pgmap v338: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:33.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:33 smithi019 ceph-mon[173579]: pgmap v338: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:34.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:34.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:35.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:35 smithi062 ceph-mon[106861]: pgmap v339: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:35.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:35 smithi019 ceph-mon[173579]: pgmap v339: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:37.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:37 smithi062 ceph-mon[106861]: pgmap v340: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:37.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:37 smithi019 ceph-mon[173579]: pgmap v340: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.365+0000 7f10d5e84700 1 -- 172.21.15.19:0/817122156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d0101e40 msgr2=0x7f10d0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.365+0000 7f10d5e84700 1 --2- 172.21.15.19:0/817122156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d0101e40 0x7f10d0102210 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10c0004940 tx=0x7f10c0009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 -- 172.21.15.19:0/817122156 shutdown_connections 2024-02-22T16:04:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 --2- 172.21.15.19:0/817122156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d01027e0 0x7f10d00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 --2- 172.21.15.19:0/817122156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d0101e40 0x7f10d0102210 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 -- 172.21.15.19:0/817122156 >> 172.21.15.19:0/817122156 conn(0x7f10d0071d90 msgr2=0x7f10d0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 -- 172.21.15.19:0/817122156 shutdown_connections 2024-02-22T16:04:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 -- 172.21.15.19:0/817122156 wait complete. 2024-02-22T16:04:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 Processor -- start 2024-02-22T16:04:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.366+0000 7f10d5e84700 1 -- start start 2024-02-22T16:04:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10d5e84700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d0101e40 0x7f10d0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10d5e84700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 0x7f10d0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10d5e84700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0114520 con 0x7f10d01027e0 2024-02-22T16:04:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10d5e84700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d010d850 con 0x7f10d0101e40 2024-02-22T16:04:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 0x7f10d0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 0x7f10d0113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38172/0 (socket says 172.21.15.19:38172) 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 -- 172.21.15.19:0/3533516119 learned_addr learned my addr 172.21.15.19:0/3533516119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10cf7fe700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d0101e40 0x7f10d0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 -- 172.21.15.19:0/3533516119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d0101e40 msgr2=0x7f10d0113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d0101e40 0x7f10d0113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 -- 172.21.15.19:0/3533516119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0005040 con 0x7f10d01027e0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10ceffd700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 0x7f10d0113ed0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10c4002a80 tx=0x7f10c4004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10c4004e80 con 0x7f10d01027e0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.367+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d010db30 con 0x7f10d01027e0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.368+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f10c400f040 con 0x7f10d01027e0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.368+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10c400da00 con 0x7f10d01027e0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.368+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d010e070 con 0x7f10d01027e0 2024-02-22T16:04:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.368+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d00738b0 con 0x7f10d01027e0 2024-02-22T16:04:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.369+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f10c4007cc0 con 0x7f10d01027e0 2024-02-22T16:04:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.370+0000 7f10b7fff700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f10b806cc30 0x7f10b806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.370+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f10c408b6c0 con 0x7f10d01027e0 2024-02-22T16:04:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.370+0000 7f10cf7fe700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f10b806cc30 0x7f10b806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.370+0000 7f10cf7fe700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f10b806cc30 0x7f10b806f0e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b3b0 tx=0x7f10c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.375+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f10c4059100 con 0x7f10d01027e0 2024-02-22T16:04:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.597+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10d0113660 con 0x7f10b806cc30 2024-02-22T16:04:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.599+0000 7f10b7fff700 1 -- 172.21.15.19:0/3533516119 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f10d0113660 con 0x7f10b806cc30 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f10b806cc30 msgr2=0x7f10b806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f10b806cc30 0x7f10b806f0e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b3b0 tx=0x7f10c0009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 msgr2=0x7f10d0113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 0x7f10d0113ed0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10c4002a80 tx=0x7f10c4004630 comp rx=0 tx=0).stop 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 shutdown_connections 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f10b806cc30 0x7f10b806f0e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10d0101e40 0x7f10d0113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 --2- 172.21.15.19:0/3533516119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d01027e0 0x7f10d0113ed0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 >> 172.21.15.19:0/3533516119 conn(0x7f10d0071d90 msgr2=0x7f10d0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 shutdown_connections 2024-02-22T16:04:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.601+0000 7f10d5e84700 1 -- 172.21.15.19:0/3533516119 wait complete. 2024-02-22T16:04:38.589 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:04:38.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 -- 172.21.15.19:0/4048724688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fe8f0 msgr2=0x7f9958107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 --2- 172.21.15.19:0/4048724688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fe8f0 0x7f9958107050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9948004970 tx=0x7f9948009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 -- 172.21.15.19:0/4048724688 shutdown_connections 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 --2- 172.21.15.19:0/4048724688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fe8f0 0x7f9958107050 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 --2- 172.21.15.19:0/4048724688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fdfe0 0x7f99580fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 -- 172.21.15.19:0/4048724688 >> 172.21.15.19:0/4048724688 conn(0x7f9958071d90 msgr2=0x7f9958072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 -- 172.21.15.19:0/4048724688 shutdown_connections 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 -- 172.21.15.19:0/4048724688 wait complete. 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 Processor -- start 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.706+0000 7f995dce3700 1 -- start start 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f995dce3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 0x7f99580fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f995dce3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fe8f0 0x7f99580fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:38.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f995dce3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99580fc9e0 con 0x7f99580fdfe0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f995dce3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99580fcb50 con 0x7f99580fe8f0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 0x7f99580fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 0x7f99580fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38184/0 (socket says 172.21.15.19:38184) 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 -- 172.21.15.19:0/1731019691 learned_addr learned my addr 172.21.15.19:0/1731019691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f9956ffd700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fe8f0 0x7f99580fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 -- 172.21.15.19:0/1731019691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fe8f0 msgr2=0x7f99580fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fe8f0 0x7f99580fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 -- 172.21.15.19:0/1731019691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948005040 con 0x7f99580fdfe0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f9956ffd700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fe8f0 0x7f99580fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f99577fe700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 0x7f99580fbea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9940002a50 tx=0x7f99400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940007cc0 con 0x7f99580fdfe0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f994000f040 con 0x7f99580fdfe0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9958119bb0 con 0x7f99580fdfe0 2024-02-22T16:04:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940009540 con 0x7f99580fdfe0 2024-02-22T16:04:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.707+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995811a060 con 0x7f99580fdfe0 2024-02-22T16:04:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.709+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99581046c0 con 0x7f99580fdfe0 2024-02-22T16:04:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.709+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f99400096a0 con 0x7f99580fdfe0 2024-02-22T16:04:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.709+0000 7f994ffff700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f994406cc20 0x7f994406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.709+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f994008be80 con 0x7f99580fdfe0 2024-02-22T16:04:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.709+0000 7f9956ffd700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f994406cc20 0x7f994406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.710+0000 7f9956ffd700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f994406cc20 0x7f994406f0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9948004f50 tx=0x7f9948009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:38.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.714+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9940055e30 con 0x7f99580fdfe0 2024-02-22T16:04:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.931+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99580fd5e0 con 0x7f994406cc20 2024-02-22T16:04:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.932+0000 7f994ffff700 1 -- 172.21.15.19:0/1731019691 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f99580fd5e0 con 0x7f994406cc20 2024-02-22T16:04:38.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.934+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f994406cc20 msgr2=0x7f994406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.934+0000 7f995dce3700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f994406cc20 0x7f994406f0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9948004f50 tx=0x7f9948009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.934+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 msgr2=0x7f99580fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.934+0000 7f995dce3700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 0x7f99580fbea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9940002a50 tx=0x7f99400046b0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 shutdown_connections 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f994406cc20 0x7f994406f0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99580fdfe0 0x7f99580fbea0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 --2- 172.21.15.19:0/1731019691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99580fe8f0 0x7f99580fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 >> 172.21.15.19:0/1731019691 conn(0x7f9958071d90 msgr2=0x7f9958101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 shutdown_connections 2024-02-22T16:04:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:38.935+0000 7f995dce3700 1 -- 172.21.15.19:0/1731019691 wait complete. 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 -- 172.21.15.19:0/1681667443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 msgr2=0x7f052c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 --2- 172.21.15.19:0/1681667443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c106fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0528004940 tx=0x7f0528009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 -- 172.21.15.19:0/1681667443 shutdown_connections 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 --2- 172.21.15.19:0/1681667443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c106fa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 --2- 172.21.15.19:0/1681667443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f052c0fdf30 0x7f052c0fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 -- 172.21.15.19:0/1681667443 >> 172.21.15.19:0/1681667443 conn(0x7f052c0f79d0 msgr2=0x7f052c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 -- 172.21.15.19:0/1681667443 shutdown_connections 2024-02-22T16:04:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 -- 172.21.15.19:0/1681667443 wait complete. 2024-02-22T16:04:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.039+0000 7f0533776700 1 Processor -- start 2024-02-22T16:04:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0533776700 1 -- start start 2024-02-22T16:04:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0533776700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f052c0fdf30 0x7f052c0fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0533776700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c0fc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0533776700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0fc930 con 0x7f052c0fe8d0 2024-02-22T16:04:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0533776700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0fcaa0 con 0x7f052c0fdf30 2024-02-22T16:04:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c0fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c0fc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38206/0 (socket says 172.21.15.19:38206) 2024-02-22T16:04:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 -- 172.21.15.19:0/1774029174 learned_addr learned my addr 172.21.15.19:0/1774029174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0531512700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f052c0fdf30 0x7f052c0fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 -- 172.21.15.19:0/1774029174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f052c0fdf30 msgr2=0x7f052c0fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f052c0fdf30 0x7f052c0fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 -- 172.21.15.19:0/1774029174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528005040 con 0x7f052c0fe8d0 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0530d11700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c0fc330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.040+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0528021070 con 0x7f052c0fe8d0 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.041+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f052801c400 con 0x7f052c0fe8d0 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.041+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c119c60 con 0x7f052c0fe8d0 2024-02-22T16:04:39.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.041+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052800e070 con 0x7f052c0fe8d0 2024-02-22T16:04:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.041+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c11a0c0 con 0x7f052c0fe8d0 2024-02-22T16:04:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.042+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c104600 con 0x7f052c0fe8d0 2024-02-22T16:04:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.042+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f052801c9c0 con 0x7f052c0fe8d0 2024-02-22T16:04:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.043+0000 7f0521ffb700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f051806cc20 0x7f051806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.043+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f052808d220 con 0x7f052c0fe8d0 2024-02-22T16:04:39.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.043+0000 7f0531512700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f051806cc20 0x7f051806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.043+0000 7f0531512700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f051806cc20 0x7f051806f0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f051c004600 tx=0x7f051c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:39.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.047+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0528057250 con 0x7f052c0fe8d0 2024-02-22T16:04:39.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.261+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f052c04bed0 con 0x7f051806cc20 2024-02-22T16:04:39.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.271+0000 7f0521ffb700 1 -- 172.21.15.19:0/1774029174 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f052c04bed0 con 0x7f051806cc20 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (11m) 5m ago 22m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (9m) 5m ago 23m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (9m) 4m ago 22m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (11m) 5m ago 22m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (5m) 5m ago 17m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (5m) 5m ago 17m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (4m) 4m ago 17m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (4m) 4m ago 17m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (13m) 5m ago 24m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:04:39.249 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (13m) 4m ago 21m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (10m) 5m ago 24m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (9m) 4m ago 21m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (12m) 5m ago 23m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (12m) 4m ago 21m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (8m) 5m ago 21m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (8m) 5m ago 20m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (7m) 5m ago 20m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (6m) 4m ago 19m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (6m) 4m ago 19m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (5m) 4m ago 18m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:04:39.250 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (12m) 5m ago 22m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f051806cc20 msgr2=0x7f051806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f051806cc20 0x7f051806f0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f051c004600 tx=0x7f051c00b040 comp rx=0 tx=0).stop 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 msgr2=0x7f052c0fc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c0fc330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f0528009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 shutdown_connections 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f051806cc20 0x7f051806f0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f052c0fdf30 0x7f052c0fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 --2- 172.21.15.19:0/1774029174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f052c0fe8d0 0x7f052c0fc330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 >> 172.21.15.19:0/1774029174 conn(0x7f052c0f79d0 msgr2=0x7f052c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 shutdown_connections 2024-02-22T16:04:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.273+0000 7f0533776700 1 -- 172.21.15.19:0/1774029174 wait complete. 2024-02-22T16:04:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 -- 172.21.15.19:0/1096586721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 msgr2=0x7f1658102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 --2- 172.21.15.19:0/1096586721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658102210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f164c004940 tx=0x7f164c009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 -- 172.21.15.19:0/1096586721 shutdown_connections 2024-02-22T16:04:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 --2- 172.21.15.19:0/1096586721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 0x7f16580fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 --2- 172.21.15.19:0/1096586721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658102210 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 -- 172.21.15.19:0/1096586721 >> 172.21.15.19:0/1096586721 conn(0x7f16580f79d0 msgr2=0x7f16580f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:39.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 -- 172.21.15.19:0/1096586721 shutdown_connections 2024-02-22T16:04:39.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 -- 172.21.15.19:0/1096586721 wait complete. 2024-02-22T16:04:39.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 Processor -- start 2024-02-22T16:04:39.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.375+0000 7f165eca8700 1 -- start start 2024-02-22T16:04:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165eca8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165eca8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 0x7f1658116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165eca8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16581166b0 con 0x7f1658101e40 2024-02-22T16:04:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165eca8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165810f9e0 con 0x7f16581027e0 2024-02-22T16:04:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658115b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38222/0 (socket says 172.21.15.19:38222) 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 -- 172.21.15.19:0/2319168068 learned_addr learned my addr 172.21.15.19:0/2319168068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f1657fff700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 0x7f1658116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 -- 172.21.15.19:0/2319168068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 msgr2=0x7f1658116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 0x7f1658116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 -- 172.21.15.19:0/2319168068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c005040 con 0x7f1658101e40 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f1657fff700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 0x7f1658116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f165ca44700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658115b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f164c009180 tx=0x7f164c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.376+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164c00e040 con 0x7f1658101e40 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.377+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f164c013420 con 0x7f1658101e40 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.377+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165810fc60 con 0x7f1658101e40 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.377+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164c01c630 con 0x7f1658101e40 2024-02-22T16:04:39.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.377+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16581101a0 con 0x7f1658101e40 2024-02-22T16:04:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.377+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16580fb5d0 con 0x7f1658101e40 2024-02-22T16:04:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.378+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f164c0139e0 con 0x7f1658101e40 2024-02-22T16:04:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.378+0000 7f16557fa700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f164806cc20 0x7f164806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.379+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f164c08d380 con 0x7f1658101e40 2024-02-22T16:04:39.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.379+0000 7f1657fff700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f164806cc20 0x7f164806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.379+0000 7f1657fff700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f164806cc20 0x7f164806f0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1644009b80 tx=0x7f164400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.383+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f164c05ae40 con 0x7f1658101e40 2024-02-22T16:04:39.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:39 smithi062 ceph-mon[106861]: pgmap v341: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:39.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:39 smithi019 ceph-mon[173579]: pgmap v341: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.668+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1658108e80 con 0x7f1658101e40 2024-02-22T16:04:39.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.669+0000 7f16557fa700 1 -- 172.21.15.19:0/2319168068 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f1658108e80 con 0x7f1658101e40 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:04:39.890 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:04:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:04:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:04:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:04:39.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:04:39.891 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f164806cc20 msgr2=0x7f164806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f164806cc20 0x7f164806f0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1644009b80 tx=0x7f164400b040 comp rx=0 tx=0).stop 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 msgr2=0x7f1658115b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658115b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f164c009180 tx=0x7f164c009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 shutdown_connections 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f164806cc20 0x7f164806f0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1658101e40 0x7f1658115b20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 --2- 172.21.15.19:0/2319168068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16581027e0 0x7f1658116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 >> 172.21.15.19:0/2319168068 conn(0x7f16580f79d0 msgr2=0x7f1658065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 shutdown_connections 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.671+0000 7f165eca8700 1 -- 172.21.15.19:0/2319168068 wait complete. 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 -- 172.21.15.19:0/3853539867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 msgr2=0x7f5b340fe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3853539867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b340fe310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5b24004940 tx=0x7f5b24009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 -- 172.21.15.19:0/3853539867 shutdown_connections 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3853539867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b340fe8e0 0x7f5b34106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3853539867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b340fe310 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 -- 172.21.15.19:0/3853539867 >> 172.21.15.19:0/3853539867 conn(0x7f5b340f79d0 msgr2=0x7f5b340f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 -- 172.21.15.19:0/3853539867 shutdown_connections 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 -- 172.21.15.19:0/3853539867 wait complete. 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 Processor -- start 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.775+0000 7f5b39f67700 1 -- start start 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b39f67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b34113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b39f67700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b340fe8e0 0x7f5b34113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b39f67700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34114500 con 0x7f5b340fdf40 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b39f67700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3410d830 con 0x7f5b340fe8e0 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b34113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b34113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38240/0 (socket says 172.21.15.19:38240) 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 -- 172.21.15.19:0/3235620570 learned_addr learned my addr 172.21.15.19:0/3235620570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b32ffd700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b340fe8e0 0x7f5b34113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 -- 172.21.15.19:0/3235620570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b340fe8e0 msgr2=0x7f5b34113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b340fe8e0 0x7f5b34113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 -- 172.21.15.19:0/3235620570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24005040 con 0x7f5b340fdf40 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b337fe700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b34113970 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b3b0 tx=0x7f5b24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2400e040 con 0x7f5b340fdf40 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b24004290 con 0x7f5b340fdf40 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3410dab0 con 0x7f5b340fdf40 2024-02-22T16:04:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.776+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2401c3f0 con 0x7f5b340fdf40 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.777+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3410dff0 con 0x7f5b340fdf40 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.777+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b34104530 con 0x7f5b340fdf40 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.778+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b24013820 con 0x7f5b340fdf40 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.779+0000 7f5b2bfff700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b2006cc20 0x7f5b2006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.779+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5b2408ded0 con 0x7f5b340fdf40 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.779+0000 7f5b32ffd700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b2006cc20 0x7f5b2006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.779+0000 7f5b32ffd700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b2006cc20 0x7f5b2006f0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b3410f1a0 tx=0x7f5b1c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:39.783+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5b2405b970 con 0x7f5b340fdf40 2024-02-22T16:04:40.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.030+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b340fb5d0 con 0x7f5b340fdf40 2024-02-22T16:04:40.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.031+0000 7f5b2bfff700 1 -- 172.21.15.19:0/3235620570 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f5b2405b500 con 0x7f5b340fdf40 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:04:40.009 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:04:40.010 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:40.011 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:40.012 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:04:40.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b2006cc20 msgr2=0x7f5b2006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b2006cc20 0x7f5b2006f0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b3410f1a0 tx=0x7f5b1c00b040 comp rx=0 tx=0).stop 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 msgr2=0x7f5b34113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b34113970 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b3b0 tx=0x7f5b24009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 shutdown_connections 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b2006cc20 0x7f5b2006f0d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b340fdf40 0x7f5b34113970 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 --2- 172.21.15.19:0/3235620570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b340fe8e0 0x7f5b34113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 >> 172.21.15.19:0/3235620570 conn(0x7f5b340f79d0 msgr2=0x7f5b341016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 shutdown_connections 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.033+0000 7f5b39f67700 1 -- 172.21.15.19:0/3235620570 wait complete. 2024-02-22T16:04:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:04:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 -- 172.21.15.19:0/3988474997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 msgr2=0x7f700c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 --2- 172.21.15.19:0/3988474997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c102210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7008004940 tx=0x7f7008009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 -- 172.21.15.19:0/3988474997 shutdown_connections 2024-02-22T16:04:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 --2- 172.21.15.19:0/3988474997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f700c1027e0 0x7f700c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 --2- 172.21.15.19:0/3988474997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c102210 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 -- 172.21.15.19:0/3988474997 >> 172.21.15.19:0/3988474997 conn(0x7f700c071d90 msgr2=0x7f700c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 -- 172.21.15.19:0/3988474997 shutdown_connections 2024-02-22T16:04:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.137+0000 7f7013e3d700 1 -- 172.21.15.19:0/3988474997 wait complete. 2024-02-22T16:04:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7013e3d700 1 Processor -- start 2024-02-22T16:04:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7013e3d700 1 -- start start 2024-02-22T16:04:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7013e3d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7013e3d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f700c1027e0 0x7f700c116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7013e3d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c116750 con 0x7f700c101e40 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7013e3d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c10fa80 con 0x7f700c1027e0 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7011bd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7011bd9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38264/0 (socket says 172.21.15.19:38264) 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.138+0000 7f7011bd9700 1 -- 172.21.15.19:0/1854052858 learned_addr learned my addr 172.21.15.19:0/1854052858 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f70113d8700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f700c1027e0 0x7f700c116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f7011bd9700 1 -- 172.21.15.19:0/1854052858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f700c1027e0 msgr2=0x7f700c116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f7011bd9700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f700c1027e0 0x7f700c116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f7011bd9700 1 -- 172.21.15.19:0/1854052858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7008005040 con 0x7f700c101e40 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f7011bd9700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c115bc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7008009fd0 tx=0x7f7008009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f700800e040 con 0x7f700c101e40 2024-02-22T16:04:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f700800bc50 con 0x7f700c101e40 2024-02-22T16:04:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c10fd00 con 0x7f700c101e40 2024-02-22T16:04:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7008013ea0 con 0x7f700c101e40 2024-02-22T16:04:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.139+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c110240 con 0x7f700c101e40 2024-02-22T16:04:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.140+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c0738b0 con 0x7f700c101e40 2024-02-22T16:04:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.141+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7008013420 con 0x7f700c101e40 2024-02-22T16:04:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.141+0000 7f6ffe7fc700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ff806cc30 0x7f6ff806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.141+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f700808d5c0 con 0x7f700c101e40 2024-02-22T16:04:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.141+0000 7f70113d8700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ff806cc30 0x7f6ff806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.141+0000 7f70113d8700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ff806cc30 0x7f6ff806f0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f700c1113f0 tx=0x7f7000009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:40.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.146+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f700805b000 con 0x7f700c101e40 2024-02-22T16:04:40.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.367+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f700c110460 con 0x7f6ff806cc30 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.368+0000 7f6ffe7fc700 1 -- 172.21.15.19:0/1854052858 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f700c110460 con 0x7f6ff806cc30 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:04:40.346 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:04:40.347 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ff806cc30 msgr2=0x7f6ff806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ff806cc30 0x7f6ff806f0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f700c1113f0 tx=0x7f7000009ee0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 msgr2=0x7f700c115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c115bc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7008009fd0 tx=0x7f7008009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 shutdown_connections 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6ff806cc30 0x7f6ff806f0e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c101e40 0x7f700c115bc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 --2- 172.21.15.19:0/1854052858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f700c1027e0 0x7f700c116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 >> 172.21.15.19:0/1854052858 conn(0x7f700c071d90 msgr2=0x7f700c065720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 shutdown_connections 2024-02-22T16:04:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.370+0000 7f7013e3d700 1 -- 172.21.15.19:0/1854052858 wait complete. 2024-02-22T16:04:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.473+0000 7f668674a700 1 -- 172.21.15.19:0/329063937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6680101e40 msgr2=0x7f6680102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.473+0000 7f668674a700 1 --2- 172.21.15.19:0/329063937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6680101e40 0x7f6680102210 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6670004940 tx=0x7f6670009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- 172.21.15.19:0/329063937 shutdown_connections 2024-02-22T16:04:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 --2- 172.21.15.19:0/329063937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66801027e0 0x7f66800fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 --2- 172.21.15.19:0/329063937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6680101e40 0x7f6680102210 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- 172.21.15.19:0/329063937 >> 172.21.15.19:0/329063937 conn(0x7f6680071d90 msgr2=0x7f6680072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- 172.21.15.19:0/329063937 shutdown_connections 2024-02-22T16:04:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- 172.21.15.19:0/329063937 wait complete. 2024-02-22T16:04:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 Processor -- start 2024-02-22T16:04:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- start start 2024-02-22T16:04:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6680101e40 0x7f668010ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 0x7f668010ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668010f4f0 con 0x7f66801027e0 2024-02-22T16:04:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.474+0000 7f668674a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668010f630 con 0x7f6680101e40 2024-02-22T16:04:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 0x7f668010ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 0x7f668010ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38278/0 (socket says 172.21.15.19:38278) 2024-02-22T16:04:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 -- 172.21.15.19:0/2972215716 learned_addr learned my addr 172.21.15.19:0/2972215716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:04:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f667ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6680101e40 0x7f668010ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 -- 172.21.15.19:0/2972215716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6680101e40 msgr2=0x7f668010ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6680101e40 0x7f668010ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 -- 172.21.15.19:0/2972215716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670005040 con 0x7f66801027e0 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f6677fff700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 0x7f668010ef60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6668002a50 tx=0x7f6668004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6668007cc0 con 0x7f66801027e0 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f666800f040 con 0x7f66801027e0 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6680117990 con 0x7f66801027e0 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666800d920 con 0x7f66801027e0 2024-02-22T16:04:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.475+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6680117ea0 con 0x7f66801027e0 2024-02-22T16:04:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:40 smithi062 ceph-mon[106861]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:40 smithi062 ceph-mon[106861]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:40 smithi062 ceph-mon[106861]: pgmap v342: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:40 smithi062 ceph-mon[106861]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:40 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2319168068' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:04:40.458 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:40 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3235620570' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:04:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.476+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66800ff150 con 0x7f66801027e0 2024-02-22T16:04:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.477+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6668009680 con 0x7f66801027e0 2024-02-22T16:04:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.477+0000 7f667d7fa700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f666c06cc20 0x7f666c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:04:40.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.477+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f666808bd20 con 0x7f66801027e0 2024-02-22T16:04:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.477+0000 7f667ffff700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f666c06cc20 0x7f666c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:04:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.478+0000 7f667ffff700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f666c06cc20 0x7f666c06f0d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6670004f90 tx=0x7f6670009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:04:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.482+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6668055d50 con 0x7f66801027e0 2024-02-22T16:04:40.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:40 smithi019 ceph-mon[173579]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:40.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:40 smithi019 ceph-mon[173579]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:40.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:40 smithi019 ceph-mon[173579]: pgmap v342: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:40.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:40 smithi019 ceph-mon[173579]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:40.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:40 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2319168068' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:04:40.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:40 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3235620570' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:04:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.768+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f66800738b0 con 0x7f66801027e0 2024-02-22T16:04:40.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.768+0000 7f667d7fa700 1 -- 172.21.15.19:0/2972215716 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6668059370 con 0x7f66801027e0 2024-02-22T16:04:40.747 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:04:40.747 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:04:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:04:40.747 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:04:40.747 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f666c06cc20 msgr2=0x7f666c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f666c06cc20 0x7f666c06f0d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6670004f90 tx=0x7f6670009e30 comp rx=0 tx=0).stop 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 msgr2=0x7f668010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 0x7f668010ef60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6668002a50 tx=0x7f6668004600 comp rx=0 tx=0).stop 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 shutdown_connections 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f666c06cc20 0x7f666c06f0d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6680101e40 0x7f668010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 --2- 172.21.15.19:0/2972215716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66801027e0 0x7f668010ef60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 >> 172.21.15.19:0/2972215716 conn(0x7f6680071d90 msgr2=0x7f66800fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:04:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 shutdown_connections 2024-02-22T16:04:40.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:04:40.771+0000 7f668674a700 1 -- 172.21.15.19:0/2972215716 wait complete. 2024-02-22T16:04:41.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:41 smithi019 ceph-mon[173579]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:41.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:41 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2972215716' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:04:41.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:41 smithi062 ceph-mon[106861]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:04:41.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:41 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2972215716' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:04:42.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:42 smithi062 ceph-mon[106861]: pgmap v343: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:42.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:42 smithi019 ceph-mon[173579]: pgmap v343: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:45.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:45 smithi062 ceph-mon[106861]: pgmap v344: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:45.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:45 smithi019 ceph-mon[173579]: pgmap v344: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:47.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:47 smithi062 ceph-mon[106861]: pgmap v345: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:47.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:47 smithi019 ceph-mon[173579]: pgmap v345: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:49 smithi062 ceph-mon[106861]: pgmap v346: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:49 smithi019 ceph-mon[173579]: pgmap v346: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:04:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:51 smithi019 ceph-mon[173579]: pgmap v347: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:51.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:51 smithi062 ceph-mon[106861]: pgmap v347: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:53.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:53 smithi062 ceph-mon[106861]: pgmap v348: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:53.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:53 smithi019 ceph-mon[173579]: pgmap v348: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:55.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:55 smithi062 ceph-mon[106861]: pgmap v349: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:55.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:55 smithi019 ceph-mon[173579]: pgmap v349: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:57.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:57 smithi062 ceph-mon[106861]: pgmap v350: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:57.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:57 smithi019 ceph-mon[173579]: pgmap v350: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:59.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:04:59 smithi062 ceph-mon[106861]: pgmap v351: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:04:59.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:04:59 smithi019 ceph-mon[173579]: pgmap v351: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:01.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:01 smithi019 ceph-mon[173579]: pgmap v352: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:01.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:01 smithi062 ceph-mon[106861]: pgmap v352: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:03.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:03 smithi062 ceph-mon[106861]: pgmap v353: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:03.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:03 smithi019 ceph-mon[173579]: pgmap v353: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:04.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:04.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:05.202 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:05 smithi062 ceph-mon[106861]: pgmap v354: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:05.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:05 smithi019 ceph-mon[173579]: pgmap v354: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:07.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:07 smithi062 ceph-mon[106861]: pgmap v355: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:07.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:07 smithi019 ceph-mon[173579]: pgmap v355: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:09.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:09 smithi062 ceph-mon[106861]: pgmap v356: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:09.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:09 smithi019 ceph-mon[173579]: pgmap v356: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 -- 172.21.15.19:0/275002679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fdfe0 msgr2=0x7fb02c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 --2- 172.21.15.19:0/275002679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c0fe3b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb020004970 tx=0x7fb020009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 -- 172.21.15.19:0/275002679 shutdown_connections 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 --2- 172.21.15.19:0/275002679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 --2- 172.21.15.19:0/275002679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c0fe3b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 -- 172.21.15.19:0/275002679 >> 172.21.15.19:0/275002679 conn(0x7fb02c071d90 msgr2=0x7fb02c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 -- 172.21.15.19:0/275002679 shutdown_connections 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.889+0000 7fb032c7d700 1 -- 172.21.15.19:0/275002679 wait complete. 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb032c7d700 1 Processor -- start 2024-02-22T16:05:10.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb032c7d700 1 -- start start 2024-02-22T16:05:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb032c7d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb032c7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb032c7d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c1144c0 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb032c7d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c10d7f0 con 0x7fb02c0fdfe0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52742/0 (socket says 172.21.15.19:52742) 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb02bfff700 1 -- 172.21.15.19:0/674823280 learned_addr learned my addr 172.21.15.19:0/674823280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.890+0000 7fb030a19700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb02bfff700 1 -- 172.21.15.19:0/674823280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 msgr2=0x7fb02c113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb02bfff700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb02bfff700 1 -- 172.21.15.19:0/674823280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020005040 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb02bfff700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113e70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb01800eb40 tx=0x7fb01800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb018007cc0 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb01800f040 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c10da70 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01800c990 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.891+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c10dfb0 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.892+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c1046c0 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.893+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb01800caf0 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.893+0000 7fb0297fa700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01c06cc30 0x7fb01c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.893+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb01808bba0 con 0x7fb02c0fe8f0 2024-02-22T16:05:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.893+0000 7fb030a19700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01c06cc30 0x7fb01c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.894+0000 7fb030a19700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01c06cc30 0x7fb01c06f0e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb020000c00 tx=0x7fb020009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:10.898+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb018059710 con 0x7fb02c0fe8f0 2024-02-22T16:05:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.122+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb02c113600 con 0x7fb01c06cc30 2024-02-22T16:05:11.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.123+0000 7fb0297fa700 1 -- 172.21.15.19:0/674823280 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb02c113600 con 0x7fb01c06cc30 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01c06cc30 msgr2=0x7fb01c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01c06cc30 0x7fb01c06f0e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb020000c00 tx=0x7fb020009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 msgr2=0x7fb02c113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113e70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb01800eb40 tx=0x7fb01800b040 comp rx=0 tx=0).stop 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 shutdown_connections 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01c06cc30 0x7fb01c06f0e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 --2- 172.21.15.19:0/674823280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113e70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 >> 172.21.15.19:0/674823280 conn(0x7fb02c071d90 msgr2=0x7fb02c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:11.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 shutdown_connections 2024-02-22T16:05:11.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.125+0000 7fb032c7d700 1 -- 172.21.15.19:0/674823280 wait complete. 2024-02-22T16:05:11.112 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:05:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 -- 172.21.15.19:0/2836820571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fdfe0 msgr2=0x7f78040fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2836820571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fdfe0 0x7f78040fe3b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f77f4004970 tx=0x7f77f4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 -- 172.21.15.19:0/2836820571 shutdown_connections 2024-02-22T16:05:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2836820571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78040fe8f0 0x7f7804107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2836820571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fdfe0 0x7f78040fe3b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 -- 172.21.15.19:0/2836820571 >> 172.21.15.19:0/2836820571 conn(0x7f7804071d90 msgr2=0x7f7804072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:11.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 -- 172.21.15.19:0/2836820571 shutdown_connections 2024-02-22T16:05:11.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 -- 172.21.15.19:0/2836820571 wait complete. 2024-02-22T16:05:11.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 Processor -- start 2024-02-22T16:05:11.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.230+0000 7f780a7a6700 1 -- start start 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f780a7a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78040fdfe0 0x7f7804113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f780a7a6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 0x7f7804113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f780a7a6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7804114500 con 0x7f78040fe8f0 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f780a7a6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780410d830 con 0x7f78040fdfe0 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 0x7f7804113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 0x7f7804113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52756/0 (socket says 172.21.15.19:52756) 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 -- 172.21.15.19:0/2776990928 learned_addr learned my addr 172.21.15.19:0/2776990928 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f7803fff700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78040fdfe0 0x7f7804113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 -- 172.21.15.19:0/2776990928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78040fdfe0 msgr2=0x7f7804113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78040fdfe0 0x7f7804113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 -- 172.21.15.19:0/2776990928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4005040 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f78037fe700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 0x7f7804113eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77ec002a50 tx=0x7f77ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77ec007cc0 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.231+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780410db10 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.232+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77ec00f040 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.232+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77ec00da70 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.232+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780410e050 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.233+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78041046c0 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.233+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f77ec00dc10 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.233+0000 7f7800ff9700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f77f006cc30 0x7f77f006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.233+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f77ec08b7e0 con 0x7f78040fe8f0 2024-02-22T16:05:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.233+0000 7f7803fff700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f77f006cc30 0x7f77f006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.234+0000 7f7803fff700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f77f006cc30 0x7f77f006f0e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009fd0 tx=0x7f77f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.238+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f77ec059220 con 0x7f78040fe8f0 2024-02-22T16:05:11.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:11 smithi019 ceph-mon[173579]: pgmap v357: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:11.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.459+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7804109f10 con 0x7f77f006cc30 2024-02-22T16:05:11.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.460+0000 7f7800ff9700 1 -- 172.21.15.19:0/2776990928 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7804109f10 con 0x7f77f006cc30 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f77f006cc30 msgr2=0x7f77f006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f77f006cc30 0x7f77f006f0e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009fd0 tx=0x7f77f4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 msgr2=0x7f7804113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 0x7f7804113eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77ec002a50 tx=0x7f77ec0046b0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 shutdown_connections 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f77f006cc30 0x7f77f006f0e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78040fdfe0 0x7f7804113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 --2- 172.21.15.19:0/2776990928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78040fe8f0 0x7f7804113eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 >> 172.21.15.19:0/2776990928 conn(0x7f7804071d90 msgr2=0x7f7804101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 shutdown_connections 2024-02-22T16:05:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.462+0000 7f780a7a6700 1 -- 172.21.15.19:0/2776990928 wait complete. 2024-02-22T16:05:11.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:11 smithi062 ceph-mon[106861]: pgmap v357: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:11.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 -- 172.21.15.19:0/270147041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 msgr2=0x7f0b000fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 --2- 172.21.15.19:0/270147041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b000fb9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0af4004940 tx=0x7f0af4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:11.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 -- 172.21.15.19:0/270147041 shutdown_connections 2024-02-22T16:05:11.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 --2- 172.21.15.19:0/270147041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b000fb9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 --2- 172.21.15.19:0/270147041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 0x7f0b00103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 -- 172.21.15.19:0/270147041 >> 172.21.15.19:0/270147041 conn(0x7f0b00071560 msgr2=0x7f0b00071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 -- 172.21.15.19:0/270147041 shutdown_connections 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.565+0000 7f0b05d36700 1 -- 172.21.15.19:0/270147041 wait complete. 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.566+0000 7f0b05d36700 1 Processor -- start 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.566+0000 7f0b05d36700 1 -- start start 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.566+0000 7f0b05d36700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b00113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.566+0000 7f0b05d36700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 0x7f0b00114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.566+0000 7f0b05d36700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00114660 con 0x7f0b000fb570 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.566+0000 7f0b05d36700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0010d990 con 0x7f0b00103140 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b00113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b00113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52772/0 (socket says 172.21.15.19:52772) 2024-02-22T16:05:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 -- 172.21.15.19:0/1333071997 learned_addr learned my addr 172.21.15.19:0/1333071997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:11.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0afeffd700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 0x7f0b00114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 -- 172.21.15.19:0/1333071997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 msgr2=0x7f0b00114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 0x7f0b00114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 -- 172.21.15.19:0/1333071997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af4005040 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0afeffd700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 0x7f0b00114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0aff7fe700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b00113ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c8c0 tx=0x7f0af00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af000a980 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0010dc70 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0af000ce60 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af00113a0 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.567+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0010e1b0 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.568+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b001054d0 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.569+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0af0012400 con 0x7f0b000fb570 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.569+0000 7f0ae7fff700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ae806cc20 0x7f0ae806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.569+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0af008c170 con 0x7f0b000fb570 2024-02-22T16:05:11.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.569+0000 7f0afeffd700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ae806cc20 0x7f0ae806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.570+0000 7f0afeffd700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ae806cc20 0x7f0ae806f0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009fd0 tx=0x7f0af4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:11.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.574+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0af0059c30 con 0x7f0b000fb570 2024-02-22T16:05:11.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.786+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b0010a0e0 con 0x7f0ae806cc20 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.796+0000 7f0ae7fff700 1 -- 172.21.15.19:0/1333071997 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f0b0010a0e0 con 0x7f0ae806cc20 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (12m) 5m ago 23m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (9m) 5m ago 23m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (9m) 5m ago 22m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (12m) 5m ago 23m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (5m) 5m ago 17m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (6m) 5m ago 17m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (5m) 5m ago 17m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (5m) 5m ago 17m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:05:11.773 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (14m) 5m ago 25m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (13m) 5m ago 22m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (10m) 5m ago 25m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (10m) 5m ago 22m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (13m) 5m ago 23m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (13m) 5m ago 22m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (9m) 5m ago 21m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (8m) 5m ago 21m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (8m) 5m ago 20m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (7m) 5m ago 20m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (6m) 5m ago 19m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (6m) 5m ago 19m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:05:11.774 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (12m) 5m ago 22m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ae806cc20 msgr2=0x7f0ae806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ae806cc20 0x7f0ae806f0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009fd0 tx=0x7f0af4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 msgr2=0x7f0b00113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b00113ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c8c0 tx=0x7f0af00044c0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 shutdown_connections 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ae806cc20 0x7f0ae806f0d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b000fb570 0x7f0b00113ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 --2- 172.21.15.19:0/1333071997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b00103140 0x7f0b00114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 >> 172.21.15.19:0/1333071997 conn(0x7f0b00071560 msgr2=0x7f0b000f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 shutdown_connections 2024-02-22T16:05:11.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.798+0000 7f0b05d36700 1 -- 172.21.15.19:0/1333071997 wait complete. 2024-02-22T16:05:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 -- 172.21.15.19:0/3104410364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a04102dc0 msgr2=0x7f6a04103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3104410364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a04102dc0 0x7f6a04103190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f69f8004940 tx=0x7f69f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 -- 172.21.15.19:0/3104410364 shutdown_connections 2024-02-22T16:05:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3104410364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a040fb390 0x7f6a040fb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3104410364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a04102dc0 0x7f6a04103190 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 -- 172.21.15.19:0/3104410364 >> 172.21.15.19:0/3104410364 conn(0x7f6a04072270 msgr2=0x7f6a04072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 -- 172.21.15.19:0/3104410364 shutdown_connections 2024-02-22T16:05:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 -- 172.21.15.19:0/3104410364 wait complete. 2024-02-22T16:05:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 Processor -- start 2024-02-22T16:05:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.901+0000 7f6a03fff700 1 -- start start 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a03fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 0x7f6a04113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a04102dc0 0x7f6a04113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a03fff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04114300 con 0x7f6a040fb390 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a03fff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0410d630 con 0x7f6a04102dc0 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 0x7f6a04113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 0x7f6a04113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52798/0 (socket says 172.21.15.19:52798) 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 -- 172.21.15.19:0/3779071210 learned_addr learned my addr 172.21.15.19:0/3779071210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a027fc700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a04102dc0 0x7f6a04113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 -- 172.21.15.19:0/3779071210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a04102dc0 msgr2=0x7f6a04113cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a04102dc0 0x7f6a04113cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 -- 172.21.15.19:0/3779071210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8005040 con 0x7f6a040fb390 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a02ffd700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 0x7f6a04113770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f69f8008f80 tx=0x7f69f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69f800e070 con 0x7f6a040fb390 2024-02-22T16:05:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0410d8b0 con 0x7f6a040fb390 2024-02-22T16:05:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69f8004290 con 0x7f6a040fb390 2024-02-22T16:05:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69f801ce20 con 0x7f6a040fb390 2024-02-22T16:05:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.902+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0410dd90 con 0x7f6a040fb390 2024-02-22T16:05:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.903+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a04100060 con 0x7f6a040fb390 2024-02-22T16:05:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.904+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f69f8013a30 con 0x7f6a040fb390 2024-02-22T16:05:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.904+0000 7f69eb7fe700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69ec06cc30 0x7f69ec06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.904+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f69f808d800 con 0x7f6a040fb390 2024-02-22T16:05:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.904+0000 7f6a027fc700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69ec06cc30 0x7f69ec06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.905+0000 7f6a027fc700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69ec06cc30 0x7f69ec06f0e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69f4004600 tx=0x7f69f400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:11.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:11.909+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f69f805b2c0 con 0x7f6a040fb390 2024-02-22T16:05:12.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.197+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a0404bed0 con 0x7f6a040fb390 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.197+0000 7f69eb7fe700 1 -- 172.21.15.19:0/3779071210 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f69f805ae50 con 0x7f6a040fb390 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:05:12.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:05:12.175 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69ec06cc30 msgr2=0x7f69ec06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69ec06cc30 0x7f69ec06f0e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69f4004600 tx=0x7f69f400b040 comp rx=0 tx=0).stop 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 msgr2=0x7f6a04113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 0x7f6a04113770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f69f8008f80 tx=0x7f69f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 shutdown_connections 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f69ec06cc30 0x7f69ec06f0e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a040fb390 0x7f6a04113770 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 --2- 172.21.15.19:0/3779071210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a04102dc0 0x7f6a04113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 >> 172.21.15.19:0/3779071210 conn(0x7f6a04072270 msgr2=0x7f6a040fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 shutdown_connections 2024-02-22T16:05:12.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.199+0000 7f6a03fff700 1 -- 172.21.15.19:0/3779071210 wait complete. 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 -- 172.21.15.19:0/2723666405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 msgr2=0x7f43bc0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 --2- 172.21.15.19:0/2723666405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc0fbea0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f43b0004940 tx=0x7f43b0009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 -- 172.21.15.19:0/2723666405 shutdown_connections 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 --2- 172.21.15.19:0/2723666405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc0fbea0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 --2- 172.21.15.19:0/2723666405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43bc101e40 0x7f43bc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 -- 172.21.15.19:0/2723666405 >> 172.21.15.19:0/2723666405 conn(0x7f43bc071d90 msgr2=0x7f43bc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 -- 172.21.15.19:0/2723666405 shutdown_connections 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.303+0000 7f43c4209700 1 -- 172.21.15.19:0/2723666405 wait complete. 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c4209700 1 Processor -- start 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c4209700 1 -- start start 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c4209700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43bc101e40 0x7f43bc113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c4209700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c4209700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc114520 con 0x7f43bc1027e0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c4209700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc10d850 con 0x7f43bc101e40 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52806/0 (socket says 172.21.15.19:52806) 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 -- 172.21.15.19:0/4178487012 learned_addr learned my addr 172.21.15.19:0/4178487012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c1fa5700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43bc101e40 0x7f43bc113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 -- 172.21.15.19:0/4178487012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43bc101e40 msgr2=0x7f43bc113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43bc101e40 0x7f43bc113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 -- 172.21.15.19:0/4178487012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0005040 con 0x7f43bc1027e0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.304+0000 7f43c17a4700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc113ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009fd0 tx=0x7f43b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.305+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b000e040 con 0x7f43bc1027e0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.305+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc10dad0 con 0x7f43bc1027e0 2024-02-22T16:05:12.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.305+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43b000bc50 con 0x7f43bc1027e0 2024-02-22T16:05:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.305+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b001c3f0 con 0x7f43bc1027e0 2024-02-22T16:05:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.305+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc10e010 con 0x7f43bc1027e0 2024-02-22T16:05:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.306+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc0738b0 con 0x7f43bc1027e0 2024-02-22T16:05:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.306+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f43b0013420 con 0x7f43bc1027e0 2024-02-22T16:05:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.307+0000 7f43ae7fc700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f43a806cc20 0x7f43a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.307+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f43b008d470 con 0x7f43bc1027e0 2024-02-22T16:05:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.307+0000 7f43c1fa5700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f43a806cc20 0x7f43a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.307+0000 7f43c1fa5700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f43a806cc20 0x7f43a806f0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43b8007fd0 tx=0x7f43b8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.311+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f43b005af30 con 0x7f43bc1027e0 2024-02-22T16:05:12.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.556+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f43bc04bed0 con 0x7f43bc1027e0 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.556+0000 7f43ae7fc700 1 -- 172.21.15.19:0/4178487012 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f43b005aac0 con 0x7f43bc1027e0 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:05:12.533 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:05:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:12.536 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:12.536 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:12.536 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f43a806cc20 msgr2=0x7f43a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f43a806cc20 0x7f43a806f0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43b8007fd0 tx=0x7f43b8009040 comp rx=0 tx=0).stop 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 msgr2=0x7f43bc113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc113ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009fd0 tx=0x7f43b0009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 shutdown_connections 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f43a806cc20 0x7f43a806f0d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43bc101e40 0x7f43bc113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 --2- 172.21.15.19:0/4178487012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43bc1027e0 0x7f43bc113ed0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 >> 172.21.15.19:0/4178487012 conn(0x7f43bc071d90 msgr2=0x7f43bc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 shutdown_connections 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.558+0000 7f43c4209700 1 -- 172.21.15.19:0/4178487012 wait complete. 2024-02-22T16:05:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3966058913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d04103140 msgr2=0x7f5d04103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3966058913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d04103140 0x7f5d04103510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004940 tx=0x7f5cf8009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3966058913 shutdown_connections 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3966058913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d040fb570 0x7f5d040fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3966058913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d04103140 0x7f5d04103510 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3966058913 >> 172.21.15.19:0/3966058913 conn(0x7f5d04071560 msgr2=0x7f5d04071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:12.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3966058913 shutdown_connections 2024-02-22T16:05:12.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.663+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3966058913 wait complete. 2024-02-22T16:05:12.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d0a20b700 1 Processor -- start 2024-02-22T16:05:12.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d0a20b700 1 -- start start 2024-02-22T16:05:12.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d0a20b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 0x7f5d04113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d0a20b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d04103140 0x7f5d04113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d0a20b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d041145d0 con 0x7f5d040fb570 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d0a20b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0410d900 con 0x7f5d04103140 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d037fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 0x7f5d04113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d037fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 0x7f5d04113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52830/0 (socket says 172.21.15.19:52830) 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.664+0000 7f5d037fe700 1 -- 172.21.15.19:0/3200158041 learned_addr learned my addr 172.21.15.19:0/3200158041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d02ffd700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d04103140 0x7f5d04113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d037fe700 1 -- 172.21.15.19:0/3200158041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d04103140 msgr2=0x7f5d04113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d037fe700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d04103140 0x7f5d04113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d037fe700 1 -- 172.21.15.19:0/3200158041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8005040 con 0x7f5d040fb570 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d02ffd700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d04103140 0x7f5d04113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d037fe700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 0x7f5d04113a40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8000c00 tx=0x7f5cf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf8021070 con 0x7f5d040fb570 2024-02-22T16:05:12.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5cf801c400 con 0x7f5d040fb570 2024-02-22T16:05:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0410db80 con 0x7f5d040fb570 2024-02-22T16:05:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf800e070 con 0x7f5d040fb570 2024-02-22T16:05:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.665+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0410e060 con 0x7f5d040fb570 2024-02-22T16:05:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.666+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d04100310 con 0x7f5d040fb570 2024-02-22T16:05:12.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.666+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5cf801c9e0 con 0x7f5d040fb570 2024-02-22T16:05:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.667+0000 7f5cebfff700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5cec06cc20 0x7f5cec06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.667+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5cf808d2a0 con 0x7f5d040fb570 2024-02-22T16:05:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.667+0000 7f5d02ffd700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5cec06cc20 0x7f5cec06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.667+0000 7f5d02ffd700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5cec06cc20 0x7f5cec06f0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4004600 tx=0x7f5cf400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:12.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.671+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5cf805ad60 con 0x7f5d040fb570 2024-02-22T16:05:12.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.894+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d0404bed0 con 0x7f5cec06cc20 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.895+0000 7f5cebfff700 1 -- 172.21.15.19:0/3200158041 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5d0404bed0 con 0x7f5cec06cc20 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:05:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:05:12.873 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5cec06cc20 msgr2=0x7f5cec06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5cec06cc20 0x7f5cec06f0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4004600 tx=0x7f5cf400b040 comp rx=0 tx=0).stop 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 msgr2=0x7f5d04113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 0x7f5d04113a40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8000c00 tx=0x7f5cf8009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 shutdown_connections 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5cec06cc20 0x7f5cec06f0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d040fb570 0x7f5d04113a40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 --2- 172.21.15.19:0/3200158041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d04103140 0x7f5d04113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 >> 172.21.15.19:0/3200158041 conn(0x7f5d04071560 msgr2=0x7f5d040fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:12.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.897+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 shutdown_connections 2024-02-22T16:05:12.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:12.898+0000 7f5d0a20b700 1 -- 172.21.15.19:0/3200158041 wait complete. 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 -- 172.21.15.19:0/799431010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fe8f0 msgr2=0x7fd6b4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/799431010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4107050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0004970 tx=0x7fd6a0009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 -- 172.21.15.19:0/799431010 shutdown_connections 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/799431010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4107050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/799431010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b40fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 -- 172.21.15.19:0/799431010 >> 172.21.15.19:0/799431010 conn(0x7fd6b4071d90 msgr2=0x7fd6b4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 -- 172.21.15.19:0/799431010 shutdown_connections 2024-02-22T16:05:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.002+0000 7fd6bae6d700 1 -- 172.21.15.19:0/799431010 wait complete. 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6bae6d700 1 Processor -- start 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6bae6d700 1 -- start start 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6bae6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6bae6d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6bae6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4114500 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6bae6d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b410d830 con 0x7fd6b40fe8f0 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6b8c09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6b8c09700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b4113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52860/0 (socket says 172.21.15.19:52860) 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6b8c09700 1 -- 172.21.15.19:0/703725264 learned_addr learned my addr 172.21.15.19:0/703725264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.003+0000 7fd6b3fff700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b8c09700 1 -- 172.21.15.19:0/703725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fe8f0 msgr2=0x7fd6b4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b8c09700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b8c09700 1 -- 172.21.15.19:0/703725264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a0005040 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b3fff700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b8c09700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b4113970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8002a50 tx=0x7fd6a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a8007cc0 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b410db10 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6a800f040 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a8009540 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.004+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b410e020 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.005+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b41046c0 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.005+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd6a80096a0 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.006+0000 7fd6b17fa700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6a406cc20 0x7fd6a406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.006+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd6a808be80 con 0x7fd6b40fdfe0 2024-02-22T16:05:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.006+0000 7fd6b3fff700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6a406cc20 0x7fd6a406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.006+0000 7fd6b3fff700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6a406cc20 0x7fd6a406f0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0004f50 tx=0x7fd6a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:12.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.010+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd6a80598c0 con 0x7fd6b40fdfe0 2024-02-22T16:05:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.296+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd6b404bed0 con 0x7fd6b40fdfe0 2024-02-22T16:05:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.297+0000 7fd6b17fa700 1 -- 172.21.15.19:0/703725264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd6a8059450 con 0x7fd6b40fdfe0 2024-02-22T16:05:13.274 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:05:13.274 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:05:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:05:13.274 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:05:13.274 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6a406cc20 msgr2=0x7fd6a406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6a406cc20 0x7fd6a406f0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0004f50 tx=0x7fd6a0009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 msgr2=0x7fd6b4113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b4113970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8002a50 tx=0x7fd6a80046b0 comp rx=0 tx=0).stop 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 shutdown_connections 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd6a406cc20 0x7fd6a406f0d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6b40fdfe0 0x7fd6b4113970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 --2- 172.21.15.19:0/703725264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b40fe8f0 0x7fd6b4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 >> 172.21.15.19:0/703725264 conn(0x7fd6b4071d90 msgr2=0x7fd6b4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 shutdown_connections 2024-02-22T16:05:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:13.299+0000 7fd6bae6d700 1 -- 172.21.15.19:0/703725264 wait complete. 2024-02-22T16:05:13.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:13 smithi062 ceph-mon[106861]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:13.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:13 smithi062 ceph-mon[106861]: pgmap v358: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:13.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:13 smithi062 ceph-mon[106861]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:13.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:13 smithi062 ceph-mon[106861]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:13.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:13 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3779071210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:05:13.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:13 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4178487012' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:05:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:13 smithi019 ceph-mon[173579]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:13 smithi019 ceph-mon[173579]: pgmap v358: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:13 smithi019 ceph-mon[173579]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:13 smithi019 ceph-mon[173579]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:13 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3779071210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:05:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:13 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4178487012' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:05:14.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:14 smithi062 ceph-mon[106861]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:14.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:14 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/703725264' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:05:14.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:14 smithi019 ceph-mon[173579]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:14.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:14 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/703725264' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:05:15.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:15 smithi062 ceph-mon[106861]: pgmap v359: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:15.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:15 smithi019 ceph-mon[173579]: pgmap v359: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:17.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:17 smithi062 ceph-mon[106861]: pgmap v360: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:17.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:17 smithi019 ceph-mon[173579]: pgmap v360: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:19 smithi062 ceph-mon[106861]: pgmap v361: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:19 smithi019 ceph-mon[173579]: pgmap v361: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:21.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:21 smithi019 ceph-mon[173579]: pgmap v362: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:21.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:21 smithi062 ceph-mon[106861]: pgmap v362: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:23 smithi062 ceph-mon[106861]: pgmap v363: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:23 smithi019 ceph-mon[173579]: pgmap v363: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:25.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:25 smithi062 ceph-mon[106861]: pgmap v364: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:25.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:25 smithi019 ceph-mon[173579]: pgmap v364: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:27.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:27 smithi062 ceph-mon[106861]: pgmap v365: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:27.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:27 smithi019 ceph-mon[173579]: pgmap v365: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:29.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:29 smithi062 ceph-mon[106861]: pgmap v366: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:29.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:29 smithi019 ceph-mon[173579]: pgmap v366: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:31.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:31 smithi019 ceph-mon[173579]: pgmap v367: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:31.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:31 smithi062 ceph-mon[106861]: pgmap v367: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:33.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:33 smithi062 ceph-mon[106861]: pgmap v368: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:33.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:33 smithi019 ceph-mon[173579]: pgmap v368: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:34.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:34.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:35.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:35 smithi062 ceph-mon[106861]: pgmap v369: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:35.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:35 smithi019 ceph-mon[173579]: pgmap v369: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:37.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:37 smithi062 ceph-mon[106861]: pgmap v370: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:37.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:37 smithi019 ceph-mon[173579]: pgmap v370: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:39.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:39 smithi062 ceph-mon[106861]: pgmap v371: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:39.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:39 smithi019 ceph-mon[173579]: pgmap v371: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:41.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:41 smithi019 ceph-mon[173579]: pgmap v372: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:41.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:41 smithi062 ceph-mon[106861]: pgmap v372: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.417+0000 7f5ac0502700 1 -- 172.21.15.19:0/1191996117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 msgr2=0x7f5ab8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.417+0000 7f5ac0502700 1 --2- 172.21.15.19:0/1191996117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8103320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5aac004940 tx=0x7f5aac009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 -- 172.21.15.19:0/1191996117 shutdown_connections 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 --2- 172.21.15.19:0/1191996117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ab81038f0 0x7f5ab80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 --2- 172.21.15.19:0/1191996117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8103320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 -- 172.21.15.19:0/1191996117 >> 172.21.15.19:0/1191996117 conn(0x7f5ab80f8ae0 msgr2=0x7f5ab80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 -- 172.21.15.19:0/1191996117 shutdown_connections 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 -- 172.21.15.19:0/1191996117 wait complete. 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 Processor -- start 2024-02-22T16:05:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 -- start start 2024-02-22T16:05:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.418+0000 7f5ac0502700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ab81038f0 0x7f5ab8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5ac0502700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab8114500 con 0x7f5ab8102f50 2024-02-22T16:05:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5ac0502700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab810d830 con 0x7f5ab81038f0 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37082/0 (socket says 172.21.15.19:37082) 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 -- 172.21.15.19:0/2615945046 learned_addr learned my addr 172.21.15.19:0/2615945046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abda9d700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ab81038f0 0x7f5ab8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 -- 172.21.15.19:0/2615945046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ab81038f0 msgr2=0x7f5ab8113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ab81038f0 0x7f5ab8113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 -- 172.21.15.19:0/2615945046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac005040 con 0x7f5ab8102f50 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5abe29e700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8113970 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b3b0 tx=0x7f5aac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aac00e040 con 0x7f5ab8102f50 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5aac00bc50 con 0x7f5ab8102f50 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab810dab0 con 0x7f5ab8102f50 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aac013420 con 0x7f5ab8102f50 2024-02-22T16:05:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.419+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab810dff0 con 0x7f5ab8102f50 2024-02-22T16:05:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.420+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab8100260 con 0x7f5ab8102f50 2024-02-22T16:05:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.421+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5aac01c430 con 0x7f5ab8102f50 2024-02-22T16:05:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.421+0000 7f5aaaffd700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5aa406cc20 0x7f5aa406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.421+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5aac08ce70 con 0x7f5ab8102f50 2024-02-22T16:05:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.421+0000 7f5abda9d700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5aa406cc20 0x7f5aa406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.422+0000 7f5abda9d700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5aa406cc20 0x7f5aa406f0d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ab810f1a0 tx=0x7f5ab4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.426+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5aac05a9e0 con 0x7f5ab8102f50 2024-02-22T16:05:43.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:43 smithi062 ceph-mon[106861]: pgmap v373: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:43.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:43 smithi019 ceph-mon[173579]: pgmap v373: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:43.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.648+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ab804bf40 con 0x7f5aa406cc20 2024-02-22T16:05:43.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.649+0000 7f5aaaffd700 1 -- 172.21.15.19:0/2615945046 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ab804bf40 con 0x7f5aa406cc20 2024-02-22T16:05:43.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.651+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5aa406cc20 msgr2=0x7f5aa406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5aa406cc20 0x7f5aa406f0d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ab810f1a0 tx=0x7f5ab4009ee0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 msgr2=0x7f5ab8113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8113970 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b3b0 tx=0x7f5aac009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 shutdown_connections 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5aa406cc20 0x7f5aa406f0d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ab8102f50 0x7f5ab8113970 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 --2- 172.21.15.19:0/2615945046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ab81038f0 0x7f5ab8113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 >> 172.21.15.19:0/2615945046 conn(0x7f5ab80f8ae0 msgr2=0x7f5ab80fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 shutdown_connections 2024-02-22T16:05:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.652+0000 7f5ac0502700 1 -- 172.21.15.19:0/2615945046 wait complete. 2024-02-22T16:05:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:05:43.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 -- 172.21.15.19:0/2389162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 msgr2=0x7f3c100fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 --2- 172.21.15.19:0/2389162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c100fcf50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c04004940 tx=0x7f3c04009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 -- 172.21.15.19:0/2389162374 shutdown_connections 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 --2- 172.21.15.19:0/2389162374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c100fcf50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 --2- 172.21.15.19:0/2389162374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 0x7f3c10103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 -- 172.21.15.19:0/2389162374 >> 172.21.15.19:0/2389162374 conn(0x7f3c100f8af0 msgr2=0x7f3c100faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 -- 172.21.15.19:0/2389162374 shutdown_connections 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.757+0000 7f3c1587e700 1 -- 172.21.15.19:0/2389162374 wait complete. 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c1587e700 1 Processor -- start 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c1587e700 1 -- start start 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c1587e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 0x7f3c10113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c1587e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c10113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:43.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c1587e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101144b0 con 0x7f3c101038e0 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c1587e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1010d7e0 con 0x7f3c10102f40 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c0e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c10113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c0e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c10113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37098/0 (socket says 172.21.15.19:37098) 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c0e7fc700 1 -- 172.21.15.19:0/4018607277 learned_addr learned my addr 172.21.15.19:0/4018607277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c0effd700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 0x7f3c10113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c0e7fc700 1 -- 172.21.15.19:0/4018607277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 msgr2=0x7f3c10113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.758+0000 7f3c0e7fc700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 0x7f3c10113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3c0e7fc700 1 -- 172.21.15.19:0/4018607277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c04005040 con 0x7f3c101038e0 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3c0effd700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 0x7f3c10113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3c0e7fc700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c10113e60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009fd0 tx=0x7f3c04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0400e040 con 0x7f3c101038e0 2024-02-22T16:05:43.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1010da60 con 0x7f3c101038e0 2024-02-22T16:05:43.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c0400bc50 con 0x7f3c101038e0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0401c3f0 con 0x7f3c101038e0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.759+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1010dfa0 con 0x7f3c101038e0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.760+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c10105410 con 0x7f3c101038e0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.760+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3c04013420 con 0x7f3c101038e0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.761+0000 7f3bf77fe700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3bf806cc20 0x7f3bf806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.761+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3c0408d4d0 con 0x7f3c101038e0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.761+0000 7f3c0effd700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3bf806cc20 0x7f3bf806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:43.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.761+0000 7f3c0effd700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3bf806cc20 0x7f3bf806f0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c00007fd0 tx=0x7f3c00009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:43.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.765+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3c0405af30 con 0x7f3c101038e0 2024-02-22T16:05:43.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.989+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c101135f0 con 0x7f3bf806cc20 2024-02-22T16:05:43.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.990+0000 7f3bf77fe700 1 -- 172.21.15.19:0/4018607277 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3c101135f0 con 0x7f3bf806cc20 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3bf806cc20 msgr2=0x7f3bf806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3bf806cc20 0x7f3bf806f0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c00007fd0 tx=0x7f3c00009f60 comp rx=0 tx=0).stop 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 msgr2=0x7f3c10113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c10113e60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009fd0 tx=0x7f3c04009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 shutdown_connections 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3bf806cc20 0x7f3bf806f0d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c10102f40 0x7f3c10113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 --2- 172.21.15.19:0/4018607277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c101038e0 0x7f3c10113e60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 >> 172.21.15.19:0/4018607277 conn(0x7f3c100f8af0 msgr2=0x7f3c100fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.992+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 shutdown_connections 2024-02-22T16:05:43.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:43.993+0000 7f3c1587e700 1 -- 172.21.15.19:0/4018607277 wait complete. 2024-02-22T16:05:44.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 -- 172.21.15.19:0/1353518514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0570103140 msgr2=0x7f0570103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 --2- 172.21.15.19:0/1353518514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0570103140 0x7f0570103510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0564004940 tx=0x7f0564009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 -- 172.21.15.19:0/1353518514 shutdown_connections 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 --2- 172.21.15.19:0/1353518514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05700fb570 0x7f05700fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 --2- 172.21.15.19:0/1353518514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0570103140 0x7f0570103510 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 -- 172.21.15.19:0/1353518514 >> 172.21.15.19:0/1353518514 conn(0x7f0570071560 msgr2=0x7f0570071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 -- 172.21.15.19:0/1353518514 shutdown_connections 2024-02-22T16:05:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 -- 172.21.15.19:0/1353518514 wait complete. 2024-02-22T16:05:44.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 Processor -- start 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.096+0000 7f0577607700 1 -- start start 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0577607700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 0x7f05700fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0577607700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570103140 0x7f05700fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0577607700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05700fd290 con 0x7f05700fb570 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0577607700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05700fd400 con 0x7f0570103140 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 0x7f05700fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 0x7f05700fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37120/0 (socket says 172.21.15.19:37120) 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 -- 172.21.15.19:0/1194546472 learned_addr learned my addr 172.21.15.19:0/1194546472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0574ba2700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570103140 0x7f05700fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 -- 172.21.15.19:0/1194546472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570103140 msgr2=0x7f05700fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570103140 0x7f05700fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 -- 172.21.15.19:0/1194546472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564005040 con 0x7f05700fb570 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0574ba2700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570103140 0x7f05700fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f05753a3700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 0x7f05700fc750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f056400b3b0 tx=0x7f0564009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.097+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056400e040 con 0x7f05700fb570 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.098+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0570119c80 con 0x7f05700fb570 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.098+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f056400bc50 con 0x7f05700fb570 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.098+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0564013420 con 0x7f05700fb570 2024-02-22T16:05:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.098+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057011a0e0 con 0x7f05700fb570 2024-02-22T16:05:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.099+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0570100310 con 0x7f05700fb570 2024-02-22T16:05:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.099+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f056401c430 con 0x7f05700fb570 2024-02-22T16:05:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.100+0000 7f0561ffb700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f055c06cc20 0x7f055c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.100+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f056408cd40 con 0x7f05700fb570 2024-02-22T16:05:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.100+0000 7f0574ba2700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f055c06cc20 0x7f055c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.100+0000 7f0574ba2700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f055c06cc20 0x7f055c06f0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f056c007fd0 tx=0x7f056c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.104+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0564056ec0 con 0x7f05700fb570 2024-02-22T16:05:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.316+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f05700fde90 con 0x7f055c06cc20 2024-02-22T16:05:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.326+0000 7f0561ffb700 1 -- 172.21.15.19:0/1194546472 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f05700fde90 con 0x7f055c06cc20 2024-02-22T16:05:44.302 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:05:44.302 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (12m) 6m ago 23m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:05:44.302 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (10m) 6m ago 24m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:05:44.302 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (10m) 5m ago 23m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (12m) 6m ago 23m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (6m) 6m ago 18m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (6m) 6m ago 18m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (5m) 5m ago 18m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (6m) 5m ago 18m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (14m) 6m ago 25m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (14m) 5m ago 23m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (11m) 6m ago 25m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (10m) 5m ago 22m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (13m) 6m ago 24m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (13m) 5m ago 23m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (9m) 6m ago 22m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (9m) 6m ago 21m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:05:44.303 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (8m) 6m ago 21m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:05:44.304 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (8m) 5m ago 20m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:05:44.304 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (7m) 5m ago 20m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:05:44.304 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (6m) 5m ago 19m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:05:44.304 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (13m) 6m ago 23m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f055c06cc20 msgr2=0x7f055c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f055c06cc20 0x7f055c06f0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f056c007fd0 tx=0x7f056c009040 comp rx=0 tx=0).stop 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 msgr2=0x7f05700fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 0x7f05700fc750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f056400b3b0 tx=0x7f0564009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 shutdown_connections 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f055c06cc20 0x7f055c06f0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05700fb570 0x7f05700fc750 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 --2- 172.21.15.19:0/1194546472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570103140 0x7f05700fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 >> 172.21.15.19:0/1194546472 conn(0x7f0570071560 msgr2=0x7f05700fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 shutdown_connections 2024-02-22T16:05:44.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.328+0000 7f0577607700 1 -- 172.21.15.19:0/1194546472 wait complete. 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 -- 172.21.15.19:0/749488576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8101ee0 msgr2=0x7f70b81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/749488576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8101ee0 0x7f70b81022b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004940 tx=0x7f70b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 -- 172.21.15.19:0/749488576 shutdown_connections 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/749488576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b8102880 0x7f70b80fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/749488576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8101ee0 0x7f70b81022b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 -- 172.21.15.19:0/749488576 >> 172.21.15.19:0/749488576 conn(0x7f70b8071560 msgr2=0x7f70b8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 -- 172.21.15.19:0/749488576 shutdown_connections 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 -- 172.21.15.19:0/749488576 wait complete. 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 Processor -- start 2024-02-22T16:05:44.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.431+0000 7f70bfdf6700 1 -- start start 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bfdf6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b8101ee0 0x7f70b810ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bfdf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 0x7f70b810ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bfdf6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b810f4b0 con 0x7f70b8102880 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bfdf6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b810f5f0 con 0x7f70b8101ee0 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 0x7f70b810ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 0x7f70b810ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37132/0 (socket says 172.21.15.19:37132) 2024-02-22T16:05:44.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 -- 172.21.15.19:0/797794777 learned_addr learned my addr 172.21.15.19:0/797794777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bdb92700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b8101ee0 0x7f70b810ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 -- 172.21.15.19:0/797794777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b8101ee0 msgr2=0x7f70b810ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b8101ee0 0x7f70b810ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 -- 172.21.15.19:0/797794777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4005040 con 0x7f70b8102880 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bd391700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 0x7f70b810ef70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f70a8002a50 tx=0x7f70a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a8007cc0 con 0x7f70b8102880 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70a800f040 con 0x7f70b8102880 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b81178e0 con 0x7f70b8102880 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a800d920 con 0x7f70b8102880 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.432+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b8117e20 con 0x7f70b8102880 2024-02-22T16:05:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.433+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b80ff1f0 con 0x7f70b8102880 2024-02-22T16:05:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.434+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f70a8009680 con 0x7f70b8102880 2024-02-22T16:05:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.434+0000 7f70ae7fc700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f70a406cc20 0x7f70a406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.435+0000 7f70bdb92700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f70a406cc20 0x7f70a406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.435+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f70a808be90 con 0x7f70b8102880 2024-02-22T16:05:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.435+0000 7f70bdb92700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f70a406cc20 0x7f70a406f0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004f90 tx=0x7f70b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:44.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.439+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f70a8055da0 con 0x7f70b8102880 2024-02-22T16:05:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.726+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f70b8108f00 con 0x7f70b8102880 2024-02-22T16:05:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.727+0000 7f70ae7fc700 1 -- 172.21.15.19:0/797794777 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f70a80593c0 con 0x7f70b8102880 2024-02-22T16:05:44.703 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:05:44.703 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:05:44.704 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:05:44.705 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:05:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f70a406cc20 msgr2=0x7f70a406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f70a406cc20 0x7f70a406f0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004f90 tx=0x7f70b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 msgr2=0x7f70b810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 0x7f70b810ef70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f70a8002a50 tx=0x7f70a8004600 comp rx=0 tx=0).stop 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 shutdown_connections 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f70a406cc20 0x7f70a406f0d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b8101ee0 0x7f70b810ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 --2- 172.21.15.19:0/797794777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102880 0x7f70b810ef70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 >> 172.21.15.19:0/797794777 conn(0x7f70b8071560 msgr2=0x7f70b80fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 shutdown_connections 2024-02-22T16:05:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.729+0000 7f70bfdf6700 1 -- 172.21.15.19:0/797794777 wait complete. 2024-02-22T16:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 -- 172.21.15.19:0/2544218076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 msgr2=0x7f15d0107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 --2- 172.21.15.19:0/2544218076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d0107050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f15c4004970 tx=0x7f15c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 -- 172.21.15.19:0/2544218076 shutdown_connections 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 --2- 172.21.15.19:0/2544218076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d0107050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 --2- 172.21.15.19:0/2544218076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 0x7f15d00fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 -- 172.21.15.19:0/2544218076 >> 172.21.15.19:0/2544218076 conn(0x7f15d0071d90 msgr2=0x7f15d0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 -- 172.21.15.19:0/2544218076 shutdown_connections 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.832+0000 7f15d69e0700 1 -- 172.21.15.19:0/2544218076 wait complete. 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15d69e0700 1 Processor -- start 2024-02-22T16:05:44.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15d69e0700 1 -- start start 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15d69e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 0x7f15d0115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15d69e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d01160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15d69e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0116700 con 0x7f15d00fe8f0 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15d69e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d010fa30 con 0x7f15d00fdfe0 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.833+0000 7f15cf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d01160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d01160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37160/0 (socket says 172.21.15.19:37160) 2024-02-22T16:05:44.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cf7fe700 1 -- 172.21.15.19:0/1133916399 learned_addr learned my addr 172.21.15.19:0/1133916399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cffff700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 0x7f15d0115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cf7fe700 1 -- 172.21.15.19:0/1133916399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 msgr2=0x7f15d0115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cf7fe700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 0x7f15d0115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cf7fe700 1 -- 172.21.15.19:0/1133916399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c4005040 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cffff700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 0x7f15d0115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15cf7fe700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d01160b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f15c400b370 tx=0x7f15c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c400e040 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d010fcb0 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15c400bc90 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c401c650 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.834+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d01101f0 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.836+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d0106ac0 con 0x7f15d00fe8f0 2024-02-22T16:05:44.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.836+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f15c40139d0 con 0x7f15d00fe8f0 2024-02-22T16:05:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.836+0000 7f15ccff9700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15b806cc20 0x7f15b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.836+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f15c408d420 con 0x7f15d00fe8f0 2024-02-22T16:05:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.836+0000 7f15cffff700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15b806cc20 0x7f15b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:44.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.837+0000 7f15cffff700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15b806cc20 0x7f15b806f0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009fd0 tx=0x7f15c0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:44.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:44.840+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f15c405aee0 con 0x7f15d00fe8f0 2024-02-22T16:05:45.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.092+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f15d0109ef0 con 0x7f15d00fe8f0 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.092+0000 7f15ccff9700 1 -- 172.21.15.19:0/1133916399 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f15c405fe60 con 0x7f15d00fe8f0 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:05:45.069 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:05:45.070 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:05:45.071 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:45.072 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:45.072 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:45.072 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:05:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15b806cc20 msgr2=0x7f15b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15b806cc20 0x7f15b806f0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009fd0 tx=0x7f15c0009ee0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 msgr2=0x7f15d01160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d01160b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f15c400b370 tx=0x7f15c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 shutdown_connections 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f15b806cc20 0x7f15b806f0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15d00fdfe0 0x7f15d0115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 --2- 172.21.15.19:0/1133916399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d00fe8f0 0x7f15d01160b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 >> 172.21.15.19:0/1133916399 conn(0x7f15d0071d90 msgr2=0x7f15d0103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 shutdown_connections 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.094+0000 7f15d69e0700 1 -- 172.21.15.19:0/1133916399 wait complete. 2024-02-22T16:05:45.073 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 -- 172.21.15.19:0/3101029754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 msgr2=0x7ff40c0fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 --2- 172.21.15.19:0/3101029754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c0fcf50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8004940 tx=0x7ff3f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 -- 172.21.15.19:0/3101029754 shutdown_connections 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 --2- 172.21.15.19:0/3101029754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c0fcf50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 --2- 172.21.15.19:0/3101029754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 0x7ff40c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 -- 172.21.15.19:0/3101029754 >> 172.21.15.19:0/3101029754 conn(0x7ff40c0f8af0 msgr2=0x7ff40c0faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 -- 172.21.15.19:0/3101029754 shutdown_connections 2024-02-22T16:05:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.199+0000 7ff412e98700 1 -- 172.21.15.19:0/3101029754 wait complete. 2024-02-22T16:05:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff412e98700 1 Processor -- start 2024-02-22T16:05:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff412e98700 1 -- start start 2024-02-22T16:05:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff412e98700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 0x7ff40c113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff412e98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff412e98700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c1145d0 con 0x7ff40c1038e0 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff412e98700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c10d900 con 0x7ff40c102f40 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff40bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff40bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37178/0 (socket says 172.21.15.19:37178) 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.200+0000 7ff40bfff700 1 -- 172.21.15.19:0/2974551211 learned_addr learned my addr 172.21.15.19:0/2974551211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff410c34700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 0x7ff40c113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff40bfff700 1 -- 172.21.15.19:0/2974551211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 msgr2=0x7ff40c113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff40bfff700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 0x7ff40c113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff40bfff700 1 -- 172.21.15.19:0/2974551211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f8005040 con 0x7ff40c1038e0 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff410c34700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 0x7ff40c113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff40bfff700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c113f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009fd0 tx=0x7ff3f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:45.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f800e040 con 0x7ff40c1038e0 2024-02-22T16:05:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c10db80 con 0x7ff40c1038e0 2024-02-22T16:05:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3f800bc50 con 0x7ff40c1038e0 2024-02-22T16:05:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f801c3f0 con 0x7ff40c1038e0 2024-02-22T16:05:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.201+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c10e0c0 con 0x7ff40c1038e0 2024-02-22T16:05:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.202+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c105410 con 0x7ff40c1038e0 2024-02-22T16:05:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.203+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff3f8013420 con 0x7ff40c1038e0 2024-02-22T16:05:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.203+0000 7ff4097fa700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff3fc06cc20 0x7ff3fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.203+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff3f808d470 con 0x7ff40c1038e0 2024-02-22T16:05:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.203+0000 7ff410c34700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff3fc06cc20 0x7ff3fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.204+0000 7ff410c34700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff3fc06cc20 0x7ff3fc06f0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff400009fd0 tx=0x7ff400009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:45.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.208+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff3f805af30 con 0x7ff40c1038e0 2024-02-22T16:05:45.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.430+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff40c10f090 con 0x7ff3fc06cc20 2024-02-22T16:05:45.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.431+0000 7ff4097fa700 1 -- 172.21.15.19:0/2974551211 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff40c10f090 con 0x7ff3fc06cc20 2024-02-22T16:05:45.407 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:05:45.408 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff3fc06cc20 msgr2=0x7ff3fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff3fc06cc20 0x7ff3fc06f0d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff400009fd0 tx=0x7ff400009ee0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 msgr2=0x7ff40c113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c113f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009fd0 tx=0x7ff3f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 shutdown_connections 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff3fc06cc20 0x7ff3fc06f0d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff40c102f40 0x7ff40c113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 --2- 172.21.15.19:0/2974551211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40c1038e0 0x7ff40c113f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 >> 172.21.15.19:0/2974551211 conn(0x7ff40c0f8af0 msgr2=0x7ff40c0fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 shutdown_connections 2024-02-22T16:05:45.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.433+0000 7ff412e98700 1 -- 172.21.15.19:0/2974551211 wait complete. 2024-02-22T16:05:45.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:45 smithi062 ceph-mon[106861]: pgmap v374: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:45.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:45 smithi062 ceph-mon[106861]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:45.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:45 smithi062 ceph-mon[106861]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:45.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:45 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/797794777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:05:45.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:45 smithi019 ceph-mon[173579]: pgmap v374: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:45.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:45 smithi019 ceph-mon[173579]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:45.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:45 smithi019 ceph-mon[173579]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:45.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:45 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/797794777' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.536+0000 7f821d404700 1 -- 172.21.15.19:0/2820345148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 msgr2=0x7f82180fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.536+0000 7f821d404700 1 --2- 172.21.15.19:0/2820345148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f82180fcf60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f820c004940 tx=0x7f820c009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 -- 172.21.15.19:0/2820345148 shutdown_connections 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 --2- 172.21.15.19:0/2820345148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f82180fcf60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 --2- 172.21.15.19:0/2820345148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 0x7f8218103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 -- 172.21.15.19:0/2820345148 >> 172.21.15.19:0/2820345148 conn(0x7f82180f8ae0 msgr2=0x7f82180faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 -- 172.21.15.19:0/2820345148 shutdown_connections 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 -- 172.21.15.19:0/2820345148 wait complete. 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 Processor -- start 2024-02-22T16:05:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.537+0000 7f821d404700 1 -- start start 2024-02-22T16:05:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f821d404700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 0x7f8218113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f821d404700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f8218113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f821d404700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181144e0 con 0x7f82181038f0 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f821d404700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821810d810 con 0x7f8218102f50 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f8218113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f8218113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37186/0 (socket says 172.21.15.19:37186) 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 -- 172.21.15.19:0/1979247288 learned_addr learned my addr 172.21.15.19:0/1979247288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f8216ffd700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 0x7f8218113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 -- 172.21.15.19:0/1979247288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 msgr2=0x7f8218113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 0x7f8218113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 -- 172.21.15.19:0/1979247288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c005040 con 0x7f82181038f0 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f8216ffd700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 0x7f8218113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f82167fc700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f8218113e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f820c008d80 tx=0x7f820c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f820c021070 con 0x7f82181038f0 2024-02-22T16:05:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f820c01c400 con 0x7f82181038f0 2024-02-22T16:05:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821810da90 con 0x7f82181038f0 2024-02-22T16:05:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f820c00e070 con 0x7f82181038f0 2024-02-22T16:05:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.538+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821810df70 con 0x7f82181038f0 2024-02-22T16:05:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.539+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82180fc6e0 con 0x7f82181038f0 2024-02-22T16:05:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.540+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f820c01c950 con 0x7f82181038f0 2024-02-22T16:05:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.540+0000 7f81ff7fe700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f820006cc30 0x7f820006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:05:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.540+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f820c08d2a0 con 0x7f82181038f0 2024-02-22T16:05:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.540+0000 7f8216ffd700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f820006cc30 0x7f820006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:05:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.541+0000 7f8216ffd700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f820006cc30 0x7f820006f0e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8208004600 tx=0x7f820800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:05:45.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.545+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f820c05acc0 con 0x7f82181038f0 2024-02-22T16:05:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.829+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f821804bed0 con 0x7f82181038f0 2024-02-22T16:05:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.829+0000 7f81ff7fe700 1 -- 172.21.15.19:0/1979247288 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f820c05a850 con 0x7f82181038f0 2024-02-22T16:05:45.806 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:05:45.806 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:05:45.806 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:05:45.806 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:05:45.806 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f820006cc30 msgr2=0x7f820006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f820006cc30 0x7f820006f0e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8208004600 tx=0x7f820800b040 comp rx=0 tx=0).stop 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 msgr2=0x7f8218113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f8218113e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f820c008d80 tx=0x7f820c009e30 comp rx=0 tx=0).stop 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 shutdown_connections 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f820006cc30 0x7f820006f0e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8218102f50 0x7f8218113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 --2- 172.21.15.19:0/1979247288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82181038f0 0x7f8218113e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 >> 172.21.15.19:0/1979247288 conn(0x7f82180f8ae0 msgr2=0x7f82180fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:05:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 shutdown_connections 2024-02-22T16:05:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:05:45.831+0000 7f821d404700 1 -- 172.21.15.19:0/1979247288 wait complete. 2024-02-22T16:05:46.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:46 smithi062 ceph-mon[106861]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:46.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:46 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1133916399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:05:46.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:46 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1979247288' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:05:46.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:46 smithi019 ceph-mon[173579]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:46.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:46 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1133916399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:05:46.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:46 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1979247288' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:05:47.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:47 smithi062 ceph-mon[106861]: pgmap v375: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:47.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:47 smithi062 ceph-mon[106861]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:47.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:47 smithi019 ceph-mon[173579]: pgmap v375: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:47.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:47 smithi019 ceph-mon[173579]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:05:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:49 smithi062 ceph-mon[106861]: pgmap v376: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:49 smithi019 ceph-mon[173579]: pgmap v376: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:05:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:51 smithi019 ceph-mon[173579]: pgmap v377: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:51.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:51 smithi062 ceph-mon[106861]: pgmap v377: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:53.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:53 smithi062 ceph-mon[106861]: pgmap v378: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:53.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:53 smithi019 ceph-mon[173579]: pgmap v378: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:55.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:55 smithi062 ceph-mon[106861]: pgmap v379: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:55.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:55 smithi019 ceph-mon[173579]: pgmap v379: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:57.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:57 smithi062 ceph-mon[106861]: pgmap v380: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:57.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:57 smithi019 ceph-mon[173579]: pgmap v380: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:59.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:05:59 smithi062 ceph-mon[106861]: pgmap v381: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:05:59.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:05:59 smithi019 ceph-mon[173579]: pgmap v381: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:01.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:01 smithi019 ceph-mon[173579]: pgmap v382: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:01.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:01 smithi062 ceph-mon[106861]: pgmap v382: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:03.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:03 smithi062 ceph-mon[106861]: pgmap v383: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:03.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:03 smithi019 ceph-mon[173579]: pgmap v383: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:04.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:04.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:05.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:05 smithi062 ceph-mon[106861]: pgmap v384: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:05.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:05 smithi019 ceph-mon[173579]: pgmap v384: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:06.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:06 smithi062 ceph-mon[106861]: pgmap v385: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:06.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:06 smithi019 ceph-mon[173579]: pgmap v385: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:09.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:09 smithi062 ceph-mon[106861]: pgmap v386: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:09.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:09 smithi019 ceph-mon[173579]: pgmap v386: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:11.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:11 smithi019 ceph-mon[173579]: pgmap v387: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:11.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:11 smithi062 ceph-mon[106861]: pgmap v387: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:13.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:13 smithi062 ceph-mon[106861]: pgmap v388: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:13.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:13 smithi019 ceph-mon[173579]: pgmap v388: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:15.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:15 smithi062 ceph-mon[106861]: pgmap v389: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:15.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:15 smithi019 ceph-mon[173579]: pgmap v389: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 -- 172.21.15.19:0/1719683962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 msgr2=0x7fa34c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1719683962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c107050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa340004970 tx=0x7fa340009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 -- 172.21.15.19:0/1719683962 shutdown_connections 2024-02-22T16:06:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1719683962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c107050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1719683962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c0fdfe0 0x7fa34c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:15.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 -- 172.21.15.19:0/1719683962 >> 172.21.15.19:0/1719683962 conn(0x7fa34c071d90 msgr2=0x7fa34c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 -- 172.21.15.19:0/1719683962 shutdown_connections 2024-02-22T16:06:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.948+0000 7fa350eb1700 1 -- 172.21.15.19:0/1719683962 wait complete. 2024-02-22T16:06:15.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.949+0000 7fa350eb1700 1 Processor -- start 2024-02-22T16:06:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.949+0000 7fa350eb1700 1 -- start start 2024-02-22T16:06:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.949+0000 7fa350eb1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c0fdfe0 0x7fa34c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:15.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.949+0000 7fa350eb1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.949+0000 7fa350eb1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c114500 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.949+0000 7fa350eb1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c10d830 con 0x7fa34c0fdfe0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53786/0 (socket says 172.21.15.19:53786) 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 -- 172.21.15.19:0/1293143284 learned_addr learned my addr 172.21.15.19:0/1293143284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa34a59c700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c0fdfe0 0x7fa34c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 -- 172.21.15.19:0/1293143284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c0fdfe0 msgr2=0x7fa34c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c0fdfe0 0x7fa34c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 -- 172.21.15.19:0/1293143284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340005040 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa349d9b700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c113eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa340000c00 tx=0x7fa340009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34000e040 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c10dab0 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa34000bc10 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34001c3f0 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.950+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c10dff0 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.951+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34c1046c0 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.952+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa340013420 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.952+0000 7fa33affd700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa33406cc20 0x7fa33406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.952+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa34008d400 con 0x7fa34c0fe8f0 2024-02-22T16:06:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.952+0000 7fa34a59c700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa33406cc20 0x7fa33406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:15.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.953+0000 7fa34a59c700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa33406cc20 0x7fa33406f0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa33c007fd0 tx=0x7fa33c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:15.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:15.957+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa34005aec0 con 0x7fa34c0fe8f0 2024-02-22T16:06:16.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.178+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa34c109ef0 con 0x7fa33406cc20 2024-02-22T16:06:16.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.180+0000 7fa33affd700 1 -- 172.21.15.19:0/1293143284 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa34c109ef0 con 0x7fa33406cc20 2024-02-22T16:06:16.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa33406cc20 msgr2=0x7fa33406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa33406cc20 0x7fa33406f0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa33c007fd0 tx=0x7fa33c009040 comp rx=0 tx=0).stop 2024-02-22T16:06:16.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 msgr2=0x7fa34c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c113eb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa340000c00 tx=0x7fa340009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 shutdown_connections 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa33406cc20 0x7fa33406f0d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa34c0fdfe0 0x7fa34c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 --2- 172.21.15.19:0/1293143284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa34c0fe8f0 0x7fa34c113eb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 >> 172.21.15.19:0/1293143284 conn(0x7fa34c071d90 msgr2=0x7fa34c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 shutdown_connections 2024-02-22T16:06:16.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.182+0000 7fa350eb1700 1 -- 172.21.15.19:0/1293143284 wait complete. 2024-02-22T16:06:16.169 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:06:16.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 -- 172.21.15.19:0/1704149531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 msgr2=0x7f8338102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 --2- 172.21.15.19:0/1704149531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f8338102210 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f832c004940 tx=0x7f832c009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 -- 172.21.15.19:0/1704149531 shutdown_connections 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 --2- 172.21.15.19:0/1704149531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83381027e0 0x7f83380fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 --2- 172.21.15.19:0/1704149531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f8338102210 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 -- 172.21.15.19:0/1704149531 >> 172.21.15.19:0/1704149531 conn(0x7f8338071d90 msgr2=0x7f8338072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 -- 172.21.15.19:0/1704149531 shutdown_connections 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 -- 172.21.15.19:0/1704149531 wait complete. 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 Processor -- start 2024-02-22T16:06:16.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.286+0000 7f833d41d700 1 -- start start 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f833d41d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f833810ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f833d41d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83381027e0 0x7f833810ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f833d41d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833810f4d0 con 0x7f8338101e40 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f833d41d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833810f640 con 0x7f83381027e0 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f833810ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f833810ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53816/0 (socket says 172.21.15.19:53816) 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 -- 172.21.15.19:0/4141549180 learned_addr learned my addr 172.21.15.19:0/4141549180 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f83367fc700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83381027e0 0x7f833810ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 -- 172.21.15.19:0/4141549180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83381027e0 msgr2=0x7f833810ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83381027e0 0x7f833810ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 -- 172.21.15.19:0/4141549180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c005040 con 0x7f8338101e40 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f8336ffd700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f833810ea00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f832c009fd0 tx=0x7f832c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.287+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832c00e040 con 0x7f8338101e40 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.288+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338117970 con 0x7f8338101e40 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.288+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f832c013420 con 0x7f8338101e40 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.288+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832c01c630 con 0x7f8338101e40 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.288+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8338117e30 con 0x7f8338101e40 2024-02-22T16:06:16.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.289+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83380ff150 con 0x7f8338101e40 2024-02-22T16:06:16.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.289+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f832c013590 con 0x7f8338101e40 2024-02-22T16:06:16.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.289+0000 7f831f7fe700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f832006cc30 0x7f832006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.290+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f832c08d370 con 0x7f8338101e40 2024-02-22T16:06:16.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.290+0000 7f83367fc700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f832006cc30 0x7f832006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.290+0000 7f83367fc700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f832006cc30 0x7f832006f0e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f83380661a0 tx=0x7f832800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:16.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.294+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f832c057300 con 0x7f8338101e40 2024-02-22T16:06:16.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.514+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8338065ee0 con 0x7f832006cc30 2024-02-22T16:06:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.516+0000 7f831f7fe700 1 -- 172.21.15.19:0/4141549180 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8338065ee0 con 0x7f832006cc30 2024-02-22T16:06:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f832006cc30 msgr2=0x7f832006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f832006cc30 0x7f832006f0e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f83380661a0 tx=0x7f832800b040 comp rx=0 tx=0).stop 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 msgr2=0x7f833810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f833810ea00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f832c009fd0 tx=0x7f832c009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 shutdown_connections 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f832006cc30 0x7f832006f0e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8338101e40 0x7f833810ea00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 --2- 172.21.15.19:0/4141549180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83381027e0 0x7f833810ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 >> 172.21.15.19:0/4141549180 conn(0x7f8338071d90 msgr2=0x7f83380fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 shutdown_connections 2024-02-22T16:06:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.518+0000 7f833d41d700 1 -- 172.21.15.19:0/4141549180 wait complete. 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 -- 172.21.15.19:0/3244341269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fe8f0 msgr2=0x7f9a5c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/3244341269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fe8f0 0x7f9a5c107050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9a48004970 tx=0x7f9a48009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 -- 172.21.15.19:0/3244341269 shutdown_connections 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/3244341269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fe8f0 0x7f9a5c107050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/3244341269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 -- 172.21.15.19:0/3244341269 >> 172.21.15.19:0/3244341269 conn(0x7f9a5c071d90 msgr2=0x7f9a5c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 -- 172.21.15.19:0/3244341269 shutdown_connections 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.621+0000 7f9a62cd7700 1 -- 172.21.15.19:0/3244341269 wait complete. 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a62cd7700 1 Processor -- start 2024-02-22T16:06:16.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a62cd7700 1 -- start start 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a62cd7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a62cd7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a5c0fe8f0 0x7f9a5c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a62cd7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c114500 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a62cd7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c10d830 con 0x7f9a5c0fe8f0 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a60a73700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a60a73700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53832/0 (socket says 172.21.15.19:53832) 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a60a73700 1 -- 172.21.15.19:0/2902180531 learned_addr learned my addr 172.21.15.19:0/2902180531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.622+0000 7f9a5bfff700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a5c0fe8f0 0x7f9a5c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a60a73700 1 -- 172.21.15.19:0/2902180531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a5c0fe8f0 msgr2=0x7f9a5c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a60a73700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a5c0fe8f0 0x7f9a5c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a60a73700 1 -- 172.21.15.19:0/2902180531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48005040 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a60a73700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c113970 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9a50002a50 tx=0x7f9a500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a50007cc0 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c10db10 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a5000f040 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a5000d9c0 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.623+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c10e020 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.624+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c1046c0 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.625+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9a5000db20 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.625+0000 7f9a597fa700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4c06cc20 0x7f9a4c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.625+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9a5008c490 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.625+0000 7f9a5bfff700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4c06cc20 0x7f9a4c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.626+0000 7f9a5bfff700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4c06cc20 0x7f9a4c06f0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a48004f50 tx=0x7f9a48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.629+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9a50059f60 con 0x7f9a5c0fdfe0 2024-02-22T16:06:16.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.840+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a5c04bf40 con 0x7f9a4c06cc20 2024-02-22T16:06:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.850+0000 7f9a597fa700 1 -- 172.21.15.19:0/2902180531 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f9a5c04bf40 con 0x7f9a4c06cc20 2024-02-22T16:06:16.827 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:06:16.827 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (13m) 6m ago 24m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (10m) 6m ago 24m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (10m) 6m ago 23m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (13m) 6m ago 24m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (6m) 6m ago 18m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (7m) 6m ago 18m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (6m) 6m ago 18m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (6m) 6m ago 18m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (15m) 6m ago 26m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (14m) 6m ago 23m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (11m) 6m ago 26m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (11m) 6m ago 23m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (14m) 6m ago 24m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:06:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (14m) 6m ago 23m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (10m) 6m ago 22m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (9m) 6m ago 22m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (9m) 6m ago 21m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (8m) 6m ago 21m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (8m) 6m ago 20m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (7m) 6m ago 20m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:06:16.829 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (13m) 6m ago 24m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4c06cc20 msgr2=0x7f9a4c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4c06cc20 0x7f9a4c06f0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9a48004f50 tx=0x7f9a48009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 msgr2=0x7f9a5c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c113970 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9a50002a50 tx=0x7f9a500046b0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 shutdown_connections 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9a4c06cc20 0x7f9a4c06f0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a5c0fdfe0 0x7f9a5c113970 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 --2- 172.21.15.19:0/2902180531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a5c0fe8f0 0x7f9a5c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 >> 172.21.15.19:0/2902180531 conn(0x7f9a5c071d90 msgr2=0x7f9a5c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 shutdown_connections 2024-02-22T16:06:16.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.852+0000 7f9a62cd7700 1 -- 172.21.15.19:0/2902180531 wait complete. 2024-02-22T16:06:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 -- 172.21.15.19:0/163670197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 msgr2=0x7fb7dc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 --2- 172.21.15.19:0/163670197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc102210 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0004940 tx=0x7fb7d0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 -- 172.21.15.19:0/163670197 shutdown_connections 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 --2- 172.21.15.19:0/163670197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7dc1027e0 0x7fb7dc0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 --2- 172.21.15.19:0/163670197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc102210 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 -- 172.21.15.19:0/163670197 >> 172.21.15.19:0/163670197 conn(0x7fb7dc071d90 msgr2=0x7fb7dc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 -- 172.21.15.19:0/163670197 shutdown_connections 2024-02-22T16:06:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.954+0000 7fb7e3653700 1 -- 172.21.15.19:0/163670197 wait complete. 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e3653700 1 Processor -- start 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e3653700 1 -- start start 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e3653700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e3653700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7dc1027e0 0x7fb7dc112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e3653700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc113410 con 0x7fb7dc101e40 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e3653700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc10c740 con 0x7fb7dc1027e0 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e13ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e13ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc112880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53856/0 (socket says 172.21.15.19:53856) 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e13ef700 1 -- 172.21.15.19:0/868179604 learned_addr learned my addr 172.21.15.19:0/868179604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e0bee700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7dc1027e0 0x7fb7dc112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e13ef700 1 -- 172.21.15.19:0/868179604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7dc1027e0 msgr2=0x7fb7dc112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e13ef700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7dc1027e0 0x7fb7dc112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.955+0000 7fb7e13ef700 1 -- 172.21.15.19:0/868179604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0005040 con 0x7fb7dc101e40 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.956+0000 7fb7e13ef700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc112880 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009fd0 tx=0x7fb7d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.956+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d000e040 con 0x7fb7dc101e40 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.956+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc10c9c0 con 0x7fb7dc101e40 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.956+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7d0013420 con 0x7fb7dc101e40 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.956+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d001c630 con 0x7fb7dc101e40 2024-02-22T16:06:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.956+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc10cf00 con 0x7fb7dc101e40 2024-02-22T16:06:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.957+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc0738b0 con 0x7fb7dc101e40 2024-02-22T16:06:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.958+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb7d0013590 con 0x7fb7dc101e40 2024-02-22T16:06:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.958+0000 7fb7cdffb700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb7c806cc30 0x7fb7c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.958+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb7d008d230 con 0x7fb7dc101e40 2024-02-22T16:06:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.958+0000 7fb7e0bee700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb7c806cc30 0x7fb7c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:16.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.959+0000 7fb7e0bee700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb7c806cc30 0x7fb7c806f0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc10e0b0 tx=0x7fb7d8002040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:16.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:16.963+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb7d005ad90 con 0x7fb7dc101e40 2024-02-22T16:06:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.251+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb7dc10de60 con 0x7fb7dc101e40 2024-02-22T16:06:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.252+0000 7fb7cdffb700 1 -- 172.21.15.19:0/868179604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fb7d005a920 con 0x7fb7dc101e40 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:06:17.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:17.231 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:06:17.231 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:06:17.231 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:06:17.231 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:06:17.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb7c806cc30 msgr2=0x7fb7c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb7c806cc30 0x7fb7c806f0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc10e0b0 tx=0x7fb7d8002040 comp rx=0 tx=0).stop 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 msgr2=0x7fb7dc112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc112880 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009fd0 tx=0x7fb7d0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 shutdown_connections 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb7c806cc30 0x7fb7c806f0e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7dc101e40 0x7fb7dc112880 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 --2- 172.21.15.19:0/868179604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7dc1027e0 0x7fb7dc112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 >> 172.21.15.19:0/868179604 conn(0x7fb7dc071d90 msgr2=0x7fb7dc0fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 shutdown_connections 2024-02-22T16:06:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.254+0000 7fb7e3653700 1 -- 172.21.15.19:0/868179604 wait complete. 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.357+0000 7f2304097700 1 -- 172.21.15.19:0/3139283365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc102f40 msgr2=0x7f22fc103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.357+0000 7f2304097700 1 --2- 172.21.15.19:0/3139283365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc102f40 0x7f22fc103310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f22f8004940 tx=0x7f22f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.357+0000 7f2304097700 1 -- 172.21.15.19:0/3139283365 shutdown_connections 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.357+0000 7f2304097700 1 --2- 172.21.15.19:0/3139283365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc1038e0 0x7f22fc0fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.357+0000 7f2304097700 1 --2- 172.21.15.19:0/3139283365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc102f40 0x7f22fc103310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.357+0000 7f2304097700 1 -- 172.21.15.19:0/3139283365 >> 172.21.15.19:0/3139283365 conn(0x7f22fc0f8af0 msgr2=0x7f22fc0faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 -- 172.21.15.19:0/3139283365 shutdown_connections 2024-02-22T16:06:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 -- 172.21.15.19:0/3139283365 wait complete. 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 Processor -- start 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 -- start start 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc102f40 0x7f22fc1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 0x7f22fc113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc114440 con 0x7f22fc1038e0 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2304097700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc10d7e0 con 0x7f22fc102f40 2024-02-22T16:06:17.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2301632700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 0x7f22fc113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2301632700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 0x7f22fc113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53888/0 (socket says 172.21.15.19:53888) 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.358+0000 7f2301632700 1 -- 172.21.15.19:0/2444380283 learned_addr learned my addr 172.21.15.19:0/2444380283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2301e33700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc102f40 0x7f22fc1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2301632700 1 -- 172.21.15.19:0/2444380283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc102f40 msgr2=0x7f22fc1138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2301632700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc102f40 0x7f22fc1138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2301632700 1 -- 172.21.15.19:0/2444380283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8005040 con 0x7f22fc1038e0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2301e33700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc102f40 0x7f22fc1138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2301632700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 0x7f22fc113df0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22ec002a50 tx=0x7f22ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ec007cc0 con 0x7f22fc1038e0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc10dac0 con 0x7f22fc1038e0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22ec00f040 con 0x7f22fc1038e0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ec0095d0 con 0x7f22fc1038e0 2024-02-22T16:06:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.359+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc10e000 con 0x7f22fc1038e0 2024-02-22T16:06:17.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.360+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc0fc6f0 con 0x7f22fc1038e0 2024-02-22T16:06:17.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.361+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f22ec009730 con 0x7f22fc1038e0 2024-02-22T16:06:17.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.361+0000 7f22f27fc700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f22e806cc30 0x7f22e806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:17.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.361+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f22ec08bdb0 con 0x7f22fc1038e0 2024-02-22T16:06:17.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.361+0000 7f2301e33700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f22e806cc30 0x7f22e806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:17.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.361+0000 7f2301e33700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f22e806cc30 0x7f22e806f0e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22f800b3b0 tx=0x7f22f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:17.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.366+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f22ec059790 con 0x7f22fc1038e0 2024-02-22T16:06:17.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:17 smithi062 ceph-mon[106861]: pgmap v390: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:17.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:17 smithi019 ceph-mon[173579]: pgmap v390: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:17.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.611+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f22fc04bed0 con 0x7f22fc1038e0 2024-02-22T16:06:17.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.611+0000 7f22f27fc700 1 -- 172.21.15.19:0/2444380283 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f22ec059320 con 0x7f22fc1038e0 2024-02-22T16:06:17.589 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:06:17.589 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:06:17.589 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:06:17.590 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:06:17.591 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:17.592 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.613+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f22e806cc30 msgr2=0x7f22e806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.613+0000 7f2304097700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f22e806cc30 0x7f22e806f0e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22f800b3b0 tx=0x7f22f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.613+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 msgr2=0x7f22fc113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.613+0000 7f2304097700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 0x7f22fc113df0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f22ec002a50 tx=0x7f22ec0046b0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 shutdown_connections 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f22e806cc30 0x7f22e806f0e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22fc102f40 0x7f22fc1138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 --2- 172.21.15.19:0/2444380283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22fc1038e0 0x7f22fc113df0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 >> 172.21.15.19:0/2444380283 conn(0x7f22fc0f8af0 msgr2=0x7f22fc0fa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 shutdown_connections 2024-02-22T16:06:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.614+0000 7f2304097700 1 -- 172.21.15.19:0/2444380283 wait complete. 2024-02-22T16:06:17.594 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:06:17.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 -- 172.21.15.19:0/1311815698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 msgr2=0x7f367c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 --2- 172.21.15.19:0/1311815698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c102230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3670004940 tx=0x7f3670009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 -- 172.21.15.19:0/1311815698 shutdown_connections 2024-02-22T16:06:17.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 --2- 172.21.15.19:0/1311815698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 0x7f367c0fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 --2- 172.21.15.19:0/1311815698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c102230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 -- 172.21.15.19:0/1311815698 >> 172.21.15.19:0/1311815698 conn(0x7f367c0f7a10 msgr2=0x7f367c0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 -- 172.21.15.19:0/1311815698 shutdown_connections 2024-02-22T16:06:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.718+0000 7f36812e2700 1 -- 172.21.15.19:0/1311815698 wait complete. 2024-02-22T16:06:17.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f36812e2700 1 Processor -- start 2024-02-22T16:06:17.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f36812e2700 1 -- start start 2024-02-22T16:06:17.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f36812e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c10ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:17.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f36812e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 0x7f367c10efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:17.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f36812e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c10f570 con 0x7f367c101e60 2024-02-22T16:06:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f36812e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c10f6e0 con 0x7f367c102800 2024-02-22T16:06:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f367affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c10ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f367affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c10ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59288/0 (socket says 172.21.15.19:59288) 2024-02-22T16:06:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f367affd700 1 -- 172.21.15.19:0/3514532571 learned_addr learned my addr 172.21.15.19:0/3514532571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.719+0000 7f367a7fc700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 0x7f367c10efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f367affd700 1 -- 172.21.15.19:0/3514532571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 msgr2=0x7f367c10efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f367affd700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 0x7f367c10efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f367affd700 1 -- 172.21.15.19:0/3514532571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670005040 con 0x7f367c101e60 2024-02-22T16:06:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f367a7fc700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 0x7f367c10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:06:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f367affd700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c10ea80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f367000b3b0 tx=0x7f3670009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367000e040 con 0x7f367c101e60 2024-02-22T16:06:17.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f367000bc50 con 0x7f367c101e60 2024-02-22T16:06:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c1179f0 con 0x7f367c101e60 2024-02-22T16:06:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3670013420 con 0x7f367c101e60 2024-02-22T16:06:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.720+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c117f00 con 0x7f367c101e60 2024-02-22T16:06:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.721+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c0ff170 con 0x7f367c101e60 2024-02-22T16:06:17.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.722+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f367001c430 con 0x7f367c101e60 2024-02-22T16:06:17.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.722+0000 7f36637fe700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366406cc20 0x7f366406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:17.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.722+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f367008ce70 con 0x7f367c101e60 2024-02-22T16:06:17.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.722+0000 7f367a7fc700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366406cc20 0x7f366406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:17.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.722+0000 7f367a7fc700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366406cc20 0x7f366406f0d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f367c066a10 tx=0x7f366c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.727+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3670056f50 con 0x7f367c101e60 2024-02-22T16:06:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.948+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f367c066750 con 0x7f366406cc20 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.950+0000 7f36637fe700 1 -- 172.21.15.19:0/3514532571 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f367c066750 con 0x7f366406cc20 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:06:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:06:17.928 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.951+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366406cc20 msgr2=0x7f366406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.951+0000 7f36812e2700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366406cc20 0x7f366406f0d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f367c066a10 tx=0x7f366c009040 comp rx=0 tx=0).stop 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.951+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 msgr2=0x7f367c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.951+0000 7f36812e2700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c10ea80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f367000b3b0 tx=0x7f3670009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 shutdown_connections 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366406cc20 0x7f366406f0d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c101e60 0x7f367c10ea80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 --2- 172.21.15.19:0/3514532571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f367c102800 0x7f367c10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 >> 172.21.15.19:0/3514532571 conn(0x7f367c0f7a10 msgr2=0x7f367c0f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 shutdown_connections 2024-02-22T16:06:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:17.952+0000 7f36812e2700 1 -- 172.21.15.19:0/3514532571 wait complete. 2024-02-22T16:06:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 -- 172.21.15.19:0/896957785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 msgr2=0x7fb424102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/896957785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424102210 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb414004940 tx=0x7fb414009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 -- 172.21.15.19:0/896957785 shutdown_connections 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/896957785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4241027e0 0x7fb4240fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/896957785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424102210 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 -- 172.21.15.19:0/896957785 >> 172.21.15.19:0/896957785 conn(0x7fb424071d90 msgr2=0x7fb424072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 -- 172.21.15.19:0/896957785 shutdown_connections 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 -- 172.21.15.19:0/896957785 wait complete. 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.056+0000 7fb42a8cf700 1 Processor -- start 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb42a8cf700 1 -- start start 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb42a8cf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb42a8cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4241027e0 0x7fb424113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb42a8cf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb424114510 con 0x7fb424101e40 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb42a8cf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410d840 con 0x7fb4241027e0 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59296/0 (socket says 172.21.15.19:59296) 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 -- 172.21.15.19:0/3558425942 learned_addr learned my addr 172.21.15.19:0/3558425942 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb4237fe700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4241027e0 0x7fb424113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 -- 172.21.15.19:0/3558425942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4241027e0 msgr2=0x7fb424113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4241027e0 0x7fb424113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 -- 172.21.15.19:0/3558425942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414005040 con 0x7fb424101e40 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.057+0000 7fb423fff700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424113980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb414009360 tx=0x7fb414009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.058+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41400e040 con 0x7fb424101e40 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.058+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42410dac0 con 0x7fb424101e40 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.058+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb414013420 con 0x7fb424101e40 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.058+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41401c610 con 0x7fb424101e40 2024-02-22T16:06:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.058+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42410e000 con 0x7fb424101e40 2024-02-22T16:06:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.059+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4240738b0 con 0x7fb424101e40 2024-02-22T16:06:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.059+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb414013590 con 0x7fb424101e40 2024-02-22T16:06:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.060+0000 7fb420ff9700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb40c06cc30 0x7fb40c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.060+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb41408d3c0 con 0x7fb424101e40 2024-02-22T16:06:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.060+0000 7fb4237fe700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb40c06cc30 0x7fb40c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.060+0000 7fb4237fe700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb40c06cc30 0x7fb40c06f0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb42410f1b0 tx=0x7fb41800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.064+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb41405ad60 con 0x7fb424101e40 2024-02-22T16:06:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.354+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb42410ef60 con 0x7fb424101e40 2024-02-22T16:06:18.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.354+0000 7fb420ff9700 1 -- 172.21.15.19:0/3558425942 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb41405a8f0 con 0x7fb424101e40 2024-02-22T16:06:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:06:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:06:18.332 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:06:18.332 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:06:18.332 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb40c06cc30 msgr2=0x7fb40c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb40c06cc30 0x7fb40c06f0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb42410f1b0 tx=0x7fb41800b040 comp rx=0 tx=0).stop 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 msgr2=0x7fb424113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424113980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb414009360 tx=0x7fb414009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 shutdown_connections 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb40c06cc30 0x7fb40c06f0e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424101e40 0x7fb424113980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 --2- 172.21.15.19:0/3558425942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4241027e0 0x7fb424113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 >> 172.21.15.19:0/3558425942 conn(0x7fb424071d90 msgr2=0x7fb4240657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 shutdown_connections 2024-02-22T16:06:18.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:18.356+0000 7fb42a8cf700 1 -- 172.21.15.19:0/3558425942 wait complete. 2024-02-22T16:06:18.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:18 smithi062 ceph-mon[106861]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:18.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:18 smithi062 ceph-mon[106861]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:18.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:18 smithi062 ceph-mon[106861]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:18.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:18 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/868179604' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:06:18.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:18 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2444380283' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:06:18.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:18 smithi019 ceph-mon[173579]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:18.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:18 smithi019 ceph-mon[173579]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:18.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:18 smithi019 ceph-mon[173579]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:18.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:18 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/868179604' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:06:18.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:18 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2444380283' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:06:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:19 smithi062 ceph-mon[106861]: pgmap v391: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:19 smithi062 ceph-mon[106861]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:19 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3558425942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:06:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:19 smithi019 ceph-mon[173579]: pgmap v391: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:19 smithi019 ceph-mon[173579]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:19.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:19 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3558425942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:06:21.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:21 smithi019 ceph-mon[173579]: pgmap v392: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:21.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:21 smithi062 ceph-mon[106861]: pgmap v392: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:23 smithi062 ceph-mon[106861]: pgmap v393: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:23 smithi019 ceph-mon[173579]: pgmap v393: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:25.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:25 smithi062 ceph-mon[106861]: pgmap v394: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:25.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:25 smithi019 ceph-mon[173579]: pgmap v394: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:27.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:27 smithi062 ceph-mon[106861]: pgmap v395: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:27.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:27 smithi019 ceph-mon[173579]: pgmap v395: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:29.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:29 smithi062 ceph-mon[106861]: pgmap v396: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:29.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:29 smithi019 ceph-mon[173579]: pgmap v396: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:31.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:31 smithi019 ceph-mon[173579]: pgmap v397: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:31.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:31 smithi062 ceph-mon[106861]: pgmap v397: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:33.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:33 smithi062 ceph-mon[106861]: pgmap v398: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:33.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:33 smithi019 ceph-mon[173579]: pgmap v398: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:34.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:34.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:35.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:35 smithi062 ceph-mon[106861]: pgmap v399: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:35.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:35 smithi019 ceph-mon[173579]: pgmap v399: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:37.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:37 smithi062 ceph-mon[106861]: pgmap v400: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:37.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:37 smithi019 ceph-mon[173579]: pgmap v400: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:39.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:39 smithi062 ceph-mon[106861]: pgmap v401: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:39.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:39 smithi019 ceph-mon[173579]: pgmap v401: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:41.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:41 smithi019 ceph-mon[173579]: pgmap v402: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:41.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:41 smithi062 ceph-mon[106861]: pgmap v402: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:43.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:43 smithi062 ceph-mon[106861]: pgmap v403: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:43.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:43 smithi019 ceph-mon[173579]: pgmap v403: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:45.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:45 smithi062 ceph-mon[106861]: pgmap v404: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:45.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:45 smithi019 ceph-mon[173579]: pgmap v404: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:46.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:46 smithi062 ceph-mon[106861]: pgmap v405: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:46.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:46 smithi019 ceph-mon[173579]: pgmap v405: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:48.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.475+0000 7fda04a6b700 1 -- 172.21.15.19:0/2826521692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fdfe0 msgr2=0x7fda000fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.475+0000 7fda04a6b700 1 --2- 172.21.15.19:0/2826521692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fdfe0 0x7fda000fe3b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0004970 tx=0x7fd9f0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 -- 172.21.15.19:0/2826521692 shutdown_connections 2024-02-22T16:06:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 --2- 172.21.15.19:0/2826521692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda000fe8f0 0x7fda00107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 --2- 172.21.15.19:0/2826521692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fdfe0 0x7fda000fe3b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 -- 172.21.15.19:0/2826521692 >> 172.21.15.19:0/2826521692 conn(0x7fda00071d90 msgr2=0x7fda00072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:48.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 -- 172.21.15.19:0/2826521692 shutdown_connections 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 -- 172.21.15.19:0/2826521692 wait complete. 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 Processor -- start 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.476+0000 7fda04a6b700 1 -- start start 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fda04a6b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda000fdfe0 0x7fda000fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fda04a6b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 0x7fda000fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fda04a6b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000fc9e0 con 0x7fda000fe8f0 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fda04a6b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000fcb50 con 0x7fda000fdfe0 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 0x7fda000fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 0x7fda000fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53784/0 (socket says 172.21.15.19:53784) 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 -- 172.21.15.19:0/3238608298 learned_addr learned my addr 172.21.15.19:0/3238608298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fe59c700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda000fdfe0 0x7fda000fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 -- 172.21.15.19:0/3238608298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda000fdfe0 msgr2=0x7fda000fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda000fdfe0 0x7fda000fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 -- 172.21.15.19:0/3238608298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0005040 con 0x7fda000fe8f0 2024-02-22T16:06:48.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.477+0000 7fd9fdd9b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 0x7fda000fc3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400eb40 tx=0x7fd9f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:48.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.478+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9f4007cc0 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.478+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9f400f040 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.478+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda00119b70 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.478+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9f400c990 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.478+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda00119fd0 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.479+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda001046c0 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.479+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd9f400cc00 con 0x7fda000fe8f0 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.480+0000 7fd9eeffd700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e806cc20 0x7fd9e806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:48.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.480+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd9f408bd10 con 0x7fda000fe8f0 2024-02-22T16:06:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.480+0000 7fd9fe59c700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e806cc20 0x7fd9e806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.480+0000 7fd9fe59c700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e806cc20 0x7fd9e806f0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009fd0 tx=0x7fd9f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:48.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.484+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd9f4055cd0 con 0x7fda000fe8f0 2024-02-22T16:06:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.709+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda000fd5e0 con 0x7fd9e806cc20 2024-02-22T16:06:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.710+0000 7fd9eeffd700 1 -- 172.21.15.19:0/3238608298 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fda000fd5e0 con 0x7fd9e806cc20 2024-02-22T16:06:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e806cc20 msgr2=0x7fd9e806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e806cc20 0x7fd9e806f0d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009fd0 tx=0x7fd9f0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 msgr2=0x7fda000fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 0x7fda000fc3e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400eb40 tx=0x7fd9f400b040 comp rx=0 tx=0).stop 2024-02-22T16:06:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 shutdown_connections 2024-02-22T16:06:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e806cc20 0x7fd9e806f0d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda000fdfe0 0x7fda000fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 --2- 172.21.15.19:0/3238608298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda000fe8f0 0x7fda000fc3e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 >> 172.21.15.19:0/3238608298 conn(0x7fda00071d90 msgr2=0x7fda00101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 shutdown_connections 2024-02-22T16:06:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.712+0000 7fda04a6b700 1 -- 172.21.15.19:0/3238608298 wait complete. 2024-02-22T16:06:48.699 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:06:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 -- 172.21.15.19:0/2868631776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9041027e0 msgr2=0x7fc9040fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 --2- 172.21.15.19:0/2868631776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9041027e0 0x7fc9040fbe50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004940 tx=0x7fc8f0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:48.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 -- 172.21.15.19:0/2868631776 shutdown_connections 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 --2- 172.21.15.19:0/2868631776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9041027e0 0x7fc9040fbe50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 --2- 172.21.15.19:0/2868631776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc904101e40 0x7fc904102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 -- 172.21.15.19:0/2868631776 >> 172.21.15.19:0/2868631776 conn(0x7fc9040f79d0 msgr2=0x7fc9040f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 -- 172.21.15.19:0/2868631776 shutdown_connections 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 -- 172.21.15.19:0/2868631776 wait complete. 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 Processor -- start 2024-02-22T16:06:48.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.817+0000 7fc90aa89700 1 -- start start 2024-02-22T16:06:48.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc90aa89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 0x7fc904113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc90aa89700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9041027e0 0x7fc904113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc90aa89700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041144c0 con 0x7fc904101e40 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc90aa89700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90410d860 con 0x7fc9041027e0 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 0x7fc904113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 0x7fc904113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53792/0 (socket says 172.21.15.19:53792) 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 -- 172.21.15.19:0/62943944 learned_addr learned my addr 172.21.15.19:0/62943944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc903fff700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9041027e0 0x7fc904113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 -- 172.21.15.19:0/62943944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9041027e0 msgr2=0x7fc904113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9041027e0 0x7fc904113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 -- 172.21.15.19:0/62943944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0005040 con 0x7fc904101e40 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc908825700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 0x7fc904113930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c8c0 tx=0x7fc8f80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f800a760 con 0x7fc904101e40 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8f800ad70 con 0x7fc904101e40 2024-02-22T16:06:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90410db40 con 0x7fc904101e40 2024-02-22T16:06:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.818+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f80113a0 con 0x7fc904101e40 2024-02-22T16:06:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.819+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90410e050 con 0x7fc904101e40 2024-02-22T16:06:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.819+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9040fb5d0 con 0x7fc904101e40 2024-02-22T16:06:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.820+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc8f8012400 con 0x7fc904101e40 2024-02-22T16:06:48.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.820+0000 7fc9017fa700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8f406cc20 0x7fc8f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.821+0000 7fc903fff700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8f406cc20 0x7fc8f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.821+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc8f808c0a0 con 0x7fc904101e40 2024-02-22T16:06:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.821+0000 7fc903fff700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8f406cc20 0x7fc8f406f0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc90410f260 tx=0x7fc8f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:48.825+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc8f8059b60 con 0x7fc904101e40 2024-02-22T16:06:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.057+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc90404bed0 con 0x7fc8f406cc20 2024-02-22T16:06:49.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.058+0000 7fc9017fa700 1 -- 172.21.15.19:0/62943944 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc90404bed0 con 0x7fc8f406cc20 2024-02-22T16:06:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8f406cc20 msgr2=0x7fc8f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8f406cc20 0x7fc8f406f0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc90410f260 tx=0x7fc8f0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 msgr2=0x7fc904113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 0x7fc904113930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800c8c0 tx=0x7fc8f80044c0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 shutdown_connections 2024-02-22T16:06:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8f406cc20 0x7fc8f406f0d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc904101e40 0x7fc904113930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 --2- 172.21.15.19:0/62943944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9041027e0 0x7fc904113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 >> 172.21.15.19:0/62943944 conn(0x7fc9040f79d0 msgr2=0x7fc904066050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 shutdown_connections 2024-02-22T16:06:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.060+0000 7fc90aa89700 1 -- 172.21.15.19:0/62943944 wait complete. 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 -- 172.21.15.19:0/1023963697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 msgr2=0x7f49840fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 --2- 172.21.15.19:0/1023963697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fb9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f496c004940 tx=0x7f496c009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 -- 172.21.15.19:0/1023963697 shutdown_connections 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 --2- 172.21.15.19:0/1023963697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fb9c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 --2- 172.21.15.19:0/1023963697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984103140 0x7f4984103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 -- 172.21.15.19:0/1023963697 >> 172.21.15.19:0/1023963697 conn(0x7f4984071560 msgr2=0x7f4984071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 -- 172.21.15.19:0/1023963697 shutdown_connections 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 -- 172.21.15.19:0/1023963697 wait complete. 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.164+0000 7f498994a700 1 Processor -- start 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f498994a700 1 -- start start 2024-02-22T16:06:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f498994a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f498994a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984103140 0x7f49840fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f498994a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840fd290 con 0x7f49840fb570 2024-02-22T16:06:49.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f498994a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840fd400 con 0x7f4984103140 2024-02-22T16:06:49.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53804/0 (socket says 172.21.15.19:53804) 2024-02-22T16:06:49.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 -- 172.21.15.19:0/2979106311 learned_addr learned my addr 172.21.15.19:0/2979106311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:49.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f49827fc700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984103140 0x7f49840fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 -- 172.21.15.19:0/2979106311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984103140 msgr2=0x7f49840fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984103140 0x7f49840fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 -- 172.21.15.19:0/2979106311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c005040 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.165+0000 7f4982ffd700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fc750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4974002a50 tx=0x7f49740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.166+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4974007cc0 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.166+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4984119d10 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.166+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f497400f040 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.166+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497400d9c0 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.166+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498411a1f0 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.167+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49841054d0 con 0x7f49840fb570 2024-02-22T16:06:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.167+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f497400db20 con 0x7f49840fb570 2024-02-22T16:06:49.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.168+0000 7f497b7fe700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f497006cc30 0x7f497006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.168+0000 7f49827fc700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f497006cc30 0x7f497006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.168+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f497408bca0 con 0x7f49840fb570 2024-02-22T16:06:49.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.168+0000 7f49827fc700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f497006cc30 0x7f497006f0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49840fe150 tx=0x7f496c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:49.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.172+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4974090070 con 0x7f49840fb570 2024-02-22T16:06:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.382+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f498404bf40 con 0x7f497006cc30 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.392+0000 7f497b7fe700 1 -- 172.21.15.19:0/2979106311 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f498404bf40 con 0x7f497006cc30 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (13m) 7m ago 25m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (11m) 7m ago 25m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (11m) 6m ago 24m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (13m) 7m ago 24m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (7m) 7m ago 19m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (7m) 7m ago 19m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:06:49.368 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (6m) 6m ago 19m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (7m) 6m ago 19m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (15m) 7m ago 26m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (15m) 6m ago 24m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (12m) 7m ago 26m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (12m) 6m ago 24m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (14m) 7m ago 25m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (14m) 6m ago 24m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (10m) 7m ago 23m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (10m) 7m ago 22m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (9m) 7m ago 22m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (9m) 6m ago 21m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (8m) 6m ago 21m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (8m) 6m ago 20m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:06:49.369 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (14m) 7m ago 24m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:06:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f497006cc30 msgr2=0x7f497006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f497006cc30 0x7f497006f0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49840fe150 tx=0x7f496c009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 msgr2=0x7f49840fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fc750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4974002a50 tx=0x7f49740046b0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 shutdown_connections 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f497006cc30 0x7f497006f0e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49840fb570 0x7f49840fc750 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 --2- 172.21.15.19:0/2979106311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4984103140 0x7f49840fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 >> 172.21.15.19:0/2979106311 conn(0x7f4984071560 msgr2=0x7f49840f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 shutdown_connections 2024-02-22T16:06:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.394+0000 7f498994a700 1 -- 172.21.15.19:0/2979106311 wait complete. 2024-02-22T16:06:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:49 smithi062 ceph-mon[106861]: pgmap v406: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 -- 172.21.15.19:0/3339772012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 msgr2=0x7f34b8107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/3339772012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8107050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34b4004970 tx=0x7f34b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 -- 172.21.15.19:0/3339772012 shutdown_connections 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/3339772012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8107050 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/3339772012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b80fdfe0 0x7f34b80fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 -- 172.21.15.19:0/3339772012 >> 172.21.15.19:0/3339772012 conn(0x7f34b8071d90 msgr2=0x7f34b8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 -- 172.21.15.19:0/3339772012 shutdown_connections 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 -- 172.21.15.19:0/3339772012 wait complete. 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 Processor -- start 2024-02-22T16:06:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.496+0000 7f34bf2ff700 1 -- start start 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bf2ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b80fdfe0 0x7f34b8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bf2ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bf2ff700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8114500 con 0x7f34b80fe8f0 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bf2ff700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b810d830 con 0x7f34b80fdfe0 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53822/0 (socket says 172.21.15.19:53822) 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 -- 172.21.15.19:0/1972245578 learned_addr learned my addr 172.21.15.19:0/1972245578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bd09b700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b80fdfe0 0x7f34b8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 -- 172.21.15.19:0/1972245578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b80fdfe0 msgr2=0x7f34b8113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b80fdfe0 0x7f34b8113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 -- 172.21.15.19:0/1972245578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4005040 con 0x7f34b80fe8f0 2024-02-22T16:06:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bc89a700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8113eb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b400e040 con 0x7f34b80fe8f0 2024-02-22T16:06:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34b4009b10 con 0x7f34b80fe8f0 2024-02-22T16:06:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b810dab0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.497+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b4013e60 con 0x7f34b80fe8f0 2024-02-22T16:06:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.498+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b810dff0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.499+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b81046c0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.499+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f34b40136b0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.499+0000 7f34adffb700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f34a406cc20 0x7f34a406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.500+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f34b408d2a0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.500+0000 7f34bd09b700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f34a406cc20 0x7f34a406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.500+0000 7f34bd09b700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f34a406cc20 0x7f34a406f0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009b80 tx=0x7f34a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:49.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.504+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f34b405ad40 con 0x7f34b80fe8f0 2024-02-22T16:06:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:49 smithi019 ceph-mon[173579]: pgmap v406: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.792+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f34b8109ef0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.793+0000 7f34adffb700 1 -- 172.21.15.19:0/1972245578 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f34b405a8d0 con 0x7f34b80fe8f0 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:49.769 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:06:49.770 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:06:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f34a406cc20 msgr2=0x7f34a406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f34a406cc20 0x7f34a406f0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009b80 tx=0x7f34a800b040 comp rx=0 tx=0).stop 2024-02-22T16:06:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 msgr2=0x7f34b8113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8113eb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 shutdown_connections 2024-02-22T16:06:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f34a406cc20 0x7f34a406f0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34b80fdfe0 0x7f34b8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 --2- 172.21.15.19:0/1972245578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b80fe8f0 0x7f34b8113eb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 >> 172.21.15.19:0/1972245578 conn(0x7f34b8071d90 msgr2=0x7f34b8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 shutdown_connections 2024-02-22T16:06:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.795+0000 7f34bf2ff700 1 -- 172.21.15.19:0/1972245578 wait complete. 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 -- 172.21.15.19:0/3879643318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c101e40 msgr2=0x7f5b5c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 --2- 172.21.15.19:0/3879643318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c101e40 0x7f5b5c102210 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c004940 tx=0x7f5b4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 -- 172.21.15.19:0/3879643318 shutdown_connections 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 --2- 172.21.15.19:0/3879643318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 --2- 172.21.15.19:0/3879643318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c101e40 0x7f5b5c102210 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 -- 172.21.15.19:0/3879643318 >> 172.21.15.19:0/3879643318 conn(0x7f5b5c071d90 msgr2=0x7f5b5c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 -- 172.21.15.19:0/3879643318 shutdown_connections 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.899+0000 7f5b630c9700 1 -- 172.21.15.19:0/3879643318 wait complete. 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b630c9700 1 Processor -- start 2024-02-22T16:06:49.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b630c9700 1 -- start start 2024-02-22T16:06:49.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b630c9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5c101e40 0x7f5b5c10ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b630c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c10ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b630c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10f4d0 con 0x7f5b5c1027e0 2024-02-22T16:06:49.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b630c9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10f640 con 0x7f5b5c101e40 2024-02-22T16:06:49.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c10ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b5bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c10ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53842/0 (socket says 172.21.15.19:53842) 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b5bfff700 1 -- 172.21.15.19:0/1595890730 learned_addr learned my addr 172.21.15.19:0/1595890730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.900+0000 7f5b60e65700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5c101e40 0x7f5b5c10ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b5bfff700 1 -- 172.21.15.19:0/1595890730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5c101e40 msgr2=0x7f5b5c10ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b5bfff700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5c101e40 0x7f5b5c10ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b5bfff700 1 -- 172.21.15.19:0/1595890730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c005040 con 0x7f5b5c1027e0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b5bfff700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c10ef40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b50002a50 tx=0x7f5b500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b50007cc0 con 0x7f5b5c1027e0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c1179d0 con 0x7f5b5c1027e0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b5000f040 con 0x7f5b5c1027e0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5000da30 con 0x7f5b5c1027e0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.901+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c117e90 con 0x7f5b5c1027e0 2024-02-22T16:06:49.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.902+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c0ff150 con 0x7f5b5c1027e0 2024-02-22T16:06:49.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.903+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b5000db90 con 0x7f5b5c1027e0 2024-02-22T16:06:49.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.903+0000 7f5b597fa700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b4406cc20 0x7f5b4406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:49.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.903+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5b5008bc90 con 0x7f5b5c1027e0 2024-02-22T16:06:49.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.903+0000 7f5b60e65700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b4406cc20 0x7f5b4406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:49.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.904+0000 7f5b60e65700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b4406cc20 0x7f5b4406f0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c004f90 tx=0x7f5b4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:49.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:49.907+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5b50055c40 con 0x7f5b5c1027e0 2024-02-22T16:06:50.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.158+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b5c0738b0 con 0x7f5b5c1027e0 2024-02-22T16:06:50.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.158+0000 7f5b597fa700 1 -- 172.21.15.19:0/1595890730 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f5b50077e60 con 0x7f5b5c1027e0 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:06:50.135 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:06:50.136 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:50.137 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b4406cc20 msgr2=0x7f5b4406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b4406cc20 0x7f5b4406f0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c004f90 tx=0x7f5b4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 msgr2=0x7f5b5c10ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c10ef40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b50002a50 tx=0x7f5b500046b0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 shutdown_connections 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b4406cc20 0x7f5b4406f0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b5c101e40 0x7f5b5c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 --2- 172.21.15.19:0/1595890730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b5c1027e0 0x7f5b5c10ef40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 >> 172.21.15.19:0/1595890730 conn(0x7f5b5c071d90 msgr2=0x7f5b5c0fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:50.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 shutdown_connections 2024-02-22T16:06:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.160+0000 7f5b630c9700 1 -- 172.21.15.19:0/1595890730 wait complete. 2024-02-22T16:06:50.141 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:06:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2011701711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 msgr2=0x7fc8ec0fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2011701711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec0fb9e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0004940 tx=0x7fc8e0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:50.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2011701711 shutdown_connections 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2011701711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec0fb9e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2011701711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8ec0fde50 0x7fc8ec0fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2011701711 >> 172.21.15.19:0/2011701711 conn(0x7fc8ec071560 msgr2=0x7fc8ec071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2011701711 shutdown_connections 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2011701711 wait complete. 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 Processor -- start 2024-02-22T16:06:50.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.265+0000 7fc8f08f9700 1 -- start start 2024-02-22T16:06:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8f08f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8f08f9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8ec0fde50 0x7fc8ec1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:50.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8f08f9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec116710 con 0x7fc8ec0fb570 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8f08f9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec10fa40 con 0x7fc8ec0fde50 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53866/0 (socket says 172.21.15.19:53866) 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 -- 172.21.15.19:0/2504770785 learned_addr learned my addr 172.21.15.19:0/2504770785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8e9d9b700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8ec0fde50 0x7fc8ec1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 -- 172.21.15.19:0/2504770785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8ec0fde50 msgr2=0x7fc8ec1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8ec0fde50 0x7fc8ec1160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 -- 172.21.15.19:0/2504770785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0005040 con 0x7fc8ec0fb570 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8ea59c700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec115b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00eb40 tx=0x7fc8dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8dc007cc0 con 0x7fc8ec0fb570 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8dc00f040 con 0x7fc8ec0fb570 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec10fcc0 con 0x7fc8ec0fb570 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8dc00c990 con 0x7fc8ec0fb570 2024-02-22T16:06:50.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.266+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec1101d0 con 0x7fc8ec0fb570 2024-02-22T16:06:50.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.267+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec107420 con 0x7fc8ec0fb570 2024-02-22T16:06:50.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.268+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc8dc00caf0 con 0x7fc8ec0fb570 2024-02-22T16:06:50.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.268+0000 7fc8daffd700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8d406cc20 0x7fc8d406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:50.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.268+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc8dc08bba0 con 0x7fc8ec0fb570 2024-02-22T16:06:50.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.268+0000 7fc8e9d9b700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8d406cc20 0x7fc8d406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:50.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.269+0000 7fc8e9d9b700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8d406cc20 0x7fc8d406f0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec110d70 tx=0x7fc8e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:50.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.273+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc8dc059710 con 0x7fc8ec0fb570 2024-02-22T16:06:50.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:50 smithi062 ceph-mon[106861]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:50.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:50 smithi062 ceph-mon[106861]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:50.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:50 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1972245578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:06:50.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.496+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8ec04bf40 con 0x7fc8d406cc20 2024-02-22T16:06:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.497+0000 7fc8daffd700 1 -- 172.21.15.19:0/2504770785 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc8ec04bf40 con 0x7fc8d406cc20 2024-02-22T16:06:50.473 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:06:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:06:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:06:50.473 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:06:50.474 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:06:50.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8d406cc20 msgr2=0x7fc8d406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8d406cc20 0x7fc8d406f0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec110d70 tx=0x7fc8e0009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 msgr2=0x7fc8ec115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec115b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00eb40 tx=0x7fc8dc00b040 comp rx=0 tx=0).stop 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 shutdown_connections 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc8d406cc20 0x7fc8d406f0d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8ec0fb570 0x7fc8ec115b80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 --2- 172.21.15.19:0/2504770785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8ec0fde50 0x7fc8ec1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 >> 172.21.15.19:0/2504770785 conn(0x7fc8ec071560 msgr2=0x7fc8ec0f9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 shutdown_connections 2024-02-22T16:06:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.499+0000 7fc8f08f9700 1 -- 172.21.15.19:0/2504770785 wait complete. 2024-02-22T16:06:50.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:50 smithi019 ceph-mon[173579]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:50.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:50 smithi019 ceph-mon[173579]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:50.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:50 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1972245578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 -- 172.21.15.19:0/1933960040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 msgr2=0x7f24e80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 --2- 172.21.15.19:0/1933960040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fbea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f24d4004940 tx=0x7f24d4009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 -- 172.21.15.19:0/1933960040 shutdown_connections 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 --2- 172.21.15.19:0/1933960040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fbea0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 --2- 172.21.15.19:0/1933960040 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e8101e40 0x7f24e8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 -- 172.21.15.19:0/1933960040 >> 172.21.15.19:0/1933960040 conn(0x7f24e8071d90 msgr2=0x7f24e8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 -- 172.21.15.19:0/1933960040 shutdown_connections 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.603+0000 7f24ef11b700 1 -- 172.21.15.19:0/1933960040 wait complete. 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24ef11b700 1 Processor -- start 2024-02-22T16:06:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24ef11b700 1 -- start start 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24ef11b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e8101e40 0x7f24e80fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24ef11b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24ef11b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e80fe600 con 0x7f24e81027e0 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24ef11b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e80fe770 con 0x7f24e8101e40 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24e7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53882/0 (socket says 172.21.15.19:53882) 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24e7fff700 1 -- 172.21.15.19:0/3967008093 learned_addr learned my addr 172.21.15.19:0/3967008093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24eceb7700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e8101e40 0x7f24e80fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24e7fff700 1 -- 172.21.15.19:0/3967008093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e8101e40 msgr2=0x7f24e80fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24e7fff700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e8101e40 0x7f24e80fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.604+0000 7f24e7fff700 1 -- 172.21.15.19:0/3967008093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d4005040 con 0x7f24e81027e0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.605+0000 7f24e7fff700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fe000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f24d4000c00 tx=0x7f24d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.605+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24d4021070 con 0x7f24e81027e0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.605+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24d401c400 con 0x7f24e81027e0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.605+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e8119c20 con 0x7f24e81027e0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.605+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24d400e070 con 0x7f24e81027e0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.605+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e811a020 con 0x7f24e81027e0 2024-02-22T16:06:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.606+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e80738b0 con 0x7f24e81027e0 2024-02-22T16:06:50.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.606+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f24d401c570 con 0x7f24e81027e0 2024-02-22T16:06:50.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.607+0000 7f24e57fa700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f24d806cc20 0x7f24d806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:06:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.607+0000 7f24eceb7700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f24d806cc20 0x7f24d806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:06:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.607+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f24d408d160 con 0x7f24e81027e0 2024-02-22T16:06:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.608+0000 7f24eceb7700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f24d806cc20 0x7f24d806f0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009b80 tx=0x7f24dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:06:50.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.612+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f24d4057190 con 0x7f24e81027e0 2024-02-22T16:06:50.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.895+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f24e8108de0 con 0x7f24e81027e0 2024-02-22T16:06:50.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.896+0000 7f24e57fa700 1 -- 172.21.15.19:0/3967008093 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f24d405a7b0 con 0x7f24e81027e0 2024-02-22T16:06:50.872 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:06:50.872 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:06:50.872 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:06:50.873 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:06:50.873 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f24d806cc20 msgr2=0x7f24d806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f24d806cc20 0x7f24d806f0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009b80 tx=0x7f24dc00b040 comp rx=0 tx=0).stop 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 msgr2=0x7f24e80fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fe000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f24d4000c00 tx=0x7f24d4009e30 comp rx=0 tx=0).stop 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 shutdown_connections 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f24d806cc20 0x7f24d806f0d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24e8101e40 0x7f24e80fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 --2- 172.21.15.19:0/3967008093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24e81027e0 0x7f24e80fe000 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:06:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 >> 172.21.15.19:0/3967008093 conn(0x7f24e8071d90 msgr2=0x7f24e8065840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:06:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 shutdown_connections 2024-02-22T16:06:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:06:50.898+0000 7f24ef11b700 1 -- 172.21.15.19:0/3967008093 wait complete. 2024-02-22T16:06:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:51 smithi019 ceph-mon[173579]: pgmap v407: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:51 smithi019 ceph-mon[173579]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:51 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1595890730' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:06:51.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:51 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3967008093' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:06:51.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:51 smithi062 ceph-mon[106861]: pgmap v407: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:51.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:51 smithi062 ceph-mon[106861]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:51.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:51 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1595890730' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:06:51.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:51 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3967008093' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:06:52.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:52 smithi062 ceph-mon[106861]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:52.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:52 smithi019 ceph-mon[173579]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:06:53.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:53 smithi062 ceph-mon[106861]: pgmap v408: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:53.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:53 smithi019 ceph-mon[173579]: pgmap v408: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:55.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:55 smithi062 ceph-mon[106861]: pgmap v409: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:55.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:55 smithi019 ceph-mon[173579]: pgmap v409: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:57.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:57 smithi062 ceph-mon[106861]: pgmap v410: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:57.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:57 smithi019 ceph-mon[173579]: pgmap v410: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:59.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:06:59 smithi062 ceph-mon[106861]: pgmap v411: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:06:59.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:06:59 smithi019 ceph-mon[173579]: pgmap v411: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:01.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:01 smithi019 ceph-mon[173579]: pgmap v412: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:01 smithi062 ceph-mon[106861]: pgmap v412: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:03.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:03 smithi062 ceph-mon[106861]: pgmap v413: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:03.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:03 smithi019 ceph-mon[173579]: pgmap v413: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:04.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:04.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:05.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:05 smithi062 ceph-mon[106861]: pgmap v414: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:05.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:05 smithi019 ceph-mon[173579]: pgmap v414: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:07.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:07 smithi062 ceph-mon[106861]: pgmap v415: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:07.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:07 smithi019 ceph-mon[173579]: pgmap v415: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:09.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:09 smithi062 ceph-mon[106861]: pgmap v416: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:09.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:09 smithi019 ceph-mon[173579]: pgmap v416: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:11.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:11 smithi019 ceph-mon[173579]: pgmap v417: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:11.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:11 smithi062 ceph-mon[106861]: pgmap v417: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:13.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:13 smithi062 ceph-mon[106861]: pgmap v418: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:13.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:13 smithi019 ceph-mon[173579]: pgmap v418: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:15.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:15 smithi062 ceph-mon[106861]: pgmap v419: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:15.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:15 smithi019 ceph-mon[173579]: pgmap v419: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:17.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:17 smithi062 ceph-mon[106861]: pgmap v420: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:17.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:17 smithi019 ceph-mon[173579]: pgmap v420: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:19.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:19 smithi062 ceph-mon[106861]: pgmap v421: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:19.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:19.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:19 smithi019 ceph-mon[173579]: pgmap v421: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:19.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:20.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 -- 172.21.15.19:0/1587606263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 msgr2=0x7f6c2c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/1587606263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c004940 tx=0x7f6c1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 -- 172.21.15.19:0/1587606263 shutdown_connections 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/1587606263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c1027e0 0x7f6c2c0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/1587606263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102210 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 -- 172.21.15.19:0/1587606263 >> 172.21.15.19:0/1587606263 conn(0x7f6c2c071d90 msgr2=0x7f6c2c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 -- 172.21.15.19:0/1587606263 shutdown_connections 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.017+0000 7f6c30a6b700 1 -- 172.21.15.19:0/1587606263 wait complete. 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c30a6b700 1 Processor -- start 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c30a6b700 1 -- start start 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c30a6b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c30a6b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c1027e0 0x7f6c2c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c30a6b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1037e0 con 0x7f6c2c101e40 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c30a6b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c103950 con 0x7f6c2c1027e0 2024-02-22T16:07:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c2a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c2a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39422/0 (socket says 172.21.15.19:39422) 2024-02-22T16:07:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c2a59c700 1 -- 172.21.15.19:0/2876201580 learned_addr learned my addr 172.21.15.19:0/2876201580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.018+0000 7f6c29d9b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c1027e0 0x7f6c2c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c2a59c700 1 -- 172.21.15.19:0/2876201580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c1027e0 msgr2=0x7f6c2c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c2a59c700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c1027e0 0x7f6c2c1031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:20.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c2a59c700 1 -- 172.21.15.19:0/2876201580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c005040 con 0x7f6c2c101e40 2024-02-22T16:07:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c2a59c700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102ca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c000c00 tx=0x7f6c1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c021070 con 0x7f6c2c101e40 2024-02-22T16:07:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c1c01c400 con 0x7f6c2c101e40 2024-02-22T16:07:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c119b30 con 0x7f6c2c101e40 2024-02-22T16:07:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c00e070 con 0x7f6c2c101e40 2024-02-22T16:07:20.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.019+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c119f30 con 0x7f6c2c101e40 2024-02-22T16:07:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.020+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c0738b0 con 0x7f6c2c101e40 2024-02-22T16:07:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.021+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6c1c01c970 con 0x7f6c2c101e40 2024-02-22T16:07:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.021+0000 7f6c22ffd700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c1806cc20 0x7f6c1806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.021+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6c1c08d160 con 0x7f6c2c101e40 2024-02-22T16:07:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.021+0000 7f6c29d9b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c1806cc20 0x7f6c1806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:20.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.022+0000 7f6c29d9b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c1806cc20 0x7f6c1806f0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c1046a0 tx=0x7f6c1400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:21.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.026+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6c1c057170 con 0x7f6c2c101e40 2024-02-22T16:07:21.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.249+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c2c065df0 con 0x7f6c1806cc20 2024-02-22T16:07:21.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.250+0000 7f6c22ffd700 1 -- 172.21.15.19:0/2876201580 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6c2c065df0 con 0x7f6c1806cc20 2024-02-22T16:07:21.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c1806cc20 msgr2=0x7f6c1806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c1806cc20 0x7f6c1806f0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c1046a0 tx=0x7f6c1400b040 comp rx=0 tx=0).stop 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 msgr2=0x7f6c2c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102ca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c000c00 tx=0x7f6c1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 shutdown_connections 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c1806cc20 0x7f6c1806f0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c2c101e40 0x7f6c2c102ca0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 --2- 172.21.15.19:0/2876201580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c2c1027e0 0x7f6c2c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.252+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 >> 172.21.15.19:0/2876201580 conn(0x7f6c2c071d90 msgr2=0x7f6c2c0656e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.253+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 shutdown_connections 2024-02-22T16:07:21.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.253+0000 7f6c30a6b700 1 -- 172.21.15.19:0/2876201580 wait complete. 2024-02-22T16:07:21.239 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:07:21.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:21 smithi019 ceph-mon[173579]: pgmap v422: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 -- 172.21.15.19:0/2904449132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 msgr2=0x7f1a40102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 --2- 172.21.15.19:0/2904449132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a40102210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a30004940 tx=0x7f1a30009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 -- 172.21.15.19:0/2904449132 shutdown_connections 2024-02-22T16:07:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 --2- 172.21.15.19:0/2904449132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401027e0 0x7f1a400fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 --2- 172.21.15.19:0/2904449132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a40102210 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 -- 172.21.15.19:0/2904449132 >> 172.21.15.19:0/2904449132 conn(0x7f1a40071d90 msgr2=0x7f1a40072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:21.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 -- 172.21.15.19:0/2904449132 shutdown_connections 2024-02-22T16:07:21.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.357+0000 7f1a478e5700 1 -- 172.21.15.19:0/2904449132 wait complete. 2024-02-22T16:07:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a478e5700 1 Processor -- start 2024-02-22T16:07:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a478e5700 1 -- start start 2024-02-22T16:07:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a478e5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a4010eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a478e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401027e0 0x7f1a4010f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:21.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a478e5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4010f6b0 con 0x7f1a40101e40 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a478e5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4010f820 con 0x7f1a401027e0 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a45681700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a4010eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a45681700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a4010eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39444/0 (socket says 172.21.15.19:39444) 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a45681700 1 -- 172.21.15.19:0/3592168805 learned_addr learned my addr 172.21.15.19:0/3592168805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a44e80700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401027e0 0x7f1a4010f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a45681700 1 -- 172.21.15.19:0/3592168805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401027e0 msgr2=0x7f1a4010f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a45681700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401027e0 0x7f1a4010f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.358+0000 7f1a45681700 1 -- 172.21.15.19:0/3592168805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30005040 con 0x7f1a40101e40 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.359+0000 7f1a45681700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a4010eb20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009360 tx=0x7f1a30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.359+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3000e040 con 0x7f1a40101e40 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.359+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a3000bc50 con 0x7f1a40101e40 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.359+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a40117a90 con 0x7f1a40101e40 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.359+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a30013ea0 con 0x7f1a40101e40 2024-02-22T16:07:21.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.359+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a40117fd0 con 0x7f1a40101e40 2024-02-22T16:07:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.360+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a400ff150 con 0x7f1a40101e40 2024-02-22T16:07:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.361+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1a30013420 con 0x7f1a40101e40 2024-02-22T16:07:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.361+0000 7f1a3dffb700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2806cc30 0x7f1a2806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.361+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1a3008ded0 con 0x7f1a40101e40 2024-02-22T16:07:21.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.361+0000 7f1a44e80700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2806cc30 0x7f1a2806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:21.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.362+0000 7f1a44e80700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2806cc30 0x7f1a2806f0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a400661a0 tx=0x7f1a34009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.365+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1a30057ee0 con 0x7f1a40101e40 2024-02-22T16:07:21.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:21 smithi062 ceph-mon[106861]: pgmap v422: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.587+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a40065ee0 con 0x7f1a2806cc30 2024-02-22T16:07:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.589+0000 7f1a3dffb700 1 -- 172.21.15.19:0/3592168805 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1a40065ee0 con 0x7f1a2806cc30 2024-02-22T16:07:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2806cc30 msgr2=0x7f1a2806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2806cc30 0x7f1a2806f0e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a400661a0 tx=0x7f1a34009040 comp rx=0 tx=0).stop 2024-02-22T16:07:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 msgr2=0x7f1a4010eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a4010eb20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009360 tx=0x7f1a30009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 shutdown_connections 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1a2806cc30 0x7f1a2806f0e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40101e40 0x7f1a4010eb20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 --2- 172.21.15.19:0/3592168805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a401027e0 0x7f1a4010f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 >> 172.21.15.19:0/3592168805 conn(0x7f1a40071d90 msgr2=0x7f1a400faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 shutdown_connections 2024-02-22T16:07:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.591+0000 7f1a478e5700 1 -- 172.21.15.19:0/3592168805 wait complete. 2024-02-22T16:07:21.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 -- 172.21.15.19:0/2888855102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 msgr2=0x7fd2980fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/2888855102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980fcf60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd280004940 tx=0x7fd280009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 -- 172.21.15.19:0/2888855102 shutdown_connections 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/2888855102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980fcf60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/2888855102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd298102f50 0x7fd298103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 -- 172.21.15.19:0/2888855102 >> 172.21.15.19:0/2888855102 conn(0x7fd2980f8ae0 msgr2=0x7fd2980faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 -- 172.21.15.19:0/2888855102 shutdown_connections 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.694+0000 7fd29dfcf700 1 -- 172.21.15.19:0/2888855102 wait complete. 2024-02-22T16:07:21.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd29dfcf700 1 Processor -- start 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd29dfcf700 1 -- start start 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd29dfcf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd298102f50 0x7fd298070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd29dfcf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd29dfcf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298070dd0 con 0x7fd2981038f0 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd29dfcf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd298070f40 con 0x7fd298102f50 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd296ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd296ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39464/0 (socket says 172.21.15.19:39464) 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd296ffd700 1 -- 172.21.15.19:0/1690919492 learned_addr learned my addr 172.21.15.19:0/1690919492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd2977fe700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd298102f50 0x7fd298070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd296ffd700 1 -- 172.21.15.19:0/1690919492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd298102f50 msgr2=0x7fd298070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd296ffd700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd298102f50 0x7fd298070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.695+0000 7fd296ffd700 1 -- 172.21.15.19:0/1690919492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280005040 con 0x7fd2981038f0 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.696+0000 7fd296ffd700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980707d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd280009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.696+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd280021070 con 0x7fd2981038f0 2024-02-22T16:07:21.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.696+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298119b60 con 0x7fd2981038f0 2024-02-22T16:07:21.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.696+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd28001c400 con 0x7fd2981038f0 2024-02-22T16:07:21.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.696+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28000e070 con 0x7fd2981038f0 2024-02-22T16:07:21.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.696+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd298119f60 con 0x7fd2981038f0 2024-02-22T16:07:21.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.697+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298105420 con 0x7fd2981038f0 2024-02-22T16:07:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.697+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd28001c570 con 0x7fd2981038f0 2024-02-22T16:07:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.698+0000 7fd28ffff700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd28406cc20 0x7fd28406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.698+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd28008d140 con 0x7fd2981038f0 2024-02-22T16:07:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.698+0000 7fd2977fe700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd28406cc20 0x7fd28406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.698+0000 7fd2977fe700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd28406cc20 0x7fd28406f0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd288009b80 tx=0x7fd28800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:21.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.702+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd280057170 con 0x7fd2981038f0 2024-02-22T16:07:21.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.923+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2980719d0 con 0x7fd28406cc20 2024-02-22T16:07:21.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.934+0000 7fd28ffff700 1 -- 172.21.15.19:0/1690919492 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fd2980719d0 con 0x7fd28406cc20 2024-02-22T16:07:21.910 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (14m) 7m ago 25m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (11m) 7m ago 25m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (11m) 7m ago 24m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (14m) 7m ago 25m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (7m) 7m ago 19m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (8m) 7m ago 19m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:07:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (7m) 7m ago 19m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (7m) 7m ago 19m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (16m) 7m ago 27m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (15m) 7m ago 24m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (12m) 7m ago 27m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (12m) 7m ago 24m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (15m) 7m ago 25m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:07:21.912 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (15m) 7m ago 24m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (11m) 7m ago 23m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (10m) 7m ago 23m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (10m) 7m ago 22m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (9m) 7m ago 22m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (9m) 7m ago 21m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (8m) 7m ago 21m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:07:21.913 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (14m) 7m ago 25m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:07:21.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd28406cc20 msgr2=0x7fd28406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd28406cc20 0x7fd28406f0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd288009b80 tx=0x7fd28800b040 comp rx=0 tx=0).stop 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 msgr2=0x7fd2980707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980707d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd280000c00 tx=0x7fd280009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 shutdown_connections 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd28406cc20 0x7fd28406f0d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd298102f50 0x7fd298070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 --2- 172.21.15.19:0/1690919492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2981038f0 0x7fd2980707d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:21.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 >> 172.21.15.19:0/1690919492 conn(0x7fd2980f8ae0 msgr2=0x7fd2980fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 shutdown_connections 2024-02-22T16:07:21.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:21.936+0000 7fd29dfcf700 1 -- 172.21.15.19:0/1690919492 wait complete. 2024-02-22T16:07:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 -- 172.21.15.19:0/4225578083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fe8f0 msgr2=0x7fdde8107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 --2- 172.21.15.19:0/4225578083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fe8f0 0x7fdde8107050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fddd8004970 tx=0x7fddd8009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 -- 172.21.15.19:0/4225578083 shutdown_connections 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 --2- 172.21.15.19:0/4225578083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fe8f0 0x7fdde8107050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 --2- 172.21.15.19:0/4225578083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 -- 172.21.15.19:0/4225578083 >> 172.21.15.19:0/4225578083 conn(0x7fdde8071d90 msgr2=0x7fdde8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 -- 172.21.15.19:0/4225578083 shutdown_connections 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 -- 172.21.15.19:0/4225578083 wait complete. 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 Processor -- start 2024-02-22T16:07:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.039+0000 7fddedfdc700 1 -- start start 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fddedfdc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fddedfdc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde80fe8f0 0x7fdde80fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fddedfdc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80fc9e0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fddedfdc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80fcb50 con 0x7fdde80fe8f0 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39482/0 (socket says 172.21.15.19:39482) 2024-02-22T16:07:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 -- 172.21.15.19:0/1285633800 learned_addr learned my addr 172.21.15.19:0/1285633800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde6ffd700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde80fe8f0 0x7fdde80fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 -- 172.21.15.19:0/1285633800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde80fe8f0 msgr2=0x7fdde80fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde80fe8f0 0x7fdde80fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 -- 172.21.15.19:0/1285633800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8005040 con 0x7fdde80fdfe0 2024-02-22T16:07:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdde77fe700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fbea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fddd0002a50 tx=0x7fddd00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddd0007cc0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.040+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde8119bb0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.041+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fddd000f040 con 0x7fdde80fdfe0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.041+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddd000d9c0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.041+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde811a060 con 0x7fdde80fdfe0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.042+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde81046c0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.042+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fddd000db20 con 0x7fdde80fdfe0 2024-02-22T16:07:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.043+0000 7fdddffff700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fddd406cc20 0x7fddd406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.043+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fddd008bcc0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.043+0000 7fdde6ffd700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fddd406cc20 0x7fddd406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.043+0000 7fdde6ffd700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fddd406cc20 0x7fddd406f0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fddd8004f50 tx=0x7fddd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.047+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fddd0055cf0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.342+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdde80738b0 con 0x7fdde80fdfe0 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.343+0000 7fdddffff700 1 -- 172.21.15.19:0/1285633800 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fddd0059310 con 0x7fdde80fdfe0 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:07:22.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:07:22.320 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fddd406cc20 msgr2=0x7fddd406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fddd406cc20 0x7fddd406f0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fddd8004f50 tx=0x7fddd8009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 msgr2=0x7fdde80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fbea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fddd0002a50 tx=0x7fddd00046b0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 shutdown_connections 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fddd406cc20 0x7fddd406f0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdde80fdfe0 0x7fdde80fbea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 --2- 172.21.15.19:0/1285633800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdde80fe8f0 0x7fdde80fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 >> 172.21.15.19:0/1285633800 conn(0x7fdde8071d90 msgr2=0x7fdde8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 shutdown_connections 2024-02-22T16:07:22.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.345+0000 7fddedfdc700 1 -- 172.21.15.19:0/1285633800 wait complete. 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 -- 172.21.15.19:0/1510920330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 msgr2=0x7f3b940fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/1510920330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b940fe3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b88004970 tx=0x7f3b88009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 -- 172.21.15.19:0/1510920330 shutdown_connections 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/1510920330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b940fe8f0 0x7f3b94107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/1510920330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b940fe3b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 -- 172.21.15.19:0/1510920330 >> 172.21.15.19:0/1510920330 conn(0x7f3b94071d90 msgr2=0x7f3b94072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 -- 172.21.15.19:0/1510920330 shutdown_connections 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.448+0000 7f3b9adc9700 1 -- 172.21.15.19:0/1510920330 wait complete. 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b9adc9700 1 Processor -- start 2024-02-22T16:07:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b9adc9700 1 -- start start 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b9adc9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b94113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b9adc9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b940fe8f0 0x7f3b94113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b9adc9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941144c0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b9adc9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9410d7f0 con 0x7f3b940fe8f0 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b98b65700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b94113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b98b65700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b94113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39502/0 (socket says 172.21.15.19:39502) 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b98b65700 1 -- 172.21.15.19:0/3661978761 learned_addr learned my addr 172.21.15.19:0/3661978761 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.449+0000 7f3b93fff700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b940fe8f0 0x7f3b94113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b98b65700 1 -- 172.21.15.19:0/3661978761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b940fe8f0 msgr2=0x7f3b94113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b98b65700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b940fe8f0 0x7f3b94113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b98b65700 1 -- 172.21.15.19:0/3661978761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88005040 con 0x7f3b940fdfe0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b98b65700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b94113930 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b88000c00 tx=0x7f3b88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8800e040 con 0x7f3b940fdfe0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9410da70 con 0x7f3b940fdfe0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b88009b10 con 0x7f3b940fdfe0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b88013e60 con 0x7f3b940fdfe0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.450+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9410dfb0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.451+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b941046c0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.452+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3b880136b0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.452+0000 7f3b917fa700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3b8406cbb0 0x7f3b8406f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.452+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3b8808d2f0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.452+0000 7f3b93fff700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3b8406cbb0 0x7f3b8406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.453+0000 7f3b93fff700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3b8406cbb0 0x7f3b8406f060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b9410f160 tx=0x7f3b8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.457+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3b8805adb0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.704+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b9404bed0 con 0x7f3b940fdfe0 2024-02-22T16:07:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.704+0000 7f3b917fa700 1 -- 172.21.15.19:0/3661978761 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f3b8805a940 con 0x7f3b940fdfe0 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:07:22.681 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:07:22.682 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:22.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3b8406cbb0 msgr2=0x7f3b8406f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3b8406cbb0 0x7f3b8406f060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b9410f160 tx=0x7f3b8000b040 comp rx=0 tx=0).stop 2024-02-22T16:07:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 msgr2=0x7f3b94113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b94113930 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b88000c00 tx=0x7f3b88009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 shutdown_connections 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3b8406cbb0 0x7f3b8406f060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b940fdfe0 0x7f3b94113930 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 --2- 172.21.15.19:0/3661978761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b940fe8f0 0x7f3b94113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 >> 172.21.15.19:0/3661978761 conn(0x7f3b94071d90 msgr2=0x7f3b94101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 shutdown_connections 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.706+0000 7f3b9adc9700 1 -- 172.21.15.19:0/3661978761 wait complete. 2024-02-22T16:07:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:07:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 -- 172.21.15.19:0/3963873308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 msgr2=0x7f4d5c0fb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 --2- 172.21.15.19:0/3963873308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c0fb510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4d50004940 tx=0x7f4d50009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 -- 172.21.15.19:0/3963873308 shutdown_connections 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 --2- 172.21.15.19:0/3963873308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c0fb510 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 --2- 172.21.15.19:0/3963873308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5c102f70 0x7f4d5c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 -- 172.21.15.19:0/3963873308 >> 172.21.15.19:0/3963873308 conn(0x7f4d5c071500 msgr2=0x7f4d5c071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 -- 172.21.15.19:0/3963873308 shutdown_connections 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.810+0000 7f4d6293b700 1 -- 172.21.15.19:0/3963873308 wait complete. 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d6293b700 1 Processor -- start 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d6293b700 1 -- start start 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d6293b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5c102f70 0x7f4d5c113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d6293b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d6293b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1144d0 con 0x7f4d5c103910 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d6293b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c10d800 con 0x7f4d5c102f70 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d5b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d5b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39518/0 (socket says 172.21.15.19:39518) 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.811+0000 7f4d5b7fe700 1 -- 172.21.15.19:0/2070965185 learned_addr learned my addr 172.21.15.19:0/2070965185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d5bfff700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5c102f70 0x7f4d5c113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d5b7fe700 1 -- 172.21.15.19:0/2070965185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5c102f70 msgr2=0x7f4d5c113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d5b7fe700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5c102f70 0x7f4d5c113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d5b7fe700 1 -- 172.21.15.19:0/2070965185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50005040 con 0x7f4d5c103910 2024-02-22T16:07:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d5b7fe700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c113e80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009360 tx=0x7f4d50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5000e040 con 0x7f4d5c103910 2024-02-22T16:07:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c10da80 con 0x7f4d5c103910 2024-02-22T16:07:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d5000bc50 con 0x7f4d5c103910 2024-02-22T16:07:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5001c630 con 0x7f4d5c103910 2024-02-22T16:07:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.812+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c10dfc0 con 0x7f4d5c103910 2024-02-22T16:07:22.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.813+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c1002b0 con 0x7f4d5c103910 2024-02-22T16:07:22.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.814+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d50013420 con 0x7f4d5c103910 2024-02-22T16:07:22.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.814+0000 7f4d58ff9700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d4406cc30 0x7f4d4406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:22.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.814+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4d5008d500 con 0x7f4d5c103910 2024-02-22T16:07:22.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.814+0000 7f4d5bfff700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d4406cc30 0x7f4d4406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:22.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.815+0000 7f4d5bfff700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d4406cc30 0x7f4d4406f0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0652a0 tx=0x7f4d4c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:22.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:22.819+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4d5005afc0 con 0x7f4d5c103910 2024-02-22T16:07:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.038+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d5c109f30 con 0x7f4d4406cc30 2024-02-22T16:07:23.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.039+0000 7f4d58ff9700 1 -- 172.21.15.19:0/2070965185 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4d5c109f30 con 0x7f4d4406cc30 2024-02-22T16:07:23.015 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:07:23.015 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:07:23.015 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:07:23.015 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:07:23.016 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:07:23.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d4406cc30 msgr2=0x7f4d4406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:23.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d4406cc30 0x7f4d4406f0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0652a0 tx=0x7f4d4c00b040 comp rx=0 tx=0).stop 2024-02-22T16:07:23.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 msgr2=0x7f4d5c113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:23.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c113e80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009360 tx=0x7f4d50009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:23.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 shutdown_connections 2024-02-22T16:07:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d4406cc30 0x7f4d4406f0e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5c102f70 0x7f4d5c113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 --2- 172.21.15.19:0/2070965185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d5c103910 0x7f4d5c113e80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 >> 172.21.15.19:0/2070965185 conn(0x7f4d5c071500 msgr2=0x7f4d5c0fab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 shutdown_connections 2024-02-22T16:07:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.041+0000 7f4d6293b700 1 -- 172.21.15.19:0/2070965185 wait complete. 2024-02-22T16:07:23.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 -- 172.21.15.19:0/1783964198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 msgr2=0x7fa4a00fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:23.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 --2- 172.21.15.19:0/1783964198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a00fcf60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa49c004940 tx=0x7fa49c009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 -- 172.21.15.19:0/1783964198 shutdown_connections 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 --2- 172.21.15.19:0/1783964198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a00fcf60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 --2- 172.21.15.19:0/1783964198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4a0102f50 0x7fa4a0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 -- 172.21.15.19:0/1783964198 >> 172.21.15.19:0/1783964198 conn(0x7fa4a00f8ae0 msgr2=0x7fa4a00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 -- 172.21.15.19:0/1783964198 shutdown_connections 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.146+0000 7fa4a7983700 1 -- 172.21.15.19:0/1783964198 wait complete. 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a7983700 1 Processor -- start 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a7983700 1 -- start start 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a7983700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4a0102f50 0x7fa4a01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a7983700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a7983700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0114530 con 0x7fa4a01038f0 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a7983700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a010d860 con 0x7fa4a0102f50 2024-02-22T16:07:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a4f1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a4f1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a0113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39530/0 (socket says 172.21.15.19:39530) 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.147+0000 7fa4a4f1e700 1 -- 172.21.15.19:0/900155356 learned_addr learned my addr 172.21.15.19:0/900155356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a571f700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4a0102f50 0x7fa4a01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a4f1e700 1 -- 172.21.15.19:0/900155356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4a0102f50 msgr2=0x7fa4a01139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a4f1e700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4a0102f50 0x7fa4a01139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a4f1e700 1 -- 172.21.15.19:0/900155356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c005040 con 0x7fa4a01038f0 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a4f1e700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a0113ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009180 tx=0x7fa49c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49c00e040 con 0x7fa4a01038f0 2024-02-22T16:07:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa49c004290 con 0x7fa4a01038f0 2024-02-22T16:07:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a010dae0 con 0x7fa4a01038f0 2024-02-22T16:07:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49c01c3f0 con 0x7fa4a01038f0 2024-02-22T16:07:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.148+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a010e020 con 0x7fa4a01038f0 2024-02-22T16:07:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.149+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a0105420 con 0x7fa4a01038f0 2024-02-22T16:07:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.150+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa49c00bc50 con 0x7fa4a01038f0 2024-02-22T16:07:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.150+0000 7fa491ffb700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa48c06cc30 0x7fa48c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.150+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa49c08d440 con 0x7fa4a01038f0 2024-02-22T16:07:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.150+0000 7fa4a571f700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa48c06cc30 0x7fa48c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.151+0000 7fa4a571f700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa48c06cc30 0x7fa48c06f0e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa494004990 tx=0x7fa494009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:23.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.155+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa49c05af50 con 0x7fa4a01038f0 2024-02-22T16:07:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.442+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa4a0109ef0 con 0x7fa4a01038f0 2024-02-22T16:07:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.442+0000 7fa491ffb700 1 -- 172.21.15.19:0/900155356 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa49c05aae0 con 0x7fa4a01038f0 2024-02-22T16:07:23.418 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:07:23.419 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:07:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:07:23.419 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:07:23.419 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa48c06cc30 msgr2=0x7fa48c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa48c06cc30 0x7fa48c06f0e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa494004990 tx=0x7fa494009ee0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 msgr2=0x7fa4a0113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a0113ee0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009180 tx=0x7fa49c009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 shutdown_connections 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa48c06cc30 0x7fa48c06f0e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4a0102f50 0x7fa4a01139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 --2- 172.21.15.19:0/900155356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a01038f0 0x7fa4a0113ee0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 >> 172.21.15.19:0/900155356 conn(0x7fa4a00f8ae0 msgr2=0x7fa4a00fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 shutdown_connections 2024-02-22T16:07:23.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:23.444+0000 7fa4a7983700 1 -- 172.21.15.19:0/900155356 wait complete. 2024-02-22T16:07:23.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:23 smithi062 ceph-mon[106861]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:23 smithi062 ceph-mon[106861]: pgmap v423: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:23 smithi062 ceph-mon[106861]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:23 smithi062 ceph-mon[106861]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:23 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1285633800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:07:23.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:23 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3661978761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:07:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:23 smithi019 ceph-mon[173579]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:23 smithi019 ceph-mon[173579]: pgmap v423: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:23 smithi019 ceph-mon[173579]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:23 smithi019 ceph-mon[173579]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:23 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1285633800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:07:23.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:23 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3661978761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:07:24.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:24 smithi062 ceph-mon[106861]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:24.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:24 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/900155356' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:07:24.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:24 smithi019 ceph-mon[173579]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:24.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:24 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/900155356' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:07:25.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:25 smithi062 ceph-mon[106861]: pgmap v424: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:25.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:25 smithi019 ceph-mon[173579]: pgmap v424: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:26.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:26 smithi062 ceph-mon[106861]: pgmap v425: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:26.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:26 smithi019 ceph-mon[173579]: pgmap v425: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:29.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:29 smithi062 ceph-mon[106861]: pgmap v426: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:29.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:29 smithi019 ceph-mon[173579]: pgmap v426: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:31.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:31 smithi019 ceph-mon[173579]: pgmap v427: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:31.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:31 smithi062 ceph-mon[106861]: pgmap v427: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:33 smithi062 ceph-mon[106861]: pgmap v428: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:33 smithi019 ceph-mon[173579]: pgmap v428: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:34.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:34.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:35.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:35 smithi062 ceph-mon[106861]: pgmap v429: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:35.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:35 smithi019 ceph-mon[173579]: pgmap v429: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:37.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:37 smithi062 ceph-mon[106861]: pgmap v430: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:37.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:37 smithi019 ceph-mon[173579]: pgmap v430: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:39.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:39 smithi062 ceph-mon[106861]: pgmap v431: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:39.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:39 smithi019 ceph-mon[173579]: pgmap v431: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:41.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:41 smithi019 ceph-mon[173579]: pgmap v432: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:41.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:41 smithi062 ceph-mon[106861]: pgmap v432: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:43.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:43 smithi062 ceph-mon[106861]: pgmap v433: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:43.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:43 smithi019 ceph-mon[173579]: pgmap v433: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:45.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:45 smithi062 ceph-mon[106861]: pgmap v434: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:45.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:45 smithi019 ceph-mon[173579]: pgmap v434: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:47.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:47 smithi062 ceph-mon[106861]: pgmap v435: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:47.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:47 smithi019 ceph-mon[173579]: pgmap v435: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:49 smithi062 ceph-mon[106861]: pgmap v436: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:49.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:49.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:49 smithi019 ceph-mon[173579]: pgmap v436: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:49.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:07:51.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:51 smithi019 ceph-mon[173579]: pgmap v437: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:51.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:51 smithi062 ceph-mon[106861]: pgmap v437: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:53.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:53 smithi062 ceph-mon[106861]: pgmap v438: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:53.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:53 smithi019 ceph-mon[173579]: pgmap v438: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 -- 172.21.15.19:0/324714554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 msgr2=0x7fd9f8101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/324714554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f8101a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8004940 tx=0x7fd9e8009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 -- 172.21.15.19:0/324714554 shutdown_connections 2024-02-22T16:07:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/324714554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 0x7fd9f80fbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/324714554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f8101a80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 -- 172.21.15.19:0/324714554 >> 172.21.15.19:0/324714554 conn(0x7fd9f8071560 msgr2=0x7fd9f8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:53.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 -- 172.21.15.19:0/324714554 shutdown_connections 2024-02-22T16:07:53.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 -- 172.21.15.19:0/324714554 wait complete. 2024-02-22T16:07:53.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.562+0000 7fd9fe2db700 1 Processor -- start 2024-02-22T16:07:53.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9fe2db700 1 -- start start 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9fe2db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 0x7fd9f80fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9fe2db700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f80fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9fe2db700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80fddc0 con 0x7fd9f81016b0 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9fe2db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80fdf30 con 0x7fd9f80fb6c0 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f80fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f80fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57240/0 (socket says 172.21.15.19:57240) 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f77fe700 1 -- 172.21.15.19:0/802551855 learned_addr learned my addr 172.21.15.19:0/802551855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f7fff700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 0x7fd9f80fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f77fe700 1 -- 172.21.15.19:0/802551855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 msgr2=0x7fd9f80fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f77fe700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 0x7fd9f80fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f77fe700 1 -- 172.21.15.19:0/802551855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8005040 con 0x7fd9f81016b0 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.563+0000 7fd9f7fff700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 0x7fd9f80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:07:53.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.564+0000 7fd9f77fe700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f80fd7c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00a6e0 tx=0x7fd9ec00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:53.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.564+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9ec00f040 con 0x7fd9f81016b0 2024-02-22T16:07:53.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.564+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f8119bc0 con 0x7fd9f81016b0 2024-02-22T16:07:53.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.564+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9ec00de50 con 0x7fd9f81016b0 2024-02-22T16:07:53.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.564+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9ec0133e0 con 0x7fd9f81016b0 2024-02-22T16:07:53.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.564+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f811a0a0 con 0x7fd9f81016b0 2024-02-22T16:07:53.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.565+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f8105b40 con 0x7fd9f81016b0 2024-02-22T16:07:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.565+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd9ec014480 con 0x7fd9f81016b0 2024-02-22T16:07:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.566+0000 7fd9f4ff9700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e006cc30 0x7fd9e006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.566+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd9ec08bdc0 con 0x7fd9f81016b0 2024-02-22T16:07:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.566+0000 7fd9f7fff700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e006cc30 0x7fd9e006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:53.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.566+0000 7fd9f7fff700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e006cc30 0x7fd9e006f0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800b840 tx=0x7fd9e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:53.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.571+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd9ec055ea0 con 0x7fd9f81016b0 2024-02-22T16:07:53.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.792+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9f80fe9c0 con 0x7fd9e006cc30 2024-02-22T16:07:53.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.793+0000 7fd9f4ff9700 1 -- 172.21.15.19:0/802551855 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd9f80fe9c0 con 0x7fd9e006cc30 2024-02-22T16:07:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e006cc30 msgr2=0x7fd9e006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:53.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e006cc30 0x7fd9e006f0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800b840 tx=0x7fd9e8009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 msgr2=0x7fd9f80fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f80fd7c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00a6e0 tx=0x7fd9ec00c040 comp rx=0 tx=0).stop 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 shutdown_connections 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd9e006cc30 0x7fd9e006f0e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9f80fb6c0 0x7fd9f80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 --2- 172.21.15.19:0/802551855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f81016b0 0x7fd9f80fd7c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.795+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 >> 172.21.15.19:0/802551855 conn(0x7fd9f8071560 msgr2=0x7fd9f81009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.796+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 shutdown_connections 2024-02-22T16:07:53.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.796+0000 7fd9fe2db700 1 -- 172.21.15.19:0/802551855 wait complete. 2024-02-22T16:07:53.783 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:07:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/3306472600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 msgr2=0x7f3ee40fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/3306472600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee40fcf50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4004940 tx=0x7f3ed4009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/3306472600 shutdown_connections 2024-02-22T16:07:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/3306472600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee40fcf50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/3306472600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee4102f40 0x7f3ee4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/3306472600 >> 172.21.15.19:0/3306472600 conn(0x7f3ee40f8af0 msgr2=0x7f3ee40faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/3306472600 shutdown_connections 2024-02-22T16:07:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.900+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/3306472600 wait complete. 2024-02-22T16:07:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3eeb2f7700 1 Processor -- start 2024-02-22T16:07:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3eeb2f7700 1 -- start start 2024-02-22T16:07:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3eeb2f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee4102f40 0x7f3ee4113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3eeb2f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee4113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3eeb2f7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41144b0 con 0x7f3ee41038e0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3eeb2f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee410d7e0 con 0x7f3ee4102f40 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3ee8892700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee4113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3ee8892700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee4113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57270/0 (socket says 172.21.15.19:57270) 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.901+0000 7f3ee8892700 1 -- 172.21.15.19:0/758834961 learned_addr learned my addr 172.21.15.19:0/758834961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ee9093700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee4102f40 0x7f3ee4113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ee8892700 1 -- 172.21.15.19:0/758834961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee4102f40 msgr2=0x7f3ee4113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ee8892700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee4102f40 0x7f3ee4113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ee8892700 1 -- 172.21.15.19:0/758834961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4005040 con 0x7f3ee41038e0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ee8892700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee4113e60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009360 tx=0x7f3ed4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ed400e040 con 0x7f3ee41038e0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee410da60 con 0x7f3ee41038e0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3ed400bc50 con 0x7f3ee41038e0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ed401c630 con 0x7f3ee41038e0 2024-02-22T16:07:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.902+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee410dfa0 con 0x7f3ee41038e0 2024-02-22T16:07:53.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.903+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee4105410 con 0x7f3ee41038e0 2024-02-22T16:07:53.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.904+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3ed4013420 con 0x7f3ee41038e0 2024-02-22T16:07:53.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.904+0000 7f3ed9ffb700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ed006cc30 0x7f3ed006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:53.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.904+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3ed408d500 con 0x7f3ee41038e0 2024-02-22T16:07:53.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.904+0000 7f3ee9093700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ed006cc30 0x7f3ed006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:53.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.905+0000 7f3ee9093700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ed006cc30 0x7f3ed006f0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009fd0 tx=0x7f3ee0009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:53.909+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3ed405afc0 con 0x7f3ee41038e0 2024-02-22T16:07:54.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.129+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ee4109ee0 con 0x7f3ed006cc30 2024-02-22T16:07:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.130+0000 7f3ed9ffb700 1 -- 172.21.15.19:0/758834961 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ee4109ee0 con 0x7f3ed006cc30 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ed006cc30 msgr2=0x7f3ed006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ed006cc30 0x7f3ed006f0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009fd0 tx=0x7f3ee0009f60 comp rx=0 tx=0).stop 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 msgr2=0x7f3ee4113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee4113e60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009360 tx=0x7f3ed4009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 shutdown_connections 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ed006cc30 0x7f3ed006f0e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ee4102f40 0x7f3ee4113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 --2- 172.21.15.19:0/758834961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ee41038e0 0x7f3ee4113e60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 >> 172.21.15.19:0/758834961 conn(0x7f3ee40f8af0 msgr2=0x7f3ee40fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 shutdown_connections 2024-02-22T16:07:54.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.132+0000 7f3eeb2f7700 1 -- 172.21.15.19:0/758834961 wait complete. 2024-02-22T16:07:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 -- 172.21.15.19:0/2681523041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 msgr2=0x7f80f0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 --2- 172.21.15.19:0/2681523041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0102210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80e4004940 tx=0x7f80e4009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 -- 172.21.15.19:0/2681523041 shutdown_connections 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 --2- 172.21.15.19:0/2681523041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f01027e0 0x7f80f00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 --2- 172.21.15.19:0/2681523041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0102210 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 -- 172.21.15.19:0/2681523041 >> 172.21.15.19:0/2681523041 conn(0x7f80f0071d90 msgr2=0x7f80f0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 -- 172.21.15.19:0/2681523041 shutdown_connections 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 -- 172.21.15.19:0/2681523041 wait complete. 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 Processor -- start 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.236+0000 7f80f716f700 1 -- start start 2024-02-22T16:07:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f716f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f716f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f01027e0 0x7f80f0113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f716f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01144a0 con 0x7f80f0101e40 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f716f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f010d7d0 con 0x7f80f01027e0 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57290/0 (socket says 172.21.15.19:57290) 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 -- 172.21.15.19:0/2220778261 learned_addr learned my addr 172.21.15.19:0/2220778261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80effff700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f01027e0 0x7f80f0113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 -- 172.21.15.19:0/2220778261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f01027e0 msgr2=0x7f80f0113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f01027e0 0x7f80f0113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 -- 172.21.15.19:0/2220778261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4005040 con 0x7f80f0101e40 2024-02-22T16:07:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f4f0b700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0113910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80e4008d80 tx=0x7f80e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e400e030 con 0x7f80f0101e40 2024-02-22T16:07:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80e4004290 con 0x7f80f0101e40 2024-02-22T16:07:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.237+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f010da50 con 0x7f80f0101e40 2024-02-22T16:07:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.238+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e401ce60 con 0x7f80f0101e40 2024-02-22T16:07:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.238+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f010df90 con 0x7f80f0101e40 2024-02-22T16:07:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.238+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f00738b0 con 0x7f80f0101e40 2024-02-22T16:07:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.239+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f80e401c690 con 0x7f80f0101e40 2024-02-22T16:07:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.239+0000 7f80ed7fa700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f80e006cbc0 0x7f80e006f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.240+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f80e408da00 con 0x7f80f0101e40 2024-02-22T16:07:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.240+0000 7f80effff700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f80e006cbc0 0x7f80e006f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.240+0000 7f80effff700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f80e006cbc0 0x7f80e006f070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80dc004600 tx=0x7f80dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.244+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f80e405b4c0 con 0x7f80f0101e40 2024-02-22T16:07:54.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.454+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80f00738b0 con 0x7f80e006cbc0 2024-02-22T16:07:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.464+0000 7f80ed7fa700 1 -- 172.21.15.19:0/2220778261 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f80f00738b0 con 0x7f80e006cbc0 2024-02-22T16:07:54.441 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (15m) 8m ago 26m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (12m) 8m ago 26m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (12m) 7m ago 25m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (14m) 8m ago 25m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (8m) 8m ago 20m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (8m) 8m ago 20m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (7m) 7m ago 20m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (8m) 7m ago 20m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (16m) 8m ago 27m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (16m) 7m ago 25m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:07:54.442 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (13m) 8m ago 27m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (13m) 7m ago 25m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (16m) 8m ago 26m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (15m) 7m ago 25m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (12m) 8m ago 24m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (11m) 8m ago 23m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (10m) 8m ago 23m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (10m) 7m ago 22m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (9m) 7m ago 22m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (9m) 7m ago 22m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:07:54.443 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (15m) 8m ago 25m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:07:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.466+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f80e006cbc0 msgr2=0x7f80e006f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.466+0000 7f80f716f700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f80e006cbc0 0x7f80e006f070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80dc004600 tx=0x7f80dc00b040 comp rx=0 tx=0).stop 2024-02-22T16:07:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.466+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 msgr2=0x7f80f0113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.466+0000 7f80f716f700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0113910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80e4008d80 tx=0x7f80e4009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 shutdown_connections 2024-02-22T16:07:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f80e006cbc0 0x7f80e006f070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80f0101e40 0x7f80f0113910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 --2- 172.21.15.19:0/2220778261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f80f01027e0 0x7f80f0113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 >> 172.21.15.19:0/2220778261 conn(0x7f80f0071d90 msgr2=0x7f80f00656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 shutdown_connections 2024-02-22T16:07:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.467+0000 7f80f716f700 1 -- 172.21.15.19:0/2220778261 wait complete. 2024-02-22T16:07:54.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 -- 172.21.15.19:0/1119316055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 msgr2=0x7fe9c40fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 --2- 172.21.15.19:0/1119316055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c40fcf60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0004940 tx=0x7fe9c0009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 -- 172.21.15.19:0/1119316055 shutdown_connections 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 --2- 172.21.15.19:0/1119316055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c40fcf60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 --2- 172.21.15.19:0/1119316055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c4102f50 0x7fe9c4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 -- 172.21.15.19:0/1119316055 >> 172.21.15.19:0/1119316055 conn(0x7fe9c40f8ae0 msgr2=0x7fe9c40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 -- 172.21.15.19:0/1119316055 shutdown_connections 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 -- 172.21.15.19:0/1119316055 wait complete. 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.569+0000 7fe9cb353700 1 Processor -- start 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9cb353700 1 -- start start 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9cb353700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c4102f50 0x7fe9c41139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9cb353700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c4113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9cb353700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4114530 con 0x7fe9c41038f0 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9cb353700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c410d860 con 0x7fe9c4102f50 2024-02-22T16:07:54.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c4113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c4113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57306/0 (socket says 172.21.15.19:57306) 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 -- 172.21.15.19:0/3384220852 learned_addr learned my addr 172.21.15.19:0/3384220852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c90ef700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c4102f50 0x7fe9c41139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 -- 172.21.15.19:0/3384220852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c4102f50 msgr2=0x7fe9c41139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c4102f50 0x7fe9c41139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 -- 172.21.15.19:0/3384220852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0005040 con 0x7fe9c41038f0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9c88ee700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c4113ee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009fd0 tx=0x7fe9c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.570+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9c000e040 con 0x7fe9c41038f0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.571+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9c0013420 con 0x7fe9c41038f0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.571+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c410dae0 con 0x7fe9c41038f0 2024-02-22T16:07:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.571+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9c001c630 con 0x7fe9c41038f0 2024-02-22T16:07:54.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.571+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c410e020 con 0x7fe9c41038f0 2024-02-22T16:07:54.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.572+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c4105420 con 0x7fe9c41038f0 2024-02-22T16:07:54.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.572+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe9c0013a00 con 0x7fe9c41038f0 2024-02-22T16:07:54.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.573+0000 7fe9b5ffb700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe9b006cc20 0x7fe9b006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.573+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe9c008d250 con 0x7fe9c41038f0 2024-02-22T16:07:54.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.573+0000 7fe9c90ef700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe9b006cc20 0x7fe9b006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.573+0000 7fe9c90ef700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe9b006cc20 0x7fe9b006f0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8004f40 tx=0x7fe9b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:54.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.577+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe9c005acf0 con 0x7fe9c41038f0 2024-02-22T16:07:54.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.864+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe9c410efb0 con 0x7fe9c41038f0 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.865+0000 7fe9b5ffb700 1 -- 172.21.15.19:0/3384220852 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fe9c005a880 con 0x7fe9c41038f0 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:07:54.842 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:07:54.843 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe9b006cc20 msgr2=0x7fe9b006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe9b006cc20 0x7fe9b006f0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8004f40 tx=0x7fe9b800b040 comp rx=0 tx=0).stop 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 msgr2=0x7fe9c4113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c4113ee0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009fd0 tx=0x7fe9c0009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 shutdown_connections 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe9b006cc20 0x7fe9b006f0d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9c4102f50 0x7fe9c41139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 --2- 172.21.15.19:0/3384220852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9c41038f0 0x7fe9c4113ee0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 >> 172.21.15.19:0/3384220852 conn(0x7fe9c40f8ae0 msgr2=0x7fe9c40fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:54.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 shutdown_connections 2024-02-22T16:07:54.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.867+0000 7fe9cb353700 1 -- 172.21.15.19:0/3384220852 wait complete. 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 -- 172.21.15.19:0/244745731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 msgr2=0x7f08580fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 --2- 172.21.15.19:0/244745731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fbe50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0844004940 tx=0x7f0844009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 -- 172.21.15.19:0/244745731 shutdown_connections 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 --2- 172.21.15.19:0/244745731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fbe50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 --2- 172.21.15.19:0/244745731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 0x7f0858102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 -- 172.21.15.19:0/244745731 >> 172.21.15.19:0/244745731 conn(0x7f08580f79d0 msgr2=0x7f08580f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 -- 172.21.15.19:0/244745731 shutdown_connections 2024-02-22T16:07:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 -- 172.21.15.19:0/244745731 wait complete. 2024-02-22T16:07:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 Processor -- start 2024-02-22T16:07:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.971+0000 7f085f01a700 1 -- start start 2024-02-22T16:07:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f085f01a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 0x7f08580fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f085f01a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f085f01a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580fe600 con 0x7f08581027e0 2024-02-22T16:07:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f085f01a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580fe770 con 0x7f0858101e40 2024-02-22T16:07:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57322/0 (socket says 172.21.15.19:57322) 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 -- 172.21.15.19:0/3042776643 learned_addr learned my addr 172.21.15.19:0/3042776643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f085cdb6700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 0x7f08580fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 -- 172.21.15.19:0/3042776643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 msgr2=0x7f08580fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 0x7f08580fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 -- 172.21.15.19:0/3042776643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844005040 con 0x7f08581027e0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f085cdb6700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 0x7f08580fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f0857fff700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fe000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0844009fd0 tx=0x7f0844009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.972+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084400e040 con 0x7f08581027e0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.973+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0858119bc0 con 0x7f08581027e0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.973+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f084400bc50 con 0x7f08581027e0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.973+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084401c3f0 con 0x7f08581027e0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.973+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085811a020 con 0x7f08581027e0 2024-02-22T16:07:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.974+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08580fb5d0 con 0x7f08581027e0 2024-02-22T16:07:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.974+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0844013420 con 0x7f08581027e0 2024-02-22T16:07:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.974+0000 7f08557fa700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084806cc20 0x7f084806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.974+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f084408d460 con 0x7f08581027e0 2024-02-22T16:07:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.975+0000 7f085cdb6700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084806cc20 0x7f084806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.975+0000 7f085cdb6700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084806cc20 0x7f084806f0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f084c009fd0 tx=0x7f084c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:54.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:54.979+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0844057490 con 0x7f08581027e0 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.223+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0858108de0 con 0x7f08581027e0 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.224+0000 7f08557fa700 1 -- 172.21.15.19:0/3042776643 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f084405aab0 con 0x7f08581027e0 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:07:55.201 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:07:55.202 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:07:55.203 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:07:55.204 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:55.204 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:55.204 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:55.204 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:07:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084806cc20 msgr2=0x7f084806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084806cc20 0x7f084806f0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f084c009fd0 tx=0x7f084c009ee0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 msgr2=0x7f08580fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fe000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0844009fd0 tx=0x7f0844009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 shutdown_connections 2024-02-22T16:07:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084806cc20 0x7f084806f0d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0858101e40 0x7f08580fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 --2- 172.21.15.19:0/3042776643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581027e0 0x7f08580fe000 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 >> 172.21.15.19:0/3042776643 conn(0x7f08580f79d0 msgr2=0x7f0858065fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 shutdown_connections 2024-02-22T16:07:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.226+0000 7f085f01a700 1 -- 172.21.15.19:0/3042776643 wait complete. 2024-02-22T16:07:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:07:55.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 -- 172.21.15.19:0/2503394432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 msgr2=0x7fd594103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 --2- 172.21.15.19:0/2503394432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd594103510 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd588004940 tx=0x7fd588009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 -- 172.21.15.19:0/2503394432 shutdown_connections 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 --2- 172.21.15.19:0/2503394432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5940fb570 0x7fd5940fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 --2- 172.21.15.19:0/2503394432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd594103510 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 -- 172.21.15.19:0/2503394432 >> 172.21.15.19:0/2503394432 conn(0x7fd594071560 msgr2=0x7fd594071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 -- 172.21.15.19:0/2503394432 shutdown_connections 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 -- 172.21.15.19:0/2503394432 wait complete. 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.331+0000 7fd5992aa700 1 Processor -- start 2024-02-22T16:07:55.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5992aa700 1 -- start start 2024-02-22T16:07:55.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5992aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5940fb570 0x7fd5940fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5992aa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd5940fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5992aa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940fd290 con 0x7fd594103140 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5992aa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940fd400 con 0x7fd5940fb570 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd5940fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd5940fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57344/0 (socket says 172.21.15.19:57344) 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 -- 172.21.15.19:0/227195693 learned_addr learned my addr 172.21.15.19:0/227195693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd592ffd700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5940fb570 0x7fd5940fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 -- 172.21.15.19:0/227195693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5940fb570 msgr2=0x7fd5940fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5940fb570 0x7fd5940fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 -- 172.21.15.19:0/227195693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588005040 con 0x7fd594103140 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.332+0000 7fd5927fc700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd5940fcc90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd58400eb40 tx=0x7fd58400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.333+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd584007cc0 con 0x7fd594103140 2024-02-22T16:07:55.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.333+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd58400f040 con 0x7fd594103140 2024-02-22T16:07:55.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.333+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594119c80 con 0x7fd594103140 2024-02-22T16:07:55.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.333+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58400c990 con 0x7fd594103140 2024-02-22T16:07:55.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.333+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59411a0e0 con 0x7fd594103140 2024-02-22T16:07:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.334+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd594100310 con 0x7fd594103140 2024-02-22T16:07:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.334+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd58400cc00 con 0x7fd594103140 2024-02-22T16:07:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.334+0000 7fd57b7fe700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd57c06cc20 0x7fd57c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.335+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd58408bc50 con 0x7fd594103140 2024-02-22T16:07:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.335+0000 7fd592ffd700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd57c06cc20 0x7fd57c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:55.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.335+0000 7fd592ffd700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd57c06cc20 0x7fd57c06f0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd588000c00 tx=0x7fd588009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:55.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.339+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd584055cb0 con 0x7fd594103140 2024-02-22T16:07:55.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:55 smithi062 ceph-mon[106861]: pgmap v439: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:55.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:55 smithi062 ceph-mon[106861]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:55.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:55 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3384220852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:07:55.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:55 smithi019 ceph-mon[173579]: pgmap v439: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:55.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:55 smithi019 ceph-mon[173579]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:55.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:55 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3384220852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:07:55.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.559+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd59404bf40 con 0x7fd57c06cc20 2024-02-22T16:07:55.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.560+0000 7fd57b7fe700 1 -- 172.21.15.19:0/227195693 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd59404bf40 con 0x7fd57c06cc20 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:07:55.537 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:07:55.538 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:07:55.538 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:07:55.538 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:07:55.538 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:07:55.538 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd57c06cc20 msgr2=0x7fd57c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd57c06cc20 0x7fd57c06f0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd588000c00 tx=0x7fd588009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 msgr2=0x7fd5940fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd5940fcc90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd58400eb40 tx=0x7fd58400b040 comp rx=0 tx=0).stop 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 shutdown_connections 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd57c06cc20 0x7fd57c06f0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5940fb570 0x7fd5940fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 --2- 172.21.15.19:0/227195693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594103140 0x7fd5940fcc90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 >> 172.21.15.19:0/227195693 conn(0x7fd594071560 msgr2=0x7fd5940fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 shutdown_connections 2024-02-22T16:07:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.562+0000 7fd5992aa700 1 -- 172.21.15.19:0/227195693 wait complete. 2024-02-22T16:07:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.665+0000 7f13e8640700 1 -- 172.21.15.19:0/662602729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 msgr2=0x7f13e0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 --2- 172.21.15.19:0/662602729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e0103320 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f13dc004940 tx=0x7f13dc009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 -- 172.21.15.19:0/662602729 shutdown_connections 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 --2- 172.21.15.19:0/662602729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01038f0 0x7f13e00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 --2- 172.21.15.19:0/662602729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e0103320 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 -- 172.21.15.19:0/662602729 >> 172.21.15.19:0/662602729 conn(0x7f13e00f8ae0 msgr2=0x7f13e00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 -- 172.21.15.19:0/662602729 shutdown_connections 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 -- 172.21.15.19:0/662602729 wait complete. 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 Processor -- start 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.666+0000 7f13e8640700 1 -- start start 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e8640700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e8640700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01038f0 0x7f13e0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e8640700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0114530 con 0x7f13e0102f50 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e8640700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e010d860 con 0x7f13e01038f0 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e01139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57370/0 (socket says 172.21.15.19:57370) 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 -- 172.21.15.19:0/1525752985 learned_addr learned my addr 172.21.15.19:0/1525752985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e5bdb700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01038f0 0x7f13e0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 -- 172.21.15.19:0/1525752985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01038f0 msgr2=0x7f13e0113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01038f0 0x7f13e0113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 -- 172.21.15.19:0/1525752985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc005040 con 0x7f13e0102f50 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e63dc700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e01139a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000c00 tx=0x7f13dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13dc021070 con 0x7f13e0102f50 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e010dae0 con 0x7f13e0102f50 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13dc01c400 con 0x7f13e0102f50 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13dc00e070 con 0x7f13e0102f50 2024-02-22T16:07:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.667+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e010dfc0 con 0x7f13e0102f50 2024-02-22T16:07:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.669+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e0100260 con 0x7f13e0102f50 2024-02-22T16:07:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.669+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f13dc01c970 con 0x7f13e0102f50 2024-02-22T16:07:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.669+0000 7f13d6ffd700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f13cc06cc20 0x7f13cc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:07:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.669+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f13dc08cd80 con 0x7f13e0102f50 2024-02-22T16:07:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.669+0000 7f13e5bdb700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f13cc06cc20 0x7f13cc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:07:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.670+0000 7f13e5bdb700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f13cc06cc20 0x7f13cc06f0d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13e010f1d0 tx=0x7f13d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:07:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.674+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f13dc05a900 con 0x7f13e0102f50 2024-02-22T16:07:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.960+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f13e00fc6b0 con 0x7f13e0102f50 2024-02-22T16:07:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.960+0000 7f13d6ffd700 1 -- 172.21.15.19:0/1525752985 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f13dc05a490 con 0x7f13e0102f50 2024-02-22T16:07:55.937 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:07:55.937 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:07:55.938 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:07:55.938 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:07:55.938 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:07:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f13cc06cc20 msgr2=0x7f13cc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f13cc06cc20 0x7f13cc06f0d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13e010f1d0 tx=0x7f13d000b040 comp rx=0 tx=0).stop 2024-02-22T16:07:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 msgr2=0x7f13e01139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:07:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e01139a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000c00 tx=0x7f13dc009e30 comp rx=0 tx=0).stop 2024-02-22T16:07:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 shutdown_connections 2024-02-22T16:07:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f13cc06cc20 0x7f13cc06f0d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e0102f50 0x7f13e01139a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 --2- 172.21.15.19:0/1525752985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13e01038f0 0x7f13e0113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:07:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.962+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 >> 172.21.15.19:0/1525752985 conn(0x7f13e00f8ae0 msgr2=0x7f13e00fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:07:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.963+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 shutdown_connections 2024-02-22T16:07:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:07:55.963+0000 7f13e8640700 1 -- 172.21.15.19:0/1525752985 wait complete. 2024-02-22T16:07:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:56 smithi062 ceph-mon[106861]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:56 smithi062 ceph-mon[106861]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:56 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3042776643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:07:56.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:56 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1525752985' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:07:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:56 smithi019 ceph-mon[173579]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:56 smithi019 ceph-mon[173579]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:56 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3042776643' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:07:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:56 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1525752985' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:07:57.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:57 smithi062 ceph-mon[106861]: pgmap v440: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:57.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:57 smithi062 ceph-mon[106861]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:57.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:57 smithi019 ceph-mon[173579]: pgmap v440: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:57.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:57 smithi019 ceph-mon[173579]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:07:59.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:07:59 smithi062 ceph-mon[106861]: pgmap v441: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:07:59.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:07:59 smithi019 ceph-mon[173579]: pgmap v441: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:01 smithi019 ceph-mon[173579]: pgmap v442: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:01 smithi062 ceph-mon[106861]: pgmap v442: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:02.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:02 smithi062 ceph-mon[106861]: pgmap v443: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:02.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:02 smithi019 ceph-mon[173579]: pgmap v443: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:04.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:04.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:05.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:05 smithi062 ceph-mon[106861]: pgmap v444: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:05.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:05 smithi019 ceph-mon[173579]: pgmap v444: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:07.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:07 smithi062 ceph-mon[106861]: pgmap v445: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:07.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:07 smithi019 ceph-mon[173579]: pgmap v445: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:09.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:09 smithi019 ceph-mon[173579]: pgmap v446: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:09.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:09 smithi062 ceph-mon[106861]: pgmap v446: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:11.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:11 smithi019 ceph-mon[173579]: pgmap v447: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:11.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:11 smithi062 ceph-mon[106861]: pgmap v447: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:13.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:13 smithi062 ceph-mon[106861]: pgmap v448: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:13.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:13 smithi019 ceph-mon[173579]: pgmap v448: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:15.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:15 smithi062 ceph-mon[106861]: pgmap v449: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:15.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:15 smithi019 ceph-mon[173579]: pgmap v449: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:17.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:17 smithi062 ceph-mon[106861]: pgmap v450: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:17.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:17 smithi019 ceph-mon[173579]: pgmap v450: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:19.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:19 smithi062 ceph-mon[106861]: pgmap v451: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:19.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:19.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:19 smithi019 ceph-mon[173579]: pgmap v451: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:19.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:21.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:21 smithi019 ceph-mon[173579]: pgmap v452: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:21.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:21 smithi062 ceph-mon[106861]: pgmap v452: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:23.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:23 smithi062 ceph-mon[106861]: pgmap v453: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:23.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:23 smithi019 ceph-mon[173579]: pgmap v453: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:25.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:25 smithi062 ceph-mon[106861]: pgmap v454: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:25.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:25 smithi019 ceph-mon[173579]: pgmap v454: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.082+0000 7f03cbd2d700 1 -- 172.21.15.19:0/1375638604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c4101e40 msgr2=0x7f03c4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.082+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/1375638604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c4101e40 0x7f03c4102210 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ce0 tx=0x7f03bc004d10 comp rx=0 tx=0).stop 2024-02-22T16:08:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.082+0000 7f03cbd2d700 1 -- 172.21.15.19:0/1375638604 shutdown_connections 2024-02-22T16:08:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.082+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/1375638604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c41027e0 0x7f03c40fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.082+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/1375638604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c4101e40 0x7f03c4102210 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.082+0000 7f03cbd2d700 1 -- 172.21.15.19:0/1375638604 >> 172.21.15.19:0/1375638604 conn(0x7f03c4071d90 msgr2=0x7f03c4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 -- 172.21.15.19:0/1375638604 shutdown_connections 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 -- 172.21.15.19:0/1375638604 wait complete. 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 Processor -- start 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 -- start start 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c4101e40 0x7f03c4113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 0x7f03c41140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c41146f0 con 0x7f03c41027e0 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03cbd2d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c410da20 con 0x7f03c4101e40 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.083+0000 7f03c92c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 0x7f03c41140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c92c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 0x7f03c41140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55234/0 (socket says 172.21.15.19:55234) 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c92c8700 1 -- 172.21.15.19:0/3863103668 learned_addr learned my addr 172.21.15.19:0/3863103668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c9ac9700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c4101e40 0x7f03c4113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c92c8700 1 -- 172.21.15.19:0/3863103668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c4101e40 msgr2=0x7f03c4113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c92c8700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c4101e40 0x7f03c4113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c92c8700 1 -- 172.21.15.19:0/3863103668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc009940 con 0x7f03c41027e0 2024-02-22T16:08:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c92c8700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 0x7f03c41140a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f03b8002a50 tx=0x7f03b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b8007cc0 con 0x7f03c41027e0 2024-02-22T16:08:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03b800f040 con 0x7f03c41027e0 2024-02-22T16:08:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c410dd00 con 0x7f03c41027e0 2024-02-22T16:08:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b800d920 con 0x7f03c41027e0 2024-02-22T16:08:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.084+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c410e240 con 0x7f03c41027e0 2024-02-22T16:08:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.085+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c40738b0 con 0x7f03c41027e0 2024-02-22T16:08:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.086+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f03b8009680 con 0x7f03c41027e0 2024-02-22T16:08:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.086+0000 7f03c27fc700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03ac06cc30 0x7f03ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.086+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f03b808bcb0 con 0x7f03c41027e0 2024-02-22T16:08:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.086+0000 7f03c9ac9700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03ac06cc30 0x7f03ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.087+0000 7f03c9ac9700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03ac06cc30 0x7f03ac06f0e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005010 tx=0x7f03bc004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.091+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f03b8059770 con 0x7f03c41027e0 2024-02-22T16:08:26.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.314+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03c404bf40 con 0x7f03ac06cc30 2024-02-22T16:08:26.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.315+0000 7f03c27fc700 1 -- 172.21.15.19:0/3863103668 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f03c404bf40 con 0x7f03ac06cc30 2024-02-22T16:08:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03ac06cc30 msgr2=0x7f03ac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03ac06cc30 0x7f03ac06f0e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005010 tx=0x7f03bc004d10 comp rx=0 tx=0).stop 2024-02-22T16:08:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 msgr2=0x7f03c41140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 0x7f03c41140a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f03b8002a50 tx=0x7f03b8004600 comp rx=0 tx=0).stop 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 shutdown_connections 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f03ac06cc30 0x7f03ac06f0e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03c4101e40 0x7f03c4113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 --2- 172.21.15.19:0/3863103668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03c41027e0 0x7f03c41140a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 >> 172.21.15.19:0/3863103668 conn(0x7f03c4071d90 msgr2=0x7f03c4065800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 shutdown_connections 2024-02-22T16:08:26.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.317+0000 7f03cbd2d700 1 -- 172.21.15.19:0/3863103668 wait complete. 2024-02-22T16:08:26.303 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:08:26.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 -- 172.21.15.19:0/3012867342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe78101e40 msgr2=0x7fbe78102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 --2- 172.21.15.19:0/3012867342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe78101e40 0x7fbe78102210 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbe68004940 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 -- 172.21.15.19:0/3012867342 shutdown_connections 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 --2- 172.21.15.19:0/3012867342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe781027e0 0x7fbe780fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 --2- 172.21.15.19:0/3012867342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe78101e40 0x7fbe78102210 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 -- 172.21.15.19:0/3012867342 >> 172.21.15.19:0/3012867342 conn(0x7fbe78071d90 msgr2=0x7fbe78072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 -- 172.21.15.19:0/3012867342 shutdown_connections 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.422+0000 7fbe7df42700 1 -- 172.21.15.19:0/3012867342 wait complete. 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe7df42700 1 Processor -- start 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe7df42700 1 -- start start 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe7df42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe78101e40 0x7fbe78113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe7df42700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 0x7fbe78113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe7df42700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781144a0 con 0x7fbe781027e0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe7df42700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7810d7d0 con 0x7fbe78101e40 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 0x7fbe78113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe76ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 0x7fbe78113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55248/0 (socket says 172.21.15.19:55248) 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe76ffd700 1 -- 172.21.15.19:0/2896361732 learned_addr learned my addr 172.21.15.19:0/2896361732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe777fe700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe78101e40 0x7fbe78113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe76ffd700 1 -- 172.21.15.19:0/2896361732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe78101e40 msgr2=0x7fbe78113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe76ffd700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe78101e40 0x7fbe78113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.423+0000 7fbe76ffd700 1 -- 172.21.15.19:0/2896361732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68005040 con 0x7fbe781027e0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.424+0000 7fbe76ffd700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 0x7fbe78113e50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000eb40 tx=0x7fbe6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.424+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe60007cc0 con 0x7fbe781027e0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.424+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe6000f040 con 0x7fbe781027e0 2024-02-22T16:08:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.424+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6000c8f0 con 0x7fbe781027e0 2024-02-22T16:08:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.424+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7810da50 con 0x7fbe781027e0 2024-02-22T16:08:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.424+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7810df90 con 0x7fbe781027e0 2024-02-22T16:08:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.425+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe780738b0 con 0x7fbe781027e0 2024-02-22T16:08:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.425+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbe6000cb60 con 0x7fbe781027e0 2024-02-22T16:08:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.426+0000 7fbe6ffff700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe6406cc20 0x7fbe6406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.426+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe6008bb90 con 0x7fbe781027e0 2024-02-22T16:08:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.426+0000 7fbe777fe700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe6406cc20 0x7fbe6406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.426+0000 7fbe777fe700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe6406cc20 0x7fbe6406f0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c00 tx=0x7fbe68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.430+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fbe600595e0 con 0x7fbe781027e0 2024-02-22T16:08:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.652+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe7804bed0 con 0x7fbe6406cc20 2024-02-22T16:08:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.653+0000 7fbe6ffff700 1 -- 172.21.15.19:0/2896361732 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbe7804bed0 con 0x7fbe6406cc20 2024-02-22T16:08:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe6406cc20 msgr2=0x7fbe6406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe6406cc20 0x7fbe6406f0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c00 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:26.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 msgr2=0x7fbe78113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 0x7fbe78113e50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000eb40 tx=0x7fbe6000b040 comp rx=0 tx=0).stop 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 shutdown_connections 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbe6406cc20 0x7fbe6406f0d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe78101e40 0x7fbe78113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 --2- 172.21.15.19:0/2896361732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe781027e0 0x7fbe78113e50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.655+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 >> 172.21.15.19:0/2896361732 conn(0x7fbe78071d90 msgr2=0x7fbe78065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.656+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 shutdown_connections 2024-02-22T16:08:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.656+0000 7fbe7df42700 1 -- 172.21.15.19:0/2896361732 wait complete. 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 -- 172.21.15.19:0/1785649254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 msgr2=0x7f6238103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 --2- 172.21.15.19:0/1785649254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f6238103310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6228004940 tx=0x7f6228009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 -- 172.21.15.19:0/1785649254 shutdown_connections 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 --2- 172.21.15.19:0/1785649254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 0x7f62380fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 --2- 172.21.15.19:0/1785649254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f6238103310 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 -- 172.21.15.19:0/1785649254 >> 172.21.15.19:0/1785649254 conn(0x7f62380f8af0 msgr2=0x7f62380faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 -- 172.21.15.19:0/1785649254 shutdown_connections 2024-02-22T16:08:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 -- 172.21.15.19:0/1785649254 wait complete. 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 Processor -- start 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.759+0000 7f623f737700 1 -- start start 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623f737700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f62381139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623f737700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 0x7f6238113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623f737700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238114560 con 0x7f6238102f40 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623f737700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623810d890 con 0x7f62381038e0 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f62381139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f62381139d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55260/0 (socket says 172.21.15.19:55260) 2024-02-22T16:08:26.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 -- 172.21.15.19:0/2829344530 learned_addr learned my addr 172.21.15.19:0/2829344530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623ccd2700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 0x7f6238113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 -- 172.21.15.19:0/2829344530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 msgr2=0x7f6238113f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 0x7f6238113f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 -- 172.21.15.19:0/2829344530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228005040 con 0x7f6238102f40 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623ccd2700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 0x7f6238113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623d4d3700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f62381139d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f622800b3b0 tx=0x7f6228009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f622800e040 con 0x7f6238102f40 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f622800bc50 con 0x7f6238102f40 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623810db10 con 0x7f6238102f40 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.760+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6228013420 con 0x7f6238102f40 2024-02-22T16:08:26.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.761+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623810e050 con 0x7f6238102f40 2024-02-22T16:08:26.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.762+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6238100250 con 0x7f6238102f40 2024-02-22T16:08:26.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.762+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f622801c430 con 0x7f6238102f40 2024-02-22T16:08:26.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.763+0000 7f622dffb700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f622406cc20 0x7f622406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:26.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.763+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f622808cd40 con 0x7f6238102f40 2024-02-22T16:08:26.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.763+0000 7f623ccd2700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f622406cc20 0x7f622406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:26.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.763+0000 7f623ccd2700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f622406cc20 0x7f622406f0d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6234007fd0 tx=0x7f6234009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.767+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f622805a950 con 0x7f6238102f40 2024-02-22T16:08:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.978+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f623804bf40 con 0x7f622406cc20 2024-02-22T16:08:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.988+0000 7f622dffb700 1 -- 172.21.15.19:0/2829344530 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f623804bf40 con 0x7f622406cc20 2024-02-22T16:08:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:08:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (15m) 8m ago 26m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:08:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (13m) 8m ago 26m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:08:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (12m) 8m ago 25m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (15m) 8m ago 26m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (9m) 8m ago 20m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (9m) 8m ago 21m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (8m) 8m ago 20m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (8m) 8m ago 20m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (17m) 8m ago 28m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (16m) 8m ago 25m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (14m) 8m ago 28m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (13m) 8m ago 25m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (16m) 8m ago 26m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (16m) 8m ago 25m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (12m) 8m ago 24m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:08:26.965 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (11m) 8m ago 24m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:08:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (11m) 8m ago 23m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:08:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (10m) 8m ago 23m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:08:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (10m) 8m ago 23m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:08:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (9m) 8m ago 22m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:08:26.966 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (16m) 8m ago 26m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f622406cc20 msgr2=0x7f622406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f622406cc20 0x7f622406f0d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6234007fd0 tx=0x7f6234009040 comp rx=0 tx=0).stop 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 msgr2=0x7f62381139d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f62381139d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f622800b3b0 tx=0x7f6228009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 shutdown_connections 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f622406cc20 0x7f622406f0d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6238102f40 0x7f62381139d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 --2- 172.21.15.19:0/2829344530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381038e0 0x7f6238113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.990+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 >> 172.21.15.19:0/2829344530 conn(0x7f62380f8af0 msgr2=0x7f62380fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.991+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 shutdown_connections 2024-02-22T16:08:26.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:26.991+0000 7f623f737700 1 -- 172.21.15.19:0/2829344530 wait complete. 2024-02-22T16:08:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.093+0000 7f35c31fd700 1 -- 172.21.15.19:0/3978794900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc1038f0 msgr2=0x7f35bc0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.093+0000 7f35c31fd700 1 --2- 172.21.15.19:0/3978794900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc1038f0 0x7f35bc0fcf60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f35a8004940 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.093+0000 7f35c31fd700 1 -- 172.21.15.19:0/3978794900 shutdown_connections 2024-02-22T16:08:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.093+0000 7f35c31fd700 1 --2- 172.21.15.19:0/3978794900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc1038f0 0x7f35bc0fcf60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.093+0000 7f35c31fd700 1 --2- 172.21.15.19:0/3978794900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35bc102f50 0x7f35bc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.093+0000 7f35c31fd700 1 -- 172.21.15.19:0/3978794900 >> 172.21.15.19:0/3978794900 conn(0x7f35bc0f8ae0 msgr2=0x7f35bc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.094+0000 7f35c31fd700 1 -- 172.21.15.19:0/3978794900 shutdown_connections 2024-02-22T16:08:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.094+0000 7f35c31fd700 1 -- 172.21.15.19:0/3978794900 wait complete. 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.094+0000 7f35c31fd700 1 Processor -- start 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.094+0000 7f35c31fd700 1 -- start start 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c31fd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 0x7f35bc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c31fd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35bc1038f0 0x7f35bc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c31fd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc114530 con 0x7f35bc102f50 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c31fd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc10d860 con 0x7f35bc1038f0 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 0x7f35bc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 0x7f35bc1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55284/0 (socket says 172.21.15.19:55284) 2024-02-22T16:08:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 -- 172.21.15.19:0/60935241 learned_addr learned my addr 172.21.15.19:0/60935241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35bbfff700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35bc1038f0 0x7f35bc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 -- 172.21.15.19:0/60935241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35bc1038f0 msgr2=0x7f35bc113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35bc1038f0 0x7f35bc113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 -- 172.21.15.19:0/60935241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8005040 con 0x7f35bc102f50 2024-02-22T16:08:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.095+0000 7f35c0f99700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 0x7f35bc1139a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35b0002a50 tx=0x7f35b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.097+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35b0007cc0 con 0x7f35bc102f50 2024-02-22T16:08:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.097+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc10db40 con 0x7f35bc102f50 2024-02-22T16:08:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.097+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35b000f040 con 0x7f35bc102f50 2024-02-22T16:08:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.097+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35b000d9c0 con 0x7f35bc102f50 2024-02-22T16:08:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.097+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc10e050 con 0x7f35bc102f50 2024-02-22T16:08:27.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.098+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc105420 con 0x7f35bc102f50 2024-02-22T16:08:27.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.099+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f35b0009070 con 0x7f35bc102f50 2024-02-22T16:08:27.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.099+0000 7f35b97fa700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f35ac06cc20 0x7f35ac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.099+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f35b008be20 con 0x7f35bc102f50 2024-02-22T16:08:27.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.099+0000 7f35bbfff700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f35ac06cc20 0x7f35ac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.100+0000 7f35bbfff700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f35ac06cc20 0x7f35ac06f0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f35bc10f2d0 tx=0x7f35a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:27.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.104+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f35b0059860 con 0x7f35bc102f50 2024-02-22T16:08:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.391+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f35bc04bed0 con 0x7f35bc102f50 2024-02-22T16:08:27.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.392+0000 7f35b97fa700 1 -- 172.21.15.19:0/60935241 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f35b00593f0 con 0x7f35bc102f50 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:27.368 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:08:27.369 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:08:27.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:08:27.369 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f35ac06cc20 msgr2=0x7f35ac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f35ac06cc20 0x7f35ac06f0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f35bc10f2d0 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 msgr2=0x7f35bc1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 0x7f35bc1139a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35b0002a50 tx=0x7f35b00046b0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 shutdown_connections 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f35ac06cc20 0x7f35ac06f0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35bc102f50 0x7f35bc1139a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 --2- 172.21.15.19:0/60935241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35bc1038f0 0x7f35bc113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 >> 172.21.15.19:0/60935241 conn(0x7f35bc0f8ae0 msgr2=0x7f35bc0fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 shutdown_connections 2024-02-22T16:08:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.394+0000 7f35c31fd700 1 -- 172.21.15.19:0/60935241 wait complete. 2024-02-22T16:08:27.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:27 smithi062 ceph-mon[106861]: pgmap v455: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:27.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 -- 172.21.15.19:0/360312780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8594102f50 msgr2=0x7f8594103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 --2- 172.21.15.19:0/360312780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8594102f50 0x7f8594103320 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8584004940 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 -- 172.21.15.19:0/360312780 shutdown_connections 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 --2- 172.21.15.19:0/360312780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85941038f0 0x7f85940fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 --2- 172.21.15.19:0/360312780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8594102f50 0x7f8594103320 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 -- 172.21.15.19:0/360312780 >> 172.21.15.19:0/360312780 conn(0x7f85940f8ae0 msgr2=0x7f85940faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 -- 172.21.15.19:0/360312780 shutdown_connections 2024-02-22T16:08:27.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 -- 172.21.15.19:0/360312780 wait complete. 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 Processor -- start 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.498+0000 7f859a3c8700 1 -- start start 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f859a3c8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8594102f50 0x7f8594115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f859a3c8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 0x7f85941160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f859a3c8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594116710 con 0x7f85941038f0 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f859a3c8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859410fa40 con 0x7f8594102f50 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 0x7f85941160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 0x7f85941160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51048/0 (socket says 172.21.15.19:51048) 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 -- 172.21.15.19:0/3150965833 learned_addr learned my addr 172.21.15.19:0/3150965833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f8593fff700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8594102f50 0x7f8594115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 -- 172.21.15.19:0/3150965833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8594102f50 msgr2=0x7f8594115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8594102f50 0x7f8594115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 -- 172.21.15.19:0/3150965833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584005040 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f85937fe700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 0x7f85941160c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f858800eb40 tx=0x7f858800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8588007cc0 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f858800f040 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.499+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859410fcc0 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.500+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858800c8f0 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.500+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8594110200 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.501+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85940fc6e0 con 0x7f85941038f0 2024-02-22T16:08:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.501+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f858800cb60 con 0x7f85941038f0 2024-02-22T16:08:27.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.501+0000 7f8590ff9700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f857c06cc20 0x7f857c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.501+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f858808baf0 con 0x7f85941038f0 2024-02-22T16:08:27.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.502+0000 7f8593fff700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f857c06cc20 0x7f857c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.502+0000 7f8593fff700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f857c06cc20 0x7f857c06f0d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8584009fd0 tx=0x7f8584009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.506+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f85880595e0 con 0x7f85941038f0 2024-02-22T16:08:27.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:27 smithi019 ceph-mon[173579]: pgmap v455: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:27.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.755+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f859404bed0 con 0x7f85941038f0 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.756+0000 7f8590ff9700 1 -- 172.21.15.19:0/3150965833 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f8588059170 con 0x7f85941038f0 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:08:27.732 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:08:27.733 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:08:27.734 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:08:27.735 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:08:27.735 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:08:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f857c06cc20 msgr2=0x7f857c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f857c06cc20 0x7f857c06f0d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8584009fd0 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 msgr2=0x7f85941160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 0x7f85941160c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f858800eb40 tx=0x7f858800b040 comp rx=0 tx=0).stop 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 shutdown_connections 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f857c06cc20 0x7f857c06f0d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8594102f50 0x7f8594115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 --2- 172.21.15.19:0/3150965833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85941038f0 0x7f85941160c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 >> 172.21.15.19:0/3150965833 conn(0x7f85940f8ae0 msgr2=0x7f85940fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 shutdown_connections 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.758+0000 7f859a3c8700 1 -- 172.21.15.19:0/3150965833 wait complete. 2024-02-22T16:08:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 -- 172.21.15.19:0/539652639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 msgr2=0x7f2c900fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 --2- 172.21.15.19:0/539652639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fe3b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c84004970 tx=0x7f2c84009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 -- 172.21.15.19:0/539652639 shutdown_connections 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 --2- 172.21.15.19:0/539652639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900fe8f0 0x7f2c90107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 --2- 172.21.15.19:0/539652639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fe3b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 -- 172.21.15.19:0/539652639 >> 172.21.15.19:0/539652639 conn(0x7f2c90071d90 msgr2=0x7f2c90072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:27.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 -- 172.21.15.19:0/539652639 shutdown_connections 2024-02-22T16:08:27.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 -- 172.21.15.19:0/539652639 wait complete. 2024-02-22T16:08:27.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 Processor -- start 2024-02-22T16:08:27.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.863+0000 7f2c96c93700 1 -- start start 2024-02-22T16:08:27.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c96c93700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c96c93700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900fe8f0 0x7f2c900fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c96c93700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900fc9e0 con 0x7f2c900fdfe0 2024-02-22T16:08:27.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c96c93700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900fcb50 con 0x7f2c900fe8f0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51076/0 (socket says 172.21.15.19:51076) 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 -- 172.21.15.19:0/2930342487 learned_addr learned my addr 172.21.15.19:0/2930342487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c8ffff700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900fe8f0 0x7f2c900fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 -- 172.21.15.19:0/2930342487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900fe8f0 msgr2=0x7f2c900fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900fe8f0 0x7f2c900fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 -- 172.21.15.19:0/2930342487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84005040 con 0x7f2c900fdfe0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c94a2f700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fbea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8400e040 con 0x7f2c900fdfe0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c84009b10 con 0x7f2c900fdfe0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.864+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c90119b70 con 0x7f2c900fdfe0 2024-02-22T16:08:27.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.865+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c84013e60 con 0x7f2c900fdfe0 2024-02-22T16:08:27.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.865+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c90119fd0 con 0x7f2c900fdfe0 2024-02-22T16:08:27.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.866+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c901046c0 con 0x7f2c900fdfe0 2024-02-22T16:08:27.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.866+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2c840136b0 con 0x7f2c900fdfe0 2024-02-22T16:08:27.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.866+0000 7f2c8d7fa700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2c8006cbb0 0x7f2c8006f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:27.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.867+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2c8408d2d0 con 0x7f2c900fdfe0 2024-02-22T16:08:27.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.867+0000 7f2c8ffff700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2c8006cbb0 0x7f2c8006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:27.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.867+0000 7f2c8ffff700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2c8006cbb0 0x7f2c8006f060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2c900fd8a0 tx=0x7f2c7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:27.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:27.871+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2c84057300 con 0x7f2c900fdfe0 2024-02-22T16:08:28.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.092+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c900fd5e0 con 0x7f2c8006cbb0 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.093+0000 7f2c8d7fa700 1 -- 172.21.15.19:0/2930342487 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2c900fd5e0 con 0x7f2c8006cbb0 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:08:28.069 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:08:28.070 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:08:28.070 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:08:28.070 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:08:28.070 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:08:28.070 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2c8006cbb0 msgr2=0x7f2c8006f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2c8006cbb0 0x7f2c8006f060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2c900fd8a0 tx=0x7f2c7c00b040 comp rx=0 tx=0).stop 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 msgr2=0x7f2c900fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fbea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 shutdown_connections 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2c8006cbb0 0x7f2c8006f060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c900fdfe0 0x7f2c900fbea0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 --2- 172.21.15.19:0/2930342487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c900fe8f0 0x7f2c900fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 >> 172.21.15.19:0/2930342487 conn(0x7f2c90071d90 msgr2=0x7f2c90101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 shutdown_connections 2024-02-22T16:08:28.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.095+0000 7f2c96c93700 1 -- 172.21.15.19:0/2930342487 wait complete. 2024-02-22T16:08:28.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.199+0000 7f42c367d700 1 -- 172.21.15.19:0/979703442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 msgr2=0x7f42bc0fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:28.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.199+0000 7f42c367d700 1 --2- 172.21.15.19:0/979703442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc0fc680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f42ac004940 tx=0x7f42ac009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:28.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.199+0000 7f42c367d700 1 -- 172.21.15.19:0/979703442 shutdown_connections 2024-02-22T16:08:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.199+0000 7f42c367d700 1 --2- 172.21.15.19:0/979703442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc0fc680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.199+0000 7f42c367d700 1 --2- 172.21.15.19:0/979703442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc101ee0 0x7f42bc1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.199+0000 7f42c367d700 1 -- 172.21.15.19:0/979703442 >> 172.21.15.19:0/979703442 conn(0x7f42bc071560 msgr2=0x7f42bc071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.200+0000 7f42c367d700 1 -- 172.21.15.19:0/979703442 shutdown_connections 2024-02-22T16:08:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.200+0000 7f42c367d700 1 -- 172.21.15.19:0/979703442 wait complete. 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.200+0000 7f42c367d700 1 Processor -- start 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.200+0000 7f42c367d700 1 -- start start 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c367d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc101ee0 0x7f42bc10ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c367d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc10efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c367d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc10f530 con 0x7f42bc102880 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c367d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc10f6a0 con 0x7f42bc101ee0 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc10efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc10efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51104/0 (socket says 172.21.15.19:51104) 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 -- 172.21.15.19:0/1662746005 learned_addr learned my addr 172.21.15.19:0/1662746005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c1419700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc101ee0 0x7f42bc10ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 -- 172.21.15.19:0/1662746005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc101ee0 msgr2=0x7f42bc10ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc101ee0 0x7f42bc10ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 -- 172.21.15.19:0/1662746005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac005040 con 0x7f42bc102880 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c0c18700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc10efa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009180 tx=0x7f42ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42ac00e040 con 0x7f42bc102880 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc1179d0 con 0x7f42bc102880 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f42ac004290 con 0x7f42bc102880 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.201+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42ac01c3f0 con 0x7f42bc102880 2024-02-22T16:08:28.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.202+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc117e90 con 0x7f42bc102880 2024-02-22T16:08:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.202+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc1043b0 con 0x7f42bc102880 2024-02-22T16:08:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.203+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f42ac013770 con 0x7f42bc102880 2024-02-22T16:08:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.204+0000 7f42b1ffb700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f42a806cc30 0x7f42a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.204+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f42ac08d380 con 0x7f42bc102880 2024-02-22T16:08:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.204+0000 7f42c1419700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f42a806cc30 0x7f42a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.204+0000 7f42c1419700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f42a806cc30 0x7f42a806f0e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42b8004f40 tx=0x7f42b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:28.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.208+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f42ac057350 con 0x7f42bc102880 2024-02-22T16:08:28.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:28 smithi062 ceph-mon[106861]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:28.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:28 smithi062 ceph-mon[106861]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:28.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:28 smithi062 ceph-mon[106861]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:28.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:28 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/60935241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:08:28.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:28 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3150965833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:08:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.491+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f42bc073080 con 0x7f42bc102880 2024-02-22T16:08:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.491+0000 7f42b1ffb700 1 -- 172.21.15.19:0/1662746005 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f42ac05a970 con 0x7f42bc102880 2024-02-22T16:08:28.467 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:08:28.467 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:08:28.467 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:08:28.467 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:08:28.467 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f42a806cc30 msgr2=0x7f42a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f42a806cc30 0x7f42a806f0e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42b8004f40 tx=0x7f42b800b040 comp rx=0 tx=0).stop 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 msgr2=0x7f42bc10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc10efa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009180 tx=0x7f42ac009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 shutdown_connections 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f42a806cc30 0x7f42a806f0e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42bc101ee0 0x7f42bc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 --2- 172.21.15.19:0/1662746005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc102880 0x7f42bc10efa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 >> 172.21.15.19:0/1662746005 conn(0x7f42bc071560 msgr2=0x7f42bc0fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 shutdown_connections 2024-02-22T16:08:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:28.493+0000 7f42c367d700 1 -- 172.21.15.19:0/1662746005 wait complete. 2024-02-22T16:08:28.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:28 smithi019 ceph-mon[173579]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:28.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:28 smithi019 ceph-mon[173579]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:28.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:28 smithi019 ceph-mon[173579]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:28.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:28 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/60935241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:08:28.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:28 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3150965833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:08:29.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:29 smithi062 ceph-mon[106861]: pgmap v456: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:29.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:29 smithi062 ceph-mon[106861]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:29.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:29 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1662746005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:08:29.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:29 smithi019 ceph-mon[173579]: pgmap v456: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:29.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:29 smithi019 ceph-mon[173579]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:08:29.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:29 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1662746005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:08:31.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:31 smithi019 ceph-mon[173579]: pgmap v457: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:31.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:31 smithi062 ceph-mon[106861]: pgmap v457: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:33 smithi062 ceph-mon[106861]: pgmap v458: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:33 smithi019 ceph-mon[173579]: pgmap v458: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:34.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:34.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:35.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:35 smithi062 ceph-mon[106861]: pgmap v459: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:35.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:35 smithi019 ceph-mon[173579]: pgmap v459: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:36.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:36 smithi062 ceph-mon[106861]: pgmap v460: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:36.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:36 smithi019 ceph-mon[173579]: pgmap v460: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:39.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:39 smithi062 ceph-mon[106861]: pgmap v461: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:39.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:39 smithi019 ceph-mon[173579]: pgmap v461: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:41.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:41 smithi019 ceph-mon[173579]: pgmap v462: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:41.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:41 smithi062 ceph-mon[106861]: pgmap v462: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:43.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:43 smithi062 ceph-mon[106861]: pgmap v463: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:43.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:43 smithi019 ceph-mon[173579]: pgmap v463: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:45.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:45 smithi062 ceph-mon[106861]: pgmap v464: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:45.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:45 smithi019 ceph-mon[173579]: pgmap v464: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:47.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:47 smithi062 ceph-mon[106861]: pgmap v465: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:47.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:47 smithi019 ceph-mon[173579]: pgmap v465: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:49 smithi062 ceph-mon[106861]: pgmap v466: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:49.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:49 smithi019 ceph-mon[173579]: pgmap v466: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:49.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:08:51.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:51 smithi019 ceph-mon[173579]: pgmap v467: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:51.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:51 smithi062 ceph-mon[106861]: pgmap v467: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:53.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:53 smithi062 ceph-mon[106861]: pgmap v468: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:53.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:53 smithi019 ceph-mon[173579]: pgmap v468: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:55.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:55 smithi062 ceph-mon[106861]: pgmap v469: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:55.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:55 smithi019 ceph-mon[173579]: pgmap v469: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:57.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:57 smithi062 ceph-mon[106861]: pgmap v470: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:57.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:57 smithi019 ceph-mon[173579]: pgmap v470: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/358831893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88103140 msgr2=0x7f7a88103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/358831893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88103140 0x7f7a88103510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7a78004940 tx=0x7f7a78009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/358831893 shutdown_connections 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/358831893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a880fb570 0x7f7a880fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/358831893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a88103140 0x7f7a88103510 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/358831893 >> 172.21.15.19:0/358831893 conn(0x7f7a88071560 msgr2=0x7f7a88071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/358831893 shutdown_connections 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.611+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/358831893 wait complete. 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a8d4d9700 1 Processor -- start 2024-02-22T16:08:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a8d4d9700 1 -- start start 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a8d4d9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 0x7f7a880fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a8d4d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a88103140 0x7f7a880fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a8d4d9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880fd290 con 0x7f7a880fb570 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a8d4d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880fd400 con 0x7f7a88103140 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 0x7f7a880fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a86ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 0x7f7a880fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35618/0 (socket says 172.21.15.19:35618) 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.612+0000 7f7a86ffd700 1 -- 172.21.15.19:0/3463693659 learned_addr learned my addr 172.21.15.19:0/3463693659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a867fc700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a88103140 0x7f7a880fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a86ffd700 1 -- 172.21.15.19:0/3463693659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a88103140 msgr2=0x7f7a880fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a86ffd700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a88103140 0x7f7a880fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a86ffd700 1 -- 172.21.15.19:0/3463693659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78005040 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a867fc700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a88103140 0x7f7a880fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a86ffd700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 0x7f7a880fc750 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800b3b0 tx=0x7f7a78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7800e040 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a7800bc50 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a88119c30 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a78013420 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.613+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8811a090 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.614+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a88100310 con 0x7f7a880fb570 2024-02-22T16:08:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.615+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7a7801c430 con 0x7f7a880fb570 2024-02-22T16:08:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.615+0000 7f7a7f7fe700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7a7406cc20 0x7f7a7406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.615+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7a7808cd80 con 0x7f7a880fb570 2024-02-22T16:08:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.615+0000 7f7a867fc700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7a7406cc20 0x7f7a7406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.616+0000 7f7a867fc700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7a7406cc20 0x7f7a7406f0d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a880fe150 tx=0x7f7a70009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.620+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7a78056f00 con 0x7f7a880fb570 2024-02-22T16:08:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.845+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a880fde90 con 0x7f7a7406cc20 2024-02-22T16:08:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.846+0000 7f7a7f7fe700 1 -- 172.21.15.19:0/3463693659 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7a880fde90 con 0x7f7a7406cc20 2024-02-22T16:08:58.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7a7406cc20 msgr2=0x7f7a7406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:58.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7a7406cc20 0x7f7a7406f0d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a880fe150 tx=0x7f7a70009040 comp rx=0 tx=0).stop 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 msgr2=0x7f7a880fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 0x7f7a880fc750 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800b3b0 tx=0x7f7a78009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 shutdown_connections 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7a7406cc20 0x7f7a7406f0d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a880fb570 0x7f7a880fc750 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 --2- 172.21.15.19:0/3463693659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a88103140 0x7f7a880fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 >> 172.21.15.19:0/3463693659 conn(0x7f7a88071560 msgr2=0x7f7a880fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 shutdown_connections 2024-02-22T16:08:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.848+0000 7f7a8d4d9700 1 -- 172.21.15.19:0/3463693659 wait complete. 2024-02-22T16:08:58.834 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:08:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 -- 172.21.15.19:0/279978528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 msgr2=0x7f28a00fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 --2- 172.21.15.19:0/279978528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a00fe3b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2890004970 tx=0x7f2890009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 -- 172.21.15.19:0/279978528 shutdown_connections 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 --2- 172.21.15.19:0/279978528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 0x7f28a0107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 --2- 172.21.15.19:0/279978528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a00fe3b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 -- 172.21.15.19:0/279978528 >> 172.21.15.19:0/279978528 conn(0x7f28a0071d90 msgr2=0x7f28a0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 -- 172.21.15.19:0/279978528 shutdown_connections 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.952+0000 7f28a6a59700 1 -- 172.21.15.19:0/279978528 wait complete. 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f28a6a59700 1 Processor -- start 2024-02-22T16:08:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f28a6a59700 1 -- start start 2024-02-22T16:08:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f28a6a59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a0113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f28a6a59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 0x7f28a0113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f28a6a59700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a01144c0 con 0x7f28a00fdfe0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f28a6a59700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a010d7f0 con 0x7f28a00fe8f0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f289ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a0113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f289ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a0113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35642/0 (socket says 172.21.15.19:35642) 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.953+0000 7f289ffff700 1 -- 172.21.15.19:0/332915332 learned_addr learned my addr 172.21.15.19:0/332915332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289f7fe700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 0x7f28a0113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289ffff700 1 -- 172.21.15.19:0/332915332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 msgr2=0x7f28a0113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289ffff700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 0x7f28a0113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289ffff700 1 -- 172.21.15.19:0/332915332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890005040 con 0x7f28a00fdfe0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289f7fe700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 0x7f28a0113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289ffff700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a0113930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2890005010 tx=0x7f2890009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f289000e040 con 0x7f28a00fdfe0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2890004290 con 0x7f28a00fdfe0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a010da70 con 0x7f28a00fdfe0 2024-02-22T16:08:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f289001c3f0 con 0x7f28a00fdfe0 2024-02-22T16:08:58.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.954+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a010dfb0 con 0x7f28a00fdfe0 2024-02-22T16:08:58.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.955+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a01046c0 con 0x7f28a00fdfe0 2024-02-22T16:08:58.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.955+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2890013420 con 0x7f28a00fdfe0 2024-02-22T16:08:58.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.956+0000 7f289cff9700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f288c06cc30 0x7f288c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:58.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.956+0000 7f289f7fe700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f288c06cc30 0x7f288c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.956+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f289008d2c0 con 0x7f28a00fdfe0 2024-02-22T16:08:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.956+0000 7f289f7fe700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f288c06cc30 0x7f288c06f0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28a010f160 tx=0x7f288800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:58.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:58.960+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f289005ae20 con 0x7f28a00fdfe0 2024-02-22T16:08:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.184+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28a004bed0 con 0x7f288c06cc30 2024-02-22T16:08:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.185+0000 7f289cff9700 1 -- 172.21.15.19:0/332915332 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f28a004bed0 con 0x7f288c06cc30 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f288c06cc30 msgr2=0x7f288c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f288c06cc30 0x7f288c06f0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28a010f160 tx=0x7f288800b040 comp rx=0 tx=0).stop 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 msgr2=0x7f28a0113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a0113930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2890005010 tx=0x7f2890009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 shutdown_connections 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f288c06cc30 0x7f288c06f0e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00fdfe0 0x7f28a0113930 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 --2- 172.21.15.19:0/332915332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28a00fe8f0 0x7f28a0113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 >> 172.21.15.19:0/332915332 conn(0x7f28a0071d90 msgr2=0x7f28a0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 shutdown_connections 2024-02-22T16:08:59.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.187+0000 7f28a6a59700 1 -- 172.21.15.19:0/332915332 wait complete. 2024-02-22T16:08:59.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 -- 172.21.15.19:0/2898674127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 msgr2=0x7f0e60102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2898674127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e60102210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c004940 tx=0x7f0e5c009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:59.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 -- 172.21.15.19:0/2898674127 shutdown_connections 2024-02-22T16:08:59.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2898674127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 0x7f0e600fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2898674127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e60102210 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 -- 172.21.15.19:0/2898674127 >> 172.21.15.19:0/2898674127 conn(0x7f0e60071d90 msgr2=0x7f0e60072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 -- 172.21.15.19:0/2898674127 shutdown_connections 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 -- 172.21.15.19:0/2898674127 wait complete. 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 Processor -- start 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.291+0000 7f0e678bc700 1 -- start start 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e678bc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e6010e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e678bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 0x7f0e6010ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e678bc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6010f490 con 0x7f0e60101e40 2024-02-22T16:08:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e678bc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6010f600 con 0x7f0e601027e0 2024-02-22T16:08:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e6010e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e6010e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35660/0 (socket says 172.21.15.19:35660) 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 -- 172.21.15.19:0/2440850819 learned_addr learned my addr 172.21.15.19:0/2440850819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e64e57700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 0x7f0e6010ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 -- 172.21.15.19:0/2440850819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 msgr2=0x7f0e6010ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 0x7f0e6010ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 -- 172.21.15.19:0/2440850819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c005040 con 0x7f0e60101e40 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e64e57700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 0x7f0e6010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e65658700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e6010e9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009180 tx=0x7f0e5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5c00e040 con 0x7f0e60101e40 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e5c013420 con 0x7f0e60101e40 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e60117930 con 0x7f0e60101e40 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.292+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5c01c630 con 0x7f0e60101e40 2024-02-22T16:08:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.293+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e60117e20 con 0x7f0e60101e40 2024-02-22T16:08:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.294+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e600ff150 con 0x7f0e60101e40 2024-02-22T16:08:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.294+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0e5c0139e0 con 0x7f0e60101e40 2024-02-22T16:08:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.295+0000 7f0e51ffb700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0e4c06cc20 0x7f0e4c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.295+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0e5c08d420 con 0x7f0e60101e40 2024-02-22T16:08:59.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.295+0000 7f0e64e57700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0e4c06cc20 0x7f0e4c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.295+0000 7f0e64e57700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0e4c06cc20 0x7f0e4c06f0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0e54007fd0 tx=0x7f0e54009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.299+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0e5c0573b0 con 0x7f0e60101e40 2024-02-22T16:08:59.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:08:59 smithi062 ceph-mon[106861]: pgmap v471: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.509+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e60065f20 con 0x7f0e4c06cc20 2024-02-22T16:08:59.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:08:59 smithi019 ceph-mon[173579]: pgmap v471: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:08:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.519+0000 7f0e51ffb700 1 -- 172.21.15.19:0/2440850819 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f0e60065f20 con 0x7f0e4c06cc20 2024-02-22T16:08:59.494 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:08:59.494 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (16m) 9m ago 27m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:08:59.494 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (13m) 9m ago 27m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (13m) 9m ago 26m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (15m) 9m ago 26m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (9m) 9m ago 21m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (9m) 9m ago 21m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (9m) 9m ago 21m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (9m) 9m ago 21m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (18m) 9m ago 28m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (17m) 9m ago 26m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (14m) 9m ago 28m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (14m) 9m ago 26m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (17m) 9m ago 27m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (16m) 9m ago 26m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (13m) 9m ago 25m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (12m) 9m ago 24m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:08:59.495 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (11m) 9m ago 24m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:08:59.496 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (11m) 9m ago 23m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:08:59.496 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (10m) 9m ago 23m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:08:59.496 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (10m) 9m ago 23m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:08:59.496 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (16m) 9m ago 26m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0e4c06cc20 msgr2=0x7f0e4c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0e4c06cc20 0x7f0e4c06f0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0e54007fd0 tx=0x7f0e54009040 comp rx=0 tx=0).stop 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 msgr2=0x7f0e6010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e6010e9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009180 tx=0x7f0e5c009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 shutdown_connections 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0e4c06cc20 0x7f0e4c06f0d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e60101e40 0x7f0e6010e9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 --2- 172.21.15.19:0/2440850819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e601027e0 0x7f0e6010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 >> 172.21.15.19:0/2440850819 conn(0x7f0e60071d90 msgr2=0x7f0e600fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 shutdown_connections 2024-02-22T16:08:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.521+0000 7f0e678bc700 1 -- 172.21.15.19:0/2440850819 wait complete. 2024-02-22T16:08:59.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 -- 172.21.15.19:0/720751819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 msgr2=0x7f7b78102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/720751819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b78102210 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7b74004940 tx=0x7f7b74009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 -- 172.21.15.19:0/720751819 shutdown_connections 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/720751819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b781027e0 0x7f7b780fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/720751819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b78102210 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 -- 172.21.15.19:0/720751819 >> 172.21.15.19:0/720751819 conn(0x7f7b78071d90 msgr2=0x7f7b78072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 -- 172.21.15.19:0/720751819 shutdown_connections 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.623+0000 7f7b7ff49700 1 -- 172.21.15.19:0/720751819 wait complete. 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7ff49700 1 Processor -- start 2024-02-22T16:08:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7ff49700 1 -- start start 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7ff49700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b7810ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7ff49700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b781027e0 0x7f7b7810ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7ff49700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7810f4d0 con 0x7f7b78101e40 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7ff49700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7810f640 con 0x7f7b781027e0 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7dce5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b7810ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7dce5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b7810ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35692/0 (socket says 172.21.15.19:35692) 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7dce5700 1 -- 172.21.15.19:0/1803192119 learned_addr learned my addr 172.21.15.19:0/1803192119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7d4e4700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b781027e0 0x7f7b7810ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7dce5700 1 -- 172.21.15.19:0/1803192119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b781027e0 msgr2=0x7f7b7810ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7dce5700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b781027e0 0x7f7b7810ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.624+0000 7f7b7dce5700 1 -- 172.21.15.19:0/1803192119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74005040 con 0x7f7b78101e40 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.625+0000 7f7b7dce5700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b7810ea00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009fd0 tx=0x7f7b74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.625+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b7400e040 con 0x7f7b78101e40 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.625+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b74013420 con 0x7f7b78101e40 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.625+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b78117970 con 0x7f7b78101e40 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.625+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b7401c630 con 0x7f7b78101e40 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.625+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78117e30 con 0x7f7b78101e40 2024-02-22T16:08:59.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.626+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b780ff150 con 0x7f7b78101e40 2024-02-22T16:08:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.626+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7b74013590 con 0x7f7b78101e40 2024-02-22T16:08:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.627+0000 7f7b6a7fc700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7b6406cc30 0x7f7b6406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:08:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.627+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7b7408d2d0 con 0x7f7b78101e40 2024-02-22T16:08:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.627+0000 7f7b7d4e4700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7b6406cc30 0x7f7b6406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:08:59.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.627+0000 7f7b7d4e4700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7b6406cc30 0x7f7b6406f0e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b780661a0 tx=0x7f7b6c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:08:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.631+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7b74057300 con 0x7f7b78101e40 2024-02-22T16:08:59.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.920+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b78065ee0 con 0x7f7b78101e40 2024-02-22T16:08:59.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.921+0000 7f7b6a7fc700 1 -- 172.21.15.19:0/1803192119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f7b7405a920 con 0x7f7b78101e40 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:08:59.897 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:08:59.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:08:59.898 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:08:59.899 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:08:59.899 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:08:59.899 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7b6406cc30 msgr2=0x7f7b6406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7b6406cc30 0x7f7b6406f0e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b780661a0 tx=0x7f7b6c00b040 comp rx=0 tx=0).stop 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 msgr2=0x7f7b7810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b7810ea00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009fd0 tx=0x7f7b74009e30 comp rx=0 tx=0).stop 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 shutdown_connections 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7b6406cc30 0x7f7b6406f0e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b78101e40 0x7f7b7810ea00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 --2- 172.21.15.19:0/1803192119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b781027e0 0x7f7b7810ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 >> 172.21.15.19:0/1803192119 conn(0x7f7b78071d90 msgr2=0x7f7b780fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:08:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 shutdown_connections 2024-02-22T16:08:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:08:59.923+0000 7f7b7ff49700 1 -- 172.21.15.19:0/1803192119 wait complete. 2024-02-22T16:09:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 -- 172.21.15.19:0/1199239676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c1027e0 msgr2=0x7f794c0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1199239676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c1027e0 0x7f794c0fbea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7934004940 tx=0x7f7934009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 -- 172.21.15.19:0/1199239676 shutdown_connections 2024-02-22T16:09:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1199239676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c1027e0 0x7f794c0fbea0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1199239676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c101e40 0x7f794c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 -- 172.21.15.19:0/1199239676 >> 172.21.15.19:0/1199239676 conn(0x7f794c071d90 msgr2=0x7f794c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 -- 172.21.15.19:0/1199239676 shutdown_connections 2024-02-22T16:09:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.026+0000 7f7950bdf700 1 -- 172.21.15.19:0/1199239676 wait complete. 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.027+0000 7f7950bdf700 1 Processor -- start 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.027+0000 7f7950bdf700 1 -- start start 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.027+0000 7f7950bdf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 0x7f794c10ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.027+0000 7f7950bdf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c1027e0 0x7f794c10ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.027+0000 7f7950bdf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c10f520 con 0x7f794c101e40 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.027+0000 7f7950bdf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c10f690 con 0x7f794c1027e0 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 0x7f794c10ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 0x7f794c10ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35718/0 (socket says 172.21.15.19:35718) 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 -- 172.21.15.19:0/1290499043 learned_addr learned my addr 172.21.15.19:0/1290499043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7949d9b700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c1027e0 0x7f794c10ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 -- 172.21.15.19:0/1290499043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c1027e0 msgr2=0x7f794c10ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c1027e0 0x7f794c10ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 -- 172.21.15.19:0/1290499043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7934005040 con 0x7f794c101e40 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7949d9b700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c1027e0 0x7f794c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f794a59c700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 0x7f794c10ea50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c830 tx=0x7f793c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793c00a760 con 0x7f794c101e40 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f793c00ad70 con 0x7f794c101e40 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c117a20 con 0x7f794c101e40 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793c0113a0 con 0x7f794c101e40 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.028+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c117ee0 con 0x7f794c101e40 2024-02-22T16:09:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.029+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c104310 con 0x7f794c101e40 2024-02-22T16:09:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.030+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f793c012400 con 0x7f794c101e40 2024-02-22T16:09:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.030+0000 7f7942ffd700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f793806cc20 0x7f793806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.030+0000 7f7949d9b700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f793806cc20 0x7f793806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.030+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f793c08c0c0 con 0x7f794c101e40 2024-02-22T16:09:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.031+0000 7f7949d9b700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f793806cc20 0x7f793806f0d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f793400b3b0 tx=0x7f7934009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:00.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.035+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f793c0560f0 con 0x7f794c101e40 2024-02-22T16:09:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.283+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f794c065ee0 con 0x7f794c101e40 2024-02-22T16:09:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.283+0000 7f7942ffd700 1 -- 172.21.15.19:0/1290499043 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f793c059710 con 0x7f794c101e40 2024-02-22T16:09:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:09:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:09:00.259 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:09:00.260 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:09:00.261 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:09:00.262 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:09:00.262 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:09:00.262 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:09:00.262 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:09:00.262 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:09:00.262 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:00.263 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f793806cc20 msgr2=0x7f793806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f793806cc20 0x7f793806f0d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f793400b3b0 tx=0x7f7934009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 msgr2=0x7f794c10ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 0x7f794c10ea50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c830 tx=0x7f793c004470 comp rx=0 tx=0).stop 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 shutdown_connections 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f793806cc20 0x7f793806f0d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f794c101e40 0x7f794c10ea50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 --2- 172.21.15.19:0/1290499043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f794c1027e0 0x7f794c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 >> 172.21.15.19:0/1290499043 conn(0x7f794c071d90 msgr2=0x7f794c0faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 shutdown_connections 2024-02-22T16:09:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.286+0000 7f7950bdf700 1 -- 172.21.15.19:0/1290499043 wait complete. 2024-02-22T16:09:00.265 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:09:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.390+0000 7fa6a8211700 1 -- 172.21.15.19:0/2433504019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 msgr2=0x7fa6a0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.390+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2433504019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a0102210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009ce0 tx=0x7fa69c004d10 comp rx=0 tx=0).stop 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.390+0000 7fa6a8211700 1 -- 172.21.15.19:0/2433504019 shutdown_connections 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.390+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2433504019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6a01027e0 0x7fa6a00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.390+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2433504019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a0102210 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.390+0000 7fa6a8211700 1 -- 172.21.15.19:0/2433504019 >> 172.21.15.19:0/2433504019 conn(0x7fa6a0071d90 msgr2=0x7fa6a0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 -- 172.21.15.19:0/2433504019 shutdown_connections 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 -- 172.21.15.19:0/2433504019 wait complete. 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 Processor -- start 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 -- start start 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a010c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6a01027e0 0x7fa6a010cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a010d160 con 0x7fa6a0101e40 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a8211700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a010d2d0 con 0x7fa6a01027e0 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a5fad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a010c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a5fad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a010c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35732/0 (socket says 172.21.15.19:35732) 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a5fad700 1 -- 172.21.15.19:0/2375908452 learned_addr learned my addr 172.21.15.19:0/2375908452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.391+0000 7fa6a57ac700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6a01027e0 0x7fa6a010cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6a5fad700 1 -- 172.21.15.19:0/2375908452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6a01027e0 msgr2=0x7fa6a010cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6a5fad700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6a01027e0 0x7fa6a010cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6a5fad700 1 -- 172.21.15.19:0/2375908452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009940 con 0x7fa6a0101e40 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6a5fad700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a010c6e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009360 tx=0x7fa69c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa69c00e040 con 0x7fa6a0101e40 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa69c004610 con 0x7fa6a0101e40 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a010d550 con 0x7fa6a0101e40 2024-02-22T16:09:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa69c012a50 con 0x7fa6a0101e40 2024-02-22T16:09:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.392+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a011a100 con 0x7fa6a0101e40 2024-02-22T16:09:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.393+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a00ff150 con 0x7fa6a0101e40 2024-02-22T16:09:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.393+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa69c01c650 con 0x7fa6a0101e40 2024-02-22T16:09:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.394+0000 7fa6967fc700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa68c06cc30 0x7fa68c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.394+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa69c08e000 con 0x7fa6a0101e40 2024-02-22T16:09:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.394+0000 7fa6a57ac700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa68c06cc30 0x7fa68c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.394+0000 7fa6a57ac700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa68c06cc30 0x7fa68c06f0e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0066180 tx=0x7fa69000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.398+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa69c05bac0 con 0x7fa6a0101e40 2024-02-22T16:09:00.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:00 smithi062 ceph-mon[106861]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:00.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:00 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1803192119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:09:00.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:00 smithi019 ceph-mon[173579]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:00.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:00 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1803192119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:09:00.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.619+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6a010d550 con 0x7fa68c06cc30 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.620+0000 7fa6967fc700 1 -- 172.21.15.19:0/2375908452 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa6a010d550 con 0x7fa68c06cc30 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:09:00.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:09:00.597 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:09:00.597 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:09:00.597 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:09:00.597 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:09:00.597 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa68c06cc30 msgr2=0x7fa68c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa68c06cc30 0x7fa68c06f0e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0066180 tx=0x7fa69000b040 comp rx=0 tx=0).stop 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 msgr2=0x7fa6a010c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a010c6e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009360 tx=0x7fa69c004d10 comp rx=0 tx=0).stop 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 shutdown_connections 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa68c06cc30 0x7fa68c06f0e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6a0101e40 0x7fa6a010c6e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 --2- 172.21.15.19:0/2375908452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6a01027e0 0x7fa6a010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 >> 172.21.15.19:0/2375908452 conn(0x7fa6a0071d90 msgr2=0x7fa6a00faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.622+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 shutdown_connections 2024-02-22T16:09:00.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.623+0000 7fa6a8211700 1 -- 172.21.15.19:0/2375908452 wait complete. 2024-02-22T16:09:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 -- 172.21.15.19:0/1910064215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 msgr2=0x7f6378102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 --2- 172.21.15.19:0/1910064215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f6378102210 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6374004940 tx=0x7f6374009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 -- 172.21.15.19:0/1910064215 shutdown_connections 2024-02-22T16:09:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 --2- 172.21.15.19:0/1910064215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63781027e0 0x7f63780fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 --2- 172.21.15.19:0/1910064215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f6378102210 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 -- 172.21.15.19:0/1910064215 >> 172.21.15.19:0/1910064215 conn(0x7f6378071d90 msgr2=0x7f6378072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 -- 172.21.15.19:0/1910064215 shutdown_connections 2024-02-22T16:09:00.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 -- 172.21.15.19:0/1910064215 wait complete. 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 Processor -- start 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.726+0000 7f637f34d700 1 -- start start 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637f34d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f63781139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637f34d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63781027e0 0x7f6378113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637f34d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378114540 con 0x7f6378101e40 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637f34d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637810d870 con 0x7f63781027e0 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f63781139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f63781139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35748/0 (socket says 172.21.15.19:35748) 2024-02-22T16:09:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 -- 172.21.15.19:0/2168536955 learned_addr learned my addr 172.21.15.19:0/2168536955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637c8e8700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63781027e0 0x7f6378113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 -- 172.21.15.19:0/2168536955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63781027e0 msgr2=0x7f6378113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63781027e0 0x7f6378113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 -- 172.21.15.19:0/2168536955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374005040 con 0x7f6378101e40 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637d0e9700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f63781139b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6374009180 tx=0x7f6374009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637400e040 con 0x7f6378101e40 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637810daf0 con 0x7f6378101e40 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6374004290 con 0x7f6378101e40 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.727+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637401c3f0 con 0x7f6378101e40 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.728+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637810e030 con 0x7f6378101e40 2024-02-22T16:09:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.728+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63780738b0 con 0x7f6378101e40 2024-02-22T16:09:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.729+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6374013420 con 0x7f6378101e40 2024-02-22T16:09:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.729+0000 7f636dffb700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f636406cc20 0x7f636406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.730+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f637408d230 con 0x7f6378101e40 2024-02-22T16:09:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.730+0000 7f637c8e8700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f636406cc20 0x7f636406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.730+0000 7f637c8e8700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f636406cc20 0x7f636406f0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f637810f1e0 tx=0x7f636800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:00.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:00.734+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f637405ad90 con 0x7f6378101e40 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.022+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f637810ef90 con 0x7f6378101e40 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.022+0000 7f636dffb700 1 -- 172.21.15.19:0/2168536955 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f637405a920 con 0x7f6378101e40 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:09:00.998 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:09:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f636406cc20 msgr2=0x7f636406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f636406cc20 0x7f636406f0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f637810f1e0 tx=0x7f636800b040 comp rx=0 tx=0).stop 2024-02-22T16:09:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 msgr2=0x7f63781139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f63781139b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6374009180 tx=0x7f6374009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:01.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 shutdown_connections 2024-02-22T16:09:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f636406cc20 0x7f636406f0d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6378101e40 0x7f63781139b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 --2- 172.21.15.19:0/2168536955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63781027e0 0x7f6378113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 >> 172.21.15.19:0/2168536955 conn(0x7f6378071d90 msgr2=0x7f6378065720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 shutdown_connections 2024-02-22T16:09:01.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:01.025+0000 7f637f34d700 1 -- 172.21.15.19:0/2168536955 wait complete. 2024-02-22T16:09:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:01 smithi019 ceph-mon[173579]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:01 smithi019 ceph-mon[173579]: pgmap v472: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:01 smithi019 ceph-mon[173579]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:01 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1290499043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:09:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:01 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2168536955' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:09:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:01 smithi062 ceph-mon[106861]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:01 smithi062 ceph-mon[106861]: pgmap v472: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:01 smithi062 ceph-mon[106861]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:01 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1290499043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:09:01.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:01 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2168536955' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:09:02.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:02 smithi062 ceph-mon[106861]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:02.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:02 smithi019 ceph-mon[173579]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:03.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:03 smithi062 ceph-mon[106861]: pgmap v473: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:03.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:03 smithi019 ceph-mon[173579]: pgmap v473: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:04.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:04.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:05.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:05 smithi062 ceph-mon[106861]: pgmap v474: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:05.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:05 smithi019 ceph-mon[173579]: pgmap v474: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:06.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:06 smithi062 ceph-mon[106861]: pgmap v475: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:06.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:06 smithi019 ceph-mon[173579]: pgmap v475: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:09.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:09 smithi062 ceph-mon[106861]: pgmap v476: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:09.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:09 smithi019 ceph-mon[173579]: pgmap v476: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:11.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:11 smithi019 ceph-mon[173579]: pgmap v477: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:11.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:11 smithi062 ceph-mon[106861]: pgmap v477: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:13.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:13 smithi062 ceph-mon[106861]: pgmap v478: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:13.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:13 smithi019 ceph-mon[173579]: pgmap v478: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:15.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:15 smithi062 ceph-mon[106861]: pgmap v479: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:15.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:15 smithi019 ceph-mon[173579]: pgmap v479: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:17.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:17 smithi062 ceph-mon[106861]: pgmap v480: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:17.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:17 smithi019 ceph-mon[173579]: pgmap v480: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:19.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:19 smithi062 ceph-mon[106861]: pgmap v481: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:19.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:19.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:19 smithi019 ceph-mon[173579]: pgmap v481: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:19.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:21.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:21 smithi019 ceph-mon[173579]: pgmap v482: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:21.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:21 smithi062 ceph-mon[106861]: pgmap v482: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:23.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:23 smithi062 ceph-mon[106861]: pgmap v483: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:23.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:23 smithi019 ceph-mon[173579]: pgmap v483: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:25.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:25 smithi062 ceph-mon[106861]: pgmap v484: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:25.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:25 smithi019 ceph-mon[173579]: pgmap v484: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:27.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:27 smithi062 ceph-mon[106861]: pgmap v485: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:27.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:27 smithi019 ceph-mon[173579]: pgmap v485: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:29.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:29 smithi062 ceph-mon[106861]: pgmap v486: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:29.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:29 smithi019 ceph-mon[173579]: pgmap v486: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 -- 172.21.15.19:0/3029011425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 msgr2=0x7f2cfc0fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 --2- 172.21.15.19:0/3029011425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc0fbea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4004940 tx=0x7f2ce4009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 -- 172.21.15.19:0/3029011425 shutdown_connections 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 --2- 172.21.15.19:0/3029011425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc0fbea0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 --2- 172.21.15.19:0/3029011425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cfc101e40 0x7f2cfc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 -- 172.21.15.19:0/3029011425 >> 172.21.15.19:0/3029011425 conn(0x7f2cfc071d90 msgr2=0x7f2cfc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 -- 172.21.15.19:0/3029011425 shutdown_connections 2024-02-22T16:09:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.142+0000 7f2d008b3700 1 -- 172.21.15.19:0/3029011425 wait complete. 2024-02-22T16:09:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.143+0000 7f2d008b3700 1 Processor -- start 2024-02-22T16:09:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.143+0000 7f2d008b3700 1 -- start start 2024-02-22T16:09:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2d008b3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cfc101e40 0x7f2cfc112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2d008b3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2d008b3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc113390 con 0x7f2cfc1027e0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2d008b3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc10c6c0 con 0x7f2cfc101e40 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43138/0 (socket says 172.21.15.19:43138) 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 -- 172.21.15.19:0/1448948346 learned_addr learned my addr 172.21.15.19:0/1448948346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cfa59c700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cfc101e40 0x7f2cfc112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 -- 172.21.15.19:0/1448948346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cfc101e40 msgr2=0x7f2cfc112800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cfc101e40 0x7f2cfc112800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 -- 172.21.15.19:0/1448948346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4005040 con 0x7f2cfc1027e0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf1d9b700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc112d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009380 tx=0x7f2ce4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce400e040 con 0x7f2cfc1027e0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.144+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ce4013420 con 0x7f2cfc1027e0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.145+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc10c940 con 0x7f2cfc1027e0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.145+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce401c630 con 0x7f2cfc1027e0 2024-02-22T16:09:31.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.145+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc10ce80 con 0x7f2cfc1027e0 2024-02-22T16:09:31.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.146+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc0738b0 con 0x7f2cfc1027e0 2024-02-22T16:09:31.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.146+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2ce4013590 con 0x7f2cfc1027e0 2024-02-22T16:09:31.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.146+0000 7f2cf37fe700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce806cc20 0x7f2ce806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.147+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2ce408d200 con 0x7f2cfc1027e0 2024-02-22T16:09:31.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.147+0000 7f2cfa59c700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce806cc20 0x7f2ce806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.147+0000 7f2cfa59c700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce806cc20 0x7f2ce806f0d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009fd0 tx=0x7f2cec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:31.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.151+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2ce405ae10 con 0x7f2cfc1027e0 2024-02-22T16:09:31.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:31 smithi019 ceph-mon[173579]: pgmap v487: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:31.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.373+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cfc108e50 con 0x7f2ce806cc20 2024-02-22T16:09:31.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.374+0000 7f2cf37fe700 1 -- 172.21.15.19:0/1448948346 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2cfc108e50 con 0x7f2ce806cc20 2024-02-22T16:09:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce806cc20 msgr2=0x7f2ce806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce806cc20 0x7f2ce806f0d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009fd0 tx=0x7f2cec00b040 comp rx=0 tx=0).stop 2024-02-22T16:09:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 msgr2=0x7f2cfc112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc112d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009380 tx=0x7f2ce4009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 shutdown_connections 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce806cc20 0x7f2ce806f0d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cfc101e40 0x7f2cfc112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 --2- 172.21.15.19:0/1448948346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cfc1027e0 0x7f2cfc112d40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 >> 172.21.15.19:0/1448948346 conn(0x7f2cfc071d90 msgr2=0x7f2cfc0fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 shutdown_connections 2024-02-22T16:09:31.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.376+0000 7f2d008b3700 1 -- 172.21.15.19:0/1448948346 wait complete. 2024-02-22T16:09:31.363 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:09:31.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:31 smithi062 ceph-mon[106861]: pgmap v487: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.480+0000 7f4a6bb58700 1 -- 172.21.15.19:0/1155765359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a64101e50 msgr2=0x7f4a64102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.480+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/1155765359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a64101e50 0x7f4a64102220 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4a60004940 tx=0x7f4a60009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.480+0000 7f4a6bb58700 1 -- 172.21.15.19:0/1155765359 shutdown_connections 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.480+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/1155765359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a641027f0 0x7f4a640fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.480+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/1155765359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a64101e50 0x7f4a64102220 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.480+0000 7f4a6bb58700 1 -- 172.21.15.19:0/1155765359 >> 172.21.15.19:0/1155765359 conn(0x7f4a640f79e0 msgr2=0x7f4a640f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 -- 172.21.15.19:0/1155765359 shutdown_connections 2024-02-22T16:09:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 -- 172.21.15.19:0/1155765359 wait complete. 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 Processor -- start 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 -- start start 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64101e50 0x7f4a64102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 0x7f4a641031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a641037f0 con 0x7f4a641027f0 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a6bb58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a64103960 con 0x7f4a64101e50 2024-02-22T16:09:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a690f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 0x7f4a641031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.481+0000 7f4a690f3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 0x7f4a641031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43156/0 (socket says 172.21.15.19:43156) 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a690f3700 1 -- 172.21.15.19:0/2581784912 learned_addr learned my addr 172.21.15.19:0/2581784912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a698f4700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64101e50 0x7f4a64102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a690f3700 1 -- 172.21.15.19:0/2581784912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64101e50 msgr2=0x7f4a64102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a690f3700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64101e50 0x7f4a64102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a690f3700 1 -- 172.21.15.19:0/2581784912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a60005040 con 0x7f4a641027f0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a690f3700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 0x7f4a641031f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400eb40 tx=0x7f4a5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a54007cc0 con 0x7f4a641027f0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a64119b20 con 0x7f4a641027f0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a5400f040 con 0x7f4a641027f0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a54009570 con 0x7f4a641027f0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.482+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a64119f80 con 0x7f4a641027f0 2024-02-22T16:09:31.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.483+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a640fb5e0 con 0x7f4a641027f0 2024-02-22T16:09:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.484+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4a540096d0 con 0x7f4a641027f0 2024-02-22T16:09:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.484+0000 7f4a5a7fc700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4a5006cc20 0x7f4a5006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.484+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4a5408bbb0 con 0x7f4a641027f0 2024-02-22T16:09:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.484+0000 7f4a698f4700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4a5006cc20 0x7f4a5006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.485+0000 7f4a698f4700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4a5006cc20 0x7f4a5006f0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b0d0 tx=0x7f4a60009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:31.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.489+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4a54092080 con 0x7f4a641027f0 2024-02-22T16:09:31.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.710+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a64066610 con 0x7f4a5006cc20 2024-02-22T16:09:31.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.712+0000 7f4a5a7fc700 1 -- 172.21.15.19:0/2581784912 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4a64066610 con 0x7f4a5006cc20 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.713+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4a5006cc20 msgr2=0x7f4a5006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.713+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4a5006cc20 0x7f4a5006f0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b0d0 tx=0x7f4a60009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.713+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 msgr2=0x7f4a641031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.713+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 0x7f4a641031f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400eb40 tx=0x7f4a5400b040 comp rx=0 tx=0).stop 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 shutdown_connections 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4a5006cc20 0x7f4a5006f0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64101e50 0x7f4a64102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 --2- 172.21.15.19:0/2581784912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a641027f0 0x7f4a641031f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 >> 172.21.15.19:0/2581784912 conn(0x7f4a640f79e0 msgr2=0x7f4a64065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 shutdown_connections 2024-02-22T16:09:31.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.714+0000 7f4a6bb58700 1 -- 172.21.15.19:0/2581784912 wait complete. 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 -- 172.21.15.19:0/1581580351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 msgr2=0x7f9998103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 --2- 172.21.15.19:0/1581580351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f9998103510 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9994004940 tx=0x7f9994009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 -- 172.21.15.19:0/1581580351 shutdown_connections 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 --2- 172.21.15.19:0/1581580351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99980fb570 0x7f99980fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 --2- 172.21.15.19:0/1581580351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f9998103510 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 -- 172.21.15.19:0/1581580351 >> 172.21.15.19:0/1581580351 conn(0x7f9998071560 msgr2=0x7f9998071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 -- 172.21.15.19:0/1581580351 shutdown_connections 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.818+0000 7f999f2a0700 1 -- 172.21.15.19:0/1581580351 wait complete. 2024-02-22T16:09:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999f2a0700 1 Processor -- start 2024-02-22T16:09:31.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999f2a0700 1 -- start start 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999f2a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99980fb570 0x7f99980fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999f2a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f99980fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999f2a0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980fd290 con 0x7f9998103140 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999f2a0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980fd400 con 0x7f99980fb570 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999c83b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f99980fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999c83b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f99980fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43172/0 (socket says 172.21.15.19:43172) 2024-02-22T16:09:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999c83b700 1 -- 172.21.15.19:0/2397051823 learned_addr learned my addr 172.21.15.19:0/2397051823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999d03c700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99980fb570 0x7f99980fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999c83b700 1 -- 172.21.15.19:0/2397051823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99980fb570 msgr2=0x7f99980fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999c83b700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99980fb570 0x7f99980fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.819+0000 7f999c83b700 1 -- 172.21.15.19:0/2397051823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994005040 con 0x7f9998103140 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.820+0000 7f999c83b700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f99980fcc90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9988002a80 tx=0x7f99880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.820+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9988007cc0 con 0x7f9998103140 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.820+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f998800f040 con 0x7f9998103140 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.820+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9998119c30 con 0x7f9998103140 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.820+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9988009640 con 0x7f9998103140 2024-02-22T16:09:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.820+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999811a110 con 0x7f9998103140 2024-02-22T16:09:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.821+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9998100310 con 0x7f9998103140 2024-02-22T16:09:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.821+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f99880097a0 con 0x7f9998103140 2024-02-22T16:09:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.822+0000 7f998dffb700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f998406cc20 0x7f998406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.822+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f998808b770 con 0x7f9998103140 2024-02-22T16:09:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.822+0000 7f999d03c700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f998406cc20 0x7f998406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.822+0000 7f999d03c700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f998406cc20 0x7f998406f0d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9994004f90 tx=0x7f9994009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:31.826+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f99880557a0 con 0x7f9998103140 2024-02-22T16:09:32.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.036+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f999804bed0 con 0x7f998406cc20 2024-02-22T16:09:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.046+0000 7f998dffb700 1 -- 172.21.15.19:0/2397051823 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f999804bed0 con 0x7f998406cc20 2024-02-22T16:09:32.022 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (16m) 10m ago 27m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (14m) 10m ago 28m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (14m) 9m ago 26m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (16m) 10m ago 27m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (10m) 10m ago 21m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (10m) 10m ago 22m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (9m) 9m ago 21m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (9m) 9m ago 22m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (18m) 10m ago 29m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (17m) 9m ago 26m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (15m) 10m ago 29m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (14m) 9m ago 26m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (17m) 10m ago 27m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:09:32.023 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (17m) 9m ago 26m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (13m) 10m ago 25m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (13m) 10m ago 25m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (12m) 10m ago 24m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (11m) 9m ago 24m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (11m) 9m ago 24m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (10m) 9m ago 23m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:09:32.024 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (17m) 10m ago 27m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f998406cc20 msgr2=0x7f998406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f998406cc20 0x7f998406f0d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9994004f90 tx=0x7f9994009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 msgr2=0x7f99980fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f99980fcc90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9988002a80 tx=0x7f99880046b0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 shutdown_connections 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f998406cc20 0x7f998406f0d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99980fb570 0x7f99980fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 --2- 172.21.15.19:0/2397051823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9998103140 0x7f99980fcc90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 >> 172.21.15.19:0/2397051823 conn(0x7f9998071560 msgr2=0x7f99980fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 shutdown_connections 2024-02-22T16:09:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.048+0000 7f999f2a0700 1 -- 172.21.15.19:0/2397051823 wait complete. 2024-02-22T16:09:32.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 -- 172.21.15.19:0/3005033135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 msgr2=0x7f29e0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 --2- 172.21.15.19:0/3005033135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0102210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004940 tx=0x7f29d8009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 -- 172.21.15.19:0/3005033135 shutdown_connections 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 --2- 172.21.15.19:0/3005033135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 0x7f29e00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 --2- 172.21.15.19:0/3005033135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0102210 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 -- 172.21.15.19:0/3005033135 >> 172.21.15.19:0/3005033135 conn(0x7f29e0071d90 msgr2=0x7f29e0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 -- 172.21.15.19:0/3005033135 shutdown_connections 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 -- 172.21.15.19:0/3005033135 wait complete. 2024-02-22T16:09:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.151+0000 7f29e613e700 1 Processor -- start 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29e613e700 1 -- start start 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29e613e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29e613e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 0x7f29e0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29e613e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0114510 con 0x7f29e0101e40 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29e613e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e010d840 con 0x7f29e01027e0 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43188/0 (socket says 172.21.15.19:43188) 2024-02-22T16:09:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 -- 172.21.15.19:0/4008852123 learned_addr learned my addr 172.21.15.19:0/4008852123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29deffd700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 0x7f29e0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 -- 172.21.15.19:0/4008852123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 msgr2=0x7f29e0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 0x7f29e0113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 -- 172.21.15.19:0/4008852123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8005040 con 0x7f29e0101e40 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29deffd700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 0x7f29e0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.152+0000 7f29df7fe700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0113980 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009360 tx=0x7f29d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.153+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29d800e040 con 0x7f29e0101e40 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.153+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e010dac0 con 0x7f29e0101e40 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.153+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29d800bc50 con 0x7f29e0101e40 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.153+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29d801c630 con 0x7f29e0101e40 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.153+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e010e000 con 0x7f29e0101e40 2024-02-22T16:09:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.154+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e00738b0 con 0x7f29e0101e40 2024-02-22T16:09:32.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.154+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f29d8013420 con 0x7f29e0101e40 2024-02-22T16:09:32.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.155+0000 7f29d7fff700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29cc06cc30 0x7f29cc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.155+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f29d808d340 con 0x7f29e0101e40 2024-02-22T16:09:32.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.155+0000 7f29deffd700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29cc06cc30 0x7f29cc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.155+0000 7f29deffd700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29cc06cc30 0x7f29cc06f0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009fd0 tx=0x7f29c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.159+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f29d805ae00 con 0x7f29e0101e40 2024-02-22T16:09:32.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.444+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f29e010ef60 con 0x7f29e0101e40 2024-02-22T16:09:32.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.445+0000 7f29d7fff700 1 -- 172.21.15.19:0/4008852123 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f29d805a990 con 0x7f29e0101e40 2024-02-22T16:09:32.421 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:09:32.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:09:32.421 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:09:32.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:09:32.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:09:32.422 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.446+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29cc06cc30 msgr2=0x7f29cc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.446+0000 7f29e613e700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29cc06cc30 0x7f29cc06f0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009fd0 tx=0x7f29c800b040 comp rx=0 tx=0).stop 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 msgr2=0x7f29e0113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0113980 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009360 tx=0x7f29d8009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 shutdown_connections 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29cc06cc30 0x7f29cc06f0e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e0101e40 0x7f29e0113980 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 --2- 172.21.15.19:0/4008852123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e01027e0 0x7f29e0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 >> 172.21.15.19:0/4008852123 conn(0x7f29e0071d90 msgr2=0x7f29e00657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 shutdown_connections 2024-02-22T16:09:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.447+0000 7f29e613e700 1 -- 172.21.15.19:0/4008852123 wait complete. 2024-02-22T16:09:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 -- 172.21.15.19:0/225808170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c102f50 msgr2=0x7faf7c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 --2- 172.21.15.19:0/225808170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c102f50 0x7faf7c103320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faf6c004940 tx=0x7faf6c009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 -- 172.21.15.19:0/225808170 shutdown_connections 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 --2- 172.21.15.19:0/225808170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c1038f0 0x7faf7c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 --2- 172.21.15.19:0/225808170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c102f50 0x7faf7c103320 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 -- 172.21.15.19:0/225808170 >> 172.21.15.19:0/225808170 conn(0x7faf7c0f8ae0 msgr2=0x7faf7c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 -- 172.21.15.19:0/225808170 shutdown_connections 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.550+0000 7faf82353700 1 -- 172.21.15.19:0/225808170 wait complete. 2024-02-22T16:09:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf82353700 1 Processor -- start 2024-02-22T16:09:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf82353700 1 -- start start 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf82353700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c102f50 0x7faf7c070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf82353700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 0x7faf7c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf82353700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c070dd0 con 0x7faf7c1038f0 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf82353700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c070f40 con 0x7faf7c102f50 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 0x7faf7c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 0x7faf7c0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43198/0 (socket says 172.21.15.19:43198) 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7b7fe700 1 -- 172.21.15.19:0/1102709024 learned_addr learned my addr 172.21.15.19:0/1102709024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7bfff700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c102f50 0x7faf7c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7b7fe700 1 -- 172.21.15.19:0/1102709024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c102f50 msgr2=0x7faf7c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7b7fe700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c102f50 0x7faf7c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7b7fe700 1 -- 172.21.15.19:0/1102709024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c005040 con 0x7faf7c1038f0 2024-02-22T16:09:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.551+0000 7faf7bfff700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c102f50 0x7faf7c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:09:32.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.552+0000 7faf7b7fe700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 0x7faf7c0707d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faf7000eb40 tx=0x7faf7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:32.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.552+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf70007cc0 con 0x7faf7c1038f0 2024-02-22T16:09:32.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.552+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf7000f040 con 0x7faf7c1038f0 2024-02-22T16:09:32.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.552+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c119be0 con 0x7faf7c1038f0 2024-02-22T16:09:32.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.552+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf70009570 con 0x7faf7c1038f0 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.552+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c11a040 con 0x7faf7c1038f0 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.553+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c100260 con 0x7faf7c1038f0 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.553+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faf700096d0 con 0x7faf7c1038f0 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.554+0000 7faf78ff9700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faf6406cc20 0x7faf6406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.554+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7faf7008bc00 con 0x7faf7c1038f0 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.554+0000 7faf7bfff700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faf6406cc20 0x7faf6406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.554+0000 7faf7bfff700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faf6406cc20 0x7faf6406f0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b0d0 tx=0x7faf6c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:32.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.558+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7faf70055bc0 con 0x7faf7c1038f0 2024-02-22T16:09:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.804+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf7c0719d0 con 0x7faf7c1038f0 2024-02-22T16:09:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.805+0000 7faf78ff9700 1 -- 172.21.15.19:0/1102709024 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7faf700591e0 con 0x7faf7c1038f0 2024-02-22T16:09:32.781 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:09:32.782 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:09:32.783 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:32.784 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faf6406cc20 msgr2=0x7faf6406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faf6406cc20 0x7faf6406f0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00b0d0 tx=0x7faf6c009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 msgr2=0x7faf7c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 0x7faf7c0707d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faf7000eb40 tx=0x7faf7000b040 comp rx=0 tx=0).stop 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 shutdown_connections 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faf6406cc20 0x7faf6406f0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf7c102f50 0x7faf7c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 --2- 172.21.15.19:0/1102709024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf7c1038f0 0x7faf7c0707d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 >> 172.21.15.19:0/1102709024 conn(0x7faf7c0f8ae0 msgr2=0x7faf7c0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 shutdown_connections 2024-02-22T16:09:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.807+0000 7faf82353700 1 -- 172.21.15.19:0/1102709024 wait complete. 2024-02-22T16:09:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:09:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 -- 172.21.15.19:0/3220847045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 msgr2=0x7f3ac80fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3220847045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac80fcf60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4004940 tx=0x7f3ab4009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 -- 172.21.15.19:0/3220847045 shutdown_connections 2024-02-22T16:09:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3220847045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac80fcf60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3220847045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ac8102f50 0x7f3ac8103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 -- 172.21.15.19:0/3220847045 >> 172.21.15.19:0/3220847045 conn(0x7f3ac80f8ae0 msgr2=0x7f3ac80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 -- 172.21.15.19:0/3220847045 shutdown_connections 2024-02-22T16:09:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.911+0000 7f3aceea5700 1 -- 172.21.15.19:0/3220847045 wait complete. 2024-02-22T16:09:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3aceea5700 1 Processor -- start 2024-02-22T16:09:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3aceea5700 1 -- start start 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3aceea5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ac8102f50 0x7f3ac8115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3aceea5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac8116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3aceea5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81166c0 con 0x7f3ac81038f0 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3aceea5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac810f9f0 con 0x7f3ac8102f50 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3ac7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac8116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3ac7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac8116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43210/0 (socket says 172.21.15.19:43210) 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.912+0000 7f3ac7fff700 1 -- 172.21.15.19:0/3435267700 learned_addr learned my addr 172.21.15.19:0/3435267700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3accc41700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ac8102f50 0x7f3ac8115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac7fff700 1 -- 172.21.15.19:0/3435267700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ac8102f50 msgr2=0x7f3ac8115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac7fff700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ac8102f50 0x7f3ac8115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac7fff700 1 -- 172.21.15.19:0/3435267700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4005040 con 0x7f3ac81038f0 2024-02-22T16:09:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac7fff700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac8116070 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009fd0 tx=0x7f3ab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab4021070 con 0x7f3ac81038f0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac810fc70 con 0x7f3ac81038f0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3ab401c400 con 0x7f3ac81038f0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ab400e070 con 0x7f3ac81038f0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.913+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac8110150 con 0x7f3ac81038f0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.914+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac80fc6e0 con 0x7f3ac81038f0 2024-02-22T16:09:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.915+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3ab401ca00 con 0x7f3ac81038f0 2024-02-22T16:09:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.915+0000 7f3ac57fa700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ab806cc20 0x7f3ab806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.915+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3ab408d1e0 con 0x7f3ac81038f0 2024-02-22T16:09:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.915+0000 7f3accc41700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ab806cc20 0x7f3ab806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:32.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.916+0000 7f3accc41700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ab806cc20 0x7f3ab806f0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3abc004600 tx=0x7f3abc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:32.920+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3ab405aca0 con 0x7f3ac81038f0 2024-02-22T16:09:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.140+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ac804bed0 con 0x7f3ab806cc20 2024-02-22T16:09:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.141+0000 7f3ac57fa700 1 -- 172.21.15.19:0/3435267700 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ac804bed0 con 0x7f3ab806cc20 2024-02-22T16:09:33.117 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:09:33.118 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ab806cc20 msgr2=0x7f3ab806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ab806cc20 0x7f3ab806f0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3abc004600 tx=0x7f3abc00b040 comp rx=0 tx=0).stop 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 msgr2=0x7f3ac8116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac8116070 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009fd0 tx=0x7f3ab4009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 shutdown_connections 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ab806cc20 0x7f3ab806f0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ac8102f50 0x7f3ac8115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 --2- 172.21.15.19:0/3435267700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac81038f0 0x7f3ac8116070 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 >> 172.21.15.19:0/3435267700 conn(0x7f3ac80f8ae0 msgr2=0x7f3ac80fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 shutdown_connections 2024-02-22T16:09:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.143+0000 7f3aceea5700 1 -- 172.21.15.19:0/3435267700 wait complete. 2024-02-22T16:09:33.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 -- 172.21.15.19:0/2426366113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c81027e0 msgr2=0x7f20c80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:33.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2426366113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c81027e0 0x7f20c80fbea0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20b0004940 tx=0x7f20b0009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 -- 172.21.15.19:0/2426366113 shutdown_connections 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2426366113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c81027e0 0x7f20c80fbea0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2426366113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20c8101e40 0x7f20c8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 -- 172.21.15.19:0/2426366113 >> 172.21.15.19:0/2426366113 conn(0x7f20c8071d90 msgr2=0x7f20c8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 -- 172.21.15.19:0/2426366113 shutdown_connections 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 -- 172.21.15.19:0/2426366113 wait complete. 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 Processor -- start 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.249+0000 7f20ce08c700 1 -- start start 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20ce08c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 0x7f20c80fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20ce08c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20c81027e0 0x7f20c80fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:33.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20ce08c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80fe600 con 0x7f20c8101e40 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20ce08c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80fe770 con 0x7f20c81027e0 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 0x7f20c80fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 0x7f20c80fdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43230/0 (socket says 172.21.15.19:43230) 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 -- 172.21.15.19:0/2812472225 learned_addr learned my addr 172.21.15.19:0/2812472225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c6ffd700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20c81027e0 0x7f20c80fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 -- 172.21.15.19:0/2812472225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20c81027e0 msgr2=0x7f20c80fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20c81027e0 0x7f20c80fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 -- 172.21.15.19:0/2812472225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0005040 con 0x7f20c8101e40 2024-02-22T16:09:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20c77fe700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 0x7f20c80fdac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f20b8002a50 tx=0x7f20b80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:33.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20b8007cc0 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f20b800f040 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.250+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c8119c80 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.251+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20b800d9c0 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.251+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c811a130 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.251+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c80738b0 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.252+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f20b800db20 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.253+0000 7f20bffff700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20b406cc20 0x7f20b406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.253+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f20b808bd20 con 0x7f20c8101e40 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.253+0000 7f20c6ffd700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20b406cc20 0x7f20b406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:09:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.253+0000 7f20c6ffd700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20b406cc20 0x7f20b406f0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f20b000b3b0 tx=0x7f20b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:09:33.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.257+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f20b8090070 con 0x7f20c8101e40 2024-02-22T16:09:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:33 smithi062 ceph-mon[106861]: pgmap v488: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:33 smithi062 ceph-mon[106861]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:33 smithi062 ceph-mon[106861]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:33 smithi062 ceph-mon[106861]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:33 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4008852123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:09:33.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:33 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1102709024' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:09:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:33 smithi019 ceph-mon[173579]: pgmap v488: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:33 smithi019 ceph-mon[173579]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:33 smithi019 ceph-mon[173579]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:33 smithi019 ceph-mon[173579]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:33 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4008852123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:09:33.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:33 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1102709024' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:09:33.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.546+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f20c80ff2a0 con 0x7f20c8101e40 2024-02-22T16:09:33.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.546+0000 7f20bffff700 1 -- 172.21.15.19:0/2812472225 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f20b8055cf0 con 0x7f20c8101e40 2024-02-22T16:09:33.523 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:09:33.523 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:09:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:09:33.523 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:09:33.523 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20b406cc20 msgr2=0x7f20b406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20b406cc20 0x7f20b406f0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f20b000b3b0 tx=0x7f20b0009e30 comp rx=0 tx=0).stop 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 msgr2=0x7f20c80fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 0x7f20c80fdac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f20b8002a50 tx=0x7f20b80046b0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 shutdown_connections 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f20b406cc20 0x7f20b406f0d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20c8101e40 0x7f20c80fdac0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 --2- 172.21.15.19:0/2812472225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20c81027e0 0x7f20c80fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.548+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 >> 172.21.15.19:0/2812472225 conn(0x7f20c8071d90 msgr2=0x7f20c8065880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:09:33.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.549+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 shutdown_connections 2024-02-22T16:09:33.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:09:33.549+0000 7f20ce08c700 1 -- 172.21.15.19:0/2812472225 wait complete. 2024-02-22T16:09:34.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:34.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:34 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2812472225' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:09:34.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:34.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:34 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2812472225' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:09:35.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:35 smithi062 ceph-mon[106861]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:35.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:35 smithi062 ceph-mon[106861]: pgmap v489: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:35.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:35 smithi019 ceph-mon[173579]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:09:35.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:35 smithi019 ceph-mon[173579]: pgmap v489: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:37.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:37 smithi062 ceph-mon[106861]: pgmap v490: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:37.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:37 smithi019 ceph-mon[173579]: pgmap v490: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:39.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:39 smithi062 ceph-mon[106861]: pgmap v491: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:39.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:39 smithi019 ceph-mon[173579]: pgmap v491: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:41.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:41 smithi019 ceph-mon[173579]: pgmap v492: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:41.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:41 smithi062 ceph-mon[106861]: pgmap v492: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:43.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:43 smithi062 ceph-mon[106861]: pgmap v493: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:43.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:43 smithi019 ceph-mon[173579]: pgmap v493: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:45.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:45 smithi062 ceph-mon[106861]: pgmap v494: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:45.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:45 smithi019 ceph-mon[173579]: pgmap v494: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:47.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:47 smithi062 ceph-mon[106861]: pgmap v495: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:47.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:47 smithi019 ceph-mon[173579]: pgmap v495: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:49 smithi062 ceph-mon[106861]: pgmap v496: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:49.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:49 smithi019 ceph-mon[173579]: pgmap v496: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:49.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:09:51.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:51 smithi019 ceph-mon[173579]: pgmap v497: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:51.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:51 smithi062 ceph-mon[106861]: pgmap v497: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:53.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:53 smithi062 ceph-mon[106861]: pgmap v498: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:53.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:53 smithi019 ceph-mon[173579]: pgmap v498: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:55.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:55 smithi062 ceph-mon[106861]: pgmap v499: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:55.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:55 smithi019 ceph-mon[173579]: pgmap v499: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:57.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:57 smithi062 ceph-mon[106861]: pgmap v500: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:57 smithi019 ceph-mon[173579]: pgmap v500: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:59.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:09:59 smithi062 ceph-mon[106861]: pgmap v501: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:09:59.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:09:59 smithi019 ceph-mon[173579]: pgmap v501: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:00.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:00 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T16:09:59.999+0000 7f643850b700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:10:00.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:00 smithi019 ceph-mon[173579]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:10:00.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:00 smithi062 ceph-mon[106861]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:10:01.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:01 smithi019 ceph-mon[173579]: pgmap v502: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:01.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:01 smithi062 ceph-mon[106861]: pgmap v502: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:03.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:03 smithi062 ceph-mon[106861]: pgmap v503: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:03.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:03 smithi019 ceph-mon[173579]: pgmap v503: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 -- 172.21.15.19:0/2388471661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 msgr2=0x7eff4c0fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:03.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 --2- 172.21.15.19:0/2388471661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c0fc1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7eff48004940 tx=0x7eff48009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 -- 172.21.15.19:0/2388471661 shutdown_connections 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 --2- 172.21.15.19:0/2388471661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff4c0fc6e0 0x7eff4c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 --2- 172.21.15.19:0/2388471661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c0fc1a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 -- 172.21.15.19:0/2388471661 >> 172.21.15.19:0/2388471661 conn(0x7eff4c071d90 msgr2=0x7eff4c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 -- 172.21.15.19:0/2388471661 shutdown_connections 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 -- 172.21.15.19:0/2388471661 wait complete. 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.668+0000 7eff53364700 1 Processor -- start 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff53364700 1 -- start start 2024-02-22T16:10:03.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff53364700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff53364700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff4c0fc6e0 0x7eff4c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff53364700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c116730 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff53364700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c10fa60 con 0x7eff4c0fc6e0 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49852/0 (socket says 172.21.15.19:49852) 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 -- 172.21.15.19:0/344617541 learned_addr learned my addr 172.21.15.19:0/344617541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff508ff700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff4c0fc6e0 0x7eff4c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 -- 172.21.15.19:0/344617541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff4c0fc6e0 msgr2=0x7eff4c1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:03.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff4c0fc6e0 0x7eff4c1160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 -- 172.21.15.19:0/344617541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff48005040 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff51100700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c115ba0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff48009360 tx=0x7eff48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.669+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff4800e040 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.670+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c10fce0 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.670+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7eff4800bbf0 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.670+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff4801c630 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.670+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c110220 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.671+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4c106b70 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.671+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7eff4800bd60 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.672+0000 7eff41ffb700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7eff3806cc20 0x7eff3806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:03.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.672+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7eff4808d340 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.672+0000 7eff508ff700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7eff3806cc20 0x7eff3806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:03.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.672+0000 7eff508ff700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7eff3806cc20 0x7eff3806f0d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff3c007fd0 tx=0x7eff3c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:03.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.677+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7eff4805ae00 con 0x7eff4c0fbdd0 2024-02-22T16:10:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.898+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff4c04bed0 con 0x7eff3806cc20 2024-02-22T16:10:03.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.900+0000 7eff41ffb700 1 -- 172.21.15.19:0/344617541 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff4c04bed0 con 0x7eff3806cc20 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7eff3806cc20 msgr2=0x7eff3806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7eff3806cc20 0x7eff3806f0d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff3c007fd0 tx=0x7eff3c009040 comp rx=0 tx=0).stop 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 msgr2=0x7eff4c115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c115ba0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff48009360 tx=0x7eff48009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 shutdown_connections 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7eff3806cc20 0x7eff3806f0d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff4c0fbdd0 0x7eff4c115ba0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 --2- 172.21.15.19:0/344617541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff4c0fc6e0 0x7eff4c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 >> 172.21.15.19:0/344617541 conn(0x7eff4c071d90 msgr2=0x7eff4c0faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 shutdown_connections 2024-02-22T16:10:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:03.902+0000 7eff53364700 1 -- 172.21.15.19:0/344617541 wait complete. 2024-02-22T16:10:03.888 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:10:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 -- 172.21.15.19:0/427616686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 msgr2=0x7f8da0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 --2- 172.21.15.19:0/427616686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da0102220 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004940 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 -- 172.21.15.19:0/427616686 shutdown_connections 2024-02-22T16:10:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 --2- 172.21.15.19:0/427616686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8da01027f0 0x7f8da00fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 --2- 172.21.15.19:0/427616686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da0102220 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 -- 172.21.15.19:0/427616686 >> 172.21.15.19:0/427616686 conn(0x7f8da00f79e0 msgr2=0x7f8da00f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 -- 172.21.15.19:0/427616686 shutdown_connections 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.007+0000 7f8da7d40700 1 -- 172.21.15.19:0/427616686 wait complete. 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da7d40700 1 Processor -- start 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da7d40700 1 -- start start 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da7d40700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da01138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da7d40700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8da01027f0 0x7f8da0113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da7d40700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0114470 con 0x7f8da0101e50 2024-02-22T16:10:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da7d40700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da010d7a0 con 0x7f8da01027f0 2024-02-22T16:10:03.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da5adc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da01138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da5adc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da01138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49872/0 (socket says 172.21.15.19:49872) 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da5adc700 1 -- 172.21.15.19:0/2673507905 learned_addr learned my addr 172.21.15.19:0/2673507905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.008+0000 7f8da52db700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8da01027f0 0x7f8da0113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8da5adc700 1 -- 172.21.15.19:0/2673507905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8da01027f0 msgr2=0x7f8da0113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8da5adc700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8da01027f0 0x7f8da0113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8da5adc700 1 -- 172.21.15.19:0/2673507905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8da0101e50 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8da5adc700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da01138e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009180 tx=0x7f8d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c00e040 con 0x7f8da0101e50 2024-02-22T16:10:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d9c004290 con 0x7f8da0101e50 2024-02-22T16:10:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da010da20 con 0x7f8da0101e50 2024-02-22T16:10:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c01c3f0 con 0x7f8da0101e50 2024-02-22T16:10:03.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.009+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da010df60 con 0x7f8da0101e50 2024-02-22T16:10:03.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.010+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da0064b00 con 0x7f8da0101e50 2024-02-22T16:10:03.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.010+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8d9c013420 con 0x7f8da0101e50 2024-02-22T16:10:03.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.011+0000 7f8d927fc700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8d8c06cc20 0x7f8d8c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:03.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.011+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8d9c08d2b0 con 0x7f8da0101e50 2024-02-22T16:10:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.011+0000 7f8da52db700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8d8c06cc20 0x7f8d8c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.011+0000 7f8da52db700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8d8c06cc20 0x7f8d8c06f0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8da010f110 tx=0x7f8d9400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:03.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.015+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8d9c05aec0 con 0x7f8da0101e50 2024-02-22T16:10:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.235+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8da010ef30 con 0x7f8d8c06cc20 2024-02-22T16:10:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.236+0000 7f8d927fc700 1 -- 172.21.15.19:0/2673507905 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8da010ef30 con 0x7f8d8c06cc20 2024-02-22T16:10:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8d8c06cc20 msgr2=0x7f8d8c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8d8c06cc20 0x7f8d8c06f0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8da010f110 tx=0x7f8d9400b040 comp rx=0 tx=0).stop 2024-02-22T16:10:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 msgr2=0x7f8da01138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da01138e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009180 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 shutdown_connections 2024-02-22T16:10:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8d8c06cc20 0x7f8d8c06f0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0101e50 0x7f8da01138e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 --2- 172.21.15.19:0/2673507905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8da01027f0 0x7f8da0113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 >> 172.21.15.19:0/2673507905 conn(0x7f8da00f79e0 msgr2=0x7f8da0065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 shutdown_connections 2024-02-22T16:10:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.238+0000 7f8da7d40700 1 -- 172.21.15.19:0/2673507905 wait complete. 2024-02-22T16:10:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 -- 172.21.15.19:0/3057540136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fe8f0 msgr2=0x7fd884107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 --2- 172.21.15.19:0/3057540136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fe8f0 0x7fd884107050 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd878004970 tx=0x7fd878009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 -- 172.21.15.19:0/3057540136 shutdown_connections 2024-02-22T16:10:04.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 --2- 172.21.15.19:0/3057540136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fe8f0 0x7fd884107050 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 --2- 172.21.15.19:0/3057540136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8840fdfe0 0x7fd8840fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 -- 172.21.15.19:0/3057540136 >> 172.21.15.19:0/3057540136 conn(0x7fd884071d90 msgr2=0x7fd884072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 -- 172.21.15.19:0/3057540136 shutdown_connections 2024-02-22T16:10:04.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.342+0000 7fd889cc6700 1 -- 172.21.15.19:0/3057540136 wait complete. 2024-02-22T16:10:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.343+0000 7fd889cc6700 1 Processor -- start 2024-02-22T16:10:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.343+0000 7fd889cc6700 1 -- start start 2024-02-22T16:10:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd889cc6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 0x7fd884113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:04.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd889cc6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8840fe8f0 0x7fd884113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd889cc6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884114500 con 0x7fd8840fdfe0 2024-02-22T16:10:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd889cc6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88410d830 con 0x7fd8840fe8f0 2024-02-22T16:10:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 0x7fd884113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 0x7fd884113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49886/0 (socket says 172.21.15.19:49886) 2024-02-22T16:10:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 -- 172.21.15.19:0/2309221538 learned_addr learned my addr 172.21.15.19:0/2309221538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:04.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd882ffd700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8840fe8f0 0x7fd884113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 -- 172.21.15.19:0/2309221538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8840fe8f0 msgr2=0x7fd884113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8840fe8f0 0x7fd884113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 -- 172.21.15.19:0/2309221538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878005040 con 0x7fd8840fdfe0 2024-02-22T16:10:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd8837fe700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 0x7fd884113970 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c8c0 tx=0x7fd8740044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd87400a760 con 0x7fd8840fdfe0 2024-02-22T16:10:04.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd87400ad70 con 0x7fd8840fdfe0 2024-02-22T16:10:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88410db10 con 0x7fd8840fdfe0 2024-02-22T16:10:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8740113a0 con 0x7fd8840fdfe0 2024-02-22T16:10:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.344+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88410e020 con 0x7fd8840fdfe0 2024-02-22T16:10:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.345+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8841046c0 con 0x7fd8840fdfe0 2024-02-22T16:10:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.346+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd874012400 con 0x7fd8840fdfe0 2024-02-22T16:10:04.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.346+0000 7fd86bfff700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd86c06cc20 0x7fd86c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.346+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd87408bfa0 con 0x7fd8840fdfe0 2024-02-22T16:10:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.346+0000 7fd882ffd700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd86c06cc20 0x7fd86c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:04.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.347+0000 7fd882ffd700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd86c06cc20 0x7fd86c06f0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd878004940 tx=0x7fd878009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:04.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.351+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd874059b00 con 0x7fd8840fdfe0 2024-02-22T16:10:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.563+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd88404bed0 con 0x7fd86c06cc20 2024-02-22T16:10:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.573+0000 7fd86bfff700 1 -- 172.21.15.19:0/2309221538 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fd88404bed0 con 0x7fd86c06cc20 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (17m) 10m ago 28m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (14m) 10m ago 28m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (14m) 10m ago 27m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (16m) 10m ago 28m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (10m) 10m ago 22m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (11m) 10m ago 22m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (10m) 10m ago 22m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (10m) 10m ago 22m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (19m) 10m ago 29m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (18m) 10m ago 27m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (15m) 10m ago 30m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:10:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (15m) 10m ago 27m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (18m) 10m ago 28m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (18m) 10m ago 27m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (14m) 10m ago 26m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (13m) 10m ago 25m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (13m) 10m ago 25m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (12m) 10m ago 25m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (11m) 10m ago 24m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (11m) 10m ago 24m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:10:04.550 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (17m) 10m ago 27m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:10:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd86c06cc20 msgr2=0x7fd86c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd86c06cc20 0x7fd86c06f0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd878004940 tx=0x7fd878009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 msgr2=0x7fd884113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 0x7fd884113970 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c8c0 tx=0x7fd8740044c0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 shutdown_connections 2024-02-22T16:10:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd86c06cc20 0x7fd86c06f0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8840fdfe0 0x7fd884113970 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 --2- 172.21.15.19:0/2309221538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8840fe8f0 0x7fd884113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.575+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 >> 172.21.15.19:0/2309221538 conn(0x7fd884071d90 msgr2=0x7fd884101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.576+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 shutdown_connections 2024-02-22T16:10:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.576+0000 7fd889cc6700 1 -- 172.21.15.19:0/2309221538 wait complete. 2024-02-22T16:10:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 -- 172.21.15.19:0/3008426671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fdfe0 msgr2=0x7f9e5c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 --2- 172.21.15.19:0/3008426671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c0fe3b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c004970 tx=0x7f9e4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 -- 172.21.15.19:0/3008426671 shutdown_connections 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 --2- 172.21.15.19:0/3008426671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 --2- 172.21.15.19:0/3008426671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c0fe3b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 -- 172.21.15.19:0/3008426671 >> 172.21.15.19:0/3008426671 conn(0x7f9e5c071d90 msgr2=0x7f9e5c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 -- 172.21.15.19:0/3008426671 shutdown_connections 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.679+0000 7f9e60902700 1 -- 172.21.15.19:0/3008426671 wait complete. 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e60902700 1 Processor -- start 2024-02-22T16:10:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e60902700 1 -- start start 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e60902700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c1139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e60902700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e60902700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1145f0 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e60902700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c10d890 con 0x7f9e5c0fdfe0 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49898/0 (socket says 172.21.15.19:49898) 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 -- 172.21.15.19:0/1089749797 learned_addr learned my addr 172.21.15.19:0/1089749797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e5a59c700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c1139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 -- 172.21.15.19:0/1089749797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fdfe0 msgr2=0x7f9e5c1139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c1139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 -- 172.21.15.19:0/1089749797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c005040 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e5a59c700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c1139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.680+0000 7f9e59d9b700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c113f10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b790 tx=0x7f9e50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.681+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e5000f770 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.681+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c10db70 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.681+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e5000fd80 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.681+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e5000d310 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.681+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c10e0b0 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.682+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c1046c0 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.682+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9e50010420 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.683+0000 7f9e4affd700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e4406cc20 0x7f9e4406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:04.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.683+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9e5008c100 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.683+0000 7f9e5a59c700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e4406cc20 0x7f9e4406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.683+0000 7f9e5a59c700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e4406cc20 0x7f9e4406f0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000c00 tx=0x7f9e4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:04.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.687+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9e50059b20 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.977+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9e5c109f70 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.977+0000 7f9e4affd700 1 -- 172.21.15.19:0/1089749797 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f9e500596b0 con 0x7f9e5c0fe8f0 2024-02-22T16:10:04.953 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:10:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:10:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:10:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:10:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:10:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:10:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:10:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:10:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:10:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:10:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:10:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:10:04.956 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e4406cc20 msgr2=0x7f9e4406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e4406cc20 0x7f9e4406f0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000c00 tx=0x7f9e4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 msgr2=0x7f9e5c113f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c113f10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b790 tx=0x7f9e50009250 comp rx=0 tx=0).stop 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 shutdown_connections 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9e4406cc20 0x7f9e4406f0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e5c0fdfe0 0x7f9e5c1139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 --2- 172.21.15.19:0/1089749797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5c0fe8f0 0x7f9e5c113f10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 >> 172.21.15.19:0/1089749797 conn(0x7f9e5c071d90 msgr2=0x7f9e5c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 shutdown_connections 2024-02-22T16:10:04.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:04.979+0000 7f9e60902700 1 -- 172.21.15.19:0/1089749797 wait complete. 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 -- 172.21.15.19:0/720075825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 msgr2=0x7f63b0103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 --2- 172.21.15.19:0/720075825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0103510 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63a0004940 tx=0x7f63a0009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 -- 172.21.15.19:0/720075825 shutdown_connections 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 --2- 172.21.15.19:0/720075825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 0x7f63b00fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 --2- 172.21.15.19:0/720075825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0103510 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 -- 172.21.15.19:0/720075825 >> 172.21.15.19:0/720075825 conn(0x7f63b0071560 msgr2=0x7f63b0071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 -- 172.21.15.19:0/720075825 shutdown_connections 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 -- 172.21.15.19:0/720075825 wait complete. 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 Processor -- start 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.084+0000 7f63b4d94700 1 -- start start 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63b4d94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 0x7f63b0113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63b4d94700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63b4d94700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01146c0 con 0x7f63b0103140 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63b4d94700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b010d960 con 0x7f63b00fb570 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49914/0 (socket says 172.21.15.19:49914) 2024-02-22T16:10:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 -- 172.21.15.19:0/425043163 learned_addr learned my addr 172.21.15.19:0/425043163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63ae59c700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 0x7f63b0113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 -- 172.21.15.19:0/425043163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 msgr2=0x7f63b0113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 0x7f63b0113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 -- 172.21.15.19:0/425043163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0005040 con 0x7f63b0103140 2024-02-22T16:10:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63ae59c700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 0x7f63b0113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:10:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63add9b700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0113fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f63a400b790 tx=0x7f63a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a400f770 con 0x7f63b0103140 2024-02-22T16:10:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b010dc40 con 0x7f63b0103140 2024-02-22T16:10:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63a400fd80 con 0x7f63b0103140 2024-02-22T16:10:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.085+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a400d310 con 0x7f63b0103140 2024-02-22T16:10:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.086+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b010e180 con 0x7f63b0103140 2024-02-22T16:10:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.086+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b0073080 con 0x7f63b0103140 2024-02-22T16:10:05.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.087+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f63a4010420 con 0x7f63b0103140 2024-02-22T16:10:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.087+0000 7f639effd700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f639806cc30 0x7f639806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.088+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f63a408bf60 con 0x7f63b0103140 2024-02-22T16:10:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.088+0000 7f63ae59c700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f639806cc30 0x7f639806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.088+0000 7f63ae59c700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f639806cc30 0x7f639806f0e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63a000b3b0 tx=0x7f63a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.092+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f63a40bf960 con 0x7f63b0103140 2024-02-22T16:10:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.339+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f63b010a0f0 con 0x7f63b0103140 2024-02-22T16:10:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.339+0000 7f639effd700 1 -- 172.21.15.19:0/425043163 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f63a4059ac0 con 0x7f63b0103140 2024-02-22T16:10:05.314 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:10:05.315 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:10:05.316 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:05.317 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:05.318 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:05.318 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:05.318 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:05.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f639806cc30 msgr2=0x7f639806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f639806cc30 0x7f639806f0e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63a000b3b0 tx=0x7f63a0009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 msgr2=0x7f63b0113fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0113fe0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f63a400b790 tx=0x7f63a4009250 comp rx=0 tx=0).stop 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 shutdown_connections 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f639806cc30 0x7f639806f0e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b00fb570 0x7f63b0113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 --2- 172.21.15.19:0/425043163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63b0103140 0x7f63b0113fe0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 >> 172.21.15.19:0/425043163 conn(0x7f63b0071560 msgr2=0x7f63b00f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 shutdown_connections 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.341+0000 7f63b4d94700 1 -- 172.21.15.19:0/425043163 wait complete. 2024-02-22T16:10:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:10:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 -- 172.21.15.19:0/2224371703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 msgr2=0x7f1bc80fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/2224371703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fe3b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc004970 tx=0x7f1bbc009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 -- 172.21.15.19:0/2224371703 shutdown_connections 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/2224371703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc80fe8f0 0x7f1bc8107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/2224371703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fe3b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 -- 172.21.15.19:0/2224371703 >> 172.21.15.19:0/2224371703 conn(0x7f1bc8071d90 msgr2=0x7f1bc8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 -- 172.21.15.19:0/2224371703 shutdown_connections 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 -- 172.21.15.19:0/2224371703 wait complete. 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.446+0000 7f1bcddeb700 1 Processor -- start 2024-02-22T16:10:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bcddeb700 1 -- start start 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bcddeb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bcddeb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc80fe8f0 0x7f1bc80fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bcddeb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80fc9e0 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bcddeb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80fcb50 con 0x7f1bc80fe8f0 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49942/0 (socket says 172.21.15.19:49942) 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 -- 172.21.15.19:0/3899012369 learned_addr learned my addr 172.21.15.19:0/3899012369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc6ffd700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc80fe8f0 0x7f1bc80fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 -- 172.21.15.19:0/3899012369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc80fe8f0 msgr2=0x7f1bc80fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc80fe8f0 0x7f1bc80fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 -- 172.21.15.19:0/3899012369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc005040 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1bc77fe700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fbea0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.447+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bbc00e040 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.448+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc8119b70 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.448+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1bbc013420 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.448+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bbc01c630 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.448+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc8119fd0 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.448+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc81046c0 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.449+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1bbc0139e0 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.449+0000 7f1baffff700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1bb006cc20 0x7f1bb006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.450+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1bbc08d3d0 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.450+0000 7f1bc6ffd700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1bb006cc20 0x7f1bb006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.450+0000 7f1bc6ffd700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1bb006cc20 0x7f1bb006f0d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80fd8a0 tx=0x7f1bb800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:05.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.454+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1bbc057360 con 0x7f1bc80fdfe0 2024-02-22T16:10:05.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:05 smithi062 ceph-mon[106861]: pgmap v504: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:05.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:05 smithi062 ceph-mon[106861]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:05.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:05 smithi062 ceph-mon[106861]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:05.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:05 smithi062 ceph-mon[106861]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:05.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:05 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1089749797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:10:05.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:05 smithi019 ceph-mon[173579]: pgmap v504: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:05.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:05 smithi019 ceph-mon[173579]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:05.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:05 smithi019 ceph-mon[173579]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:05.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:05 smithi019 ceph-mon[173579]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:05.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:05 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1089749797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:10:05.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.674+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bc80fd5e0 con 0x7f1bb006cc20 2024-02-22T16:10:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.675+0000 7f1baffff700 1 -- 172.21.15.19:0/3899012369 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1bc80fd5e0 con 0x7f1bb006cc20 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:10:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:10:05.652 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:10:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.677+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1bb006cc20 msgr2=0x7f1bb006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.677+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1bb006cc20 0x7f1bb006f0d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80fd8a0 tx=0x7f1bb800b040 comp rx=0 tx=0).stop 2024-02-22T16:10:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.677+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 msgr2=0x7f1bc80fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.677+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fbea0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 shutdown_connections 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1bb006cc20 0x7f1bb006f0d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bc80fdfe0 0x7f1bc80fbea0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 --2- 172.21.15.19:0/3899012369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc80fe8f0 0x7f1bc80fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 >> 172.21.15.19:0/3899012369 conn(0x7f1bc8071d90 msgr2=0x7f1bc8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 shutdown_connections 2024-02-22T16:10:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.678+0000 7f1bcddeb700 1 -- 172.21.15.19:0/3899012369 wait complete. 2024-02-22T16:10:05.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.781+0000 7f6434e8a700 1 -- 172.21.15.19:0/1530845440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 msgr2=0x7f6430102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.781+0000 7f6434e8a700 1 --2- 172.21.15.19:0/1530845440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f6430102220 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6420004940 tx=0x7f6420009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- 172.21.15.19:0/1530845440 shutdown_connections 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 --2- 172.21.15.19:0/1530845440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64301027f0 0x7f64300fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 --2- 172.21.15.19:0/1530845440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f6430102220 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- 172.21.15.19:0/1530845440 >> 172.21.15.19:0/1530845440 conn(0x7f64300f79e0 msgr2=0x7f64300f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- 172.21.15.19:0/1530845440 shutdown_connections 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- 172.21.15.19:0/1530845440 wait complete. 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 Processor -- start 2024-02-22T16:10:05.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- start start 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f64301138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64301027f0 0x7f6430113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430114470 con 0x7f6430101e50 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.782+0000 7f6434e8a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643010d7a0 con 0x7f64301027f0 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f64301138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f64301138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49964/0 (socket says 172.21.15.19:49964) 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 -- 172.21.15.19:0/3029641508 learned_addr learned my addr 172.21.15.19:0/3029641508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642dd9b700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64301027f0 0x7f6430113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 -- 172.21.15.19:0/3029641508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64301027f0 msgr2=0x7f6430113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64301027f0 0x7f6430113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 -- 172.21.15.19:0/3029641508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420005040 con 0x7f6430101e50 2024-02-22T16:10:05.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f642e59c700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f64301138e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f642000b3b0 tx=0x7f6420009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642000e040 con 0x7f6430101e50 2024-02-22T16:10:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f642001c430 con 0x7f6430101e50 2024-02-22T16:10:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643010da20 con 0x7f6430101e50 2024-02-22T16:10:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6420013640 con 0x7f6430101e50 2024-02-22T16:10:05.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.783+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643010df60 con 0x7f6430101e50 2024-02-22T16:10:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.784+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6430064b00 con 0x7f6430101e50 2024-02-22T16:10:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.785+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f642001c5a0 con 0x7f6430101e50 2024-02-22T16:10:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.785+0000 7f641effd700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f641806cc20 0x7f641806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.785+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f642008cc00 con 0x7f6430101e50 2024-02-22T16:10:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.785+0000 7f642dd9b700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f641806cc20 0x7f641806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:05.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.786+0000 7f642dd9b700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f641806cc20 0x7f641806f0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f643010f110 tx=0x7f642400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:05.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:05.790+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f642005a6d0 con 0x7f6430101e50 2024-02-22T16:10:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.074+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f643010ef80 con 0x7f6430101e50 2024-02-22T16:10:06.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.074+0000 7f641effd700 1 -- 172.21.15.19:0/3029641508 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f642005a260 con 0x7f6430101e50 2024-02-22T16:10:06.049 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:10:06.050 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:10:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:10:06.050 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:10:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:10:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f641806cc20 msgr2=0x7f641806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f641806cc20 0x7f641806f0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f643010f110 tx=0x7f642400b040 comp rx=0 tx=0).stop 2024-02-22T16:10:06.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 msgr2=0x7f64301138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f64301138e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f642000b3b0 tx=0x7f6420009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 shutdown_connections 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f641806cc20 0x7f641806f0d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6430101e50 0x7f64301138e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 --2- 172.21.15.19:0/3029641508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64301027f0 0x7f6430113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 >> 172.21.15.19:0/3029641508 conn(0x7f64300f79e0 msgr2=0x7f6430065f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 shutdown_connections 2024-02-22T16:10:06.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:06.076+0000 7f6434e8a700 1 -- 172.21.15.19:0/3029641508 wait complete. 2024-02-22T16:10:06.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:06 smithi062 ceph-mon[106861]: pgmap v505: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:06.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:06 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/425043163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:10:06.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:06 smithi062 ceph-mon[106861]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:06.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:06 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3029641508' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:10:06.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:06 smithi019 ceph-mon[173579]: pgmap v505: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:06.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:06 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/425043163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:10:06.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:06 smithi019 ceph-mon[173579]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:06.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:06 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3029641508' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:10:09.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:09 smithi062 ceph-mon[106861]: pgmap v506: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:09.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:09 smithi019 ceph-mon[173579]: pgmap v506: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:11.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:11 smithi019 ceph-mon[173579]: pgmap v507: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:11.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:11 smithi062 ceph-mon[106861]: pgmap v507: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:13.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:13 smithi062 ceph-mon[106861]: pgmap v508: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:13.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:13 smithi019 ceph-mon[173579]: pgmap v508: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:15.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:15 smithi062 ceph-mon[106861]: pgmap v509: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:15.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:15 smithi019 ceph-mon[173579]: pgmap v509: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:17.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:17 smithi062 ceph-mon[106861]: pgmap v510: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:17.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:17 smithi019 ceph-mon[173579]: pgmap v510: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:19 smithi062 ceph-mon[106861]: pgmap v511: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:19.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:19 smithi019 ceph-mon[173579]: pgmap v511: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:21 smithi019 ceph-mon[173579]: pgmap v512: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:21 smithi062 ceph-mon[106861]: pgmap v512: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:23.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:23 smithi062 ceph-mon[106861]: pgmap v513: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:23.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:23 smithi019 ceph-mon[173579]: pgmap v513: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:25.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:25 smithi062 ceph-mon[106861]: pgmap v514: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:25.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:25 smithi019 ceph-mon[173579]: pgmap v514: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:27.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:27 smithi062 ceph-mon[106861]: pgmap v515: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:27.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:27 smithi019 ceph-mon[173579]: pgmap v515: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:29.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:29 smithi062 ceph-mon[106861]: pgmap v516: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:29.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:29 smithi019 ceph-mon[173579]: pgmap v516: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:31.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:31 smithi019 ceph-mon[173579]: pgmap v517: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:31.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:31 smithi062 ceph-mon[106861]: pgmap v517: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:33.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:33 smithi062 ceph-mon[106861]: pgmap v518: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:33.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:33 smithi019 ceph-mon[173579]: pgmap v518: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:34.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:34.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:35.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:35 smithi062 ceph-mon[106861]: pgmap v519: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:35.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:35 smithi019 ceph-mon[173579]: pgmap v519: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 -- 172.21.15.19:0/2675042272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 msgr2=0x7f0fc8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/2675042272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8102210 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4004940 tx=0x7f0fc4009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 -- 172.21.15.19:0/2675042272 shutdown_connections 2024-02-22T16:10:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/2675042272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc81027e0 0x7f0fc80fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/2675042272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8102210 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 -- 172.21.15.19:0/2675042272 >> 172.21.15.19:0/2675042272 conn(0x7f0fc8071d90 msgr2=0x7f0fc8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 -- 172.21.15.19:0/2675042272 shutdown_connections 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 -- 172.21.15.19:0/2675042272 wait complete. 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 Processor -- start 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.197+0000 7f0fcf41c700 1 -- start start 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcf41c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcf41c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc81027e0 0x7f0fc8116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcf41c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81166a0 con 0x7f0fc8101e40 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcf41c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc810f9d0 con 0x7f0fc81027e0 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54404/0 (socket says 172.21.15.19:54404) 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 -- 172.21.15.19:0/566593145 learned_addr learned my addr 172.21.15.19:0/566593145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcc9b7700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc81027e0 0x7f0fc8116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 -- 172.21.15.19:0/566593145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc81027e0 msgr2=0x7f0fc8116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc81027e0 0x7f0fc8116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 -- 172.21.15.19:0/566593145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4005040 con 0x7f0fc8101e40 2024-02-22T16:10:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fcd1b8700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8115b10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009180 tx=0x7f0fc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc400e040 con 0x7f0fc8101e40 2024-02-22T16:10:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.198+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fc4004290 con 0x7f0fc8101e40 2024-02-22T16:10:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.199+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc810fc50 con 0x7f0fc8101e40 2024-02-22T16:10:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.199+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc401c3f0 con 0x7f0fc8101e40 2024-02-22T16:10:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.199+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc8110190 con 0x7f0fc8101e40 2024-02-22T16:10:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.200+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc80738b0 con 0x7f0fc8101e40 2024-02-22T16:10:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.201+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0fc4013420 con 0x7f0fc8101e40 2024-02-22T16:10:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.201+0000 7f0fbdffb700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0fb406cc20 0x7f0fb406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.201+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0fc408d2b0 con 0x7f0fc8101e40 2024-02-22T16:10:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.201+0000 7f0fcc9b7700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0fb406cc20 0x7f0fb406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.202+0000 7f0fcc9b7700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0fb406cc20 0x7f0fb406f0d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8111340 tx=0x7f0fb800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.206+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0fc405aec0 con 0x7f0fc8101e40 2024-02-22T16:10:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.426+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fc8110400 con 0x7f0fb406cc20 2024-02-22T16:10:36.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.427+0000 7f0fbdffb700 1 -- 172.21.15.19:0/566593145 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0fc8110400 con 0x7f0fb406cc20 2024-02-22T16:10:36.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0fb406cc20 msgr2=0x7f0fb406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0fb406cc20 0x7f0fb406f0d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8111340 tx=0x7f0fb800b040 comp rx=0 tx=0).stop 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 msgr2=0x7f0fc8115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8115b10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009180 tx=0x7f0fc4009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 shutdown_connections 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0fb406cc20 0x7f0fb406f0d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc8101e40 0x7f0fc8115b10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 --2- 172.21.15.19:0/566593145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc81027e0 0x7f0fc8116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.429+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 >> 172.21.15.19:0/566593145 conn(0x7f0fc8071d90 msgr2=0x7f0fc8065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.430+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 shutdown_connections 2024-02-22T16:10:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.430+0000 7f0fcf41c700 1 -- 172.21.15.19:0/566593145 wait complete. 2024-02-22T16:10:36.415 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:10:36.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.534+0000 7f0861ba9700 1 -- 172.21.15.19:0/2751900374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fc630 msgr2=0x7f085c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.534+0000 7f0861ba9700 1 --2- 172.21.15.19:0/2751900374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fc630 0x7f085c109390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0850004940 tx=0x7f0850009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.534+0000 7f0861ba9700 1 -- 172.21.15.19:0/2751900374 shutdown_connections 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.534+0000 7f0861ba9700 1 --2- 172.21.15.19:0/2751900374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fc630 0x7f085c109390 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 --2- 172.21.15.19:0/2751900374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fbd20 0x7f085c0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 -- 172.21.15.19:0/2751900374 >> 172.21.15.19:0/2751900374 conn(0x7f085c0f79d0 msgr2=0x7f085c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 -- 172.21.15.19:0/2751900374 shutdown_connections 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 -- 172.21.15.19:0/2751900374 wait complete. 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 Processor -- start 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 -- start start 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.535+0000 7f0861ba9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 0x7f085c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0861ba9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fc630 0x7f085c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0861ba9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c116730 con 0x7f085c0fbd20 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0861ba9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c10fa60 con 0x7f085c0fc630 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 0x7f085c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 0x7f085c115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54430/0 (socket says 172.21.15.19:54430) 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 -- 172.21.15.19:0/357401256 learned_addr learned my addr 172.21.15.19:0/357401256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085affd700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fc630 0x7f085c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 -- 172.21.15.19:0/357401256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fc630 msgr2=0x7f085c1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fc630 0x7f085c1160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 -- 172.21.15.19:0/357401256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850005040 con 0x7f085c0fbd20 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085affd700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fc630 0x7f085c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f085b7fe700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 0x7f085c115ba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f084c00eb40 tx=0x7f084c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c007cc0 con 0x7f085c0fbd20 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f084c00f040 con 0x7f085c0fbd20 2024-02-22T16:10:36.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c10fce0 con 0x7f085c0fbd20 2024-02-22T16:10:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c009570 con 0x7f085c0fbd20 2024-02-22T16:10:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.536+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c110220 con 0x7f085c0fbd20 2024-02-22T16:10:36.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.537+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c106ac0 con 0x7f085c0fbd20 2024-02-22T16:10:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.538+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f084c0096d0 con 0x7f085c0fbd20 2024-02-22T16:10:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.538+0000 7f0843fff700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084406cc20 0x7f084406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.538+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f084c08bd40 con 0x7f085c0fbd20 2024-02-22T16:10:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.538+0000 7f085affd700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084406cc20 0x7f084406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.539+0000 7f085affd700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084406cc20 0x7f084406f0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0850000c00 tx=0x7f0850009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:36.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.543+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f084c059830 con 0x7f085c0fbd20 2024-02-22T16:10:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.762+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f085c04bf40 con 0x7f084406cc20 2024-02-22T16:10:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.763+0000 7f0843fff700 1 -- 172.21.15.19:0/357401256 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f085c04bf40 con 0x7f084406cc20 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084406cc20 msgr2=0x7f084406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084406cc20 0x7f084406f0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0850000c00 tx=0x7f0850009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 msgr2=0x7f085c115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 0x7f085c115ba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f084c00eb40 tx=0x7f084c00b040 comp rx=0 tx=0).stop 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 shutdown_connections 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f084406cc20 0x7f084406f0d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f085c0fbd20 0x7f085c115ba0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 --2- 172.21.15.19:0/357401256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f085c0fc630 0x7f085c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 >> 172.21.15.19:0/357401256 conn(0x7f085c0f79d0 msgr2=0x7f085c0f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 shutdown_connections 2024-02-22T16:10:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.766+0000 7f0861ba9700 1 -- 172.21.15.19:0/357401256 wait complete. 2024-02-22T16:10:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.870+0000 7fdaca1a6700 1 -- 172.21.15.19:0/1991299135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 msgr2=0x7fdac4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/1991299135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac4102210 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdab4004940 tx=0x7fdab4009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 -- 172.21.15.19:0/1991299135 shutdown_connections 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/1991299135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 0x7fdac40fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/1991299135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac4102210 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 -- 172.21.15.19:0/1991299135 >> 172.21.15.19:0/1991299135 conn(0x7fdac4071d90 msgr2=0x7fdac4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 -- 172.21.15.19:0/1991299135 shutdown_connections 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 -- 172.21.15.19:0/1991299135 wait complete. 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 Processor -- start 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.871+0000 7fdaca1a6700 1 -- start start 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaca1a6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaca1a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 0x7fdac410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaca1a6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac410f630 con 0x7fdac4101e40 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaca1a6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac410f7a0 con 0x7fdac41027e0 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac410eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54448/0 (socket says 172.21.15.19:54448) 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 -- 172.21.15.19:0/3721075164 learned_addr learned my addr 172.21.15.19:0/3721075164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac2ffd700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 0x7fdac410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 -- 172.21.15.19:0/3721075164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 msgr2=0x7fdac410f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 0x7fdac410f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 -- 172.21.15.19:0/3721075164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab4005040 con 0x7fdac4101e40 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac2ffd700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 0x7fdac410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdac37fe700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac410eb40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009180 tx=0x7fdab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab400e040 con 0x7fdac4101e40 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdab4013420 con 0x7fdac4101e40 2024-02-22T16:10:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac4117ab0 con 0x7fdac4101e40 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdab401c630 con 0x7fdac4101e40 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.872+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac4117fc0 con 0x7fdac4101e40 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.873+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac40ff150 con 0x7fdac4101e40 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.874+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdab40139e0 con 0x7fdac4101e40 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.874+0000 7fdaabfff700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdaac06cc20 0x7fdaac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.874+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdab408d330 con 0x7fdac4101e40 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.874+0000 7fdac2ffd700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdaac06cc20 0x7fdaac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.875+0000 7fdac2ffd700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdaac06cc20 0x7fdaac06f0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdac4066180 tx=0x7fdab8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:36.879+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fdab4057360 con 0x7fdac4101e40 2024-02-22T16:10:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.090+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdac4065ec0 con 0x7fdaac06cc20 2024-02-22T16:10:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.100+0000 7fdaabfff700 1 -- 172.21.15.19:0/3721075164 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fdac4065ec0 con 0x7fdaac06cc20 2024-02-22T16:10:37.075 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:10:37.075 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (17m) 11m ago 28m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:10:37.076 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (15m) 11m ago 29m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:10:37.076 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (15m) 10m ago 28m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:10:37.076 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (17m) 11m ago 28m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:10:37.076 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (11m) 11m ago 23m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:10:37.076 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (11m) 11m ago 23m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:10:37.076 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (10m) 10m ago 23m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:10:37.077 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (10m) 10m ago 23m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:10:37.077 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (19m) 11m ago 30m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:10:37.077 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (18m) 10m ago 27m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:10:37.077 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (16m) 11m ago 30m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:10:37.077 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (15m) 10m ago 27m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:10:37.077 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (18m) 11m ago 29m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:10:37.078 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (18m) 10m ago 27m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:10:37.078 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (14m) 11m ago 27m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:10:37.078 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (14m) 11m ago 26m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:10:37.078 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (13m) 11m ago 26m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:10:37.078 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (12m) 10m ago 25m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:10:37.078 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (12m) 10m ago 25m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:10:37.079 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (11m) 10m ago 24m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:10:37.079 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (18m) 11m ago 28m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:10:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdaac06cc20 msgr2=0x7fdaac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdaac06cc20 0x7fdaac06f0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdac4066180 tx=0x7fdab8009040 comp rx=0 tx=0).stop 2024-02-22T16:10:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 msgr2=0x7fdac410eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac410eb40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009180 tx=0x7fdab4009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 shutdown_connections 2024-02-22T16:10:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdaac06cc20 0x7fdaac06f0d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdac4101e40 0x7fdac410eb40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 --2- 172.21.15.19:0/3721075164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdac41027e0 0x7fdac410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 >> 172.21.15.19:0/3721075164 conn(0x7fdac4071d90 msgr2=0x7fdac40faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 shutdown_connections 2024-02-22T16:10:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.102+0000 7fdaca1a6700 1 -- 172.21.15.19:0/3721075164 wait complete. 2024-02-22T16:10:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 -- 172.21.15.19:0/2823212732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 msgr2=0x7f5b940fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 --2- 172.21.15.19:0/2823212732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b940fbe70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5b88004940 tx=0x7f5b88009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 -- 172.21.15.19:0/2823212732 shutdown_connections 2024-02-22T16:10:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 --2- 172.21.15.19:0/2823212732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b940fbe70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 --2- 172.21.15.19:0/2823212732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b94101e60 0x7f5b94102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 -- 172.21.15.19:0/2823212732 >> 172.21.15.19:0/2823212732 conn(0x7f5b940f7a10 msgr2=0x7f5b940f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 -- 172.21.15.19:0/2823212732 shutdown_connections 2024-02-22T16:10:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 -- 172.21.15.19:0/2823212732 wait complete. 2024-02-22T16:10:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 Processor -- start 2024-02-22T16:10:37.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.207+0000 7f5b99309700 1 -- start start 2024-02-22T16:10:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b99309700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b94101e60 0x7f5b9410eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b99309700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b9410eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b99309700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9410f580 con 0x7f5b94102800 2024-02-22T16:10:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b99309700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9410f6c0 con 0x7f5b94101e60 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b9410eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b9410eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54462/0 (socket says 172.21.15.19:54462) 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 -- 172.21.15.19:0/4282218253 learned_addr learned my addr 172.21.15.19:0/4282218253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b92ffd700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b94101e60 0x7f5b9410eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 -- 172.21.15.19:0/4282218253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b94101e60 msgr2=0x7f5b9410eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b94101e60 0x7f5b9410eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 -- 172.21.15.19:0/4282218253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88005040 con 0x7f5b94102800 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b927fc700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b9410eff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009380 tx=0x7f5b88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b8800e040 con 0x7f5b94102800 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b941179c0 con 0x7f5b94102800 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b88013420 con 0x7f5b94102800 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b8801c630 con 0x7f5b94102800 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.208+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b94117ed0 con 0x7f5b94102800 2024-02-22T16:10:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.209+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94104330 con 0x7f5b94102800 2024-02-22T16:10:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.210+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b88013590 con 0x7f5b94102800 2024-02-22T16:10:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.210+0000 7f5b7b7fe700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7c06cc20 0x7f5b7c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.211+0000 7f5b92ffd700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7c06cc20 0x7f5b7c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.211+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5b8808d2d0 con 0x7f5b94102800 2024-02-22T16:10:37.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.211+0000 7f5b92ffd700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7c06cc20 0x7f5b7c06f0d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b84004f40 tx=0x7f5b8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.216+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5b88057300 con 0x7f5b94102800 2024-02-22T16:10:37.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:37 smithi062 ceph-mon[106861]: pgmap v520: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.507+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5b9404bf00 con 0x7f5b94102800 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.508+0000 7f5b7b7fe700 1 -- 172.21.15.19:0/4282218253 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f5b8805a920 con 0x7f5b94102800 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:37.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:10:37.484 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7c06cc20 msgr2=0x7f5b7c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7c06cc20 0x7f5b7c06f0d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b84004f40 tx=0x7f5b8400b040 comp rx=0 tx=0).stop 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 msgr2=0x7f5b9410eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b9410eff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009380 tx=0x7f5b88009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 shutdown_connections 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5b7c06cc20 0x7f5b7c06f0d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b94101e60 0x7f5b9410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 --2- 172.21.15.19:0/4282218253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b94102800 0x7f5b9410eff0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 >> 172.21.15.19:0/4282218253 conn(0x7f5b940f7a10 msgr2=0x7f5b940f96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 shutdown_connections 2024-02-22T16:10:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.510+0000 7f5b99309700 1 -- 172.21.15.19:0/4282218253 wait complete. 2024-02-22T16:10:37.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:37 smithi019 ceph-mon[173579]: pgmap v520: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 -- 172.21.15.19:0/177622116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 msgr2=0x7f4d841021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 --2- 172.21.15.19:0/177622116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d841021b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4d74004940 tx=0x7f4d74009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 -- 172.21.15.19:0/177622116 shutdown_connections 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 --2- 172.21.15.19:0/177622116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d84102780 0x7f4d840fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 --2- 172.21.15.19:0/177622116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d841021b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 -- 172.21.15.19:0/177622116 >> 172.21.15.19:0/177622116 conn(0x7f4d840f7970 msgr2=0x7f4d840f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 -- 172.21.15.19:0/177622116 shutdown_connections 2024-02-22T16:10:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.614+0000 7f4d89826700 1 -- 172.21.15.19:0/177622116 wait complete. 2024-02-22T16:10:37.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.615+0000 7f4d89826700 1 Processor -- start 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.615+0000 7f4d89826700 1 -- start start 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.615+0000 7f4d89826700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d84112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.615+0000 7f4d89826700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d84102780 0x7f4d84112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.615+0000 7f4d89826700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d84113390 con 0x7f4d84101de0 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.615+0000 7f4d89826700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8410c6c0 con 0x7f4d84102780 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d84112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d84112800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60492/0 (socket says 172.21.15.19:60492) 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 -- 172.21.15.19:0/1311401826 learned_addr learned my addr 172.21.15.19:0/1311401826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 -- 172.21.15.19:0/1311401826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d84102780 msgr2=0x7f4d84112d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d827fc700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d84102780 0x7f4d84112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d84102780 0x7f4d84112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 -- 172.21.15.19:0/1311401826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74005040 con 0x7f4d84101de0 2024-02-22T16:10:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d82ffd700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d84112800 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b3b0 tx=0x7f4d74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d7400e040 con 0x7f4d84101de0 2024-02-22T16:10:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8410c940 con 0x7f4d84101de0 2024-02-22T16:10:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d74009af0 con 0x7f4d84101de0 2024-02-22T16:10:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d74013770 con 0x7f4d84101de0 2024-02-22T16:10:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.616+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8410cea0 con 0x7f4d84101de0 2024-02-22T16:10:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.617+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d840fb570 con 0x7f4d84101de0 2024-02-22T16:10:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.618+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d7401e420 con 0x7f4d84101de0 2024-02-22T16:10:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.618+0000 7f4d6b7fe700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d6c06cc20 0x7f4d6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.618+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4d74090120 con 0x7f4d84101de0 2024-02-22T16:10:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.618+0000 7f4d827fc700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d6c06cc20 0x7f4d6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.619+0000 7f4d827fc700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d6c06cc20 0x7f4d6c06f0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d8410e030 tx=0x7f4d78009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:37.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.623+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f4d7405dac0 con 0x7f4d84101de0 2024-02-22T16:10:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.870+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d8410dde0 con 0x7f4d84101de0 2024-02-22T16:10:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.870+0000 7f4d6b7fe700 1 -- 172.21.15.19:0/1311401826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f4d7405d650 con 0x7f4d84101de0 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:10:37.846 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:10:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:37.848 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:10:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d6c06cc20 msgr2=0x7f4d6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d6c06cc20 0x7f4d6c06f0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d8410e030 tx=0x7f4d78009040 comp rx=0 tx=0).stop 2024-02-22T16:10:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 msgr2=0x7f4d84112800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d84112800 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b3b0 tx=0x7f4d74009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 shutdown_connections 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f4d6c06cc20 0x7f4d6c06f0d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d84101de0 0x7f4d84112800 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 --2- 172.21.15.19:0/1311401826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d84102780 0x7f4d84112d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.872+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 >> 172.21.15.19:0/1311401826 conn(0x7f4d840f7970 msgr2=0x7f4d840f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.873+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 shutdown_connections 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.873+0000 7f4d89826700 1 -- 172.21.15.19:0/1311401826 wait complete. 2024-02-22T16:10:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:10:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 -- 172.21.15.19:0/1113494298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c103140 msgr2=0x7f082c103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1113494298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c103140 0x7f082c103510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0820004940 tx=0x7f0820009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:37.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 -- 172.21.15.19:0/1113494298 shutdown_connections 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1113494298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f082c0fb570 0x7f082c0fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1113494298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c103140 0x7f082c103510 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 -- 172.21.15.19:0/1113494298 >> 172.21.15.19:0/1113494298 conn(0x7f082c071560 msgr2=0x7f082c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 -- 172.21.15.19:0/1113494298 shutdown_connections 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 -- 172.21.15.19:0/1113494298 wait complete. 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 Processor -- start 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.978+0000 7f0830ba7700 1 -- start start 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0830ba7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 0x7f082c115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0830ba7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f082c103140 0x7f082c1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0830ba7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c116800 con 0x7f082c0fb570 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0830ba7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c10fb30 con 0x7f082c103140 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 0x7f082c115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 0x7f082c115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60514/0 (socket says 172.21.15.19:60514) 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 -- 172.21.15.19:0/1054336733 learned_addr learned my addr 172.21.15.19:0/1054336733 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0829d9b700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f082c103140 0x7f082c1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 -- 172.21.15.19:0/1054336733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f082c103140 msgr2=0x7f082c1161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f082c103140 0x7f082c1161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 -- 172.21.15.19:0/1054336733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820005040 con 0x7f082c0fb570 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f082a59c700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 0x7f082c115c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f082000b3b0 tx=0x7f0820009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f082000e040 con 0x7f082c0fb570 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f082000bc50 con 0x7f082c0fb570 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c10fdb0 con 0x7f082c0fb570 2024-02-22T16:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0820013420 con 0x7f082c0fb570 2024-02-22T16:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.979+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c1102f0 con 0x7f082c0fb570 2024-02-22T16:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.980+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c073080 con 0x7f082c0fb570 2024-02-22T16:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.981+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f082001c430 con 0x7f082c0fb570 2024-02-22T16:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.981+0000 7f081affd700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f081406cc20 0x7f081406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.981+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f082008ce70 con 0x7f082c0fb570 2024-02-22T16:10:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.981+0000 7f0829d9b700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f081406cc20 0x7f081406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.982+0000 7f0829d9b700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f081406cc20 0x7f081406f0d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f082c1114a0 tx=0x7f081c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:37.986+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f082005a9e0 con 0x7f082c0fb570 2024-02-22T16:10:38.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.206+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f082c1111e0 con 0x7f081406cc20 2024-02-22T16:10:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.208+0000 7f081affd700 1 -- 172.21.15.19:0/1054336733 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f082c1111e0 con 0x7f081406cc20 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:10:38.183 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.209+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f081406cc20 msgr2=0x7f081406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f081406cc20 0x7f081406f0d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f082c1114a0 tx=0x7f081c009ee0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 msgr2=0x7f082c115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 0x7f082c115c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f082000b3b0 tx=0x7f0820009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 shutdown_connections 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f081406cc20 0x7f081406f0d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f082c0fb570 0x7f082c115c70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 --2- 172.21.15.19:0/1054336733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f082c103140 0x7f082c1161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 >> 172.21.15.19:0/1054336733 conn(0x7f082c071560 msgr2=0x7f082c0fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 shutdown_connections 2024-02-22T16:10:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.210+0000 7f0830ba7700 1 -- 172.21.15.19:0/1054336733 wait complete. 2024-02-22T16:10:38.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 -- 172.21.15.19:0/22478067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 msgr2=0x7fb02c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:38.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 --2- 172.21.15.19:0/22478067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c107050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb020004970 tx=0x7fb020009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:38.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 -- 172.21.15.19:0/22478067 shutdown_connections 2024-02-22T16:10:38.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 --2- 172.21.15.19:0/22478067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c107050 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 --2- 172.21.15.19:0/22478067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 -- 172.21.15.19:0/22478067 >> 172.21.15.19:0/22478067 conn(0x7fb02c071d90 msgr2=0x7fb02c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:38.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 -- 172.21.15.19:0/22478067 shutdown_connections 2024-02-22T16:10:38.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 -- 172.21.15.19:0/22478067 wait complete. 2024-02-22T16:10:38.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.314+0000 7fb03120e700 1 Processor -- start 2024-02-22T16:10:38.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb03120e700 1 -- start start 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb03120e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb03120e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb03120e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c114500 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb03120e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c10d830 con 0x7fb02c0fdfe0 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60524/0 (socket says 172.21.15.19:60524) 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 -- 172.21.15.19:0/1316632706 learned_addr learned my addr 172.21.15.19:0/1316632706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:10:38.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02ad9d700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 -- 172.21.15.19:0/1316632706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 msgr2=0x7fb02c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 -- 172.21.15.19:0/1316632706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020005040 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.315+0000 7fb02a59c700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113eb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb020000c00 tx=0x7fb020009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.316+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02000e040 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.316+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb020013420 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.316+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c10dab0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.316+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02001c630 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.316+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c10dff0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.317+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c1046c0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.317+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb0200139e0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.317+0000 7fb0137fe700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01406cc20 0x7fb01406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:10:38.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.318+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb02008d330 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.318+0000 7fb02ad9d700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01406cc20 0x7fb01406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:10:38.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.318+0000 7fb02ad9d700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01406cc20 0x7fb01406f0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb01c004f40 tx=0x7fb01c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:10:38.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.322+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb02005adf0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:38 smithi062 ceph-mon[106861]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:38.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:38 smithi062 ceph-mon[106861]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:38.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:38 smithi062 ceph-mon[106861]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:38.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:38 smithi062 ceph-mon[106861]: pgmap v521: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:38.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:38 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4282218253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:10:38.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:38 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1311401826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:10:38.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:38 smithi019 ceph-mon[173579]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:38.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:38 smithi019 ceph-mon[173579]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:38.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:38 smithi019 ceph-mon[173579]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:38.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:38 smithi019 ceph-mon[173579]: pgmap v521: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:38.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:38 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4282218253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:10:38.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:38 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1311401826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:10:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.607+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb02c10efc0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.607+0000 7fb0137fe700 1 -- 172.21.15.19:0/1316632706 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb02c10efc0 con 0x7fb02c0fe8f0 2024-02-22T16:10:38.583 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:10:38.583 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:10:38.583 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:10:38.583 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:10:38.583 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:10:38.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01406cc20 msgr2=0x7fb01406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:38.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01406cc20 0x7fb01406f0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb01c004f40 tx=0x7fb01c00b040 comp rx=0 tx=0).stop 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 msgr2=0x7fb02c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113eb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb020000c00 tx=0x7fb020009e30 comp rx=0 tx=0).stop 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 shutdown_connections 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb01406cc20 0x7fb01406f0d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb02c0fdfe0 0x7fb02c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 --2- 172.21.15.19:0/1316632706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb02c0fe8f0 0x7fb02c113eb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 >> 172.21.15.19:0/1316632706 conn(0x7fb02c071d90 msgr2=0x7fb02c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 shutdown_connections 2024-02-22T16:10:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:10:38.609+0000 7fb03120e700 1 -- 172.21.15.19:0/1316632706 wait complete. 2024-02-22T16:10:39.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:39 smithi062 ceph-mon[106861]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:39.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:39 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1316632706' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:10:39.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:39 smithi019 ceph-mon[173579]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:10:39.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:39 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1316632706' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:10:40.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:40 smithi062 ceph-mon[106861]: pgmap v522: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:40.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:40 smithi019 ceph-mon[173579]: pgmap v522: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:42.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:42 smithi062 ceph-mon[106861]: pgmap v523: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:42.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:42 smithi019 ceph-mon[173579]: pgmap v523: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:45.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:45 smithi062 ceph-mon[106861]: pgmap v524: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:45.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:45 smithi019 ceph-mon[173579]: pgmap v524: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:47.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:47 smithi062 ceph-mon[106861]: pgmap v525: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:47.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:47 smithi019 ceph-mon[173579]: pgmap v525: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:49 smithi062 ceph-mon[106861]: pgmap v526: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:49.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:49 smithi019 ceph-mon[173579]: pgmap v526: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:49.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:10:51.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:51 smithi019 ceph-mon[173579]: pgmap v527: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:51.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:51 smithi062 ceph-mon[106861]: pgmap v527: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:53.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:53 smithi062 ceph-mon[106861]: pgmap v528: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:53.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:53 smithi019 ceph-mon[173579]: pgmap v528: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:55.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:55 smithi062 ceph-mon[106861]: pgmap v529: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:55.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:55 smithi019 ceph-mon[173579]: pgmap v529: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:57 smithi062 ceph-mon[106861]: pgmap v530: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:57 smithi019 ceph-mon[173579]: pgmap v530: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:10:59 smithi062 ceph-mon[106861]: pgmap v531: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:10:59.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:10:59 smithi019 ceph-mon[173579]: pgmap v531: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:01.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:01 smithi019 ceph-mon[173579]: pgmap v532: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:01.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:01 smithi062 ceph-mon[106861]: pgmap v532: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:03.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:03 smithi062 ceph-mon[106861]: pgmap v533: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:03.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:03 smithi019 ceph-mon[173579]: pgmap v533: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:05.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:05 smithi062 ceph-mon[106861]: pgmap v534: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:05.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:05 smithi019 ceph-mon[173579]: pgmap v534: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:07.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:07 smithi062 ceph-mon[106861]: pgmap v535: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:07.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:07 smithi019 ceph-mon[173579]: pgmap v535: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.727+0000 7f8292aab700 1 -- 172.21.15.19:0/1846193794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c1038e0 msgr2=0x7f828c0fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.727+0000 7f8292aab700 1 --2- 172.21.15.19:0/1846193794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c1038e0 0x7f828c0fcf50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8278004940 tx=0x7f8278009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 -- 172.21.15.19:0/1846193794 shutdown_connections 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 --2- 172.21.15.19:0/1846193794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c1038e0 0x7f828c0fcf50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 --2- 172.21.15.19:0/1846193794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c102f40 0x7f828c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 -- 172.21.15.19:0/1846193794 >> 172.21.15.19:0/1846193794 conn(0x7f828c0f8af0 msgr2=0x7f828c0faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 -- 172.21.15.19:0/1846193794 shutdown_connections 2024-02-22T16:11:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 -- 172.21.15.19:0/1846193794 wait complete. 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 Processor -- start 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.728+0000 7f8292aab700 1 -- start start 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8292aab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 0x7f828c070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8292aab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c1038e0 0x7f828c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8292aab700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c070d60 con 0x7f828c102f40 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8292aab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c070ed0 con 0x7f828c1038e0 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 0x7f828c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 0x7f828c070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33434/0 (socket says 172.21.15.19:33434) 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 -- 172.21.15.19:0/2230441284 learned_addr learned my addr 172.21.15.19:0/2230441284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f828bfff700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c1038e0 0x7f828c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 -- 172.21.15.19:0/2230441284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c1038e0 msgr2=0x7f828c0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c1038e0 0x7f828c0707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 -- 172.21.15.19:0/2230441284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278005040 con 0x7f828c102f40 2024-02-22T16:11:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f828bfff700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c1038e0 0x7f828c0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:11:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8290847700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 0x7f828c070290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8280002a80 tx=0x7f82800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8280007cc0 con 0x7f828c102f40 2024-02-22T16:11:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c119b60 con 0x7f828c102f40 2024-02-22T16:11:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f828000f040 con 0x7f828c102f40 2024-02-22T16:11:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.729+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8280009640 con 0x7f828c102f40 2024-02-22T16:11:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.730+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c119fc0 con 0x7f828c102f40 2024-02-22T16:11:08.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.731+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c105410 con 0x7f828c102f40 2024-02-22T16:11:08.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.731+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f82800097a0 con 0x7f828c102f40 2024-02-22T16:11:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.731+0000 7f82897fa700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f827c06cc20 0x7f827c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.731+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f828008b840 con 0x7f828c102f40 2024-02-22T16:11:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.732+0000 7f828bfff700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f827c06cc20 0x7f827c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.732+0000 7f828bfff700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f827c06cc20 0x7f827c06f0d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f827800b3b0 tx=0x7f8278009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:08.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.736+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f82800557f0 con 0x7f828c102f40 2024-02-22T16:11:08.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.961+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f828c0719d0 con 0x7f827c06cc20 2024-02-22T16:11:08.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.962+0000 7f82897fa700 1 -- 172.21.15.19:0/2230441284 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f828c0719d0 con 0x7f827c06cc20 2024-02-22T16:11:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f827c06cc20 msgr2=0x7f827c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f827c06cc20 0x7f827c06f0d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f827800b3b0 tx=0x7f8278009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 msgr2=0x7f828c070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 0x7f828c070290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8280002a80 tx=0x7f82800046b0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 shutdown_connections 2024-02-22T16:11:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f827c06cc20 0x7f827c06f0d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f828c102f40 0x7f828c070290 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 --2- 172.21.15.19:0/2230441284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f828c1038e0 0x7f828c0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 >> 172.21.15.19:0/2230441284 conn(0x7f828c0f8af0 msgr2=0x7f828c0fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 shutdown_connections 2024-02-22T16:11:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:08.964+0000 7f8292aab700 1 -- 172.21.15.19:0/2230441284 wait complete. 2024-02-22T16:11:08.950 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:11:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 -- 172.21.15.19:0/3339426558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fdfe0 msgr2=0x7fd78c0fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 --2- 172.21.15.19:0/3339426558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fdfe0 0x7fd78c0fe3b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd77c004970 tx=0x7fd77c009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:09.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 -- 172.21.15.19:0/3339426558 shutdown_connections 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 --2- 172.21.15.19:0/3339426558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 --2- 172.21.15.19:0/3339426558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fdfe0 0x7fd78c0fe3b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 -- 172.21.15.19:0/3339426558 >> 172.21.15.19:0/3339426558 conn(0x7fd78c071d90 msgr2=0x7fd78c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 -- 172.21.15.19:0/3339426558 shutdown_connections 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 -- 172.21.15.19:0/3339426558 wait complete. 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.070+0000 7fd792132700 1 Processor -- start 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd792132700 1 -- start start 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd792132700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78c0fdfe0 0x7fd78c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd792132700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd792132700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1144c0 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd792132700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c10d7f0 con 0x7fd78c0fdfe0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33456/0 (socket says 172.21.15.19:33456) 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 -- 172.21.15.19:0/1606720623 learned_addr learned my addr 172.21.15.19:0/1606720623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78b7fe700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78c0fdfe0 0x7fd78c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 -- 172.21.15.19:0/1606720623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78c0fdfe0 msgr2=0x7fd78c113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78c0fdfe0 0x7fd78c113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 -- 172.21.15.19:0/1606720623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c005040 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.071+0000 7fd78affd700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c113e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd77400eb40 tx=0x7fd77400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.072+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd774007cc0 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.072+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd77400f040 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.072+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c10da70 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.072+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77400c990 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.072+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c10dfb0 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.073+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c1046c0 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.073+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd77400cc00 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.074+0000 7fd783fff700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77806cc30 0x7fd77806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.074+0000 7fd78b7fe700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77806cc30 0x7fd77806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.074+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd77408bc70 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.074+0000 7fd78b7fe700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77806cc30 0x7fd77806f0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd77c000c00 tx=0x7fd77c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:09.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.079+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd774059760 con 0x7fd78c0fe8f0 2024-02-22T16:11:09.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.302+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd78c109ef0 con 0x7fd77806cc30 2024-02-22T16:11:09.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.303+0000 7fd783fff700 1 -- 172.21.15.19:0/1606720623 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd78c109ef0 con 0x7fd77806cc30 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77806cc30 msgr2=0x7fd77806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77806cc30 0x7fd77806f0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd77c000c00 tx=0x7fd77c009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 msgr2=0x7fd78c113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c113e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd77400eb40 tx=0x7fd77400b040 comp rx=0 tx=0).stop 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 shutdown_connections 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77806cc30 0x7fd77806f0e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd78c0fdfe0 0x7fd78c113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 --2- 172.21.15.19:0/1606720623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd78c0fe8f0 0x7fd78c113e70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 >> 172.21.15.19:0/1606720623 conn(0x7fd78c071d90 msgr2=0x7fd78c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:09.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 shutdown_connections 2024-02-22T16:11:09.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.305+0000 7fd792132700 1 -- 172.21.15.19:0/1606720623 wait complete. 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.409+0000 7f7d15589700 1 -- 172.21.15.19:0/1822169515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 msgr2=0x7f7d100fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.409+0000 7f7d15589700 1 --2- 172.21.15.19:0/1822169515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d100fb9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7d04004940 tx=0x7f7d04009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.409+0000 7f7d15589700 1 -- 172.21.15.19:0/1822169515 shutdown_connections 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.409+0000 7f7d15589700 1 --2- 172.21.15.19:0/1822169515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d100fb9c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.409+0000 7f7d15589700 1 --2- 172.21.15.19:0/1822169515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10103140 0x7f7d10103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.409+0000 7f7d15589700 1 -- 172.21.15.19:0/1822169515 >> 172.21.15.19:0/1822169515 conn(0x7f7d10071560 msgr2=0x7f7d10071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 -- 172.21.15.19:0/1822169515 shutdown_connections 2024-02-22T16:11:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 -- 172.21.15.19:0/1822169515 wait complete. 2024-02-22T16:11:09.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 Processor -- start 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 -- start start 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d10113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10103140 0x7f7d10113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10114600 con 0x7f7d100fb570 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.410+0000 7f7d15589700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1010d930 con 0x7f7d10103140 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d10113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d10113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33478/0 (socket says 172.21.15.19:33478) 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 -- 172.21.15.19:0/2595720326 learned_addr learned my addr 172.21.15.19:0/2595720326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0e7fc700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10103140 0x7f7d10113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 -- 172.21.15.19:0/2595720326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10103140 msgr2=0x7f7d10113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10103140 0x7f7d10113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 -- 172.21.15.19:0/2595720326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04005040 con 0x7f7d100fb570 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d0effd700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d10113a70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7d00002a50 tx=0x7f7d000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d00007cc0 con 0x7f7d100fb570 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d0000f040 con 0x7f7d100fb570 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1010dc10 con 0x7f7d100fb570 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0000d9c0 con 0x7f7d100fb570 2024-02-22T16:11:09.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.411+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1010e150 con 0x7f7d100fb570 2024-02-22T16:11:09.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.412+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d101054d0 con 0x7f7d100fb570 2024-02-22T16:11:09.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.413+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7d0000db20 con 0x7f7d100fb570 2024-02-22T16:11:09.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.413+0000 7f7cf77fe700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cf806cc30 0x7f7cf806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.413+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7d0008bcc0 con 0x7f7d100fb570 2024-02-22T16:11:09.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.413+0000 7f7d0e7fc700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cf806cc30 0x7f7cf806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.414+0000 7f7d0e7fc700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cf806cc30 0x7f7cf806f0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d1010f2a0 tx=0x7f7d04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:09.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.418+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7d00059780 con 0x7f7d100fb570 2024-02-22T16:11:09.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:09 smithi062 ceph-mon[106861]: pgmap v536: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:09.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:09 smithi019 ceph-mon[173579]: pgmap v536: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.627+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1004bed0 con 0x7f7cf806cc30 2024-02-22T16:11:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.637+0000 7f7cf77fe700 1 -- 172.21.15.19:0/2595720326 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f7d1004bed0 con 0x7f7cf806cc30 2024-02-22T16:11:09.614 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:11:09.614 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (18m) 11m ago 29m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:11:09.614 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (15m) 11m ago 29m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:11:09.614 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (15m) 11m ago 28m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:11:09.614 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (18m) 11m ago 29m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (11m) 11m ago 23m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (12m) 11m ago 23m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (11m) 11m ago 23m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (11m) 11m ago 23m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (20m) 11m ago 30m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (19m) 11m ago 28m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:11:09.615 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (16m) 11m ago 31m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:11:09.616 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (16m) 11m ago 28m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:11:09.616 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (19m) 11m ago 29m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:11:09.616 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (19m) 11m ago 28m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:11:09.616 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (15m) 11m ago 27m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:11:09.616 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (14m) 11m ago 27m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:11:09.616 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (14m) 11m ago 26m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:11:09.617 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (13m) 11m ago 26m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:11:09.617 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (12m) 11m ago 25m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:11:09.617 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (12m) 11m ago 25m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:11:09.617 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (18m) 11m ago 28m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:11:09.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.639+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cf806cc30 msgr2=0x7f7cf806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.639+0000 7f7d15589700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cf806cc30 0x7f7cf806f0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d1010f2a0 tx=0x7f7d04009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:09.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 msgr2=0x7f7d10113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d10113a70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7d00002a50 tx=0x7f7d000046b0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 shutdown_connections 2024-02-22T16:11:09.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cf806cc30 0x7f7cf806f0e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d100fb570 0x7f7d10113a70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 --2- 172.21.15.19:0/2595720326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d10103140 0x7f7d10113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 >> 172.21.15.19:0/2595720326 conn(0x7f7d10071560 msgr2=0x7f7d100f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 shutdown_connections 2024-02-22T16:11:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.640+0000 7f7d15589700 1 -- 172.21.15.19:0/2595720326 wait complete. 2024-02-22T16:11:09.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 -- 172.21.15.19:0/987843984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc101e40 msgr2=0x7feefc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 --2- 172.21.15.19:0/987843984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc101e40 0x7feefc102210 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feeec004940 tx=0x7feeec009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 -- 172.21.15.19:0/987843984 shutdown_connections 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 --2- 172.21.15.19:0/987843984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc1027e0 0x7feefc0fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 --2- 172.21.15.19:0/987843984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc101e40 0x7feefc102210 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 -- 172.21.15.19:0/987843984 >> 172.21.15.19:0/987843984 conn(0x7feefc071d90 msgr2=0x7feefc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 -- 172.21.15.19:0/987843984 shutdown_connections 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.742+0000 7fef02882700 1 -- 172.21.15.19:0/987843984 wait complete. 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7fef02882700 1 Processor -- start 2024-02-22T16:11:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7fef02882700 1 -- start start 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7fef02882700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc101e40 0x7feefc113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7fef02882700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 0x7feefc113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7fef02882700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1144f0 con 0x7feefc1027e0 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7fef02882700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc10d820 con 0x7feefc101e40 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 0x7feefc113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 0x7feefc113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33496/0 (socket says 172.21.15.19:33496) 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefb7fe700 1 -- 172.21.15.19:0/2786434354 learned_addr learned my addr 172.21.15.19:0/2786434354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefbfff700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc101e40 0x7feefc113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefb7fe700 1 -- 172.21.15.19:0/2786434354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc101e40 msgr2=0x7feefc113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefb7fe700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc101e40 0x7feefc113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.743+0000 7feefb7fe700 1 -- 172.21.15.19:0/2786434354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec005040 con 0x7feefc1027e0 2024-02-22T16:11:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.744+0000 7feefb7fe700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 0x7feefc113ea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feef000eb40 tx=0x7feef000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.744+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feef0007cc0 con 0x7feefc1027e0 2024-02-22T16:11:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.744+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feef000f040 con 0x7feefc1027e0 2024-02-22T16:11:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.744+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc10daa0 con 0x7feefc1027e0 2024-02-22T16:11:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.744+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feef000c8f0 con 0x7feefc1027e0 2024-02-22T16:11:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.744+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc10dfe0 con 0x7feefc1027e0 2024-02-22T16:11:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.745+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0738b0 con 0x7feefc1027e0 2024-02-22T16:11:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.746+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feef00090d0 con 0x7feefc1027e0 2024-02-22T16:11:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.746+0000 7feef8ff9700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feee406cc20 0x7feee406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.746+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7feef008b9f0 con 0x7feefc1027e0 2024-02-22T16:11:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.746+0000 7feefbfff700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feee406cc20 0x7feee406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.747+0000 7feefbfff700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feee406cc20 0x7feee406f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:09.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:09.751+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7feef008de60 con 0x7feefc1027e0 2024-02-22T16:11:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.054+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feefc04bed0 con 0x7feefc1027e0 2024-02-22T16:11:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.055+0000 7feef8ff9700 1 -- 172.21.15.19:0/2786434354 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7feef0059560 con 0x7feefc1027e0 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:11:10.032 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:11:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feee406cc20 msgr2=0x7feee406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feee406cc20 0x7feee406f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 msgr2=0x7feefc113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 0x7feefc113ea0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feef000eb40 tx=0x7feef000b040 comp rx=0 tx=0).stop 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 shutdown_connections 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7feee406cc20 0x7feee406f0d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feefc101e40 0x7feefc113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 --2- 172.21.15.19:0/2786434354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feefc1027e0 0x7feefc113ea0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 >> 172.21.15.19:0/2786434354 conn(0x7feefc071d90 msgr2=0x7feefc1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 shutdown_connections 2024-02-22T16:11:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.057+0000 7fef02882700 1 -- 172.21.15.19:0/2786434354 wait complete. 2024-02-22T16:11:10.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.163+0000 7f47cd960700 1 -- 172.21.15.19:0/3872492933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102f50 msgr2=0x7f47c8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.163+0000 7f47cd960700 1 --2- 172.21.15.19:0/3872492933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102f50 0x7f47c8103320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f47b8004940 tx=0x7f47b8009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.163+0000 7f47cd960700 1 -- 172.21.15.19:0/3872492933 shutdown_connections 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.163+0000 7f47cd960700 1 --2- 172.21.15.19:0/3872492933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c81038f0 0x7f47c80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.163+0000 7f47cd960700 1 --2- 172.21.15.19:0/3872492933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c8102f50 0x7f47c8103320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.163+0000 7f47cd960700 1 -- 172.21.15.19:0/3872492933 >> 172.21.15.19:0/3872492933 conn(0x7f47c80f8ae0 msgr2=0x7f47c80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.164+0000 7f47cd960700 1 -- 172.21.15.19:0/3872492933 shutdown_connections 2024-02-22T16:11:10.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.164+0000 7f47cd960700 1 -- 172.21.15.19:0/3872492933 wait complete. 2024-02-22T16:11:10.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.164+0000 7f47cd960700 1 Processor -- start 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47cd960700 1 -- start start 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47cd960700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c8102f50 0x7f47c8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47cd960700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 0x7f47c8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47cd960700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8114500 con 0x7f47c81038f0 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47cd960700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c810d830 con 0x7f47c8102f50 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 0x7f47c8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 0x7f47c8113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33514/0 (socket says 172.21.15.19:33514) 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 -- 172.21.15.19:0/103545156 learned_addr learned my addr 172.21.15.19:0/103545156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c6ffd700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c8102f50 0x7f47c8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 -- 172.21.15.19:0/103545156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c8102f50 msgr2=0x7f47c8113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c8102f50 0x7f47c8113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 -- 172.21.15.19:0/103545156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8005040 con 0x7f47c81038f0 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.165+0000 7f47c67fc700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 0x7f47c8113eb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f47bc002a50 tx=0x7f47bc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.166+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47bc007cc0 con 0x7f47c81038f0 2024-02-22T16:11:10.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.166+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47bc00f040 con 0x7f47c81038f0 2024-02-22T16:11:10.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.166+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c810db10 con 0x7f47c81038f0 2024-02-22T16:11:10.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.166+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47bc00d920 con 0x7f47c81038f0 2024-02-22T16:11:10.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.166+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c810e050 con 0x7f47c81038f0 2024-02-22T16:11:10.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.167+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c8100260 con 0x7f47c81038f0 2024-02-22T16:11:10.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.167+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f47bc009680 con 0x7f47c81038f0 2024-02-22T16:11:10.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.167+0000 7f47af7fe700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f47b006cc20 0x7f47b006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.168+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f47bc08bd20 con 0x7f47c81038f0 2024-02-22T16:11:10.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.168+0000 7f47c6ffd700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f47b006cc20 0x7f47b006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.168+0000 7f47c6ffd700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f47b006cc20 0x7f47b006f0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47b8004f90 tx=0x7f47b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.172+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f47bc0597e0 con 0x7f47c81038f0 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.425+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47c804bed0 con 0x7f47c81038f0 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.425+0000 7f47af7fe700 1 -- 172.21.15.19:0/103545156 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f47bc059370 con 0x7f47c81038f0 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:10.401 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:11:10.402 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:10.403 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:10.404 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:10.404 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:10.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f47b006cc20 msgr2=0x7f47b006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f47b006cc20 0x7f47b006f0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47b8004f90 tx=0x7f47b8009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 msgr2=0x7f47c8113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 0x7f47c8113eb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f47bc002a50 tx=0x7f47bc004600 comp rx=0 tx=0).stop 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 shutdown_connections 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f47b006cc20 0x7f47b006f0d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c8102f50 0x7f47c8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 --2- 172.21.15.19:0/103545156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47c81038f0 0x7f47c8113eb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 >> 172.21.15.19:0/103545156 conn(0x7f47c80f8ae0 msgr2=0x7f47c80fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 shutdown_connections 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.427+0000 7f47cd960700 1 -- 172.21.15.19:0/103545156 wait complete. 2024-02-22T16:11:10.405 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:11:10.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:10 smithi062 ceph-mon[106861]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:10.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:10 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2786434354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:11:10.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:10 smithi019 ceph-mon[173579]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:10.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:10 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2786434354' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:11:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 -- 172.21.15.19:0/4040163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe201027e0 msgr2=0x7efe200fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 --2- 172.21.15.19:0/4040163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe201027e0 0x7efe200fbea0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004940 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 -- 172.21.15.19:0/4040163013 shutdown_connections 2024-02-22T16:11:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 --2- 172.21.15.19:0/4040163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe201027e0 0x7efe200fbea0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 --2- 172.21.15.19:0/4040163013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe20101e40 0x7efe20102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 -- 172.21.15.19:0/4040163013 >> 172.21.15.19:0/4040163013 conn(0x7efe20071d90 msgr2=0x7efe20072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:10.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 -- 172.21.15.19:0/4040163013 shutdown_connections 2024-02-22T16:11:10.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.532+0000 7efe2789a700 1 -- 172.21.15.19:0/4040163013 wait complete. 2024-02-22T16:11:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe2789a700 1 Processor -- start 2024-02-22T16:11:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe2789a700 1 -- start start 2024-02-22T16:11:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe2789a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 0x7efe201128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe2789a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe201027e0 0x7efe20112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe2789a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20113460 con 0x7efe20101e40 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe2789a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2010c790 con 0x7efe201027e0 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe25636700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 0x7efe201128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe25636700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 0x7efe201128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33540/0 (socket says 172.21.15.19:33540) 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.533+0000 7efe25636700 1 -- 172.21.15.19:0/1818194251 learned_addr learned my addr 172.21.15.19:0/1818194251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe25636700 1 -- 172.21.15.19:0/1818194251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe201027e0 msgr2=0x7efe20112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe25636700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe201027e0 0x7efe20112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe25636700 1 -- 172.21.15.19:0/1818194251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c005040 con 0x7efe20101e40 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe25636700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 0x7efe201128d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efe1400fce0 tx=0x7efe140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1400fe90 con 0x7efe20101e40 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe140090d0 con 0x7efe20101e40 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe140126a0 con 0x7efe20101e40 2024-02-22T16:11:10.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2010ca10 con 0x7efe20101e40 2024-02-22T16:11:10.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.534+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2010cf00 con 0x7efe20101e40 2024-02-22T16:11:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.535+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe200738b0 con 0x7efe20101e40 2024-02-22T16:11:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.536+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efe14009240 con 0x7efe20101e40 2024-02-22T16:11:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.536+0000 7efe11ffb700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efe0c06cc20 0x7efe0c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.536+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efe1408e820 con 0x7efe20101e40 2024-02-22T16:11:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.536+0000 7efe24e35700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efe0c06cc20 0x7efe0c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.537+0000 7efe24e35700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efe0c06cc20 0x7efe0c06f0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b3b0 tx=0x7efe1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:10.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.541+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7efe1405c260 con 0x7efe20101e40 2024-02-22T16:11:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.762+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe2004bf40 con 0x7efe0c06cc20 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.763+0000 7efe11ffb700 1 -- 172.21.15.19:0/1818194251 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe2004bf40 con 0x7efe0c06cc20 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:11:10.739 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:11:10.740 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efe0c06cc20 msgr2=0x7efe0c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efe0c06cc20 0x7efe0c06f0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b3b0 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 msgr2=0x7efe201128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 0x7efe201128d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efe1400fce0 tx=0x7efe140046b0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 shutdown_connections 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7efe0c06cc20 0x7efe0c06f0d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe20101e40 0x7efe201128d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 --2- 172.21.15.19:0/1818194251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe201027e0 0x7efe20112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 >> 172.21.15.19:0/1818194251 conn(0x7efe20071d90 msgr2=0x7efe200faf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 shutdown_connections 2024-02-22T16:11:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.765+0000 7efe2789a700 1 -- 172.21.15.19:0/1818194251 wait complete. 2024-02-22T16:11:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/2327907385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 msgr2=0x7f3df80fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/2327907385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df80fcf60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3de4004940 tx=0x7f3de4009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/2327907385 shutdown_connections 2024-02-22T16:11:10.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/2327907385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df80fcf60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/2327907385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 0x7f3df8103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/2327907385 >> 172.21.15.19:0/2327907385 conn(0x7f3df80f8ae0 msgr2=0x7f3df80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/2327907385 shutdown_connections 2024-02-22T16:11:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.868+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/2327907385 wait complete. 2024-02-22T16:11:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfeeaf700 1 Processor -- start 2024-02-22T16:11:10.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfeeaf700 1 -- start start 2024-02-22T16:11:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfeeaf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 0x7f3df8113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfeeaf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df8113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfeeaf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81144c0 con 0x7f3df81038f0 2024-02-22T16:11:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfeeaf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df810d7f0 con 0x7f3df8102f50 2024-02-22T16:11:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df8113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df8113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33566/0 (socket says 172.21.15.19:33566) 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3df7fff700 1 -- 172.21.15.19:0/560994874 learned_addr learned my addr 172.21.15.19:0/560994874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfcc4b700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 0x7f3df8113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3df7fff700 1 -- 172.21.15.19:0/560994874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 msgr2=0x7f3df8113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3df7fff700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 0x7f3df8113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3df7fff700 1 -- 172.21.15.19:0/560994874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4005040 con 0x7f3df81038f0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.869+0000 7f3dfcc4b700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 0x7f3df8113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.870+0000 7f3df7fff700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df8113e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.870+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de4021070 con 0x7f3df81038f0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.870+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df810da70 con 0x7f3df81038f0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.870+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3de401c400 con 0x7f3df81038f0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.870+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de400e070 con 0x7f3df81038f0 2024-02-22T16:11:10.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.870+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df810df50 con 0x7f3df81038f0 2024-02-22T16:11:10.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.871+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df80fc6e0 con 0x7f3df81038f0 2024-02-22T16:11:10.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.871+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3de401c9e0 con 0x7f3df81038f0 2024-02-22T16:11:10.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.872+0000 7f3df57fa700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3de806cc20 0x7f3de806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:10.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.872+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3de408d270 con 0x7f3df81038f0 2024-02-22T16:11:10.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.872+0000 7f3dfcc4b700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3de806cc20 0x7f3de806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:10.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.872+0000 7f3dfcc4b700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3de806cc20 0x7f3de806f0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dec004600 tx=0x7f3dec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:10.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:10.876+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3de405ad30 con 0x7f3df81038f0 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.162+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3df804bed0 con 0x7f3df81038f0 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.162+0000 7f3df57fa700 1 -- 172.21.15.19:0/560994874 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3de405a8c0 con 0x7f3df81038f0 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:11:11.138 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3de806cc20 msgr2=0x7f3de806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3de806cc20 0x7f3de806f0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dec004600 tx=0x7f3dec00b040 comp rx=0 tx=0).stop 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 msgr2=0x7f3df8113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df8113e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 shutdown_connections 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3de806cc20 0x7f3de806f0d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df8102f50 0x7f3df8113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 --2- 172.21.15.19:0/560994874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3df81038f0 0x7f3df8113e70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 >> 172.21.15.19:0/560994874 conn(0x7f3df80f8ae0 msgr2=0x7f3df80fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:11.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 shutdown_connections 2024-02-22T16:11:11.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:11.164+0000 7f3dfeeaf700 1 -- 172.21.15.19:0/560994874 wait complete. 2024-02-22T16:11:11.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:11 smithi019 ceph-mon[173579]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:11.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:11 smithi019 ceph-mon[173579]: pgmap v537: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:11.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:11 smithi019 ceph-mon[173579]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:11.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:11 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/103545156' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:11:11.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:11 smithi062 ceph-mon[106861]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:11.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:11 smithi062 ceph-mon[106861]: pgmap v537: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:11.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:11 smithi062 ceph-mon[106861]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:11.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:11 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/103545156' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:11:12.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:12 smithi062 ceph-mon[106861]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:12.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:12 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/560994874' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:11:12.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:12 smithi019 ceph-mon[173579]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:12.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:12 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/560994874' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:11:13.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:13 smithi062 ceph-mon[106861]: pgmap v538: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:13.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:13 smithi019 ceph-mon[173579]: pgmap v538: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:15.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:15 smithi062 ceph-mon[106861]: pgmap v539: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:15.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:15 smithi019 ceph-mon[173579]: pgmap v539: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:17.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:17 smithi062 ceph-mon[106861]: pgmap v540: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:17.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:17 smithi019 ceph-mon[173579]: pgmap v540: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:19 smithi062 ceph-mon[106861]: pgmap v541: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:19 smithi019 ceph-mon[173579]: pgmap v541: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:21 smithi019 ceph-mon[173579]: pgmap v542: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:21 smithi062 ceph-mon[106861]: pgmap v542: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:23.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:23 smithi062 ceph-mon[106861]: pgmap v543: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:23.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:23 smithi019 ceph-mon[173579]: pgmap v543: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:25.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:25 smithi062 ceph-mon[106861]: pgmap v544: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:25.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:25 smithi019 ceph-mon[173579]: pgmap v544: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:27.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:27 smithi062 ceph-mon[106861]: pgmap v545: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:27.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:27 smithi019 ceph-mon[173579]: pgmap v545: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:29.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:29 smithi062 ceph-mon[106861]: pgmap v546: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:29.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:29 smithi019 ceph-mon[173579]: pgmap v546: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:30.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:30 smithi062 ceph-mon[106861]: pgmap v547: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:30.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:30 smithi019 ceph-mon[173579]: pgmap v547: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:32.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:32 smithi062 ceph-mon[106861]: pgmap v548: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:32.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:32 smithi019 ceph-mon[173579]: pgmap v548: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:34.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:34.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:35.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:35 smithi062 ceph-mon[106861]: pgmap v549: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:35.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:35 smithi019 ceph-mon[173579]: pgmap v549: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:37.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:37 smithi062 ceph-mon[106861]: pgmap v550: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:37.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:37 smithi019 ceph-mon[173579]: pgmap v550: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:39.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:39 smithi062 ceph-mon[106861]: pgmap v551: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:39.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:39 smithi019 ceph-mon[173579]: pgmap v551: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:41.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:41 smithi019 ceph-mon[173579]: pgmap v552: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 -- 172.21.15.19:0/1687330788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 msgr2=0x7f5ff00fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/1687330788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff00fcf60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004940 tx=0x7f5fe0009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 -- 172.21.15.19:0/1687330788 shutdown_connections 2024-02-22T16:11:41.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/1687330788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff00fcf60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/1687330788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ff0102f50 0x7f5ff0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 -- 172.21.15.19:0/1687330788 >> 172.21.15.19:0/1687330788 conn(0x7f5ff00f8ae0 msgr2=0x7f5ff00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 -- 172.21.15.19:0/1687330788 shutdown_connections 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 -- 172.21.15.19:0/1687330788 wait complete. 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 Processor -- start 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.285+0000 7f5ff86cd700 1 -- start start 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff86cd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ff0102f50 0x7f5ff01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff86cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff86cd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0114530 con 0x7f5ff01038f0 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff86cd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff010d860 con 0x7f5ff0102f50 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff0113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54076/0 (socket says 172.21.15.19:54076) 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 -- 172.21.15.19:0/344211751 learned_addr learned my addr 172.21.15.19:0/344211751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:41.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff6469700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ff0102f50 0x7f5ff01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 -- 172.21.15.19:0/344211751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ff0102f50 msgr2=0x7f5ff01139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ff0102f50 0x7f5ff01139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 -- 172.21.15.19:0/344211751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0005040 con 0x7f5ff01038f0 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff5c68700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff0113ee0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009180 tx=0x7f5fe0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fe000e040 con 0x7f5ff01038f0 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5fe0004290 con 0x7f5ff01038f0 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.286+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff010dae0 con 0x7f5ff01038f0 2024-02-22T16:11:41.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.287+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fe001c3f0 con 0x7f5ff01038f0 2024-02-22T16:11:41.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.287+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff010e020 con 0x7f5ff01038f0 2024-02-22T16:11:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.287+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff0105420 con 0x7f5ff01038f0 2024-02-22T16:11:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.288+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5fe000bc50 con 0x7f5ff01038f0 2024-02-22T16:11:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.289+0000 7f5fe6ffd700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5fdc06cc30 0x7f5fdc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.289+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5fe008d2f0 con 0x7f5ff01038f0 2024-02-22T16:11:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.289+0000 7f5ff6469700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5fdc06cc30 0x7f5fdc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.289+0000 7f5ff6469700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5fdc06cc30 0x7f5fdc06f0e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fec004990 tx=0x7f5fec009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.293+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5fe005ae50 con 0x7f5ff01038f0 2024-02-22T16:11:41.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:41 smithi062 ceph-mon[106861]: pgmap v552: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:41.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.515+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ff0109ef0 con 0x7f5fdc06cc30 2024-02-22T16:11:41.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.516+0000 7f5fe6ffd700 1 -- 172.21.15.19:0/344211751 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ff0109ef0 con 0x7f5fdc06cc30 2024-02-22T16:11:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5fdc06cc30 msgr2=0x7f5fdc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5fdc06cc30 0x7f5fdc06f0e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fec004990 tx=0x7f5fec009ee0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 msgr2=0x7f5ff0113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff0113ee0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009180 tx=0x7f5fe0009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 shutdown_connections 2024-02-22T16:11:41.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5fdc06cc30 0x7f5fdc06f0e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ff0102f50 0x7f5ff01139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 --2- 172.21.15.19:0/344211751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff01038f0 0x7f5ff0113ee0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.518+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 >> 172.21.15.19:0/344211751 conn(0x7f5ff00f8ae0 msgr2=0x7f5ff00fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.519+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 shutdown_connections 2024-02-22T16:11:41.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.519+0000 7f5ff86cd700 1 -- 172.21.15.19:0/344211751 wait complete. 2024-02-22T16:11:41.503 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 -- 172.21.15.19:0/1833291567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd741027f0 msgr2=0x7fdd740fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 --2- 172.21.15.19:0/1833291567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd741027f0 0x7fdd740fbe60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdd68004940 tx=0x7fdd68009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 -- 172.21.15.19:0/1833291567 shutdown_connections 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 --2- 172.21.15.19:0/1833291567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd741027f0 0x7fdd740fbe60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 --2- 172.21.15.19:0/1833291567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd74101e50 0x7fdd74102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 -- 172.21.15.19:0/1833291567 >> 172.21.15.19:0/1833291567 conn(0x7fdd740f79e0 msgr2=0x7fdd740f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 -- 172.21.15.19:0/1833291567 shutdown_connections 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.623+0000 7fdd7a578700 1 -- 172.21.15.19:0/1833291567 wait complete. 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd7a578700 1 Processor -- start 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd7a578700 1 -- start start 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd7a578700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 0x7fdd74113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd7a578700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd741027f0 0x7fdd74113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd7a578700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd74114500 con 0x7fdd74101e50 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd7a578700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7410d830 con 0x7fdd741027f0 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 0x7fdd74113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 0x7fdd74113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54090/0 (socket says 172.21.15.19:54090) 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.624+0000 7fdd73fff700 1 -- 172.21.15.19:0/3922148067 learned_addr learned my addr 172.21.15.19:0/3922148067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd737fe700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd741027f0 0x7fdd74113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd73fff700 1 -- 172.21.15.19:0/3922148067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd741027f0 msgr2=0x7fdd74113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd73fff700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd741027f0 0x7fdd74113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd73fff700 1 -- 172.21.15.19:0/3922148067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68005040 con 0x7fdd74101e50 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd737fe700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd741027f0 0x7fdd74113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd73fff700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 0x7fdd74113970 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c8c0 tx=0x7fdd640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6400a980 con 0x7fdd74101e50 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd6400ce60 con 0x7fdd74101e50 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7410db10 con 0x7fdd74101e50 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd640113a0 con 0x7fdd74101e50 2024-02-22T16:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.625+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7410e050 con 0x7fdd74101e50 2024-02-22T16:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.626+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd74064b00 con 0x7fdd74101e50 2024-02-22T16:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.626+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdd64012400 con 0x7fdd74101e50 2024-02-22T16:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.627+0000 7fdd70ff9700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdd5c06cc20 0x7fdd5c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.627+0000 7fdd737fe700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdd5c06cc20 0x7fdd5c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.627+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdd6408c170 con 0x7fdd74101e50 2024-02-22T16:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.628+0000 7fdd737fe700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdd5c06cc20 0x7fdd5c06f0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd7410f1a0 tx=0x7fdd68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.632+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fdd64059c30 con 0x7fdd74101e50 2024-02-22T16:11:41.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.852+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd7410ef50 con 0x7fdd5c06cc20 2024-02-22T16:11:41.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.853+0000 7fdd70ff9700 1 -- 172.21.15.19:0/3922148067 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdd7410ef50 con 0x7fdd5c06cc20 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.855+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdd5c06cc20 msgr2=0x7fdd5c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.855+0000 7fdd7a578700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdd5c06cc20 0x7fdd5c06f0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd7410f1a0 tx=0x7fdd68009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.855+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 msgr2=0x7fdd74113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.855+0000 7fdd7a578700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 0x7fdd74113970 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c8c0 tx=0x7fdd640044c0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 shutdown_connections 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdd5c06cc20 0x7fdd5c06f0d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd74101e50 0x7fdd74113970 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 --2- 172.21.15.19:0/3922148067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd741027f0 0x7fdd74113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 >> 172.21.15.19:0/3922148067 conn(0x7fdd740f79e0 msgr2=0x7fdd74065fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 shutdown_connections 2024-02-22T16:11:41.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.856+0000 7fdd7a578700 1 -- 172.21.15.19:0/3922148067 wait complete. 2024-02-22T16:11:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 -- 172.21.15.19:0/892458473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe8f0 msgr2=0x7f83d4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 --2- 172.21.15.19:0/892458473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe8f0 0x7f83d4107050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f83c4004970 tx=0x7f83c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 -- 172.21.15.19:0/892458473 shutdown_connections 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 --2- 172.21.15.19:0/892458473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe8f0 0x7f83d4107050 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 --2- 172.21.15.19:0/892458473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 -- 172.21.15.19:0/892458473 >> 172.21.15.19:0/892458473 conn(0x7f83d4071d90 msgr2=0x7f83d4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 -- 172.21.15.19:0/892458473 shutdown_connections 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.959+0000 7f83dbb04700 1 -- 172.21.15.19:0/892458473 wait complete. 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83dbb04700 1 Processor -- start 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83dbb04700 1 -- start start 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83dbb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83dbb04700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d40fe8f0 0x7f83d40fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83dbb04700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40fc9e0 con 0x7f83d40fdfe0 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83dbb04700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40fcb50 con 0x7f83d40fe8f0 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54100/0 (socket says 172.21.15.19:54100) 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 -- 172.21.15.19:0/3228141295 learned_addr learned my addr 172.21.15.19:0/3228141295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d909f700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d40fe8f0 0x7f83d40fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 -- 172.21.15.19:0/3228141295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d40fe8f0 msgr2=0x7f83d40fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d40fe8f0 0x7f83d40fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 -- 172.21.15.19:0/3228141295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4005040 con 0x7f83d40fdfe0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.960+0000 7f83d98a0700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fbea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83d000eb70 tx=0x7f83d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.961+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d0007cc0 con 0x7f83d40fdfe0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.961+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f83d000f040 con 0x7f83d40fdfe0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.961+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d4119bb0 con 0x7f83d40fdfe0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.961+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d000c9f0 con 0x7f83d40fdfe0 2024-02-22T16:11:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.961+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d411a010 con 0x7f83d40fdfe0 2024-02-22T16:11:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.962+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d41046c0 con 0x7f83d40fdfe0 2024-02-22T16:11:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.962+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f83d000cb50 con 0x7f83d40fdfe0 2024-02-22T16:11:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.963+0000 7f83ca7fc700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83c006cc30 0x7f83c006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.963+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f83d008bb80 con 0x7f83d40fdfe0 2024-02-22T16:11:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.963+0000 7f83d909f700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83c006cc30 0x7f83c006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:41.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.963+0000 7f83d909f700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83c006cc30 0x7f83c006f0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:41.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:41.967+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f83d0055bb0 con 0x7f83d40fdfe0 2024-02-22T16:11:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.180+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f83d40fd5e0 con 0x7f83c006cc30 2024-02-22T16:11:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.190+0000 7f83ca7fc700 1 -- 172.21.15.19:0/3228141295 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f83d40fd5e0 con 0x7f83c006cc30 2024-02-22T16:11:42.164 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (18m) 12m ago 29m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (16m) 12m ago 30m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (16m) 11m ago 29m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (18m) 12m ago 29m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (12m) 12m ago 24m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (12m) 12m ago 24m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (11m) 11m ago 24m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (12m) 11m ago 24m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (20m) 12m ago 31m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (20m) 11m ago 28m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (17m) 12m ago 31m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (16m) 11m ago 28m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:11:42.165 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (19m) 12m ago 30m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (19m) 11m ago 29m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (15m) 12m ago 28m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (15m) 12m ago 27m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (14m) 12m ago 27m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (14m) 11m ago 26m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (13m) 11m ago 26m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (12m) 11m ago 25m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:11:42.166 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (19m) 12m ago 29m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83c006cc30 msgr2=0x7f83c006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83c006cc30 0x7f83c006f0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f83c4000c00 tx=0x7f83c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 msgr2=0x7f83d40fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fbea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83d000eb70 tx=0x7f83d000b040 comp rx=0 tx=0).stop 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 shutdown_connections 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f83c006cc30 0x7f83c006f0e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fdfe0 0x7f83d40fbea0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 --2- 172.21.15.19:0/3228141295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83d40fe8f0 0x7f83d40fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 >> 172.21.15.19:0/3228141295 conn(0x7f83d4071d90 msgr2=0x7f83d4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 shutdown_connections 2024-02-22T16:11:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.192+0000 7f83dbb04700 1 -- 172.21.15.19:0/3228141295 wait complete. 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 -- 172.21.15.19:0/3791521338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 msgr2=0x7f3f040fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/3791521338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f040fb9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8004940 tx=0x7f3ef8009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 -- 172.21.15.19:0/3791521338 shutdown_connections 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/3791521338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f040fb9c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/3791521338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f04103140 0x7f3f04103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 -- 172.21.15.19:0/3791521338 >> 172.21.15.19:0/3791521338 conn(0x7f3f04071560 msgr2=0x7f3f04071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 -- 172.21.15.19:0/3791521338 shutdown_connections 2024-02-22T16:11:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.294+0000 7f3f0b79f700 1 -- 172.21.15.19:0/3791521338 wait complete. 2024-02-22T16:11:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0b79f700 1 Processor -- start 2024-02-22T16:11:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0b79f700 1 -- start start 2024-02-22T16:11:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0b79f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f04113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0b79f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f04103140 0x7f3f04114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0b79f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04114660 con 0x7f3f040fb570 2024-02-22T16:11:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0b79f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0410d990 con 0x7f3f04103140 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.295+0000 7f3f0953b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f04113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0953b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f04113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54124/0 (socket says 172.21.15.19:54124) 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0953b700 1 -- 172.21.15.19:0/1926668502 learned_addr learned my addr 172.21.15.19:0/1926668502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f08d3a700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f04103140 0x7f3f04114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0953b700 1 -- 172.21.15.19:0/1926668502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f04103140 msgr2=0x7f3f04114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0953b700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f04103140 0x7f3f04114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0953b700 1 -- 172.21.15.19:0/1926668502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8005040 con 0x7f3f040fb570 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0953b700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f04113ad0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c830 tx=0x7f3f00004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0000a760 con 0x7f3f040fb570 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0410dc70 con 0x7f3f040fb570 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f0000ad70 con 0x7f3f040fb570 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f000113a0 con 0x7f3f040fb570 2024-02-22T16:11:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.296+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0410e1b0 con 0x7f3f040fb570 2024-02-22T16:11:42.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.297+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f041054d0 con 0x7f3f040fb570 2024-02-22T16:11:42.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.298+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3f00012400 con 0x7f3f040fb570 2024-02-22T16:11:42.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.298+0000 7f3ef5ffb700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ef006cc20 0x7f3ef006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:42.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.298+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3f0008c0c0 con 0x7f3f040fb570 2024-02-22T16:11:42.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.298+0000 7f3f08d3a700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ef006cc20 0x7f3ef006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:42.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.299+0000 7f3f08d3a700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ef006cc20 0x7f3ef006f0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8004f90 tx=0x7f3ef8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.302+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3f00059b80 con 0x7f3f040fb570 2024-02-22T16:11:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.586+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f0404bed0 con 0x7f3f040fb570 2024-02-22T16:11:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.586+0000 7f3ef5ffb700 1 -- 172.21.15.19:0/1926668502 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f3f00059710 con 0x7f3f040fb570 2024-02-22T16:11:42.561 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:11:42.561 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:11:42.562 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:11:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ef006cc20 msgr2=0x7f3ef006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ef006cc20 0x7f3ef006f0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8004f90 tx=0x7f3ef8009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 msgr2=0x7f3f04113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f04113ad0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c830 tx=0x7f3f00004470 comp rx=0 tx=0).stop 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 shutdown_connections 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ef006cc20 0x7f3ef006f0d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f040fb570 0x7f3f04113ad0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 --2- 172.21.15.19:0/1926668502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f04103140 0x7f3f04114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 >> 172.21.15.19:0/1926668502 conn(0x7f3f04071560 msgr2=0x7f3f040f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 shutdown_connections 2024-02-22T16:11:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.589+0000 7f3f0b79f700 1 -- 172.21.15.19:0/1926668502 wait complete. 2024-02-22T16:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 -- 172.21.15.19:0/3379833411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fe8f0 msgr2=0x7f0588107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 --2- 172.21.15.19:0/3379833411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fe8f0 0x7f0588107050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f057c004970 tx=0x7f057c009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 -- 172.21.15.19:0/3379833411 shutdown_connections 2024-02-22T16:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 --2- 172.21.15.19:0/3379833411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fe8f0 0x7f0588107050 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 --2- 172.21.15.19:0/3379833411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05880fdfe0 0x7f05880fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 -- 172.21.15.19:0/3379833411 >> 172.21.15.19:0/3379833411 conn(0x7f0588071d90 msgr2=0x7f0588072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 -- 172.21.15.19:0/3379833411 shutdown_connections 2024-02-22T16:11:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 -- 172.21.15.19:0/3379833411 wait complete. 2024-02-22T16:11:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.692+0000 7f058db1b700 1 Processor -- start 2024-02-22T16:11:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f058db1b700 1 -- start start 2024-02-22T16:11:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f058db1b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 0x7f0588113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f058db1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05880fe8f0 0x7f0588113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f058db1b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0588114500 con 0x7f05880fdfe0 2024-02-22T16:11:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f058db1b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058810d830 con 0x7f05880fe8f0 2024-02-22T16:11:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f05877fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 0x7f0588113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f05877fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 0x7f0588113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54148/0 (socket says 172.21.15.19:54148) 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f05877fe700 1 -- 172.21.15.19:0/370515275 learned_addr learned my addr 172.21.15.19:0/370515275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f0586ffd700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05880fe8f0 0x7f0588113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f05877fe700 1 -- 172.21.15.19:0/370515275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05880fe8f0 msgr2=0x7f0588113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f05877fe700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05880fe8f0 0x7f0588113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.693+0000 7f05877fe700 1 -- 172.21.15.19:0/370515275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c005040 con 0x7f05880fdfe0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.694+0000 7f05877fe700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 0x7f0588113970 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0578002a50 tx=0x7f05780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.694+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0578007cc0 con 0x7f05880fdfe0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.694+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058810db10 con 0x7f05880fdfe0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.694+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f057800f040 con 0x7f05880fdfe0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.694+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f057800d9c0 con 0x7f05880fdfe0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.694+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058810e020 con 0x7f05880fdfe0 2024-02-22T16:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.695+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05740052e0 con 0x7f05880fdfe0 2024-02-22T16:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.695+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f057800db20 con 0x7f05880fdfe0 2024-02-22T16:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.696+0000 7f056ffff700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f057006cc20 0x7f057006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.696+0000 7f0586ffd700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f057006cc20 0x7f057006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.696+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f057808bd40 con 0x7f05880fdfe0 2024-02-22T16:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.696+0000 7f0586ffd700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f057006cc20 0x7f057006f0d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f057c004f50 tx=0x7f057c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.700+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0578059780 con 0x7f05880fdfe0 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.948+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0574005da0 con 0x7f05880fdfe0 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.948+0000 7f056ffff700 1 -- 172.21.15.19:0/370515275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f0578059310 con 0x7f05880fdfe0 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:11:42.923 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:11:42.924 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:11:42.925 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:11:42.926 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:11:42.927 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:42.927 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:42.927 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:42.927 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f057006cc20 msgr2=0x7f057006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f057006cc20 0x7f057006f0d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f057c004f50 tx=0x7f057c009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 msgr2=0x7f0588113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 0x7f0588113970 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0578002a50 tx=0x7f05780046b0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 shutdown_connections 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f057006cc20 0x7f057006f0d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05880fdfe0 0x7f0588113970 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 --2- 172.21.15.19:0/370515275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05880fe8f0 0x7f0588113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 >> 172.21.15.19:0/370515275 conn(0x7f0588071d90 msgr2=0x7f0588101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 shutdown_connections 2024-02-22T16:11:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:42.950+0000 7f058db1b700 1 -- 172.21.15.19:0/370515275 wait complete. 2024-02-22T16:11:42.929 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 -- 172.21.15.19:0/2934510352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce4102f50 msgr2=0x7f7ce4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/2934510352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce4102f50 0x7f7ce4103320 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4004940 tx=0x7f7cd4009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 -- 172.21.15.19:0/2934510352 shutdown_connections 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/2934510352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce41038f0 0x7f7ce40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/2934510352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce4102f50 0x7f7ce4103320 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 -- 172.21.15.19:0/2934510352 >> 172.21.15.19:0/2934510352 conn(0x7f7ce40f8ae0 msgr2=0x7f7ce40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.054+0000 7f7cec6bd700 1 -- 172.21.15.19:0/2934510352 shutdown_connections 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 -- 172.21.15.19:0/2934510352 wait complete. 2024-02-22T16:11:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 Processor -- start 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 -- start start 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce4102f50 0x7f7ce4070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 0x7f7ce40707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce4070dd0 con 0x7f7ce41038f0 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7cec6bd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce4070f40 con 0x7f7ce4102f50 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7ce9c58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 0x7f7ce40707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7ce9c58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 0x7f7ce40707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54168/0 (socket says 172.21.15.19:54168) 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.055+0000 7f7ce9c58700 1 -- 172.21.15.19:0/1017882419 learned_addr learned my addr 172.21.15.19:0/1017882419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cea459700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce4102f50 0x7f7ce4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7ce9c58700 1 -- 172.21.15.19:0/1017882419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce4102f50 msgr2=0x7f7ce4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7ce9c58700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce4102f50 0x7f7ce4070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7ce9c58700 1 -- 172.21.15.19:0/1017882419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd4005040 con 0x7f7ce41038f0 2024-02-22T16:11:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cea459700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce4102f50 0x7f7ce4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:11:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7ce9c58700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 0x7f7ce40707d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0002a50 tx=0x7f7ce00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ce0007cc0 con 0x7f7ce41038f0 2024-02-22T16:11:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ce000f040 con 0x7f7ce41038f0 2024-02-22T16:11:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ce00095d0 con 0x7f7ce41038f0 2024-02-22T16:11:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce4119be0 con 0x7f7ce41038f0 2024-02-22T16:11:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.056+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce411a0c0 con 0x7f7ce41038f0 2024-02-22T16:11:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.057+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce4100260 con 0x7f7ce41038f0 2024-02-22T16:11:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.058+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ce0009730 con 0x7f7ce41038f0 2024-02-22T16:11:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.058+0000 7f7cdaffd700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cd006cc20 0x7f7cd006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.058+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7ce008c230 con 0x7f7ce41038f0 2024-02-22T16:11:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.058+0000 7f7cea459700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cd006cc20 0x7f7cd006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.059+0000 7f7cea459700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cd006cc20 0x7f7cd006f0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4004560 tx=0x7f7cd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.062+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7ce0056300 con 0x7f7ce41038f0 2024-02-22T16:11:43.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.283+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce40fc6b0 con 0x7f7cd006cc20 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.284+0000 7f7cdaffd700 1 -- 172.21.15.19:0/1017882419 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7ce40fc6b0 con 0x7f7cd006cc20 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:11:43.259 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:11:43.260 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cd006cc20 msgr2=0x7f7cd006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cd006cc20 0x7f7cd006f0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4004560 tx=0x7f7cd4009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 msgr2=0x7f7ce40707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 0x7f7ce40707d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0002a50 tx=0x7f7ce00046b0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 shutdown_connections 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7cd006cc20 0x7f7cd006f0d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ce4102f50 0x7f7ce4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 --2- 172.21.15.19:0/1017882419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce41038f0 0x7f7ce40707d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 >> 172.21.15.19:0/1017882419 conn(0x7f7ce40f8ae0 msgr2=0x7f7ce40fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:43.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 shutdown_connections 2024-02-22T16:11:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.286+0000 7f7cec6bd700 1 -- 172.21.15.19:0/1017882419 wait complete. 2024-02-22T16:11:43.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.389+0000 7f01ab277700 1 -- 172.21.15.19:0/1204839280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a41038e0 msgr2=0x7f01a40fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.389+0000 7f01ab277700 1 --2- 172.21.15.19:0/1204839280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a41038e0 0x7f01a40fcf50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0194004940 tx=0x7f0194009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 -- 172.21.15.19:0/1204839280 shutdown_connections 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 --2- 172.21.15.19:0/1204839280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a41038e0 0x7f01a40fcf50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 --2- 172.21.15.19:0/1204839280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4102f40 0x7f01a4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 -- 172.21.15.19:0/1204839280 >> 172.21.15.19:0/1204839280 conn(0x7f01a40f8af0 msgr2=0x7f01a40faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 -- 172.21.15.19:0/1204839280 shutdown_connections 2024-02-22T16:11:43.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 -- 172.21.15.19:0/1204839280 wait complete. 2024-02-22T16:11:43.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 Processor -- start 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.390+0000 7f01ab277700 1 -- start start 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01ab277700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 0x7f01a4113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01ab277700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a41038e0 0x7f01a4113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01ab277700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a41144b0 con 0x7f01a4102f40 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01ab277700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a410d7e0 con 0x7f01a41038e0 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 0x7f01a4113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 0x7f01a4113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54176/0 (socket says 172.21.15.19:54176) 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 -- 172.21.15.19:0/1866406307 learned_addr learned my addr 172.21.15.19:0/1866406307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a8812700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a41038e0 0x7f01a4113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 -- 172.21.15.19:0/1866406307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a41038e0 msgr2=0x7f01a4113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a41038e0 0x7f01a4113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 -- 172.21.15.19:0/1866406307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194005040 con 0x7f01a4102f40 2024-02-22T16:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01a9013700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 0x7f01a4113920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f01a000eb40 tx=0x7f01a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a0007cc0 con 0x7f01a4102f40 2024-02-22T16:11:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01a000f040 con 0x7f01a4102f40 2024-02-22T16:11:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a410da60 con 0x7f01a4102f40 2024-02-22T16:11:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.391+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a000c990 con 0x7f01a4102f40 2024-02-22T16:11:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.392+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a410df70 con 0x7f01a4102f40 2024-02-22T16:11:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.393+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4105410 con 0x7f01a4102f40 2024-02-22T16:11:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.393+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f01a000cc00 con 0x7f01a4102f40 2024-02-22T16:11:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.393+0000 7f0199ffb700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f019006cc20 0x7f019006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:11:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.393+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f01a008bc70 con 0x7f01a4102f40 2024-02-22T16:11:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.393+0000 7f01a8812700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f019006cc20 0x7f019006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:11:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.394+0000 7f01a8812700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f019006cc20 0x7f019006f0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0194000c00 tx=0x7f0194009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:11:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.398+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f01a0059760 con 0x7f01a4102f40 2024-02-22T16:11:43.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:43 smithi062 ceph-mon[106861]: pgmap v553: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:43.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:43 smithi062 ceph-mon[106861]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:43.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:43 smithi062 ceph-mon[106861]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:43.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:43 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1926668502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:11:43.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:43 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/370515275' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:11:43.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:43 smithi019 ceph-mon[173579]: pgmap v553: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:43.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:43 smithi019 ceph-mon[173579]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:43.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:43 smithi019 ceph-mon[173579]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:43.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:43 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1926668502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:11:43.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:43 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/370515275' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:11:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.682+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f01a404bed0 con 0x7f01a4102f40 2024-02-22T16:11:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.683+0000 7f0199ffb700 1 -- 172.21.15.19:0/1866406307 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f01a00592f0 con 0x7f01a4102f40 2024-02-22T16:11:43.658 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:11:43.658 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:11:43.658 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:11:43.658 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:11:43.658 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f019006cc20 msgr2=0x7f019006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f019006cc20 0x7f019006f0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0194000c00 tx=0x7f0194009e30 comp rx=0 tx=0).stop 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 msgr2=0x7f01a4113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 0x7f01a4113920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f01a000eb40 tx=0x7f01a000b040 comp rx=0 tx=0).stop 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 shutdown_connections 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f019006cc20 0x7f019006f0d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01a4102f40 0x7f01a4113920 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 --2- 172.21.15.19:0/1866406307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a41038e0 0x7f01a4113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 >> 172.21.15.19:0/1866406307 conn(0x7f01a40f8af0 msgr2=0x7f01a40fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 shutdown_connections 2024-02-22T16:11:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:11:43.685+0000 7f01ab277700 1 -- 172.21.15.19:0/1866406307 wait complete. 2024-02-22T16:11:44.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:44 smithi062 ceph-mon[106861]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:44.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:44 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1866406307' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:11:44.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:44 smithi019 ceph-mon[173579]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:44.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:44 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1866406307' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:11:45.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:45 smithi062 ceph-mon[106861]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:45.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:45 smithi062 ceph-mon[106861]: pgmap v554: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:45.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:45 smithi019 ceph-mon[173579]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:11:45.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:45 smithi019 ceph-mon[173579]: pgmap v554: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:47.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:47 smithi062 ceph-mon[106861]: pgmap v555: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:47.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:47 smithi019 ceph-mon[173579]: pgmap v555: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:49 smithi062 ceph-mon[106861]: pgmap v556: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:49 smithi019 ceph-mon[173579]: pgmap v556: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:11:51.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:51 smithi019 ceph-mon[173579]: pgmap v557: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:51.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:51 smithi062 ceph-mon[106861]: pgmap v557: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:52.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:52 smithi062 ceph-mon[106861]: pgmap v558: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:52.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:52 smithi019 ceph-mon[173579]: pgmap v558: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:55.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:55 smithi062 ceph-mon[106861]: pgmap v559: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:55.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:55 smithi019 ceph-mon[173579]: pgmap v559: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:57 smithi062 ceph-mon[106861]: pgmap v560: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:57 smithi019 ceph-mon[173579]: pgmap v560: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:11:59 smithi062 ceph-mon[106861]: pgmap v561: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:11:59.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:11:59 smithi019 ceph-mon[173579]: pgmap v561: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:01.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:01 smithi019 ceph-mon[173579]: pgmap v562: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:01.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:01 smithi062 ceph-mon[106861]: pgmap v562: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:03.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:03 smithi062 ceph-mon[106861]: pgmap v563: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:03.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:03 smithi019 ceph-mon[173579]: pgmap v563: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:05.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:05 smithi062 ceph-mon[106861]: pgmap v564: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:05.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:05 smithi019 ceph-mon[173579]: pgmap v564: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:07.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:07 smithi062 ceph-mon[106861]: pgmap v565: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:07.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:07 smithi019 ceph-mon[173579]: pgmap v565: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:09.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:09 smithi062 ceph-mon[106861]: pgmap v566: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:09.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:09 smithi019 ceph-mon[173579]: pgmap v566: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:11.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:11 smithi019 ceph-mon[173579]: pgmap v567: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:11.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:11 smithi062 ceph-mon[106861]: pgmap v567: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:13.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:13 smithi062 ceph-mon[106861]: pgmap v568: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:13.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:13 smithi019 ceph-mon[173579]: pgmap v568: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 -- 172.21.15.19:0/3263678674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e01038f0 msgr2=0x7f64e00fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/3263678674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e01038f0 0x7f64e00fcf60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f64cc004940 tx=0x7f64cc009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 -- 172.21.15.19:0/3263678674 shutdown_connections 2024-02-22T16:12:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/3263678674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e01038f0 0x7f64e00fcf60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/3263678674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e0102f50 0x7f64e0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 -- 172.21.15.19:0/3263678674 >> 172.21.15.19:0/3263678674 conn(0x7f64e00f8ae0 msgr2=0x7f64e00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 -- 172.21.15.19:0/3263678674 shutdown_connections 2024-02-22T16:12:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.803+0000 7f64e6bf5700 1 -- 172.21.15.19:0/3263678674 wait complete. 2024-02-22T16:12:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.804+0000 7f64e6bf5700 1 Processor -- start 2024-02-22T16:12:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.804+0000 7f64e6bf5700 1 -- start start 2024-02-22T16:12:13.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e6bf5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 0x7f64e0113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e6bf5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e01038f0 0x7f64e0113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e6bf5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01144c0 con 0x7f64e0102f50 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e6bf5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e010d7f0 con 0x7f64e01038f0 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 0x7f64e0113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 0x7f64e0113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35770/0 (socket says 172.21.15.19:35770) 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 -- 172.21.15.19:0/1973845975 learned_addr learned my addr 172.21.15.19:0/1973845975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64dffff700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e01038f0 0x7f64e0113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 -- 172.21.15.19:0/1973845975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e01038f0 msgr2=0x7f64e0113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e01038f0 0x7f64e0113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 -- 172.21.15.19:0/1973845975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc005040 con 0x7f64e0102f50 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64e4991700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 0x7f64e0113930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f64d4002a80 tx=0x7f64d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d4007cc0 con 0x7f64e0102f50 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.805+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64d400f040 con 0x7f64e0102f50 2024-02-22T16:12:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.806+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e010dad0 con 0x7f64e0102f50 2024-02-22T16:12:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.806+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d400da60 con 0x7f64e0102f50 2024-02-22T16:12:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.806+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e010dfe0 con 0x7f64e0102f50 2024-02-22T16:12:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.807+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e00fc6e0 con 0x7f64e0102f50 2024-02-22T16:12:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.807+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f64d400dbc0 con 0x7f64e0102f50 2024-02-22T16:12:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.808+0000 7f64dd7fa700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f64d006cc20 0x7f64d006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:13.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.808+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f64d408bcc0 con 0x7f64e0102f50 2024-02-22T16:12:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.808+0000 7f64dffff700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f64d006cc20 0x7f64d006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.808+0000 7f64dffff700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f64d006cc20 0x7f64d006f0d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64e010eb20 tx=0x7f64cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:13.813+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f64d4059700 con 0x7f64e0102f50 2024-02-22T16:12:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.034+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e010ef80 con 0x7f64d006cc20 2024-02-22T16:12:14.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.035+0000 7f64dd7fa700 1 -- 172.21.15.19:0/1973845975 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f64e010ef80 con 0x7f64d006cc20 2024-02-22T16:12:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f64d006cc20 msgr2=0x7f64d006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f64d006cc20 0x7f64d006f0d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64e010eb20 tx=0x7f64cc009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 msgr2=0x7f64e0113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 0x7f64e0113930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f64d4002a80 tx=0x7f64d40046b0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 shutdown_connections 2024-02-22T16:12:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f64d006cc20 0x7f64d006f0d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0102f50 0x7f64e0113930 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 --2- 172.21.15.19:0/1973845975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64e01038f0 0x7f64e0113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 >> 172.21.15.19:0/1973845975 conn(0x7f64e00f8ae0 msgr2=0x7f64e00fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 shutdown_connections 2024-02-22T16:12:14.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.037+0000 7f64e6bf5700 1 -- 172.21.15.19:0/1973845975 wait complete. 2024-02-22T16:12:14.022 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:12:14.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 -- 172.21.15.19:0/59633928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f0102f50 msgr2=0x7fa4f0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 --2- 172.21.15.19:0/59633928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f0102f50 0x7fa4f0103320 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4004940 tx=0x7fa4e4009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 -- 172.21.15.19:0/59633928 shutdown_connections 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 --2- 172.21.15.19:0/59633928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4f01038f0 0x7fa4f00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 --2- 172.21.15.19:0/59633928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f0102f50 0x7fa4f0103320 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 -- 172.21.15.19:0/59633928 >> 172.21.15.19:0/59633928 conn(0x7fa4f00f8ae0 msgr2=0x7fa4f00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 -- 172.21.15.19:0/59633928 shutdown_connections 2024-02-22T16:12:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.142+0000 7fa4f5061700 1 -- 172.21.15.19:0/59633928 wait complete. 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4f5061700 1 Processor -- start 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4f5061700 1 -- start start 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4f5061700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4f0102f50 0x7fa4f0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4f5061700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 0x7fa4f00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4f5061700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0070dd0 con 0x7fa4f01038f0 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4f5061700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0070f40 con 0x7fa4f0102f50 2024-02-22T16:12:14.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 0x7fa4f00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 0x7fa4f00707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35782/0 (socket says 172.21.15.19:35782) 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 -- 172.21.15.19:0/1196848963 learned_addr learned my addr 172.21.15.19:0/1196848963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4eed9d700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4f0102f50 0x7fa4f0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 -- 172.21.15.19:0/1196848963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4f0102f50 msgr2=0x7fa4f0070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4f0102f50 0x7fa4f0070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 -- 172.21.15.19:0/1196848963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4005040 con 0x7fa4f01038f0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.143+0000 7fa4ee59c700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 0x7fa4f00707d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000eb40 tx=0x7fa4e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.144+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4e0007cc0 con 0x7fa4f01038f0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.144+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f0119be0 con 0x7fa4f01038f0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.144+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4e000f040 con 0x7fa4f01038f0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.144+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4e000c8f0 con 0x7fa4f01038f0 2024-02-22T16:12:14.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.144+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f011a040 con 0x7fa4f01038f0 2024-02-22T16:12:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.145+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f0100260 con 0x7fa4f01038f0 2024-02-22T16:12:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.145+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa4e000cb60 con 0x7fa4f01038f0 2024-02-22T16:12:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.146+0000 7fa4d77fe700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa4d806cc20 0x7fa4d806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.146+0000 7fa4eed9d700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa4d806cc20 0x7fa4d806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.146+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa4e008bba0 con 0x7fa4f01038f0 2024-02-22T16:12:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.146+0000 7fa4eed9d700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa4d806cc20 0x7fa4d806f0d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000c00 tx=0x7fa4e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:14.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.152+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa4e0055b60 con 0x7fa4f01038f0 2024-02-22T16:12:14.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.373+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4f00fc6b0 con 0x7fa4d806cc20 2024-02-22T16:12:14.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.374+0000 7fa4d77fe700 1 -- 172.21.15.19:0/1196848963 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa4f00fc6b0 con 0x7fa4d806cc20 2024-02-22T16:12:14.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa4d806cc20 msgr2=0x7fa4d806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa4d806cc20 0x7fa4d806f0d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4000c00 tx=0x7fa4e4009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 msgr2=0x7fa4f00707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 0x7fa4f00707d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000eb40 tx=0x7fa4e000b040 comp rx=0 tx=0).stop 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 shutdown_connections 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa4d806cc20 0x7fa4d806f0d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4f0102f50 0x7fa4f0070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 --2- 172.21.15.19:0/1196848963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f01038f0 0x7fa4f00707d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 >> 172.21.15.19:0/1196848963 conn(0x7fa4f00f8ae0 msgr2=0x7fa4f00fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 shutdown_connections 2024-02-22T16:12:14.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.376+0000 7fa4f5061700 1 -- 172.21.15.19:0/1196848963 wait complete. 2024-02-22T16:12:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 -- 172.21.15.19:0/3666386064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298101e60 msgr2=0x7f6298102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 --2- 172.21.15.19:0/3666386064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298101e60 0x7f6298102230 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6288004940 tx=0x7f6288009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 -- 172.21.15.19:0/3666386064 shutdown_connections 2024-02-22T16:12:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 --2- 172.21.15.19:0/3666386064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6298102800 0x7f62980fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 --2- 172.21.15.19:0/3666386064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298101e60 0x7f6298102230 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 -- 172.21.15.19:0/3666386064 >> 172.21.15.19:0/3666386064 conn(0x7f62980f7a10 msgr2=0x7f62980f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 -- 172.21.15.19:0/3666386064 shutdown_connections 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 -- 172.21.15.19:0/3666386064 wait complete. 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 Processor -- start 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.480+0000 7f629d2e6700 1 -- start start 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f629d2e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6298101e60 0x7f6298115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f629d2e6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 0x7f62981161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f629d2e6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298116780 con 0x7f6298102800 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f629d2e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629810fb00 con 0x7f6298101e60 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 0x7f62981161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 0x7f62981161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35788/0 (socket says 172.21.15.19:35788) 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 -- 172.21.15.19:0/1864963225 learned_addr learned my addr 172.21.15.19:0/1864963225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f6296ffd700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6298101e60 0x7f6298115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 -- 172.21.15.19:0/1864963225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6298101e60 msgr2=0x7f6298115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6298101e60 0x7f6298115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 -- 172.21.15.19:0/1864963225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288005040 con 0x7f6298102800 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f62967fc700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 0x7f62981161d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f628000c890 tx=0x7f62800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f628000a700 con 0x7f6298102800 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.481+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629810fde0 con 0x7f6298102800 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.482+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f628000ad10 con 0x7f6298102800 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.482+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6280012e70 con 0x7f6298102800 2024-02-22T16:12:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.482+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6298110320 con 0x7f6298102800 2024-02-22T16:12:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.482+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62980fb610 con 0x7f6298102800 2024-02-22T16:12:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.483+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f628000a860 con 0x7f6298102800 2024-02-22T16:12:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.484+0000 7f628f7fe700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f628406cc20 0x7f628406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.484+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f628008cc70 con 0x7f6298102800 2024-02-22T16:12:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.484+0000 7f6296ffd700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f628406cc20 0x7f628406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.485+0000 7f6296ffd700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f628406cc20 0x7f628406f0d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6288004f90 tx=0x7f6288009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.488+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6280056d40 con 0x7f6298102800 2024-02-22T16:12:14.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.699+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6298111200 con 0x7f628406cc20 2024-02-22T16:12:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.709+0000 7f628f7fe700 1 -- 172.21.15.19:0/1864963225 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f6298111200 con 0x7f628406cc20 2024-02-22T16:12:14.683 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (19m) 12m ago 30m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (16m) 12m ago 30m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (16m) 12m ago 29m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (19m) 12m ago 30m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (12m) 12m ago 24m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (13m) 12m ago 24m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:12:14.684 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (12m) 12m ago 24m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (12m) 12m ago 24m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (21m) 12m ago 32m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (20m) 12m ago 29m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (17m) 12m ago 32m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (17m) 12m ago 29m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (20m) 12m ago 30m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:12:14.685 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (20m) 12m ago 29m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (16m) 12m ago 28m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (15m) 12m ago 28m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (15m) 12m ago 27m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (14m) 12m ago 27m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (14m) 12m ago 26m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (13m) 12m ago 26m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:12:14.686 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (19m) 12m ago 30m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f628406cc20 msgr2=0x7f628406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f628406cc20 0x7f628406f0d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6288004f90 tx=0x7f6288009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 msgr2=0x7f62981161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 0x7f62981161d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f628000c890 tx=0x7f62800044c0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 shutdown_connections 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f628406cc20 0x7f628406f0d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6298101e60 0x7f6298115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 --2- 172.21.15.19:0/1864963225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6298102800 0x7f62981161d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 >> 172.21.15.19:0/1864963225 conn(0x7f62980f7a10 msgr2=0x7f6298066110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 shutdown_connections 2024-02-22T16:12:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.711+0000 7f629d2e6700 1 -- 172.21.15.19:0/1864963225 wait complete. 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 -- 172.21.15.19:0/3318313570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 msgr2=0x7fe96c0fb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 --2- 172.21.15.19:0/3318313570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c0fb9d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe95c004940 tx=0x7fe95c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 -- 172.21.15.19:0/3318313570 shutdown_connections 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 --2- 172.21.15.19:0/3318313570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c0fb9d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 --2- 172.21.15.19:0/3318313570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe96c102fe0 0x7fe96c1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 -- 172.21.15.19:0/3318313570 >> 172.21.15.19:0/3318313570 conn(0x7fe96c071500 msgr2=0x7fe96c071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 -- 172.21.15.19:0/3318313570 shutdown_connections 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.813+0000 7fe974133700 1 -- 172.21.15.19:0/3318313570 wait complete. 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe974133700 1 Processor -- start 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe974133700 1 -- start start 2024-02-22T16:12:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe974133700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe974133700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe96c102fe0 0x7fe96c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe974133700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c114510 con 0x7fe96c0fb560 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe974133700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c10d840 con 0x7fe96c102fe0 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe971ecf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe971ecf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35808/0 (socket says 172.21.15.19:35808) 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.814+0000 7fe971ecf700 1 -- 172.21.15.19:0/293640652 learned_addr learned my addr 172.21.15.19:0/293640652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe971ecf700 1 -- 172.21.15.19:0/293640652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe96c102fe0 msgr2=0x7fe96c113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe9716ce700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe96c102fe0 0x7fe96c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe971ecf700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe96c102fe0 0x7fe96c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe971ecf700 1 -- 172.21.15.19:0/293640652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c005040 con 0x7fe96c0fb560 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe971ecf700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c113980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe96800eb70 tx=0x7fe96800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe968007cc0 con 0x7fe96c0fb560 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe96800f040 con 0x7fe96c0fb560 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c10dac0 con 0x7fe96c0fb560 2024-02-22T16:12:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96800cad0 con 0x7fe96c0fb560 2024-02-22T16:12:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.815+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c10e000 con 0x7fe96c0fb560 2024-02-22T16:12:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.816+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c1053c0 con 0x7fe96c0fb560 2024-02-22T16:12:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.817+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe96800cc30 con 0x7fe96c0fb560 2024-02-22T16:12:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.817+0000 7fe9627fc700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe95806cc20 0x7fe95806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.817+0000 7fe9716ce700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe95806cc20 0x7fe95806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.817+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe96808bb70 con 0x7fe96c0fb560 2024-02-22T16:12:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.818+0000 7fe9716ce700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe95806cc20 0x7fe95806f0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00b0d0 tx=0x7fe95c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:14.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:14.822+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe968059660 con 0x7fe96c0fb560 2024-02-22T16:12:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.107+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe96c04bed0 con 0x7fe96c0fb560 2024-02-22T16:12:15.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.108+0000 7fe9627fc700 1 -- 172.21.15.19:0/293640652 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fe9680591f0 con 0x7fe96c0fb560 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:12:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:12:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:12:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:12:15.084 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:12:15.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe95806cc20 msgr2=0x7fe95806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe95806cc20 0x7fe95806f0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00b0d0 tx=0x7fe95c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 msgr2=0x7fe96c113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c113980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe96800eb70 tx=0x7fe96800b040 comp rx=0 tx=0).stop 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 shutdown_connections 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe95806cc20 0x7fe95806f0d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe96c0fb560 0x7fe96c113980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 --2- 172.21.15.19:0/293640652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe96c102fe0 0x7fe96c113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 >> 172.21.15.19:0/293640652 conn(0x7fe96c071500 msgr2=0x7fe96c0fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 shutdown_connections 2024-02-22T16:12:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.110+0000 7fe974133700 1 -- 172.21.15.19:0/293640652 wait complete. 2024-02-22T16:12:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 -- 172.21.15.19:0/1887059686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 msgr2=0x7f4574102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 --2- 172.21.15.19:0/1887059686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f4574102210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4570004940 tx=0x7f4570009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 -- 172.21.15.19:0/1887059686 shutdown_connections 2024-02-22T16:12:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 --2- 172.21.15.19:0/1887059686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 0x7f45740fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 --2- 172.21.15.19:0/1887059686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f4574102210 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 -- 172.21.15.19:0/1887059686 >> 172.21.15.19:0/1887059686 conn(0x7f4574071d90 msgr2=0x7f4574072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 -- 172.21.15.19:0/1887059686 shutdown_connections 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 -- 172.21.15.19:0/1887059686 wait complete. 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 Processor -- start 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.214+0000 7f457be24700 1 -- start start 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f457be24700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f457410e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f457be24700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 0x7f457410eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f457be24700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457410f450 con 0x7f4574101e40 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f457be24700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457410f590 con 0x7f45741027e0 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f457410e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f457410e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35832/0 (socket says 172.21.15.19:35832) 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 -- 172.21.15.19:0/1663166539 learned_addr learned my addr 172.21.15.19:0/1663166539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:15.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f45793bf700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 0x7f457410eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 -- 172.21.15.19:0/1663166539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 msgr2=0x7f457410eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 0x7f457410eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 -- 172.21.15.19:0/1663166539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570005040 con 0x7f4574101e40 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f45793bf700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 0x7f457410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f4579bc0700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f457410e980 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4570000c00 tx=0x7f4570009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4570021070 con 0x7f4574101e40 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f457001c400 con 0x7f4574101e40 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457000e070 con 0x7f4574101e40 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4574117890 con 0x7f4574101e40 2024-02-22T16:12:15.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.215+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4574117cf0 con 0x7f4574101e40 2024-02-22T16:12:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.216+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45740738b0 con 0x7f4574101e40 2024-02-22T16:12:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.217+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f457001c9e0 con 0x7f4574101e40 2024-02-22T16:12:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.217+0000 7f456a7fc700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f456006cc20 0x7f456006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.217+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f457008d2a0 con 0x7f4574101e40 2024-02-22T16:12:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.217+0000 7f45793bf700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f456006cc20 0x7f456006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.218+0000 7f45793bf700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f456006cc20 0x7f456006f0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4564004600 tx=0x7f456400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.222+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f45700572d0 con 0x7f4574101e40 2024-02-22T16:12:15.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.470+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4574065fc0 con 0x7f4574101e40 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.470+0000 7f456a7fc700 1 -- 172.21.15.19:0/1663166539 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f457005a8f0 con 0x7f4574101e40 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:12:15.445 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:12:15.446 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:15.447 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:15.448 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:15.448 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:15.448 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:15.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f456006cc20 msgr2=0x7f456006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f456006cc20 0x7f456006f0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4564004600 tx=0x7f456400b040 comp rx=0 tx=0).stop 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 msgr2=0x7f457410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f457410e980 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4570000c00 tx=0x7f4570009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 shutdown_connections 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f456006cc20 0x7f456006f0d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4574101e40 0x7f457410e980 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 --2- 172.21.15.19:0/1663166539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45741027e0 0x7f457410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 >> 172.21.15.19:0/1663166539 conn(0x7f4574071d90 msgr2=0x7f45740fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 shutdown_connections 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.472+0000 7f457be24700 1 -- 172.21.15.19:0/1663166539 wait complete. 2024-02-22T16:12:15.449 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:12:15.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:15 smithi062 ceph-mon[106861]: pgmap v569: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:15.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:15 smithi062 ceph-mon[106861]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:15.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:15 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/293640652' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:12:15.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:15 smithi019 ceph-mon[173579]: pgmap v569: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:15.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:15 smithi019 ceph-mon[173579]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:15.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:15 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/293640652' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:12:15.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.576+0000 7f1167a8a700 1 -- 172.21.15.19:0/1182139498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1160101e40 msgr2=0x7f1160102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.576+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1182139498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1160101e40 0x7f1160102210 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f115c004940 tx=0x7f115c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:15.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- 172.21.15.19:0/1182139498 shutdown_connections 2024-02-22T16:12:15.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1182139498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11601027e0 0x7f11600fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1182139498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1160101e40 0x7f1160102210 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- 172.21.15.19:0/1182139498 >> 172.21.15.19:0/1182139498 conn(0x7f1160071d90 msgr2=0x7f1160072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:15.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- 172.21.15.19:0/1182139498 shutdown_connections 2024-02-22T16:12:15.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- 172.21.15.19:0/1182139498 wait complete. 2024-02-22T16:12:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 Processor -- start 2024-02-22T16:12:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- start start 2024-02-22T16:12:15.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1160101e40 0x7f116010c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 0x7f116010cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116010d0d0 con 0x7f11601027e0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.577+0000 7f1167a8a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116010d240 con 0x7f1160101e40 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 0x7f116010cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 0x7f116010cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35844/0 (socket says 172.21.15.19:35844) 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 -- 172.21.15.19:0/1047409875 learned_addr learned my addr 172.21.15.19:0/1047409875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165826700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1160101e40 0x7f116010c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 -- 172.21.15.19:0/1047409875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1160101e40 msgr2=0x7f116010c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1160101e40 0x7f116010c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 -- 172.21.15.19:0/1047409875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c005040 con 0x7f11601027e0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1165025700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 0x7f116010cb90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1150002a50 tx=0x7f1150004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1150007cc0 con 0x7f11601027e0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f115000f040 con 0x7f11601027e0 2024-02-22T16:12:15.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1160119cc0 con 0x7f11601027e0 2024-02-22T16:12:15.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f115000d920 con 0x7f11601027e0 2024-02-22T16:12:15.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.578+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116011a0c0 con 0x7f11601027e0 2024-02-22T16:12:15.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.579+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11600ff150 con 0x7f11601027e0 2024-02-22T16:12:15.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.580+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1150009680 con 0x7f11601027e0 2024-02-22T16:12:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.580+0000 7f11567fc700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f114c06cc30 0x7f114c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.580+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f115008bd00 con 0x7f11601027e0 2024-02-22T16:12:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.580+0000 7f1165826700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f114c06cc30 0x7f114c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.581+0000 7f1165826700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f114c06cc30 0x7f114c06f0e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f115c00b3b0 tx=0x7f115c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:15.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.585+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f115008f030 con 0x7f11601027e0 2024-02-22T16:12:15.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.805+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1160065f60 con 0x7f114c06cc30 2024-02-22T16:12:15.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.806+0000 7f11567fc700 1 -- 172.21.15.19:0/1047409875 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1160065f60 con 0x7f114c06cc30 2024-02-22T16:12:15.780 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:12:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:12:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:12:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:12:15.781 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:12:15.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f114c06cc30 msgr2=0x7f114c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f114c06cc30 0x7f114c06f0e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f115c00b3b0 tx=0x7f115c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:15.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 msgr2=0x7f116010cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 0x7f116010cb90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1150002a50 tx=0x7f1150004600 comp rx=0 tx=0).stop 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 shutdown_connections 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f114c06cc30 0x7f114c06f0e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1160101e40 0x7f116010c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 --2- 172.21.15.19:0/1047409875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11601027e0 0x7f116010cb90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 >> 172.21.15.19:0/1047409875 conn(0x7f1160071d90 msgr2=0x7f11600fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 shutdown_connections 2024-02-22T16:12:15.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.808+0000 7f1167a8a700 1 -- 172.21.15.19:0/1047409875 wait complete. 2024-02-22T16:12:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.911+0000 7fc2347f8700 1 -- 172.21.15.19:0/966142250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fe8f0 msgr2=0x7fc22c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.911+0000 7fc2347f8700 1 --2- 172.21.15.19:0/966142250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fe8f0 0x7fc22c107050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc21c004970 tx=0x7fc21c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.911+0000 7fc2347f8700 1 -- 172.21.15.19:0/966142250 shutdown_connections 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.911+0000 7fc2347f8700 1 --2- 172.21.15.19:0/966142250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fe8f0 0x7fc22c107050 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.911+0000 7fc2347f8700 1 --2- 172.21.15.19:0/966142250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.911+0000 7fc2347f8700 1 -- 172.21.15.19:0/966142250 >> 172.21.15.19:0/966142250 conn(0x7fc22c071d90 msgr2=0x7fc22c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.912+0000 7fc2347f8700 1 -- 172.21.15.19:0/966142250 shutdown_connections 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.912+0000 7fc2347f8700 1 -- 172.21.15.19:0/966142250 wait complete. 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.912+0000 7fc2347f8700 1 Processor -- start 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.912+0000 7fc2347f8700 1 -- start start 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc2347f8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc2347f8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc22c0fe8f0 0x7fc22c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc2347f8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c114500 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc2347f8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c10d830 con 0x7fc22c0fe8f0 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35862/0 (socket says 172.21.15.19:35862) 2024-02-22T16:12:15.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 -- 172.21.15.19:0/3310198137 learned_addr learned my addr 172.21.15.19:0/3310198137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:15.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc231d93700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc22c0fe8f0 0x7fc22c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 -- 172.21.15.19:0/3310198137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc22c0fe8f0 msgr2=0x7fc22c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:15.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc22c0fe8f0 0x7fc22c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:15.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 -- 172.21.15.19:0/3310198137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c005040 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc232594700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c113970 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc22800c830 tx=0x7fc228004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2280106f0 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc228010d00 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc228012e90 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c10db10 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.913+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c10dfd0 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.914+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c1046c0 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.915+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc228010850 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.915+0000 7fc222ffd700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc21806cc20 0x7fc21806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.915+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc22808bfb0 con 0x7fc22c0fdfe0 2024-02-22T16:12:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.915+0000 7fc231d93700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc21806cc20 0x7fc21806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:15.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.916+0000 7fc231d93700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc21806cc20 0x7fc21806f0d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc21c004f50 tx=0x7fc21c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:15.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:15.920+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc228059b10 con 0x7fc22c0fdfe0 2024-02-22T16:12:16.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.205+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc22c04bed0 con 0x7fc22c0fdfe0 2024-02-22T16:12:16.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.205+0000 7fc222ffd700 1 -- 172.21.15.19:0/3310198137 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc2280596a0 con 0x7fc22c0fdfe0 2024-02-22T16:12:16.180 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:12:16.180 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:12:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:12:16.180 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:12:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc21806cc20 msgr2=0x7fc21806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc21806cc20 0x7fc21806f0d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc21c004f50 tx=0x7fc21c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 msgr2=0x7fc22c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c113970 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc22800c830 tx=0x7fc228004470 comp rx=0 tx=0).stop 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 shutdown_connections 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc21806cc20 0x7fc21806f0d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc22c0fdfe0 0x7fc22c113970 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 --2- 172.21.15.19:0/3310198137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc22c0fe8f0 0x7fc22c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 >> 172.21.15.19:0/3310198137 conn(0x7fc22c071d90 msgr2=0x7fc22c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 shutdown_connections 2024-02-22T16:12:16.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:16.207+0000 7fc2347f8700 1 -- 172.21.15.19:0/3310198137 wait complete. 2024-02-22T16:12:16.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:16 smithi062 ceph-mon[106861]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:16.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:16 smithi062 ceph-mon[106861]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:16.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:16 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1663166539' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:12:16.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:16 smithi019 ceph-mon[173579]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:16.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:16 smithi019 ceph-mon[173579]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:16.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:16 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1663166539' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:12:17.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:17 smithi062 ceph-mon[106861]: pgmap v570: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:17.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:17 smithi062 ceph-mon[106861]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:17.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:17 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3310198137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:12:17.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:17 smithi019 ceph-mon[173579]: pgmap v570: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:17.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:17 smithi019 ceph-mon[173579]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:17.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:17 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3310198137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:12:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:19 smithi062 ceph-mon[106861]: pgmap v571: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:19 smithi019 ceph-mon[173579]: pgmap v571: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:21 smithi019 ceph-mon[173579]: pgmap v572: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:21 smithi062 ceph-mon[106861]: pgmap v572: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:23.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:23 smithi062 ceph-mon[106861]: pgmap v573: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:23.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:23 smithi019 ceph-mon[173579]: pgmap v573: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:25.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:25 smithi062 ceph-mon[106861]: pgmap v574: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:25.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:25 smithi019 ceph-mon[173579]: pgmap v574: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:27.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:27 smithi062 ceph-mon[106861]: pgmap v575: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:27.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:27 smithi019 ceph-mon[173579]: pgmap v575: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:29.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:29 smithi062 ceph-mon[106861]: pgmap v576: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:29.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:29 smithi019 ceph-mon[173579]: pgmap v576: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:31.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:31 smithi019 ceph-mon[173579]: pgmap v577: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:31.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:31 smithi062 ceph-mon[106861]: pgmap v577: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:33.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:33 smithi062 ceph-mon[106861]: pgmap v578: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:33.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:33 smithi019 ceph-mon[173579]: pgmap v578: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:34.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:34.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:35.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:35 smithi062 ceph-mon[106861]: pgmap v579: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:35.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:35 smithi019 ceph-mon[173579]: pgmap v579: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:37.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:37 smithi062 ceph-mon[106861]: pgmap v580: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:37.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:37 smithi019 ceph-mon[173579]: pgmap v580: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:39.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:39 smithi062 ceph-mon[106861]: pgmap v581: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:39.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:39 smithi019 ceph-mon[173579]: pgmap v581: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:40.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:40 smithi062 ceph-mon[106861]: pgmap v582: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:40.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:40 smithi019 ceph-mon[173579]: pgmap v582: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:42.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:42 smithi062 ceph-mon[106861]: pgmap v583: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:42.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:42 smithi019 ceph-mon[173579]: pgmap v583: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:45.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:45 smithi062 ceph-mon[106861]: pgmap v584: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:45.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:45 smithi019 ceph-mon[173579]: pgmap v584: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.323+0000 7f19c9de3700 1 -- 172.21.15.19:0/1725538070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 msgr2=0x7f19c4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.323+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1725538070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c4103320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19ac004940 tx=0x7f19ac009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 -- 172.21.15.19:0/1725538070 shutdown_connections 2024-02-22T16:12:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1725538070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c41038f0 0x7f19c40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1725538070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c4103320 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 -- 172.21.15.19:0/1725538070 >> 172.21.15.19:0/1725538070 conn(0x7f19c40f8ae0 msgr2=0x7f19c40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 -- 172.21.15.19:0/1725538070 shutdown_connections 2024-02-22T16:12:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 -- 172.21.15.19:0/1725538070 wait complete. 2024-02-22T16:12:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.324+0000 7f19c9de3700 1 Processor -- start 2024-02-22T16:12:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c9de3700 1 -- start start 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c9de3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c41139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c9de3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c41038f0 0x7f19c4113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c9de3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4114530 con 0x7f19c4102f50 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c9de3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c410d860 con 0x7f19c41038f0 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c41139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c41139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57536/0 (socket says 172.21.15.19:57536) 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c37fe700 1 -- 172.21.15.19:0/1211436740 learned_addr learned my addr 172.21.15.19:0/1211436740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.325+0000 7f19c2ffd700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c41038f0 0x7f19c4113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19c37fe700 1 -- 172.21.15.19:0/1211436740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c41038f0 msgr2=0x7f19c4113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19c37fe700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c41038f0 0x7f19c4113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19c37fe700 1 -- 172.21.15.19:0/1211436740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac005040 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19c37fe700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c41139a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19ac008d80 tx=0x7f19ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19ac00e030 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19ac004290 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c410dae0 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19ac01cda0 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.326+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c410e020 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.327+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c4100260 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.328+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f19ac01c600 con 0x7f19c4102f50 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.328+0000 7f19bbfff700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19b006cc30 0x7f19b006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.328+0000 7f19c2ffd700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19b006cc30 0x7f19b006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.328+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19ac08d900 con 0x7f19c4102f50 2024-02-22T16:12:46.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.329+0000 7f19c2ffd700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19b006cc30 0x7f19b006f0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f19c410f1d0 tx=0x7f19b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.333+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f19ac05b3c0 con 0x7f19c4102f50 2024-02-22T16:12:46.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.553+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19c404bf40 con 0x7f19b006cc30 2024-02-22T16:12:46.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.554+0000 7f19bbfff700 1 -- 172.21.15.19:0/1211436740 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f19c404bf40 con 0x7f19b006cc30 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19b006cc30 msgr2=0x7f19b006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19b006cc30 0x7f19b006f0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f19c410f1d0 tx=0x7f19b400b040 comp rx=0 tx=0).stop 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 msgr2=0x7f19c41139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c41139a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19ac008d80 tx=0x7f19ac009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 shutdown_connections 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19b006cc30 0x7f19b006f0e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19c4102f50 0x7f19c41139a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 --2- 172.21.15.19:0/1211436740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c41038f0 0x7f19c4113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.556+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 >> 172.21.15.19:0/1211436740 conn(0x7f19c40f8ae0 msgr2=0x7f19c40fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:46.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.557+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 shutdown_connections 2024-02-22T16:12:46.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.557+0000 7f19c9de3700 1 -- 172.21.15.19:0/1211436740 wait complete. 2024-02-22T16:12:46.542 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:12:46.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 -- 172.21.15.19:0/848451303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60102f40 msgr2=0x7f6c60103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 --2- 172.21.15.19:0/848451303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60102f40 0x7f6c60103310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6c50004940 tx=0x7f6c50009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:46.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 -- 172.21.15.19:0/848451303 shutdown_connections 2024-02-22T16:12:46.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 --2- 172.21.15.19:0/848451303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c601038e0 0x7f6c600fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 --2- 172.21.15.19:0/848451303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60102f40 0x7f6c60103310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 -- 172.21.15.19:0/848451303 >> 172.21.15.19:0/848451303 conn(0x7f6c600f8af0 msgr2=0x7f6c600faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 -- 172.21.15.19:0/848451303 shutdown_connections 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.661+0000 7f6c661f7700 1 -- 172.21.15.19:0/848451303 wait complete. 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.662+0000 7f6c661f7700 1 Processor -- start 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.662+0000 7f6c661f7700 1 -- start start 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.662+0000 7f6c661f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c60102f40 0x7f6c60113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.662+0000 7f6c661f7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 0x7f6c60113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.662+0000 7f6c661f7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c601144f0 con 0x7f6c601038e0 2024-02-22T16:12:46.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.662+0000 7f6c661f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6010d820 con 0x7f6c60102f40 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 0x7f6c60113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 0x7f6c60113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57548/0 (socket says 172.21.15.19:57548) 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 -- 172.21.15.19:0/1366827885 learned_addr learned my addr 172.21.15.19:0/1366827885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5f7fe700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c60102f40 0x7f6c60113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 -- 172.21.15.19:0/1366827885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c60102f40 msgr2=0x7f6c60113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c60102f40 0x7f6c60113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 -- 172.21.15.19:0/1366827885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48005040 con 0x7f6c601038e0 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c5effd700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 0x7f6c60113ea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009700 tx=0x7f6c48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:46.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c4800e7e0 con 0x7f6c601038e0 2024-02-22T16:12:46.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c50005040 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6010de50 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c4800f040 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.663+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c4801cbb0 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.664+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c400052e0 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.665+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6c4800e940 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.665+0000 7f6c57fff700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c4c06cc30 0x7f6c4c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.665+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6c4808bdf0 con 0x7f6c601038e0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.665+0000 7f6c5f7fe700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c4c06cc30 0x7f6c4c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.665+0000 7f6c5f7fe700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c4c06cc30 0x7f6c4c06f0e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009fd0 tx=0x7f6c50009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:46.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.670+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f6c48059830 con 0x7f6c601038e0 2024-02-22T16:12:46.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.890+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c40000bf0 con 0x7f6c4c06cc30 2024-02-22T16:12:46.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.891+0000 7f6c57fff700 1 -- 172.21.15.19:0/1366827885 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6c40000bf0 con 0x7f6c4c06cc30 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c4c06cc30 msgr2=0x7f6c4c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c4c06cc30 0x7f6c4c06f0e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009fd0 tx=0x7f6c50009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 msgr2=0x7f6c60113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 0x7f6c60113ea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009700 tx=0x7f6c48009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 shutdown_connections 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f6c4c06cc30 0x7f6c4c06f0e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c60102f40 0x7f6c60113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 --2- 172.21.15.19:0/1366827885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c601038e0 0x7f6c60113ea0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 >> 172.21.15.19:0/1366827885 conn(0x7f6c600f8af0 msgr2=0x7f6c600fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 shutdown_connections 2024-02-22T16:12:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.893+0000 7f6c661f7700 1 -- 172.21.15.19:0/1366827885 wait complete. 2024-02-22T16:12:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- 172.21.15.19:0/4133229235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 msgr2=0x7f3c1c0fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 --2- 172.21.15.19:0/4133229235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c0fb9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c004940 tx=0x7f3c0c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- 172.21.15.19:0/4133229235 shutdown_connections 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 --2- 172.21.15.19:0/4133229235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c0fb9c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 --2- 172.21.15.19:0/4133229235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c103140 0x7f3c1c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- 172.21.15.19:0/4133229235 >> 172.21.15.19:0/4133229235 conn(0x7f3c1c071560 msgr2=0x7f3c1c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- 172.21.15.19:0/4133229235 shutdown_connections 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- 172.21.15.19:0/4133229235 wait complete. 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 Processor -- start 2024-02-22T16:12:46.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- start start 2024-02-22T16:12:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c103140 0x7f3c1c114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c114660 con 0x7f3c1c0fb570 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.997+0000 7f3c237ae700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c10d990 con 0x7f3c1c103140 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57556/0 (socket says 172.21.15.19:57556) 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 -- 172.21.15.19:0/3186921298 learned_addr learned my addr 172.21.15.19:0/3186921298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c20d49700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c103140 0x7f3c1c114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 -- 172.21.15.19:0/3186921298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c103140 msgr2=0x7f3c1c114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c103140 0x7f3c1c114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 -- 172.21.15.19:0/3186921298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c005040 con 0x7f3c1c0fb570 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c2154a700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c113ad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3c18002a50 tx=0x7f3c180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c18007cc0 con 0x7f3c1c0fb570 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c10dc70 con 0x7f3c1c0fb570 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c1800f040 con 0x7f3c1c0fb570 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c1800d9c0 con 0x7f3c1c0fb570 2024-02-22T16:12:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.998+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c10e180 con 0x7f3c1c0fb570 2024-02-22T16:12:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:46.999+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1c1054d0 con 0x7f3c1c0fb570 2024-02-22T16:12:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.000+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3c1800db20 con 0x7f3c1c0fb570 2024-02-22T16:12:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.000+0000 7f3c11ffb700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c0806cc20 0x7f3c0806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.000+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3c1808bd80 con 0x7f3c1c0fb570 2024-02-22T16:12:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.000+0000 7f3c20d49700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c0806cc20 0x7f3c0806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.001+0000 7f3c20d49700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c0806cc20 0x7f3c0806f0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c10ecc0 tx=0x7f3c0c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.005+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3c180597c0 con 0x7f3c1c0fb570 2024-02-22T16:12:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.219+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c1c04bed0 con 0x7f3c0806cc20 2024-02-22T16:12:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.229+0000 7f3c11ffb700 1 -- 172.21.15.19:0/3186921298 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f3c1c04bed0 con 0x7f3c0806cc20 2024-02-22T16:12:47.204 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:12:47.204 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (19m) 13m ago 31m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:12:47.204 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (17m) 13m ago 31m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:12:47.204 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (17m) 12m ago 30m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (19m) 13m ago 30m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (13m) 13m ago 25m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (13m) 13m ago 25m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (12m) 12m ago 25m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (13m) 12m ago 25m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (21m) 13m ago 32m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (21m) 12m ago 30m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (18m) 13m ago 32m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (17m) 12m ago 29m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (20m) 13m ago 31m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (20m) 12m ago 30m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (16m) 13m ago 29m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (16m) 13m ago 28m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:12:47.205 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (15m) 13m ago 28m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:12:47.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (15m) 12m ago 27m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:12:47.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (14m) 12m ago 27m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:12:47.206 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (14m) 12m ago 26m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:12:47.206 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (20m) 13m ago 30m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:12:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c0806cc20 msgr2=0x7f3c0806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c0806cc20 0x7f3c0806f0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c10ecc0 tx=0x7f3c0c009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 msgr2=0x7f3c1c113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c113ad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3c18002a50 tx=0x7f3c180046b0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 shutdown_connections 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3c0806cc20 0x7f3c0806f0d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1c0fb570 0x7f3c1c113ad0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 --2- 172.21.15.19:0/3186921298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c1c103140 0x7f3c1c114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 >> 172.21.15.19:0/3186921298 conn(0x7f3c1c071560 msgr2=0x7f3c1c0f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 shutdown_connections 2024-02-22T16:12:47.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.231+0000 7f3c237ae700 1 -- 172.21.15.19:0/3186921298 wait complete. 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.333+0000 7fba90521700 1 -- 172.21.15.19:0/2198885112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 msgr2=0x7fba880fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.333+0000 7fba90521700 1 --2- 172.21.15.19:0/2198885112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba880fbe60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fba78004940 tx=0x7fba78009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- 172.21.15.19:0/2198885112 shutdown_connections 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 --2- 172.21.15.19:0/2198885112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba880fbe60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 --2- 172.21.15.19:0/2198885112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 0x7fba88102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- 172.21.15.19:0/2198885112 >> 172.21.15.19:0/2198885112 conn(0x7fba880f79e0 msgr2=0x7fba880f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- 172.21.15.19:0/2198885112 shutdown_connections 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- 172.21.15.19:0/2198885112 wait complete. 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 Processor -- start 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- start start 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 0x7fba8810ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba8810ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8810f530 con 0x7fba881027f0 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.334+0000 7fba90521700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8810f6a0 con 0x7fba88101e50 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba8810ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba8810ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57572/0 (socket says 172.21.15.19:57572) 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 -- 172.21.15.19:0/2171988759 learned_addr learned my addr 172.21.15.19:0/2171988759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8e2bd700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 0x7fba8810ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 -- 172.21.15.19:0/2171988759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 msgr2=0x7fba8810ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 0x7fba8810ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 -- 172.21.15.19:0/2171988759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78005040 con 0x7fba881027f0 2024-02-22T16:12:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8e2bd700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 0x7fba8810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba8dabc700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba8810ef80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fba78009180 tx=0x7fba78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7800e040 con 0x7fba881027f0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba78004290 con 0x7fba881027f0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba881179b0 con 0x7fba881027f0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7801c3f0 con 0x7fba881027f0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.335+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba88117ec0 con 0x7fba881027f0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.336+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba88104320 con 0x7fba881027f0 2024-02-22T16:12:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.337+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fba7800bc50 con 0x7fba881027f0 2024-02-22T16:12:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.337+0000 7fba7effd700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fba7406cc30 0x7fba7406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.337+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fba7808d2f0 con 0x7fba881027f0 2024-02-22T16:12:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.337+0000 7fba8e2bd700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fba7406cc30 0x7fba7406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.338+0000 7fba8e2bd700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fba7406cc30 0x7fba7406f0e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fba84009fd0 tx=0x7fba84009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.342+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fba78057470 con 0x7fba881027f0 2024-02-22T16:12:47.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:47 smithi062 ceph-mon[106861]: pgmap v585: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:47.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:47 smithi019 ceph-mon[173579]: pgmap v585: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.627+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fba8804bed0 con 0x7fba881027f0 2024-02-22T16:12:47.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.628+0000 7fba7effd700 1 -- 172.21.15.19:0/2171988759 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fba7805aa90 con 0x7fba881027f0 2024-02-22T16:12:47.603 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:12:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:12:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:12:47.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:12:47.604 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:12:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.629+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fba7406cc30 msgr2=0x7fba7406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.629+0000 7fba90521700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fba7406cc30 0x7fba7406f0e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fba84009fd0 tx=0x7fba84009ee0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 msgr2=0x7fba8810ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba8810ef80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fba78009180 tx=0x7fba78009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 shutdown_connections 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fba7406cc30 0x7fba7406f0e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fba88101e50 0x7fba8810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 --2- 172.21.15.19:0/2171988759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba881027f0 0x7fba8810ef80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 >> 172.21.15.19:0/2171988759 conn(0x7fba880f79e0 msgr2=0x7fba880f95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 shutdown_connections 2024-02-22T16:12:47.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.630+0000 7fba90521700 1 -- 172.21.15.19:0/2171988759 wait complete. 2024-02-22T16:12:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 -- 172.21.15.19:0/1798391380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 msgr2=0x7ff234102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/1798391380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff234102210 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff224004940 tx=0x7ff224009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 -- 172.21.15.19:0/1798391380 shutdown_connections 2024-02-22T16:12:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/1798391380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2341027e0 0x7ff2340fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/1798391380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff234102210 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 -- 172.21.15.19:0/1798391380 >> 172.21.15.19:0/1798391380 conn(0x7ff234071d90 msgr2=0x7ff234072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 -- 172.21.15.19:0/1798391380 shutdown_connections 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.733+0000 7ff23a2c5700 1 -- 172.21.15.19:0/1798391380 wait complete. 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff23a2c5700 1 Processor -- start 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff23a2c5700 1 -- start start 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff23a2c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff2341139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff23a2c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2341027e0 0x7ff234113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff23a2c5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234114560 con 0x7ff234101e40 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff23a2c5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23410d890 con 0x7ff2341027e0 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff233fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff2341139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff233fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff2341139d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44134/0 (socket says 172.21.15.19:44134) 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff233fff700 1 -- 172.21.15.19:0/3878381016 learned_addr learned my addr 172.21.15.19:0/3878381016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff2337fe700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2341027e0 0x7ff234113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff233fff700 1 -- 172.21.15.19:0/3878381016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2341027e0 msgr2=0x7ff234113f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff233fff700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2341027e0 0x7ff234113f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.734+0000 7ff233fff700 1 -- 172.21.15.19:0/3878381016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224005040 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.735+0000 7ff233fff700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff2341139d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff224009180 tx=0x7ff224009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.735+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22400e040 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.735+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff224013420 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.735+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22401c630 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.735+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23410db10 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.735+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23410e050 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.736+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2340738b0 con 0x7ff234101e40 2024-02-22T16:12:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.736+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff2240139e0 con 0x7ff234101e40 2024-02-22T16:12:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.737+0000 7ff230ff9700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff21c06cc20 0x7ff21c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.737+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff22408d2e0 con 0x7ff234101e40 2024-02-22T16:12:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.737+0000 7ff2337fe700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff21c06cc20 0x7ff21c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.737+0000 7ff2337fe700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff21c06cc20 0x7ff21c06f0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff228007fd0 tx=0x7ff228009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.741+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff22405ae40 con 0x7ff234101e40 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.988+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff23410efb0 con 0x7ff234101e40 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.988+0000 7ff230ff9700 1 -- 172.21.15.19:0/3878381016 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7ff22405a9d0 con 0x7ff234101e40 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:47.964 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:12:47.965 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:12:47.966 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:47.967 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:12:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff21c06cc20 msgr2=0x7ff21c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff21c06cc20 0x7ff21c06f0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff228007fd0 tx=0x7ff228009040 comp rx=0 tx=0).stop 2024-02-22T16:12:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 msgr2=0x7ff2341139d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff2341139d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff224009180 tx=0x7ff224009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 shutdown_connections 2024-02-22T16:12:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff21c06cc20 0x7ff21c06f0d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff234101e40 0x7ff2341139d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 --2- 172.21.15.19:0/3878381016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2341027e0 0x7ff234113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 >> 172.21.15.19:0/3878381016 conn(0x7ff234071d90 msgr2=0x7ff234065720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 shutdown_connections 2024-02-22T16:12:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:47.991+0000 7ff23a2c5700 1 -- 172.21.15.19:0/3878381016 wait complete. 2024-02-22T16:12:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 -- 172.21.15.19:0/1238417886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fe8f0 msgr2=0x7fd788107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/1238417886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fe8f0 0x7fd788107050 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd770004970 tx=0x7fd770009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 -- 172.21.15.19:0/1238417886 shutdown_connections 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/1238417886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fe8f0 0x7fd788107050 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/1238417886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fdfe0 0x7fd7880fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 -- 172.21.15.19:0/1238417886 >> 172.21.15.19:0/1238417886 conn(0x7fd788071d90 msgr2=0x7fd788072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 -- 172.21.15.19:0/1238417886 shutdown_connections 2024-02-22T16:12:48.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.096+0000 7fd78cf3c700 1 -- 172.21.15.19:0/1238417886 wait complete. 2024-02-22T16:12:48.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78cf3c700 1 Processor -- start 2024-02-22T16:12:48.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78cf3c700 1 -- start start 2024-02-22T16:12:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78cf3c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 0x7fd788113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:48.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78cf3c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fe8f0 0x7fd788113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78cf3c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788114500 con 0x7fd7880fdfe0 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78cf3c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78810d830 con 0x7fd7880fe8f0 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 0x7fd788113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 0x7fd788113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44148/0 (socket says 172.21.15.19:44148) 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 -- 172.21.15.19:0/2765394162 learned_addr learned my addr 172.21.15.19:0/2765394162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd785d9b700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fe8f0 0x7fd788113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 -- 172.21.15.19:0/2765394162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fe8f0 msgr2=0x7fd788113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fe8f0 0x7fd788113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 -- 172.21.15.19:0/2765394162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770005040 con 0x7fd7880fdfe0 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd785d9b700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fe8f0 0x7fd788113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.097+0000 7fd78659c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 0x7fd788113970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c830 tx=0x7fd778004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:48.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.098+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77800a760 con 0x7fd7880fdfe0 2024-02-22T16:12:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.098+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd77800ad70 con 0x7fd7880fdfe0 2024-02-22T16:12:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.098+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78810db10 con 0x7fd7880fdfe0 2024-02-22T16:12:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.098+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7780113a0 con 0x7fd7880fdfe0 2024-02-22T16:12:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.098+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78810e020 con 0x7fd7880fdfe0 2024-02-22T16:12:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.099+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7881046c0 con 0x7fd7880fdfe0 2024-02-22T16:12:48.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.099+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd778012400 con 0x7fd7880fdfe0 2024-02-22T16:12:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.099+0000 7fd77effd700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77406cc20 0x7fd77406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.100+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd77808c160 con 0x7fd7880fdfe0 2024-02-22T16:12:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.100+0000 7fd785d9b700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77406cc20 0x7fd77406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:48.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.100+0000 7fd785d9b700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77406cc20 0x7fd77406f0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd770004f50 tx=0x7fd770009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:48.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.104+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd778059b80 con 0x7fd7880fdfe0 2024-02-22T16:12:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.327+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd78804bed0 con 0x7fd77406cc20 2024-02-22T16:12:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.328+0000 7fd77effd700 1 -- 172.21.15.19:0/2765394162 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd78804bed0 con 0x7fd77406cc20 2024-02-22T16:12:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:12:48.303 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:12:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:12:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77406cc20 msgr2=0x7fd77406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77406cc20 0x7fd77406f0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd770004f50 tx=0x7fd770009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 msgr2=0x7fd788113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 0x7fd788113970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c830 tx=0x7fd778004470 comp rx=0 tx=0).stop 2024-02-22T16:12:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 shutdown_connections 2024-02-22T16:12:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd77406cc20 0x7fd77406f0d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7880fdfe0 0x7fd788113970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 --2- 172.21.15.19:0/2765394162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7880fe8f0 0x7fd788113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 >> 172.21.15.19:0/2765394162 conn(0x7fd788071d90 msgr2=0x7fd788101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 shutdown_connections 2024-02-22T16:12:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.330+0000 7fd78cf3c700 1 -- 172.21.15.19:0/2765394162 wait complete. 2024-02-22T16:12:48.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.433+0000 7f29df271700 1 -- 172.21.15.19:0/3484461747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 msgr2=0x7f29d8103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.433+0000 7f29df271700 1 --2- 172.21.15.19:0/3484461747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d8103510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29d4004940 tx=0x7f29d4009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:48.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.433+0000 7f29df271700 1 -- 172.21.15.19:0/3484461747 shutdown_connections 2024-02-22T16:12:48.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.433+0000 7f29df271700 1 --2- 172.21.15.19:0/3484461747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d80fb570 0x7f29d80fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.433+0000 7f29df271700 1 --2- 172.21.15.19:0/3484461747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d8103510 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.433+0000 7f29df271700 1 -- 172.21.15.19:0/3484461747 >> 172.21.15.19:0/3484461747 conn(0x7f29d8071560 msgr2=0x7f29d8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 -- 172.21.15.19:0/3484461747 shutdown_connections 2024-02-22T16:12:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 -- 172.21.15.19:0/3484461747 wait complete. 2024-02-22T16:12:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 Processor -- start 2024-02-22T16:12:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 -- start start 2024-02-22T16:12:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d80fb570 0x7f29d80fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d80fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80fd290 con 0x7f29d8103140 2024-02-22T16:12:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29df271700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80fd400 con 0x7f29d80fb570 2024-02-22T16:12:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29dc80c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d80fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29dc80c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d80fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44168/0 (socket says 172.21.15.19:44168) 2024-02-22T16:12:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.434+0000 7f29dc80c700 1 -- 172.21.15.19:0/3712926434 learned_addr learned my addr 172.21.15.19:0/3712926434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:12:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29dd00d700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d80fb570 0x7f29d80fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29dc80c700 1 -- 172.21.15.19:0/3712926434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d80fb570 msgr2=0x7f29d80fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29dc80c700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d80fb570 0x7f29d80fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29dc80c700 1 -- 172.21.15.19:0/3712926434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4005040 con 0x7f29d8103140 2024-02-22T16:12:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29dc80c700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d80fcc90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c8c0 tx=0x7f29c80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29c800a980 con 0x7f29d8103140 2024-02-22T16:12:48.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29c800ce60 con 0x7f29d8103140 2024-02-22T16:12:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d8119c80 con 0x7f29d8103140 2024-02-22T16:12:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29c80113a0 con 0x7f29d8103140 2024-02-22T16:12:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.435+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d811a160 con 0x7f29d8103140 2024-02-22T16:12:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.436+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d8100310 con 0x7f29d8103140 2024-02-22T16:12:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.437+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f29c8012400 con 0x7f29d8103140 2024-02-22T16:12:48.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.437+0000 7f29cdffb700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29c406cc20 0x7f29c406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:12:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.437+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f29c808c180 con 0x7f29d8103140 2024-02-22T16:12:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.437+0000 7f29dd00d700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29c406cc20 0x7f29c406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:12:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.438+0000 7f29dd00d700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29c406cc20 0x7f29c406f0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29d4004f90 tx=0x7f29d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:12:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.442+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f29c80561b0 con 0x7f29d8103140 2024-02-22T16:12:48.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:48 smithi062 ceph-mon[106861]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:48.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:48 smithi062 ceph-mon[106861]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:48.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:48 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2171988759' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:12:48.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:48 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3878381016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:12:48.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:48 smithi019 ceph-mon[173579]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:48.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:48 smithi019 ceph-mon[173579]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:48.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:48 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2171988759' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:12:48.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:48 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3878381016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:12:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.728+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f29d8073080 con 0x7f29d8103140 2024-02-22T16:12:48.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.729+0000 7f29cdffb700 1 -- 172.21.15.19:0/3712926434 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f29c80597d0 con 0x7f29d8103140 2024-02-22T16:12:48.705 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:12:48.705 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:12:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:12:48.705 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:12:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:12:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29c406cc20 msgr2=0x7f29c406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29c406cc20 0x7f29c406f0d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29d4004f90 tx=0x7f29d4009e30 comp rx=0 tx=0).stop 2024-02-22T16:12:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 msgr2=0x7f29d80fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:12:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d80fcc90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29c800c8c0 tx=0x7f29c80044c0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 shutdown_connections 2024-02-22T16:12:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f29c406cc20 0x7f29c406f0d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29d80fb570 0x7f29d80fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 --2- 172.21.15.19:0/3712926434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29d8103140 0x7f29d80fcc90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:12:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 >> 172.21.15.19:0/3712926434 conn(0x7f29d8071560 msgr2=0x7f29d80fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:12:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 shutdown_connections 2024-02-22T16:12:48.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:12:48.731+0000 7f29df271700 1 -- 172.21.15.19:0/3712926434 wait complete. 2024-02-22T16:12:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:49 smithi062 ceph-mon[106861]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:49 smithi062 ceph-mon[106861]: pgmap v586: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:49 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3712926434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:12:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:49 smithi019 ceph-mon[173579]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:49 smithi019 ceph-mon[173579]: pgmap v586: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:12:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:49 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3712926434' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:12:50.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:50 smithi062 ceph-mon[106861]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:50.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:50 smithi019 ceph-mon[173579]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:12:51.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:51 smithi019 ceph-mon[173579]: pgmap v587: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:51.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:51 smithi062 ceph-mon[106861]: pgmap v587: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:53.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:53 smithi062 ceph-mon[106861]: pgmap v588: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:53.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:53 smithi019 ceph-mon[173579]: pgmap v588: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:55.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:55 smithi062 ceph-mon[106861]: pgmap v589: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:55.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:55 smithi019 ceph-mon[173579]: pgmap v589: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:57 smithi062 ceph-mon[106861]: pgmap v590: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:57 smithi019 ceph-mon[173579]: pgmap v590: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:12:59 smithi062 ceph-mon[106861]: pgmap v591: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:12:59.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:12:59 smithi019 ceph-mon[173579]: pgmap v591: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:00.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:00 smithi062 ceph-mon[106861]: pgmap v592: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:00.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:00 smithi019 ceph-mon[173579]: pgmap v592: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:02.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:02 smithi062 ceph-mon[106861]: pgmap v593: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:02.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:02 smithi019 ceph-mon[173579]: pgmap v593: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:05.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:05 smithi062 ceph-mon[106861]: pgmap v594: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:05.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:05 smithi019 ceph-mon[173579]: pgmap v594: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:07.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:07 smithi062 ceph-mon[106861]: pgmap v595: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:07.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:07 smithi019 ceph-mon[173579]: pgmap v595: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:09.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:09 smithi062 ceph-mon[106861]: pgmap v596: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:09.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:09 smithi019 ceph-mon[173579]: pgmap v596: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:11.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:11 smithi019 ceph-mon[173579]: pgmap v597: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:11.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:11 smithi062 ceph-mon[106861]: pgmap v597: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:13.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:13 smithi062 ceph-mon[106861]: pgmap v598: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:13.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:13 smithi019 ceph-mon[173579]: pgmap v598: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:15.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:15 smithi062 ceph-mon[106861]: pgmap v599: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:15.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:15 smithi019 ceph-mon[173579]: pgmap v599: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:17.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:17 smithi062 ceph-mon[106861]: pgmap v600: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:17.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:17 smithi019 ceph-mon[173579]: pgmap v600: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 -- 172.21.15.19:0/1350977742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fe8f0 msgr2=0x7f28c0107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 --2- 172.21.15.19:0/1350977742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fe8f0 0x7f28c0107050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f28b4004970 tx=0x7f28b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 -- 172.21.15.19:0/1350977742 shutdown_connections 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 --2- 172.21.15.19:0/1350977742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fe8f0 0x7f28c0107050 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 --2- 172.21.15.19:0/1350977742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fdfe0 0x7f28c00fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 -- 172.21.15.19:0/1350977742 >> 172.21.15.19:0/1350977742 conn(0x7f28c0071d90 msgr2=0x7f28c0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 -- 172.21.15.19:0/1350977742 shutdown_connections 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 -- 172.21.15.19:0/1350977742 wait complete. 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 Processor -- start 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.850+0000 7f28c86f6700 1 -- start start 2024-02-22T16:13:18.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c86f6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 0x7f28c0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c86f6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fe8f0 0x7f28c0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c86f6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0114500 con 0x7f28c00fdfe0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c86f6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c010d830 con 0x7f28c00fe8f0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 0x7f28c0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 0x7f28c0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42618/0 (socket says 172.21.15.19:42618) 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 -- 172.21.15.19:0/2575604453 learned_addr learned my addr 172.21.15.19:0/2575604453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c5c91700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fe8f0 0x7f28c0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 -- 172.21.15.19:0/2575604453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fe8f0 msgr2=0x7f28c0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fe8f0 0x7f28c0113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 -- 172.21.15.19:0/2575604453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4005040 con 0x7f28c00fdfe0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c5c91700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fe8f0 0x7f28c0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c6492700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 0x7f28c0113970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f28b0002a50 tx=0x7f28b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b0007cc0 con 0x7f28c00fdfe0 2024-02-22T16:13:18.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.851+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c010db10 con 0x7f28c00fdfe0 2024-02-22T16:13:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.852+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28b000f040 con 0x7f28c00fdfe0 2024-02-22T16:13:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.852+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b0009540 con 0x7f28c00fdfe0 2024-02-22T16:13:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.852+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c010e020 con 0x7f28c00fdfe0 2024-02-22T16:13:18.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.853+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c01046c0 con 0x7f28c00fdfe0 2024-02-22T16:13:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.853+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f28b00096a0 con 0x7f28c00fdfe0 2024-02-22T16:13:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.854+0000 7f28beffd700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28a806cc20 0x7f28a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.854+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f28b008be80 con 0x7f28c00fdfe0 2024-02-22T16:13:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.858+0000 7f28c5c91700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28a806cc20 0x7f28a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:18.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.858+0000 7f28c5c91700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28a806cc20 0x7f28a806f0d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f28b4000c00 tx=0x7f28b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:18.863+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f28b00598c0 con 0x7f28c00fdfe0 2024-02-22T16:13:19.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.083+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28c004bf40 con 0x7f28a806cc20 2024-02-22T16:13:19.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.084+0000 7f28beffd700 1 -- 172.21.15.19:0/2575604453 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f28c004bf40 con 0x7f28a806cc20 2024-02-22T16:13:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28a806cc20 msgr2=0x7f28a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28a806cc20 0x7f28a806f0d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f28b4000c00 tx=0x7f28b4009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 msgr2=0x7f28c0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 0x7f28c0113970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f28b0002a50 tx=0x7f28b00046b0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 shutdown_connections 2024-02-22T16:13:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28a806cc20 0x7f28a806f0d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28c00fdfe0 0x7f28c0113970 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 --2- 172.21.15.19:0/2575604453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28c00fe8f0 0x7f28c0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 >> 172.21.15.19:0/2575604453 conn(0x7f28c0071d90 msgr2=0x7f28c0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 shutdown_connections 2024-02-22T16:13:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.086+0000 7f28c86f6700 1 -- 172.21.15.19:0/2575604453 wait complete. 2024-02-22T16:13:19.071 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:13:19.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 -- 172.21.15.19:0/1177690962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fc6e0 msgr2=0x7f8ec4109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/1177690962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fc6e0 0x7f8ec4109440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8004940 tx=0x7f8eb8009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 -- 172.21.15.19:0/1177690962 shutdown_connections 2024-02-22T16:13:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/1177690962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fc6e0 0x7f8ec4109440 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/1177690962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec40fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 -- 172.21.15.19:0/1177690962 >> 172.21.15.19:0/1177690962 conn(0x7f8ec4071d90 msgr2=0x7f8ec4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 -- 172.21.15.19:0/1177690962 shutdown_connections 2024-02-22T16:13:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.191+0000 7f8eca3e4700 1 -- 172.21.15.19:0/1177690962 wait complete. 2024-02-22T16:13:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8eca3e4700 1 Processor -- start 2024-02-22T16:13:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8eca3e4700 1 -- start start 2024-02-22T16:13:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8eca3e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec4115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8eca3e4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec40fc6e0 0x7f8ec4116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8eca3e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec41166d0 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8eca3e4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec410fa00 con 0x7f8ec40fc6e0 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec4115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.192+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec4115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42628/0 (socket says 172.21.15.19:42628) 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec3fff700 1 -- 172.21.15.19:0/594805707 learned_addr learned my addr 172.21.15.19:0/594805707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec37fe700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec40fc6e0 0x7f8ec4116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec3fff700 1 -- 172.21.15.19:0/594805707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec40fc6e0 msgr2=0x7f8ec4116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec3fff700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec40fc6e0 0x7f8ec4116080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec3fff700 1 -- 172.21.15.19:0/594805707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb8005040 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec3fff700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec4115b40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400eb70 tx=0x7f8eb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eb4007cc0 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec410fc80 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8eb400f040 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8eb40095f0 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.193+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec41101c0 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.194+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec4106b70 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.195+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8eb4009750 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.195+0000 7f8ec0ff9700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8eac06cc30 0x7f8eac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.195+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8eb408bce0 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.195+0000 7f8ec37fe700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8eac06cc30 0x7f8eac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.196+0000 7f8ec37fe700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8eac06cc30 0x7f8eac06f0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800b0d0 tx=0x7f8eb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:19.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.200+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8eb4059720 con 0x7f8ec40fbdd0 2024-02-22T16:13:19.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.421+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ec404bf40 con 0x7f8eac06cc30 2024-02-22T16:13:19.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.422+0000 7f8ec0ff9700 1 -- 172.21.15.19:0/594805707 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ec404bf40 con 0x7f8eac06cc30 2024-02-22T16:13:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8eac06cc30 msgr2=0x7f8eac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8eac06cc30 0x7f8eac06f0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800b0d0 tx=0x7f8eb8009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 msgr2=0x7f8ec4115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec4115b40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400eb70 tx=0x7f8eb400b040 comp rx=0 tx=0).stop 2024-02-22T16:13:19.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 shutdown_connections 2024-02-22T16:13:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8eac06cc30 0x7f8eac06f0e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec40fbdd0 0x7f8ec4115b40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 --2- 172.21.15.19:0/594805707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec40fc6e0 0x7f8ec4116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 >> 172.21.15.19:0/594805707 conn(0x7f8ec4071d90 msgr2=0x7f8ec40fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 shutdown_connections 2024-02-22T16:13:19.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.424+0000 7f8eca3e4700 1 -- 172.21.15.19:0/594805707 wait complete. 2024-02-22T16:13:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:19 smithi062 ceph-mon[106861]: pgmap v601: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:19.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:19 smithi019 ceph-mon[173579]: pgmap v601: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:19.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.527+0000 7f6076e05700 1 -- 172.21.15.19:0/695605392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 msgr2=0x7f60700fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.527+0000 7f6076e05700 1 --2- 172.21.15.19:0/695605392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60700fbea0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f605c004940 tx=0x7f605c009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 -- 172.21.15.19:0/695605392 shutdown_connections 2024-02-22T16:13:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 --2- 172.21.15.19:0/695605392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60700fbea0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 --2- 172.21.15.19:0/695605392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6070101e40 0x7f6070102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 -- 172.21.15.19:0/695605392 >> 172.21.15.19:0/695605392 conn(0x7f6070071d90 msgr2=0x7f6070072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 -- 172.21.15.19:0/695605392 shutdown_connections 2024-02-22T16:13:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 -- 172.21.15.19:0/695605392 wait complete. 2024-02-22T16:13:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 Processor -- start 2024-02-22T16:13:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.528+0000 7f6076e05700 1 -- start start 2024-02-22T16:13:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6076e05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6070101e40 0x7f6070115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6076e05700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60701160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6076e05700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701166f0 con 0x7f60701027e0 2024-02-22T16:13:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6076e05700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010fa20 con 0x7f6070101e40 2024-02-22T16:13:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60701160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60701160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42650/0 (socket says 172.21.15.19:42650) 2024-02-22T16:13:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 -- 172.21.15.19:0/1689445080 learned_addr learned my addr 172.21.15.19:0/1689445080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6074ba1700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6070101e40 0x7f6070115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 -- 172.21.15.19:0/1689445080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6070101e40 msgr2=0x7f6070115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6070101e40 0x7f6070115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 -- 172.21.15.19:0/1689445080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c005040 con 0x7f60701027e0 2024-02-22T16:13:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606ffff700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60701160a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f605c009380 tx=0x7f605c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605c00e040 con 0x7f60701027e0 2024-02-22T16:13:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f605c00bc50 con 0x7f60701027e0 2024-02-22T16:13:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607010fca0 con 0x7f60701027e0 2024-02-22T16:13:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605c01c3f0 con 0x7f60701027e0 2024-02-22T16:13:19.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.529+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701101e0 con 0x7f60701027e0 2024-02-22T16:13:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.530+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60700738b0 con 0x7f60701027e0 2024-02-22T16:13:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.531+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f605c013420 con 0x7f60701027e0 2024-02-22T16:13:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.531+0000 7f606d7fa700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f606006cc20 0x7f606006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.531+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f605c08d4a0 con 0x7f60701027e0 2024-02-22T16:13:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.531+0000 7f6074ba1700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f606006cc20 0x7f606006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.532+0000 7f6074ba1700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f606006cc20 0x7f606006f0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6064009fd0 tx=0x7f6064009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:19.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.536+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f605c05af60 con 0x7f60701027e0 2024-02-22T16:13:19.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.746+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f607004bed0 con 0x7f606006cc20 2024-02-22T16:13:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.756+0000 7f606d7fa700 1 -- 172.21.15.19:0/1689445080 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f607004bed0 con 0x7f606006cc20 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (20m) 13m ago 31m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (17m) 13m ago 31m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (17m) 13m ago 30m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (20m) 13m ago 31m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (13m) 13m ago 25m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (14m) 13m ago 25m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (13m) 13m ago 25m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (13m) 13m ago 25m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (22m) 13m ago 33m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (21m) 13m ago 30m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:13:19.731 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (18m) 13m ago 33m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (18m) 13m ago 30m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (21m) 13m ago 31m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (21m) 13m ago 30m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (17m) 13m ago 29m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (16m) 13m ago 29m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (16m) 13m ago 28m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (15m) 13m ago 28m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (15m) 13m ago 27m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (14m) 13m ago 27m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:13:19.732 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (20m) 13m ago 31m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f606006cc20 msgr2=0x7f606006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f606006cc20 0x7f606006f0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6064009fd0 tx=0x7f6064009f60 comp rx=0 tx=0).stop 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 msgr2=0x7f60701160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60701160a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f605c009380 tx=0x7f605c009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 shutdown_connections 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f606006cc20 0x7f606006f0d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6070101e40 0x7f6070115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 --2- 172.21.15.19:0/1689445080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60701027e0 0x7f60701160a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 >> 172.21.15.19:0/1689445080 conn(0x7f6070071d90 msgr2=0x7f60700657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 shutdown_connections 2024-02-22T16:13:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.758+0000 7f6076e05700 1 -- 172.21.15.19:0/1689445080 wait complete. 2024-02-22T16:13:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.861+0000 7f367de7e700 1 -- 172.21.15.19:0/1867715240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36780702d0 msgr2=0x7f36780706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.861+0000 7f367de7e700 1 --2- 172.21.15.19:0/1867715240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36780702d0 0x7f36780706a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3668004970 tx=0x7f3668009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- 172.21.15.19:0/1867715240 shutdown_connections 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 --2- 172.21.15.19:0/1867715240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3678070be0 0x7f36781092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 --2- 172.21.15.19:0/1867715240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36780702d0 0x7f36780706a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- 172.21.15.19:0/1867715240 >> 172.21.15.19:0/1867715240 conn(0x7f36780f8ae0 msgr2=0x7f36780faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- 172.21.15.19:0/1867715240 shutdown_connections 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- 172.21.15.19:0/1867715240 wait complete. 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 Processor -- start 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- start start 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36780702d0 0x7f3678115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 0x7f3678116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36781166c0 con 0x7f3678070be0 2024-02-22T16:13:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.862+0000 7f367de7e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367810f9f0 con 0x7f36780702d0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 0x7f3678116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 0x7f3678116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42668/0 (socket says 172.21.15.19:42668) 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 -- 172.21.15.19:0/2480648299 learned_addr learned my addr 172.21.15.19:0/2480648299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f36777fe700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36780702d0 0x7f3678115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 -- 172.21.15.19:0/2480648299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36780702d0 msgr2=0x7f3678115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36780702d0 0x7f3678115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 -- 172.21.15.19:0/2480648299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668005040 con 0x7f3678070be0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f3676ffd700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 0x7f3678116070 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f366c002a80 tx=0x7f366c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366c007cc0 con 0x7f3678070be0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f366c00f040 con 0x7f3678070be0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367810fcd0 con 0x7f3678070be0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366c00da60 con 0x7f3678070be0 2024-02-22T16:13:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.863+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3678110210 con 0x7f3678070be0 2024-02-22T16:13:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.864+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3678106980 con 0x7f3678070be0 2024-02-22T16:13:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.865+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f366c00dbc0 con 0x7f3678070be0 2024-02-22T16:13:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.865+0000 7f365ffff700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366006cc30 0x7f366006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.865+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f366c08bcc0 con 0x7f3678070be0 2024-02-22T16:13:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.865+0000 7f36777fe700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366006cc30 0x7f366006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.866+0000 7f36777fe700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366006cc30 0x7f366006f0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3668004940 tx=0x7f3668009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:19.870+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f366c059700 con 0x7f3678070be0 2024-02-22T16:13:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.164+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3678111180 con 0x7f3678070be0 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.165+0000 7f365ffff700 1 -- 172.21.15.19:0/2480648299 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f3678111180 con 0x7f3678070be0 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:13:20.139 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:13:20.140 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366006cc30 msgr2=0x7f366006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366006cc30 0x7f366006f0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3668004940 tx=0x7f3668009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 msgr2=0x7f3678116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 0x7f3678116070 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f366c002a80 tx=0x7f366c0046b0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 shutdown_connections 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f366006cc30 0x7f366006f0e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36780702d0 0x7f3678115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 --2- 172.21.15.19:0/2480648299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3678070be0 0x7f3678116070 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 >> 172.21.15.19:0/2480648299 conn(0x7f36780f8ae0 msgr2=0x7f3678103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 shutdown_connections 2024-02-22T16:13:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.167+0000 7f367de7e700 1 -- 172.21.15.19:0/2480648299 wait complete. 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.270+0000 7f944e779700 1 -- 172.21.15.19:0/1465314437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94481038e0 msgr2=0x7f94480fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.270+0000 7f944e779700 1 --2- 172.21.15.19:0/1465314437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94481038e0 0x7f94480fcf50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f943c004940 tx=0x7f943c009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.270+0000 7f944e779700 1 -- 172.21.15.19:0/1465314437 shutdown_connections 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.270+0000 7f944e779700 1 --2- 172.21.15.19:0/1465314437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94481038e0 0x7f94480fcf50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.270+0000 7f944e779700 1 --2- 172.21.15.19:0/1465314437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9448102f40 0x7f9448103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.270+0000 7f944e779700 1 -- 172.21.15.19:0/1465314437 >> 172.21.15.19:0/1465314437 conn(0x7f94480f8af0 msgr2=0x7f94480faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 -- 172.21.15.19:0/1465314437 shutdown_connections 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 -- 172.21.15.19:0/1465314437 wait complete. 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 Processor -- start 2024-02-22T16:13:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 -- start start 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 0x7f9448113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94481038e0 0x7f9448113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94481145d0 con 0x7f9448102f40 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.271+0000 7f944e779700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944810d900 con 0x7f94481038e0 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 0x7f9448113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 0x7f9448113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42688/0 (socket says 172.21.15.19:42688) 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 -- 172.21.15.19:0/1520298618 learned_addr learned my addr 172.21.15.19:0/1520298618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f94477fe700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94481038e0 0x7f9448113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 -- 172.21.15.19:0/1520298618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94481038e0 msgr2=0x7f9448113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94481038e0 0x7f9448113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 -- 172.21.15.19:0/1520298618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c005040 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9447fff700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 0x7f9448113a40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9438002a50 tx=0x7f94380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9438007cc0 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f943800f040 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944810dbe0 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9438009540 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.272+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944810e120 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.273+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448105410 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.274+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f94380096a0 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.274+0000 7f9444ff9700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f943006cc20 0x7f943006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.274+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f943808be80 con 0x7f9448102f40 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.274+0000 7f94477fe700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f943006cc20 0x7f943006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.275+0000 7f94477fe700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f943006cc20 0x7f943006f0d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f944810f270 tx=0x7f943c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.279+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f94380598c0 con 0x7f9448102f40 2024-02-22T16:13:20.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:20 smithi062 ceph-mon[106861]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:20.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:20 smithi019 ceph-mon[173579]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.528+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f944804bed0 con 0x7f9448102f40 2024-02-22T16:13:20.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.528+0000 7f9444ff9700 1 -- 172.21.15.19:0/1520298618 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f9438059450 con 0x7f9448102f40 2024-02-22T16:13:20.502 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:13:20.503 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:13:20.504 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:20.505 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.530+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f943006cc20 msgr2=0x7f943006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.530+0000 7f944e779700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f943006cc20 0x7f943006f0d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f944810f270 tx=0x7f943c009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.530+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 msgr2=0x7f9448113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.530+0000 7f944e779700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 0x7f9448113a40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9438002a50 tx=0x7f94380046b0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.530+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 shutdown_connections 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.530+0000 7f944e779700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f943006cc20 0x7f943006f0d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.531+0000 7f944e779700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448102f40 0x7f9448113a40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.531+0000 7f944e779700 1 --2- 172.21.15.19:0/1520298618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94481038e0 0x7f9448113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.531+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 >> 172.21.15.19:0/1520298618 conn(0x7f94480f8af0 msgr2=0x7f94480fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.531+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 shutdown_connections 2024-02-22T16:13:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.531+0000 7f944e779700 1 -- 172.21.15.19:0/1520298618 wait complete. 2024-02-22T16:13:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 -- 172.21.15.19:0/694887678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 msgr2=0x7f7364102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 --2- 172.21.15.19:0/694887678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f7364102220 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7358004940 tx=0x7f7358009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 -- 172.21.15.19:0/694887678 shutdown_connections 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 --2- 172.21.15.19:0/694887678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73641027f0 0x7f73640fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 --2- 172.21.15.19:0/694887678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f7364102220 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 -- 172.21.15.19:0/694887678 >> 172.21.15.19:0/694887678 conn(0x7f73640f79e0 msgr2=0x7f73640f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.634+0000 7f736ab8a700 1 -- 172.21.15.19:0/694887678 shutdown_connections 2024-02-22T16:13:20.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 -- 172.21.15.19:0/694887678 wait complete. 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 Processor -- start 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 -- start start 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f736410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73641027f0 0x7f736410ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736410f520 con 0x7f7364101e50 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.635+0000 7f736ab8a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736410f690 con 0x7f73641027f0 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f736410ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f736410ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42708/0 (socket says 172.21.15.19:42708) 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 -- 172.21.15.19:0/3469043798 learned_addr learned my addr 172.21.15.19:0/3469043798 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7363fff700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73641027f0 0x7f736410ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 -- 172.21.15.19:0/3469043798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73641027f0 msgr2=0x7f736410ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73641027f0 0x7f736410ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 -- 172.21.15.19:0/3469043798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358005040 con 0x7f7364101e50 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f7368926700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f736410ea50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7358009fd0 tx=0x7f7358009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735800e040 con 0x7f7364101e50 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7358013420 con 0x7f7364101e50 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73641179c0 con 0x7f7364101e50 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735801c630 con 0x7f7364101e50 2024-02-22T16:13:20.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.636+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7364117e80 con 0x7f7364101e50 2024-02-22T16:13:20.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.637+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73640ff160 con 0x7f7364101e50 2024-02-22T16:13:20.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.638+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7358013590 con 0x7f7364101e50 2024-02-22T16:13:20.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.639+0000 7f73617fa700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f735406cc30 0x7f735406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.639+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f735808dcd0 con 0x7f7364101e50 2024-02-22T16:13:20.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.639+0000 7f7363fff700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f735406cc30 0x7f735406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.639+0000 7f7363fff700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f735406cc30 0x7f735406f0e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f73640669c0 tx=0x7f73500043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:20.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.643+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7358057da0 con 0x7f7364101e50 2024-02-22T16:13:20.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.862+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7364066700 con 0x7f735406cc30 2024-02-22T16:13:20.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.863+0000 7f73617fa700 1 -- 172.21.15.19:0/3469043798 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7364066700 con 0x7f735406cc30 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:13:20.838 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:13:20.839 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:13:20.839 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:13:20.839 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:13:20.839 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:13:20.839 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.865+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f735406cc30 msgr2=0x7f735406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.865+0000 7f736ab8a700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f735406cc30 0x7f735406f0e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f73640669c0 tx=0x7f73500043c0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.865+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 msgr2=0x7f736410ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.865+0000 7f736ab8a700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f736410ea50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7358009fd0 tx=0x7f7358009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 shutdown_connections 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f735406cc30 0x7f735406f0e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7364101e50 0x7f736410ea50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 --2- 172.21.15.19:0/3469043798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73641027f0 0x7f736410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 >> 172.21.15.19:0/3469043798 conn(0x7f73640f79e0 msgr2=0x7f73640f95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:20.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 shutdown_connections 2024-02-22T16:13:20.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.866+0000 7f736ab8a700 1 -- 172.21.15.19:0/3469043798 wait complete. 2024-02-22T16:13:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/2237241860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fe8f0 msgr2=0x7f1cb4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/2237241860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fe8f0 0x7f1cb4107050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004970 tx=0x7f1ca8009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/2237241860 shutdown_connections 2024-02-22T16:13:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/2237241860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fe8f0 0x7f1cb4107050 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/2237241860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb40fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/2237241860 >> 172.21.15.19:0/2237241860 conn(0x7f1cb4071d90 msgr2=0x7f1cb4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/2237241860 shutdown_connections 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.969+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/2237241860 wait complete. 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb8f1c700 1 Processor -- start 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb8f1c700 1 -- start start 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb8f1c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb8f1c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cb40fe8f0 0x7f1cb4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb8f1c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb4114500 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb8f1c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb410d830 con 0x7f1cb40fe8f0 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb4113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42716/0 (socket says 172.21.15.19:42716) 2024-02-22T16:13:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb259c700 1 -- 172.21.15.19:0/4026619702 learned_addr learned my addr 172.21.15.19:0/4026619702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb1d9b700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cb40fe8f0 0x7f1cb4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb259c700 1 -- 172.21.15.19:0/4026619702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cb40fe8f0 msgr2=0x7f1cb4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb259c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cb40fe8f0 0x7f1cb4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.970+0000 7f1cb259c700 1 -- 172.21.15.19:0/4026619702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8005040 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.971+0000 7f1cb259c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb4113970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4002a50 tx=0x7f1ca40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.971+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca4007cc0 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.971+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb410db10 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.971+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb410e020 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.971+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ca400f040 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.971+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca400d8d0 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.972+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940052e0 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.972+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1ca400da30 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.973+0000 7f1ca2ffd700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1c9c06cc20 0x7f1c9c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.973+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1ca408bcd0 con 0x7f1cb40fdfe0 2024-02-22T16:13:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.973+0000 7f1cb1d9b700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1c9c06cc20 0x7f1c9c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.973+0000 7f1cb1d9b700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1c9c06cc20 0x7f1c9c06f0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004f50 tx=0x7f1ca8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:20.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:20.977+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1ca4059710 con 0x7f1cb40fdfe0 2024-02-22T16:13:21.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:21 smithi019 ceph-mon[173579]: pgmap v602: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:21 smithi019 ceph-mon[173579]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:21 smithi019 ceph-mon[173579]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:21 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2480648299' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:13:21.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:21 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1520298618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:13:21.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.264+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c94005590 con 0x7f1cb40fdfe0 2024-02-22T16:13:21.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.265+0000 7f1ca2ffd700 1 -- 172.21.15.19:0/4026619702 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1ca40592a0 con 0x7f1cb40fdfe0 2024-02-22T16:13:21.239 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:13:21.239 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:13:21.239 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:13:21.239 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:13:21.239 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1c9c06cc20 msgr2=0x7f1c9c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1c9c06cc20 0x7f1c9c06f0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004f50 tx=0x7f1ca8009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 msgr2=0x7f1cb4113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb4113970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4002a50 tx=0x7f1ca40046b0 comp rx=0 tx=0).stop 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 shutdown_connections 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1c9c06cc20 0x7f1c9c06f0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb40fdfe0 0x7f1cb4113970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 --2- 172.21.15.19:0/4026619702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cb40fe8f0 0x7f1cb4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 >> 172.21.15.19:0/4026619702 conn(0x7f1cb4071d90 msgr2=0x7f1cb4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 shutdown_connections 2024-02-22T16:13:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:21.267+0000 7f1cb8f1c700 1 -- 172.21.15.19:0/4026619702 wait complete. 2024-02-22T16:13:21.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:21 smithi062 ceph-mon[106861]: pgmap v602: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:21 smithi062 ceph-mon[106861]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:21 smithi062 ceph-mon[106861]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:21 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2480648299' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:13:21.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:21 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1520298618' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:13:22.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:22 smithi062 ceph-mon[106861]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:22.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:22 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4026619702' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:13:22.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:22 smithi019 ceph-mon[173579]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:22.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:22 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4026619702' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:13:23.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:23 smithi062 ceph-mon[106861]: pgmap v603: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:23.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:23 smithi019 ceph-mon[173579]: pgmap v603: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:24.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:24 smithi062 ceph-mon[106861]: pgmap v604: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:24.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:24 smithi019 ceph-mon[173579]: pgmap v604: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:26.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:26 smithi062 ceph-mon[106861]: pgmap v605: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:26.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:26 smithi019 ceph-mon[173579]: pgmap v605: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:29.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:29 smithi062 ceph-mon[106861]: pgmap v606: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:29.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:29 smithi019 ceph-mon[173579]: pgmap v606: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:31.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:31 smithi019 ceph-mon[173579]: pgmap v607: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:31.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:31 smithi062 ceph-mon[106861]: pgmap v607: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:33.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:33 smithi062 ceph-mon[106861]: pgmap v608: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:33.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:33 smithi019 ceph-mon[173579]: pgmap v608: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:34.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:34.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:35.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:35 smithi062 ceph-mon[106861]: pgmap v609: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:35.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:35 smithi019 ceph-mon[173579]: pgmap v609: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:37.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:37 smithi062 ceph-mon[106861]: pgmap v610: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:37.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:37 smithi019 ceph-mon[173579]: pgmap v610: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:39.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:39 smithi062 ceph-mon[106861]: pgmap v611: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:39.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:39 smithi019 ceph-mon[173579]: pgmap v611: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:41.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:41 smithi019 ceph-mon[173579]: pgmap v612: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:41.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:41 smithi062 ceph-mon[106861]: pgmap v612: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:43.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:43 smithi062 ceph-mon[106861]: pgmap v613: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:43.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:43 smithi019 ceph-mon[173579]: pgmap v613: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:45.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:45 smithi062 ceph-mon[106861]: pgmap v614: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:45.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:45 smithi019 ceph-mon[173579]: pgmap v614: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:47.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:47 smithi062 ceph-mon[106861]: pgmap v615: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:47.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:47 smithi019 ceph-mon[173579]: pgmap v615: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:49 smithi062 ceph-mon[106861]: pgmap v616: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:49.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:49.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:49 smithi019 ceph-mon[173579]: pgmap v616: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:49.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:13:50.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:50 smithi062 ceph-mon[106861]: pgmap v617: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:50.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:50 smithi019 ceph-mon[173579]: pgmap v617: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:51.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.384+0000 7f8f87589700 1 -- 172.21.15.19:0/3568129498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 msgr2=0x7f8f80102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.384+0000 7f8f87589700 1 --2- 172.21.15.19:0/3568129498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c004940 tx=0x7f8f7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- 172.21.15.19:0/3568129498 shutdown_connections 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 --2- 172.21.15.19:0/3568129498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f801027e0 0x7f8f800fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 --2- 172.21.15.19:0/3568129498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102210 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- 172.21.15.19:0/3568129498 >> 172.21.15.19:0/3568129498 conn(0x7f8f80071d90 msgr2=0x7f8f80072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- 172.21.15.19:0/3568129498 shutdown_connections 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- 172.21.15.19:0/3568129498 wait complete. 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 Processor -- start 2024-02-22T16:13:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- start start 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f801027e0 0x7f8f801031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f801037e0 con 0x7f8f80101e40 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.385+0000 7f8f87589700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80103950 con 0x7f8f801027e0 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44530/0 (socket says 172.21.15.19:44530) 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 -- 172.21.15.19:0/3541113615 learned_addr learned my addr 172.21.15.19:0/3541113615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f84b24700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f801027e0 0x7f8f801031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 -- 172.21.15.19:0/3541113615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f801027e0 msgr2=0x7f8f801031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f801027e0 0x7f8f801031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 -- 172.21.15.19:0/3541113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c005040 con 0x7f8f80101e40 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f85325700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102ca0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00b3b0 tx=0x7f8f7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7c00e040 con 0x7f8f80101e40 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f7c01c430 con 0x7f8f80101e40 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7c013640 con 0x7f8f80101e40 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.386+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f80119be0 con 0x7f8f80101e40 2024-02-22T16:13:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.387+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8011a040 con 0x7f8f80101e40 2024-02-22T16:13:51.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.388+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f800738b0 con 0x7f8f80101e40 2024-02-22T16:13:51.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.388+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8f7c01c5a0 con 0x7f8f80101e40 2024-02-22T16:13:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.389+0000 7f8f75ffb700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8f6c06cc20 0x7f8f6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.389+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8f7c08cae0 con 0x7f8f80101e40 2024-02-22T16:13:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.389+0000 7f8f84b24700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8f6c06cc20 0x7f8f6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.389+0000 7f8f84b24700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8f6c06cc20 0x7f8f6c06f0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8f801046a0 tx=0x7f8f70009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:51.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.394+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f8f7c056c60 con 0x7f8f80101e40 2024-02-22T16:13:51.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.613+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f80104450 con 0x7f8f6c06cc20 2024-02-22T16:13:51.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.614+0000 7f8f75ffb700 1 -- 172.21.15.19:0/3541113615 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8f80104450 con 0x7f8f6c06cc20 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8f6c06cc20 msgr2=0x7f8f6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8f6c06cc20 0x7f8f6c06f0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8f801046a0 tx=0x7f8f70009040 comp rx=0 tx=0).stop 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 msgr2=0x7f8f80102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102ca0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00b3b0 tx=0x7f8f7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 shutdown_connections 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f8f6c06cc20 0x7f8f6c06f0d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f80101e40 0x7f8f80102ca0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 --2- 172.21.15.19:0/3541113615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f801027e0 0x7f8f801031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 >> 172.21.15.19:0/3541113615 conn(0x7f8f80071d90 msgr2=0x7f8f800657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 shutdown_connections 2024-02-22T16:13:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.617+0000 7f8f87589700 1 -- 172.21.15.19:0/3541113615 wait complete. 2024-02-22T16:13:51.602 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.720+0000 7f04cfde9700 1 -- 172.21.15.19:0/4160491794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fdfe0 msgr2=0x7f04c80fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.720+0000 7f04cfde9700 1 --2- 172.21.15.19:0/4160491794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fdfe0 0x7f04c80fe3b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f04c4004970 tx=0x7f04c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.720+0000 7f04cfde9700 1 -- 172.21.15.19:0/4160491794 shutdown_connections 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.720+0000 7f04cfde9700 1 --2- 172.21.15.19:0/4160491794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04c80fe8f0 0x7f04c8107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.720+0000 7f04cfde9700 1 --2- 172.21.15.19:0/4160491794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fdfe0 0x7f04c80fe3b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.720+0000 7f04cfde9700 1 -- 172.21.15.19:0/4160491794 >> 172.21.15.19:0/4160491794 conn(0x7f04c8071d90 msgr2=0x7f04c8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 -- 172.21.15.19:0/4160491794 shutdown_connections 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 -- 172.21.15.19:0/4160491794 wait complete. 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 Processor -- start 2024-02-22T16:13:51.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 -- start start 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04c80fdfe0 0x7f04c8113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 0x7f04c8113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c81144c0 con 0x7f04c80fe8f0 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cfde9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c810d7f0 con 0x7f04c80fdfe0 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cd384700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 0x7f04c8113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cd384700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 0x7f04c8113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44548/0 (socket says 172.21.15.19:44548) 2024-02-22T16:13:51.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.721+0000 7f04cd384700 1 -- 172.21.15.19:0/2950366103 learned_addr learned my addr 172.21.15.19:0/2950366103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cdb85700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04c80fdfe0 0x7f04c8113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cd384700 1 -- 172.21.15.19:0/2950366103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04c80fdfe0 msgr2=0x7f04c8113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cd384700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04c80fdfe0 0x7f04c8113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cd384700 1 -- 172.21.15.19:0/2950366103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4005040 con 0x7f04c80fe8f0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cd384700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 0x7f04c8113e70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04bc002a80 tx=0x7f04bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04bc007cc0 con 0x7f04c80fe8f0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04bc00f040 con 0x7f04c80fe8f0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c810dad0 con 0x7f04c80fe8f0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04bc009640 con 0x7f04c80fe8f0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.722+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c810e010 con 0x7f04c80fe8f0 2024-02-22T16:13:51.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.723+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c81046c0 con 0x7f04c80fe8f0 2024-02-22T16:13:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.724+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f04bc0097a0 con 0x7f04c80fe8f0 2024-02-22T16:13:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.724+0000 7f04ba7fc700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f04b406cc20 0x7f04b406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.724+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f04bc08b780 con 0x7f04c80fe8f0 2024-02-22T16:13:51.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.724+0000 7f04cdb85700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f04b406cc20 0x7f04b406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:51.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.725+0000 7f04cdb85700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f04b406cc20 0x7f04b406f0d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f04c4004f50 tx=0x7f04c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:51.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.729+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f04bc059240 con 0x7f04c80fe8f0 2024-02-22T16:13:51.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.951+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04c8109ef0 con 0x7f04b406cc20 2024-02-22T16:13:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.953+0000 7f04ba7fc700 1 -- 172.21.15.19:0/2950366103 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f04c8109ef0 con 0x7f04b406cc20 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f04b406cc20 msgr2=0x7f04b406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f04b406cc20 0x7f04b406f0d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f04c4004f50 tx=0x7f04c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 msgr2=0x7f04c8113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 0x7f04c8113e70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04bc002a80 tx=0x7f04bc0046b0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 shutdown_connections 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f04b406cc20 0x7f04b406f0d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04c80fdfe0 0x7f04c8113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 --2- 172.21.15.19:0/2950366103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04c80fe8f0 0x7f04c8113e70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 >> 172.21.15.19:0/2950366103 conn(0x7f04c8071d90 msgr2=0x7f04c8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 shutdown_connections 2024-02-22T16:13:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:51.955+0000 7f04cfde9700 1 -- 172.21.15.19:0/2950366103 wait complete. 2024-02-22T16:13:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.058+0000 7f2f33099700 1 -- 172.21.15.19:0/2182113780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 msgr2=0x7f2f2c0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.058+0000 7f2f33099700 1 --2- 172.21.15.19:0/2182113780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c0fcf60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004940 tx=0x7f2f18009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 -- 172.21.15.19:0/2182113780 shutdown_connections 2024-02-22T16:13:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 --2- 172.21.15.19:0/2182113780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c0fcf60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 --2- 172.21.15.19:0/2182113780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 0x7f2f2c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 -- 172.21.15.19:0/2182113780 >> 172.21.15.19:0/2182113780 conn(0x7f2f2c0f8ae0 msgr2=0x7f2f2c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:52.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 -- 172.21.15.19:0/2182113780 shutdown_connections 2024-02-22T16:13:52.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 -- 172.21.15.19:0/2182113780 wait complete. 2024-02-22T16:13:52.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 Processor -- start 2024-02-22T16:13:52.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.059+0000 7f2f33099700 1 -- start start 2024-02-22T16:13:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f33099700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 0x7f2f2c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f33099700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f33099700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c1144c0 con 0x7f2f2c1038f0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f33099700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c10d7f0 con 0x7f2f2c102f50 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44572/0 (socket says 172.21.15.19:44572) 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 -- 172.21.15.19:0/482426495 learned_addr learned my addr 172.21.15.19:0/482426495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f30e35700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 0x7f2f2c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 -- 172.21.15.19:0/482426495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 msgr2=0x7f2f2c113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 0x7f2f2c113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 -- 172.21.15.19:0/482426495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18005040 con 0x7f2f2c1038f0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f30e35700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 0x7f2f2c113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f2bfff700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c113e70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000c00 tx=0x7f2f18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f18021070 con 0x7f2f2c1038f0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c10da70 con 0x7f2f2c1038f0 2024-02-22T16:13:52.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f1801c400 con 0x7f2f2c1038f0 2024-02-22T16:13:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1800e070 con 0x7f2f2c1038f0 2024-02-22T16:13:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.060+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c10df50 con 0x7f2f2c1038f0 2024-02-22T16:13:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.061+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0fc6e0 con 0x7f2f2c1038f0 2024-02-22T16:13:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.062+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2f1801c9e0 con 0x7f2f2c1038f0 2024-02-22T16:13:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.062+0000 7f2f297fa700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2f1c06cc20 0x7f2f1c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.063+0000 7f2f30e35700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2f1c06cc20 0x7f2f1c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.063+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2f1808d270 con 0x7f2f2c1038f0 2024-02-22T16:13:52.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.063+0000 7f2f30e35700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2f1c06cc20 0x7f2f1c06f0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f20004600 tx=0x7f2f2000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:52.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.067+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2f1805ad30 con 0x7f2f2c1038f0 2024-02-22T16:13:52.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.277+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f2c113600 con 0x7f2f1c06cc20 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.287+0000 7f2f297fa700 1 -- 172.21.15.19:0/482426495 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f2f2c113600 con 0x7f2f1c06cc20 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (21m) 14m ago 32m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (18m) 14m ago 32m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (18m) 13m ago 31m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (20m) 14m ago 31m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:13:52.261 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (14m) 14m ago 26m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (14m) 14m ago 26m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (13m) 13m ago 26m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (14m) 13m ago 26m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (22m) 14m ago 33m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (22m) 13m ago 31m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (19m) 14m ago 33m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (19m) 13m ago 31m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (22m) 14m ago 32m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (21m) 13m ago 31m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (18m) 14m ago 30m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (17m) 14m ago 29m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (16m) 14m ago 29m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (16m) 13m ago 28m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:13:52.262 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (15m) 13m ago 28m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:13:52.263 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (15m) 13m ago 27m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:13:52.263 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (21m) 14m ago 31m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:13:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2f1c06cc20 msgr2=0x7f2f1c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2f1c06cc20 0x7f2f1c06f0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2f20004600 tx=0x7f2f2000b040 comp rx=0 tx=0).stop 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 msgr2=0x7f2f2c113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c113e70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000c00 tx=0x7f2f18009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 shutdown_connections 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2f1c06cc20 0x7f2f1c06f0d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f2c102f50 0x7f2f2c113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 --2- 172.21.15.19:0/482426495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f2c1038f0 0x7f2f2c113e70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 >> 172.21.15.19:0/482426495 conn(0x7f2f2c0f8ae0 msgr2=0x7f2f2c0fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 shutdown_connections 2024-02-22T16:13:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.289+0000 7f2f33099700 1 -- 172.21.15.19:0/482426495 wait complete. 2024-02-22T16:13:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 -- 172.21.15.19:0/2640004279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd038101e40 msgr2=0x7fd038102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 --2- 172.21.15.19:0/2640004279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd038101e40 0x7fd038102210 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd028004940 tx=0x7fd028009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 -- 172.21.15.19:0/2640004279 shutdown_connections 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 --2- 172.21.15.19:0/2640004279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0381027e0 0x7fd0380fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 --2- 172.21.15.19:0/2640004279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd038101e40 0x7fd038102210 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 -- 172.21.15.19:0/2640004279 >> 172.21.15.19:0/2640004279 conn(0x7fd0380f79d0 msgr2=0x7fd0380f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 -- 172.21.15.19:0/2640004279 shutdown_connections 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 -- 172.21.15.19:0/2640004279 wait complete. 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 Processor -- start 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.392+0000 7fd03ce59700 1 -- start start 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03ce59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101e40 0x7fd038113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03ce59700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 0x7fd038113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03ce59700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038114490 con 0x7fd0381027e0 2024-02-22T16:13:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03ce59700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03810d7c0 con 0x7fd038101e40 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 0x7fd038113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 0x7fd038113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44600/0 (socket says 172.21.15.19:44600) 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 -- 172.21.15.19:0/3357043995 learned_addr learned my addr 172.21.15.19:0/3357043995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03659c700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101e40 0x7fd038113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 -- 172.21.15.19:0/3357043995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101e40 msgr2=0x7fd038113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101e40 0x7fd038113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 -- 172.21.15.19:0/3357043995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028005040 con 0x7fd0381027e0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03659c700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101e40 0x7fd038113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd035d9b700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 0x7fd038113e40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00eb40 tx=0x7fd02c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd02c007cc0 con 0x7fd0381027e0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd02c00f040 con 0x7fd0381027e0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03810da40 con 0x7fd0381027e0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd02c009570 con 0x7fd0381027e0 2024-02-22T16:13:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.393+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03810df80 con 0x7fd0381027e0 2024-02-22T16:13:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.394+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0380fb5d0 con 0x7fd0381027e0 2024-02-22T16:13:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.395+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd02c0096d0 con 0x7fd0381027e0 2024-02-22T16:13:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.395+0000 7fd026ffd700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd02006cc20 0x7fd02006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.396+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd02c08bc10 con 0x7fd0381027e0 2024-02-22T16:13:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.396+0000 7fd03659c700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd02006cc20 0x7fd02006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.396+0000 7fd03659c700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd02006cc20 0x7fd02006f0d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd02800b0d0 tx=0x7fd028009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.400+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fd02c0596a0 con 0x7fd0381027e0 2024-02-22T16:13:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.695+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd03810e1a0 con 0x7fd0381027e0 2024-02-22T16:13:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.695+0000 7fd026ffd700 1 -- 172.21.15.19:0/3357043995 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fd02c059230 con 0x7fd0381027e0 2024-02-22T16:13:52.669 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:13:52.670 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:13:52.671 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:13:52.671 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:13:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd02006cc20 msgr2=0x7fd02006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd02006cc20 0x7fd02006f0d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd02800b0d0 tx=0x7fd028009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 msgr2=0x7fd038113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 0x7fd038113e40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00eb40 tx=0x7fd02c00b040 comp rx=0 tx=0).stop 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 shutdown_connections 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fd02006cc20 0x7fd02006f0d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd038101e40 0x7fd038113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 --2- 172.21.15.19:0/3357043995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0381027e0 0x7fd038113e40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 >> 172.21.15.19:0/3357043995 conn(0x7fd0380f79d0 msgr2=0x7fd038065fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 shutdown_connections 2024-02-22T16:13:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.697+0000 7fd03ce59700 1 -- 172.21.15.19:0/3357043995 wait complete. 2024-02-22T16:13:52.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:52 smithi062 ceph-mon[106861]: pgmap v618: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:52.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:52 smithi062 ceph-mon[106861]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:52.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:52 smithi062 ceph-mon[106861]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:52.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:52 smithi019 ceph-mon[173579]: pgmap v618: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:52.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:52 smithi019 ceph-mon[173579]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:52.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:52 smithi019 ceph-mon[173579]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 -- 172.21.15.19:0/1226090333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41027e0 msgr2=0x7f28f40fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/1226090333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41027e0 0x7f28f40fbea0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f28e8004940 tx=0x7f28e8009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 -- 172.21.15.19:0/1226090333 shutdown_connections 2024-02-22T16:13:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/1226090333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f41027e0 0x7f28f40fbea0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/1226090333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f4101e40 0x7f28f4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 -- 172.21.15.19:0/1226090333 >> 172.21.15.19:0/1226090333 conn(0x7f28f4071d90 msgr2=0x7f28f4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 -- 172.21.15.19:0/1226090333 shutdown_connections 2024-02-22T16:13:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.801+0000 7f28f9c3f700 1 -- 172.21.15.19:0/1226090333 wait complete. 2024-02-22T16:13:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f9c3f700 1 Processor -- start 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f9c3f700 1 -- start start 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f9c3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 0x7f28f4113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f9c3f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41027e0 0x7f28f4113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f9c3f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41145d0 con 0x7f28f4101e40 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f9c3f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f410d900 con 0x7f28f41027e0 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 0x7f28f4113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 0x7f28f4113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44620/0 (socket says 172.21.15.19:44620) 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f37fe700 1 -- 172.21.15.19:0/2300889788 learned_addr learned my addr 172.21.15.19:0/2300889788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.802+0000 7f28f2ffd700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41027e0 0x7f28f4113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28f37fe700 1 -- 172.21.15.19:0/2300889788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41027e0 msgr2=0x7f28f4113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28f37fe700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41027e0 0x7f28f4113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28f37fe700 1 -- 172.21.15.19:0/2300889788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8005040 con 0x7f28f4101e40 2024-02-22T16:13:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28f37fe700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 0x7f28f4113a40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f28e4002a50 tx=0x7f28e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e4007cc0 con 0x7f28f4101e40 2024-02-22T16:13:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f410dbe0 con 0x7f28f4101e40 2024-02-22T16:13:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28e400f040 con 0x7f28f4101e40 2024-02-22T16:13:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e400d9c0 con 0x7f28f4101e40 2024-02-22T16:13:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.803+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f410e0f0 con 0x7f28f4101e40 2024-02-22T16:13:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.804+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f40738b0 con 0x7f28f4101e40 2024-02-22T16:13:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.804+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f28e400db20 con 0x7f28f4101e40 2024-02-22T16:13:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.805+0000 7f28dbfff700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28dc06cc30 0x7f28dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.805+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f28e408bd80 con 0x7f28f4101e40 2024-02-22T16:13:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.805+0000 7f28f2ffd700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28dc06cc30 0x7f28dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.805+0000 7f28f2ffd700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28dc06cc30 0x7f28dc06f0e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f28f410ec30 tx=0x7f28e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:52.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:52.809+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f28e40597c0 con 0x7f28f4101e40 2024-02-22T16:13:53.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.057+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f28f410f020 con 0x7f28f4101e40 2024-02-22T16:13:53.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.058+0000 7f28dbfff700 1 -- 172.21.15.19:0/2300889788 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f28e4059350 con 0x7f28f4101e40 2024-02-22T16:13:53.032 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:13:53.032 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:13:53.032 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:13:53.032 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:13:53.033 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:13:53.034 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:13:53.035 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:53.036 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:13:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28dc06cc30 msgr2=0x7f28dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28dc06cc30 0x7f28dc06f0e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f28f410ec30 tx=0x7f28e8009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 msgr2=0x7f28f4113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 0x7f28f4113a40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f28e4002a50 tx=0x7f28e40046b0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 shutdown_connections 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f28dc06cc30 0x7f28dc06f0e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f4101e40 0x7f28f4113a40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 --2- 172.21.15.19:0/2300889788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28f41027e0 0x7f28f4113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 >> 172.21.15.19:0/2300889788 conn(0x7f28f4071d90 msgr2=0x7f28f4065820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 shutdown_connections 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.060+0000 7f28f9c3f700 1 -- 172.21.15.19:0/2300889788 wait complete. 2024-02-22T16:13:53.038 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:13:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- 172.21.15.19:0/3330558573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e84102f40 msgr2=0x7f5e84103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 --2- 172.21.15.19:0/3330558573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e84102f40 0x7f5e84103310 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e74004940 tx=0x7f5e74009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- 172.21.15.19:0/3330558573 shutdown_connections 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 --2- 172.21.15.19:0/3330558573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e841038e0 0x7f5e840fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 --2- 172.21.15.19:0/3330558573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e84102f40 0x7f5e84103310 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- 172.21.15.19:0/3330558573 >> 172.21.15.19:0/3330558573 conn(0x7f5e840f8af0 msgr2=0x7f5e840faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- 172.21.15.19:0/3330558573 shutdown_connections 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- 172.21.15.19:0/3330558573 wait complete. 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 Processor -- start 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- start start 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e84102f40 0x7f5e841139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 0x7f5e84113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84114560 con 0x7f5e841038e0 2024-02-22T16:13:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.166+0000 7f5e88b58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8410d890 con 0x7f5e84102f40 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 0x7f5e84113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 0x7f5e84113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44638/0 (socket says 172.21.15.19:44638) 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 -- 172.21.15.19:0/1009950067 learned_addr learned my addr 172.21.15.19:0/1009950067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e8259c700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e84102f40 0x7f5e841139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 -- 172.21.15.19:0/1009950067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e84102f40 msgr2=0x7f5e841139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e84102f40 0x7f5e841139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 -- 172.21.15.19:0/1009950067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74005040 con 0x7f5e841038e0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e81d9b700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 0x7f5e84113f10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800c800 tx=0x7f5e78004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e780106f0 con 0x7f5e841038e0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e78010d00 con 0x7f5e841038e0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8410db70 con 0x7f5e841038e0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e780113a0 con 0x7f5e841038e0 2024-02-22T16:13:53.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.167+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8410e0b0 con 0x7f5e841038e0 2024-02-22T16:13:53.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.168+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84100250 con 0x7f5e841038e0 2024-02-22T16:13:53.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.169+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5e78010850 con 0x7f5e841038e0 2024-02-22T16:13:53.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.169+0000 7f5e72ffd700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5e6c06cc20 0x7f5e6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:53.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.169+0000 7f5e8259c700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5e6c06cc20 0x7f5e6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.169+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5e7808bf30 con 0x7f5e841038e0 2024-02-22T16:13:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.170+0000 7f5e8259c700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5e6c06cc20 0x7f5e6c06f0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5e74004f90 tx=0x7f5e74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.174+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5e780bf960 con 0x7f5e841038e0 2024-02-22T16:13:53.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.397+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e84109f00 con 0x7f5e6c06cc20 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.398+0000 7f5e72ffd700 1 -- 172.21.15.19:0/1009950067 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e84109f00 con 0x7f5e6c06cc20 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:13:53.372 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:13:53.373 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5e6c06cc20 msgr2=0x7f5e6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5e6c06cc20 0x7f5e6c06f0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5e74004f90 tx=0x7f5e74009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 msgr2=0x7f5e84113f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 0x7f5e84113f10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800c800 tx=0x7f5e78004470 comp rx=0 tx=0).stop 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 shutdown_connections 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5e6c06cc20 0x7f5e6c06f0d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e84102f40 0x7f5e841139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 --2- 172.21.15.19:0/1009950067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e841038e0 0x7f5e84113f10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 >> 172.21.15.19:0/1009950067 conn(0x7f5e840f8af0 msgr2=0x7f5e840fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 shutdown_connections 2024-02-22T16:13:53.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.400+0000 7f5e88b58700 1 -- 172.21.15.19:0/1009950067 wait complete. 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 -- 172.21.15.19:0/2063603362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0901038e0 msgr2=0x7fa0900fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 --2- 172.21.15.19:0/2063603362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0901038e0 0x7fa0900fcf50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa080004940 tx=0x7fa080009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 -- 172.21.15.19:0/2063603362 shutdown_connections 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 --2- 172.21.15.19:0/2063603362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0901038e0 0x7fa0900fcf50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 --2- 172.21.15.19:0/2063603362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa090102f40 0x7fa090103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 -- 172.21.15.19:0/2063603362 >> 172.21.15.19:0/2063603362 conn(0x7fa0900f8af0 msgr2=0x7fa0900faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 -- 172.21.15.19:0/2063603362 shutdown_connections 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.504+0000 7fa098353700 1 -- 172.21.15.19:0/2063603362 wait complete. 2024-02-22T16:13:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa098353700 1 Processor -- start 2024-02-22T16:13:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa098353700 1 -- start start 2024-02-22T16:13:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa098353700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 0x7fa090113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa098353700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0901038e0 0x7fa090113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa098353700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0901144b0 con 0x7fa090102f40 2024-02-22T16:13:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa098353700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09010d7e0 con 0x7fa0901038e0 2024-02-22T16:13:53.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0960ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 0x7fa090113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:53.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0960ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 0x7fa090113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44656/0 (socket says 172.21.15.19:44656) 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0960ef700 1 -- 172.21.15.19:0/4160806551 learned_addr learned my addr 172.21.15.19:0/4160806551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0958ee700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0901038e0 0x7fa090113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0960ef700 1 -- 172.21.15.19:0/4160806551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0901038e0 msgr2=0x7fa090113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0960ef700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0901038e0 0x7fa090113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.505+0000 7fa0960ef700 1 -- 172.21.15.19:0/4160806551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080005040 con 0x7fa090102f40 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.506+0000 7fa0960ef700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 0x7fa090113920 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00eb40 tx=0x7fa08c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.506+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08c007cc0 con 0x7fa090102f40 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.506+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09010da60 con 0x7fa090102f40 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.506+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa08c00f040 con 0x7fa090102f40 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.506+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08c009570 con 0x7fa090102f40 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.506+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09010df70 con 0x7fa090102f40 2024-02-22T16:13:53.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.507+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa090105410 con 0x7fa090102f40 2024-02-22T16:13:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.507+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa08c0096d0 con 0x7fa090102f40 2024-02-22T16:13:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.508+0000 7fa086ffd700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa07c06cc20 0x7fa07c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:13:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.508+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa08c08bd40 con 0x7fa090102f40 2024-02-22T16:13:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.508+0000 7fa0958ee700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa07c06cc20 0x7fa07c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:13:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.508+0000 7fa0958ee700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa07c06cc20 0x7fa07c06f0d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa09010eb10 tx=0x7fa080009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:13:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.512+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fa08c059830 con 0x7fa090102f40 2024-02-22T16:13:53.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:53 smithi062 ceph-mon[106861]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:53.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:53 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3357043995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:13:53.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:53 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2300889788' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:13:53.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:53 smithi019 ceph-mon[173579]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:53.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:53 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3357043995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:13:53.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:53 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2300889788' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.800+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa09004bed0 con 0x7fa090102f40 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.800+0000 7fa086ffd700 1 -- 172.21.15.19:0/4160806551 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa08c0593c0 con 0x7fa090102f40 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:13:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa07c06cc20 msgr2=0x7fa07c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa07c06cc20 0x7fa07c06f0d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa09010eb10 tx=0x7fa080009e30 comp rx=0 tx=0).stop 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 msgr2=0x7fa090113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 0x7fa090113920 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00eb40 tx=0x7fa08c00b040 comp rx=0 tx=0).stop 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 shutdown_connections 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fa07c06cc20 0x7fa07c06f0d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa090102f40 0x7fa090113920 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 --2- 172.21.15.19:0/4160806551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0901038e0 0x7fa090113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:13:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 >> 172.21.15.19:0/4160806551 conn(0x7fa0900f8af0 msgr2=0x7fa0900fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:13:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 shutdown_connections 2024-02-22T16:13:53.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:13:53.802+0000 7fa098353700 1 -- 172.21.15.19:0/4160806551 wait complete. 2024-02-22T16:13:54.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:54 smithi062 ceph-mon[106861]: pgmap v619: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:54.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:54 smithi062 ceph-mon[106861]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:54.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:54 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4160806551' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:13:54.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:54 smithi019 ceph-mon[173579]: pgmap v619: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:54.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:54 smithi019 ceph-mon[173579]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:13:54.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:54 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4160806551' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:13:56.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:56 smithi062 ceph-mon[106861]: pgmap v620: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:56.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:56 smithi019 ceph-mon[173579]: pgmap v620: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:13:59 smithi062 ceph-mon[106861]: pgmap v621: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:13:59.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:13:59 smithi019 ceph-mon[173579]: pgmap v621: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:01.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:01 smithi019 ceph-mon[173579]: pgmap v622: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:01.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:01 smithi062 ceph-mon[106861]: pgmap v622: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:03.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:03 smithi062 ceph-mon[106861]: pgmap v623: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:03.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:03 smithi019 ceph-mon[173579]: pgmap v623: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:05 smithi062 ceph-mon[106861]: pgmap v624: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:05 smithi019 ceph-mon[173579]: pgmap v624: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:07.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:07 smithi062 ceph-mon[106861]: pgmap v625: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:07.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:07 smithi019 ceph-mon[173579]: pgmap v625: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:09.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:09 smithi062 ceph-mon[106861]: pgmap v626: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:09.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:09 smithi019 ceph-mon[173579]: pgmap v626: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:11.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:11 smithi019 ceph-mon[173579]: pgmap v627: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:11.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:11 smithi062 ceph-mon[106861]: pgmap v627: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:13.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:13 smithi062 ceph-mon[106861]: pgmap v628: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:13.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:13 smithi019 ceph-mon[173579]: pgmap v628: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:14.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:14 smithi062 ceph-mon[106861]: pgmap v629: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:14.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:14 smithi019 ceph-mon[173579]: pgmap v629: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:16.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:16 smithi062 ceph-mon[106861]: pgmap v630: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:16.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:16 smithi019 ceph-mon[173579]: pgmap v630: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:19 smithi062 ceph-mon[106861]: pgmap v631: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:19 smithi019 ceph-mon[173579]: pgmap v631: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:19.667 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:21.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:21 smithi019 ceph-mon[173579]: pgmap v632: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:21.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:21 smithi062 ceph-mon[106861]: pgmap v632: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:23.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:23 smithi062 ceph-mon[106861]: pgmap v633: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:23.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:23 smithi019 ceph-mon[173579]: pgmap v633: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:23.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.918+0000 7fc15f482700 1 -- 172.21.15.19:0/2224808890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 msgr2=0x7fc158102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.918+0000 7fc15f482700 1 --2- 172.21.15.19:0/2224808890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158102210 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc154004940 tx=0x7fc154009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 -- 172.21.15.19:0/2224808890 shutdown_connections 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 --2- 172.21.15.19:0/2224808890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1581027e0 0x7fc1580fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 --2- 172.21.15.19:0/2224808890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158102210 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 -- 172.21.15.19:0/2224808890 >> 172.21.15.19:0/2224808890 conn(0x7fc158071d90 msgr2=0x7fc158072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 -- 172.21.15.19:0/2224808890 shutdown_connections 2024-02-22T16:14:23.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 -- 172.21.15.19:0/2224808890 wait complete. 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 Processor -- start 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 -- start start 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.919+0000 7fc15f482700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15f482700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1581027e0 0x7fc1581160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15f482700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158116720 con 0x7fc158101e40 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15f482700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15810fa50 con 0x7fc1581027e0 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49582/0 (socket says 172.21.15.19:49582) 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 -- 172.21.15.19:0/1850243970 learned_addr learned my addr 172.21.15.19:0/1850243970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15ca1d700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1581027e0 0x7fc1581160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 -- 172.21.15.19:0/1850243970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1581027e0 msgr2=0x7fc1581160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:23.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1581027e0 0x7fc1581160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 -- 172.21.15.19:0/1850243970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc154005040 con 0x7fc158101e40 2024-02-22T16:14:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15d21e700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158115b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b3b0 tx=0x7fc154009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15400e040 con 0x7fc158101e40 2024-02-22T16:14:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc15400bc50 con 0x7fc158101e40 2024-02-22T16:14:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15810fcd0 con 0x7fc158101e40 2024-02-22T16:14:23.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.920+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc154013420 con 0x7fc158101e40 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.921+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc158110210 con 0x7fc158101e40 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.922+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1580738b0 con 0x7fc158101e40 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.922+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc15401c430 con 0x7fc158101e40 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.923+0000 7fc14dffb700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc14406cc20 0x7fc14406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.923+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc15408cd50 con 0x7fc158101e40 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.923+0000 7fc15ca1d700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc14406cc20 0x7fc14406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:23.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.923+0000 7fc15ca1d700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc14406cc20 0x7fc14406f0d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc1581113c0 tx=0x7fc148009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:23.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:23.928+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc15405a960 con 0x7fc158101e40 2024-02-22T16:14:24.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.148+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1581111e0 con 0x7fc14406cc20 2024-02-22T16:14:24.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.149+0000 7fc14dffb700 1 -- 172.21.15.19:0/1850243970 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc1581111e0 con 0x7fc14406cc20 2024-02-22T16:14:24.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc14406cc20 msgr2=0x7fc14406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc14406cc20 0x7fc14406f0d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc1581113c0 tx=0x7fc148009ee0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 msgr2=0x7fc158115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158115b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b3b0 tx=0x7fc154009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 shutdown_connections 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc14406cc20 0x7fc14406f0d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc158101e40 0x7fc158115b90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 --2- 172.21.15.19:0/1850243970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1581027e0 0x7fc1581160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 >> 172.21.15.19:0/1850243970 conn(0x7fc158071d90 msgr2=0x7fc1580657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 shutdown_connections 2024-02-22T16:14:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.152+0000 7fc15f482700 1 -- 172.21.15.19:0/1850243970 wait complete. 2024-02-22T16:14:24.137 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:14:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 -- 172.21.15.19:0/1362163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa80101e40 msgr2=0x7faa80102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 --2- 172.21.15.19:0/1362163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa80101e40 0x7faa80102210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faa7c004940 tx=0x7faa7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:24.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 -- 172.21.15.19:0/1362163454 shutdown_connections 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 --2- 172.21.15.19:0/1362163454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa801027e0 0x7faa800fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 --2- 172.21.15.19:0/1362163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa80101e40 0x7faa80102210 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 -- 172.21.15.19:0/1362163454 >> 172.21.15.19:0/1362163454 conn(0x7faa80071d90 msgr2=0x7faa80072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 -- 172.21.15.19:0/1362163454 shutdown_connections 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 -- 172.21.15.19:0/1362163454 wait complete. 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.257+0000 7faa883e2700 1 Processor -- start 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa883e2700 1 -- start start 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa883e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80101e40 0x7faa80113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa883e2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 0x7faa80113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa883e2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa80114520 con 0x7faa801027e0 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa883e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8010d850 con 0x7faa80101e40 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 0x7faa80113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 0x7faa80113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49594/0 (socket says 172.21.15.19:49594) 2024-02-22T16:14:24.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 -- 172.21.15.19:0/4216573011 learned_addr learned my addr 172.21.15.19:0/4216573011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8617e700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80101e40 0x7faa80113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 -- 172.21.15.19:0/4216573011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80101e40 msgr2=0x7faa80113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80101e40 0x7faa80113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 -- 172.21.15.19:0/4216573011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c005040 con 0x7faa801027e0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.258+0000 7faa8597d700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 0x7faa80113ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faa74002a50 tx=0x7faa74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.259+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa74007cc0 con 0x7faa801027e0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.259+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8010db30 con 0x7faa801027e0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.259+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa7400f040 con 0x7faa801027e0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.259+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa7400d920 con 0x7faa801027e0 2024-02-22T16:14:24.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.259+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8010e070 con 0x7faa801027e0 2024-02-22T16:14:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.259+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa800738b0 con 0x7faa801027e0 2024-02-22T16:14:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.260+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faa74009680 con 0x7faa801027e0 2024-02-22T16:14:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.260+0000 7faa72ffd700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faa6c06cc30 0x7faa6c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.260+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7faa7408bdf0 con 0x7faa801027e0 2024-02-22T16:14:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.261+0000 7faa8617e700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faa6c06cc30 0x7faa6c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.261+0000 7faa8617e700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faa6c06cc30 0x7faa6c06f0e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00b3b0 tx=0x7faa7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.265+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7faa74059830 con 0x7faa801027e0 2024-02-22T16:14:24.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.485+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa8004bed0 con 0x7faa6c06cc30 2024-02-22T16:14:24.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.486+0000 7faa72ffd700 1 -- 172.21.15.19:0/4216573011 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faa8004bed0 con 0x7faa6c06cc30 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faa6c06cc30 msgr2=0x7faa6c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faa6c06cc30 0x7faa6c06f0e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00b3b0 tx=0x7faa7c009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 msgr2=0x7faa80113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 0x7faa80113ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faa74002a50 tx=0x7faa74004600 comp rx=0 tx=0).stop 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 shutdown_connections 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7faa6c06cc30 0x7faa6c06f0e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa80101e40 0x7faa80113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 --2- 172.21.15.19:0/4216573011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa801027e0 0x7faa80113ed0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 >> 172.21.15.19:0/4216573011 conn(0x7faa80071d90 msgr2=0x7faa80105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 shutdown_connections 2024-02-22T16:14:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.488+0000 7faa883e2700 1 -- 172.21.15.19:0/4216573011 wait complete. 2024-02-22T16:14:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 -- 172.21.15.19:0/1327058288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 msgr2=0x7fab800fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 --2- 172.21.15.19:0/1327058288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab800fcf60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab68004940 tx=0x7fab68009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 -- 172.21.15.19:0/1327058288 shutdown_connections 2024-02-22T16:14:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 --2- 172.21.15.19:0/1327058288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab800fcf60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 --2- 172.21.15.19:0/1327058288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab80102f50 0x7fab80103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 -- 172.21.15.19:0/1327058288 >> 172.21.15.19:0/1327058288 conn(0x7fab800f8ae0 msgr2=0x7fab800faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:24.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 -- 172.21.15.19:0/1327058288 shutdown_connections 2024-02-22T16:14:24.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.591+0000 7fab85e5e700 1 -- 172.21.15.19:0/1327058288 wait complete. 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab85e5e700 1 Processor -- start 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab85e5e700 1 -- start start 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab85e5e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab80102f50 0x7fab80113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab85e5e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab80113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab85e5e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab80114500 con 0x7fab801038f0 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab85e5e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8010d830 con 0x7fab80102f50 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab80113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab7effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab80113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49614/0 (socket says 172.21.15.19:49614) 2024-02-22T16:14:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab7effd700 1 -- 172.21.15.19:0/3693190793 learned_addr learned my addr 172.21.15.19:0/3693190793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab7f7fe700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab80102f50 0x7fab80113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab7effd700 1 -- 172.21.15.19:0/3693190793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab80102f50 msgr2=0x7fab80113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.592+0000 7fab7effd700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab80102f50 0x7fab80113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab7effd700 1 -- 172.21.15.19:0/3693190793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68005040 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab7effd700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab80113eb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab68009fd0 tx=0x7fab68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6800e040 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab68013420 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6801c630 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8010dab0 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.593+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8010dfa0 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.594+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab80105420 con 0x7fab801038f0 2024-02-22T16:14:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.594+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fab68013a00 con 0x7fab801038f0 2024-02-22T16:14:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.595+0000 7fab77fff700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fab6c06cc20 0x7fab6c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.595+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fab6808d260 con 0x7fab801038f0 2024-02-22T16:14:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.595+0000 7fab7f7fe700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fab6c06cc20 0x7fab6c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.595+0000 7fab7f7fe700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fab6c06cc20 0x7fab6c06f0d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab70004f40 tx=0x7fab7000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.599+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fab6805ad20 con 0x7fab801038f0 2024-02-22T16:14:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.813+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab80109ef0 con 0x7fab6c06cc20 2024-02-22T16:14:24.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.823+0000 7fab77fff700 1 -- 172.21.15.19:0/3693190793 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fab80109ef0 con 0x7fab6c06cc20 2024-02-22T16:14:24.798 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:14:24.798 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (21m) 14m ago 32m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:14:24.798 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (18m) 14m ago 32m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:14:24.798 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (18m) 14m ago 31m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (21m) 14m ago 32m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (15m) 14m ago 26m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (15m) 14m ago 26m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (14m) 14m ago 26m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (14m) 14m ago 26m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (23m) 14m ago 34m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (22m) 14m ago 31m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (20m) 14m ago 34m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (19m) 14m ago 31m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (22m) 14m ago 32m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (22m) 14m ago 31m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (18m) 14m ago 30m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (17m) 14m ago 30m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:14:24.799 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (17m) 14m ago 29m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:14:24.800 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (16m) 14m ago 29m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:14:24.800 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (16m) 14m ago 28m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:14:24.800 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (15m) 14m ago 28m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:14:24.800 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (21m) 14m ago 32m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:14:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fab6c06cc20 msgr2=0x7fab6c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fab6c06cc20 0x7fab6c06f0d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fab70004f40 tx=0x7fab7000b040 comp rx=0 tx=0).stop 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 msgr2=0x7fab80113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab80113eb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab68009fd0 tx=0x7fab68009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 shutdown_connections 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fab6c06cc20 0x7fab6c06f0d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab80102f50 0x7fab80113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 --2- 172.21.15.19:0/3693190793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab801038f0 0x7fab80113eb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 >> 172.21.15.19:0/3693190793 conn(0x7fab800f8ae0 msgr2=0x7fab800fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 shutdown_connections 2024-02-22T16:14:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.825+0000 7fab85e5e700 1 -- 172.21.15.19:0/3693190793 wait complete. 2024-02-22T16:14:24.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 -- 172.21.15.19:0/3596390128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 msgr2=0x7f40f00fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 --2- 172.21.15.19:0/3596390128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00fcf50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f40ec004940 tx=0x7f40ec009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:24.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 -- 172.21.15.19:0/3596390128 shutdown_connections 2024-02-22T16:14:24.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 --2- 172.21.15.19:0/3596390128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00fcf50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 --2- 172.21.15.19:0/3596390128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f0102f40 0x7f40f0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 -- 172.21.15.19:0/3596390128 >> 172.21.15.19:0/3596390128 conn(0x7f40f00f8af0 msgr2=0x7f40f00faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 -- 172.21.15.19:0/3596390128 shutdown_connections 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 -- 172.21.15.19:0/3596390128 wait complete. 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 Processor -- start 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.928+0000 7f40f831b700 1 -- start start 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f831b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f0102f40 0x7f40f0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f831b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f831b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0070dd0 con 0x7f40f01038e0 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f831b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0070f40 con 0x7f40f0102f40 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49640/0 (socket says 172.21.15.19:49640) 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 -- 172.21.15.19:0/2924087205 learned_addr learned my addr 172.21.15.19:0/2924087205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f60b7700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f0102f40 0x7f40f0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 -- 172.21.15.19:0/2924087205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f0102f40 msgr2=0x7f40f0070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f0102f40 0x7f40f0070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:24.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 -- 172.21.15.19:0/2924087205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec005040 con 0x7f40f01038e0 2024-02-22T16:14:24.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f58b6700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00707d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009380 tx=0x7f40ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:24.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ec00e040 con 0x7f40f01038e0 2024-02-22T16:14:24.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f0119be0 con 0x7f40f01038e0 2024-02-22T16:14:24.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f40ec00bc50 con 0x7f40f01038e0 2024-02-22T16:14:24.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ec01c3f0 con 0x7f40f01038e0 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.929+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f011a040 con 0x7f40f01038e0 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.930+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f0105410 con 0x7f40f01038e0 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.931+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f40ec013420 con 0x7f40f01038e0 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.931+0000 7f40e6ffd700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f40dc06cc20 0x7f40dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.931+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f40ec08d4a0 con 0x7f40f01038e0 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.931+0000 7f40f60b7700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f40dc06cc20 0x7f40dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.932+0000 7f40f60b7700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f40dc06cc20 0x7f40dc06f0d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009fd0 tx=0x7f40e0009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:24.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:24.936+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f40ec0574d0 con 0x7f40f01038e0 2024-02-22T16:14:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.219+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40f00719d0 con 0x7f40f01038e0 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.220+0000 7f40e6ffd700 1 -- 172.21.15.19:0/2924087205 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f40ec05aaf0 con 0x7f40f01038e0 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:14:25.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:14:25.196 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f40dc06cc20 msgr2=0x7f40dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f40dc06cc20 0x7f40dc06f0d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009fd0 tx=0x7f40e0009f20 comp rx=0 tx=0).stop 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 msgr2=0x7f40f00707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00707d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009380 tx=0x7f40ec009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 shutdown_connections 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f40dc06cc20 0x7f40dc06f0d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40f0102f40 0x7f40f0070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 --2- 172.21.15.19:0/2924087205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f01038e0 0x7f40f00707d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 >> 172.21.15.19:0/2924087205 conn(0x7f40f00f8af0 msgr2=0x7f40f00fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 shutdown_connections 2024-02-22T16:14:25.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.222+0000 7f40f831b700 1 -- 172.21.15.19:0/2924087205 wait complete. 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 -- 172.21.15.19:0/1221373136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 msgr2=0x7f76700fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 --2- 172.21.15.19:0/1221373136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f76700fbea0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7664004940 tx=0x7f7664009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 -- 172.21.15.19:0/1221373136 shutdown_connections 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 --2- 172.21.15.19:0/1221373136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f76700fbea0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 --2- 172.21.15.19:0/1221373136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7670101e40 0x7f7670102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 -- 172.21.15.19:0/1221373136 >> 172.21.15.19:0/1221373136 conn(0x7f7670071d90 msgr2=0x7f7670072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 -- 172.21.15.19:0/1221373136 shutdown_connections 2024-02-22T16:14:25.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 -- 172.21.15.19:0/1221373136 wait complete. 2024-02-22T16:14:25.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.325+0000 7f76768ad700 1 Processor -- start 2024-02-22T16:14:25.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f76768ad700 1 -- start start 2024-02-22T16:14:25.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f76768ad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7670101e40 0x7f7670112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:25.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f76768ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f7670112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f76768ad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7670113390 con 0x7f76701027e0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f76768ad700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767010c6c0 con 0x7f7670101e40 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f7670112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f7670112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49654/0 (socket says 172.21.15.19:49654) 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 -- 172.21.15.19:0/2940256503 learned_addr learned my addr 172.21.15.19:0/2940256503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766ffff700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7670101e40 0x7f7670112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 -- 172.21.15.19:0/2940256503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7670101e40 msgr2=0x7f7670112800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7670101e40 0x7f7670112800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 -- 172.21.15.19:0/2940256503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664005040 con 0x7f76701027e0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.326+0000 7f766f7fe700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f7670112d40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7664000c00 tx=0x7f7664009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.327+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7664021070 con 0x7f76701027e0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.327+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76640093b0 con 0x7f76701027e0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.327+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767010c940 con 0x7f76701027e0 2024-02-22T16:14:25.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.327+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f766400e070 con 0x7f76701027e0 2024-02-22T16:14:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.327+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767010ce20 con 0x7f76701027e0 2024-02-22T16:14:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.328+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76700738b0 con 0x7f76701027e0 2024-02-22T16:14:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.328+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f766401c670 con 0x7f76701027e0 2024-02-22T16:14:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.328+0000 7f766cff9700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f765806cc20 0x7f765806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.329+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f766408d0f0 con 0x7f76701027e0 2024-02-22T16:14:25.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.329+0000 7f766ffff700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f765806cc20 0x7f765806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:25.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.329+0000 7f766ffff700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f765806cc20 0x7f765806f0d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7660009b80 tx=0x7f766000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:25.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.333+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f766405ac50 con 0x7f76701027e0 2024-02-22T16:14:25.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:25 smithi062 ceph-mon[106861]: pgmap v634: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:25.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:25 smithi062 ceph-mon[106861]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:25.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:25 smithi019 ceph-mon[173579]: pgmap v634: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:25.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:25 smithi019 ceph-mon[173579]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.580+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f767004bed0 con 0x7f76701027e0 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.580+0000 7f766cff9700 1 -- 172.21.15.19:0/2940256503 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1837 (secure 0 0 0) 0x7f766405a7e0 con 0x7f76701027e0 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout:e35 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:14:25.555 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T15:58:54.154609+0000 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:14:25.556 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:up {} 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:failed 0,1 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:14:25.557 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:25.558 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{-1:34392} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:25.558 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:25.558 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:25.558 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:25.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f765806cc20 msgr2=0x7f765806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f765806cc20 0x7f765806f0d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7660009b80 tx=0x7f766000b040 comp rx=0 tx=0).stop 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 msgr2=0x7f7670112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f7670112d40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7664000c00 tx=0x7f7664009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 shutdown_connections 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f765806cc20 0x7f765806f0d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7670101e40 0x7f7670112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 --2- 172.21.15.19:0/2940256503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76701027e0 0x7f7670112d40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 >> 172.21.15.19:0/2940256503 conn(0x7f7670071d90 msgr2=0x7f76700fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 shutdown_connections 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.582+0000 7f76768ad700 1 -- 172.21.15.19:0/2940256503 wait complete. 2024-02-22T16:14:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:14:25.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 -- 172.21.15.19:0/518061677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 msgr2=0x7f95ec107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 --2- 172.21.15.19:0/518061677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec107050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f95dc004970 tx=0x7f95dc009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 -- 172.21.15.19:0/518061677 shutdown_connections 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 --2- 172.21.15.19:0/518061677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec107050 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 --2- 172.21.15.19:0/518061677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ec0fdfe0 0x7f95ec0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 -- 172.21.15.19:0/518061677 >> 172.21.15.19:0/518061677 conn(0x7f95ec071d90 msgr2=0x7f95ec072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 -- 172.21.15.19:0/518061677 shutdown_connections 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.687+0000 7f95f35e3700 1 -- 172.21.15.19:0/518061677 wait complete. 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f35e3700 1 Processor -- start 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f35e3700 1 -- start start 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f35e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ec0fdfe0 0x7f95ec113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f35e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f35e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec114500 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f35e3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec10d830 con 0x7f95ec0fdfe0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49660/0 (socket says 172.21.15.19:49660) 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 -- 172.21.15.19:0/1788536708 learned_addr learned my addr 172.21.15.19:0/1788536708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f137f700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ec0fdfe0 0x7f95ec113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 -- 172.21.15.19:0/1788536708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ec0fdfe0 msgr2=0x7f95ec113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ec0fdfe0 0x7f95ec113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 -- 172.21.15.19:0/1788536708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc005040 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.688+0000 7f95f0b7e700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec113eb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.689+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95dc00e040 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.689+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec10dab0 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.689+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95dc013420 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.689+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95dc01c630 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.689+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec10dff0 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.690+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec1046c0 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.690+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f95dc0139e0 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.691+0000 7f95e1ffb700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f95d806cc20 0x7f95d806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.691+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f95dc08d330 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.691+0000 7f95f137f700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f95d806cc20 0x7f95d806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.691+0000 7f95f137f700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f95d806cc20 0x7f95d806f0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007fd0 tx=0x7f95e8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.695+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f95dc05adf0 con 0x7f95ec0fe8f0 2024-02-22T16:14:25.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.918+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95ec109ef0 con 0x7f95d806cc20 2024-02-22T16:14:25.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.920+0000 7f95e1ffb700 1 -- 172.21.15.19:0/1788536708 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f95ec109ef0 con 0x7f95d806cc20 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": true, 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [ 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "mon", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "osd", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "mds", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr" 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": "16/21 daemons upgraded", 2024-02-22T16:14:25.895 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "Currently upgrading mds daemons", 2024-02-22T16:14:25.896 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:14:25.896 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f95d806cc20 msgr2=0x7f95d806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f95d806cc20 0x7f95d806f0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007fd0 tx=0x7f95e8009040 comp rx=0 tx=0).stop 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 msgr2=0x7f95ec113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec113eb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 shutdown_connections 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f95d806cc20 0x7f95d806f0d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95ec0fdfe0 0x7f95ec113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 --2- 172.21.15.19:0/1788536708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95ec0fe8f0 0x7f95ec113eb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 >> 172.21.15.19:0/1788536708 conn(0x7f95ec071d90 msgr2=0x7f95ec101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:25.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 shutdown_connections 2024-02-22T16:14:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:25.922+0000 7f95f35e3700 1 -- 172.21.15.19:0/1788536708 wait complete. 2024-02-22T16:14:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 -- 172.21.15.19:0/3981826913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 msgr2=0x7fe844102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3981826913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844102220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe838004940 tx=0x7fe838009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 -- 172.21.15.19:0/3981826913 shutdown_connections 2024-02-22T16:14:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3981826913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8441027f0 0x7fe8440fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3981826913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844102220 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 -- 172.21.15.19:0/3981826913 >> 172.21.15.19:0/3981826913 conn(0x7fe8440f79e0 msgr2=0x7fe8440f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 -- 172.21.15.19:0/3981826913 shutdown_connections 2024-02-22T16:14:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.025+0000 7fe84aec6700 1 -- 172.21.15.19:0/3981826913 wait complete. 2024-02-22T16:14:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe84aec6700 1 Processor -- start 2024-02-22T16:14:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe84aec6700 1 -- start start 2024-02-22T16:14:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe84aec6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe84aec6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8441027f0 0x7fe844112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe84aec6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe844113420 con 0x7fe844101e50 2024-02-22T16:14:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe84aec6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84410c750 con 0x7fe8441027f0 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.026+0000 7fe848c62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe848c62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844112890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49678/0 (socket says 172.21.15.19:49678) 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe848c62700 1 -- 172.21.15.19:0/3076344178 learned_addr learned my addr 172.21.15.19:0/3076344178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe843fff700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8441027f0 0x7fe844112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe848c62700 1 -- 172.21.15.19:0/3076344178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8441027f0 msgr2=0x7fe844112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe848c62700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8441027f0 0x7fe844112dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe848c62700 1 -- 172.21.15.19:0/3076344178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838005040 con 0x7fe844101e50 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe848c62700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844112890 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe838009180 tx=0x7fe838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83800e040 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe838004290 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84410c9d0 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe838013660 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.027+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84410ceb0 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.028+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe844064b00 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.029+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe83801e9d0 con 0x7fe844101e50 2024-02-22T16:14:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.029+0000 7fe8417fa700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe83406cc20 0x7fe83406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.029+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe83808fed0 con 0x7fe844101e50 2024-02-22T16:14:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.029+0000 7fe843fff700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe83406cc20 0x7fe83406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.030+0000 7fe843fff700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe83406cc20 0x7fe83406f0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe84410e0c0 tx=0x7fe83000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.034+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe83805d910 con 0x7fe844101e50 2024-02-22T16:14:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.326+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8440fb5e0 con 0x7fe844101e50 2024-02-22T16:14:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.326+0000 7fe8417fa700 1 -- 172.21.15.19:0/3076344178 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fe83805d4a0 con 0x7fe844101e50 2024-02-22T16:14:26.301 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-22T16:14:26.301 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:14:26.301 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:14:26.302 INFO:teuthology.orchestra.run.smithi019.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-22T16:14:26.302 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe83406cc20 msgr2=0x7fe83406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe83406cc20 0x7fe83406f0d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe84410e0c0 tx=0x7fe83000b040 comp rx=0 tx=0).stop 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 msgr2=0x7fe844112890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844112890 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe838009180 tx=0x7fe838009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 shutdown_connections 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe83406cc20 0x7fe83406f0d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe844101e50 0x7fe844112890 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 --2- 172.21.15.19:0/3076344178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8441027f0 0x7fe844112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 >> 172.21.15.19:0/3076344178 conn(0x7fe8440f79e0 msgr2=0x7fe8440f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 shutdown_connections 2024-02-22T16:14:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:26.329+0000 7fe84aec6700 1 -- 172.21.15.19:0/3076344178 wait complete. 2024-02-22T16:14:26.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:26 smithi062 ceph-mon[106861]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:26.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:26 smithi062 ceph-mon[106861]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:26.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:26 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2924087205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:26.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:26 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2940256503' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:14:26.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:26 smithi019 ceph-mon[173579]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:26.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:26 smithi019 ceph-mon[173579]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:26.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:26 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2924087205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:26.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:26 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2940256503' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:14:27.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:27 smithi062 ceph-mon[106861]: pgmap v635: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:27.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:27 smithi062 ceph-mon[106861]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:27.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:27 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3076344178' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:14:27.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:27 smithi019 ceph-mon[173579]: pgmap v635: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:27.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:27 smithi019 ceph-mon[173579]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:27.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:27 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3076344178' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:14:29.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:29 smithi062 ceph-mon[106861]: pgmap v636: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:29.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:29 smithi019 ceph-mon[173579]: pgmap v636: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:31.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:31 smithi019 ceph-mon[173579]: pgmap v637: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:31.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:31 smithi062 ceph-mon[106861]: pgmap v637: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:32.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:32 smithi062 ceph-mon[106861]: pgmap v638: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:32.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:32 smithi019 ceph-mon[173579]: pgmap v638: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:34.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:34.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:35.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:35 smithi062 ceph-mon[106861]: pgmap v639: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:35.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:35 smithi019 ceph-mon[173579]: pgmap v639: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:37.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:37 smithi062 ceph-mon[106861]: pgmap v640: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:37.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:37 smithi019 ceph-mon[173579]: pgmap v640: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:39.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:39 smithi062 ceph-mon[106861]: pgmap v641: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:39.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:39 smithi019 ceph-mon[173579]: pgmap v641: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:41.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:41 smithi019 ceph-mon[173579]: pgmap v642: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:41.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:41 smithi062 ceph-mon[106861]: pgmap v642: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:43.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:43 smithi062 ceph-mon[106861]: pgmap v643: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:43.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:43 smithi019 ceph-mon[173579]: pgmap v643: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:45.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:45 smithi062 ceph-mon[106861]: pgmap v644: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:45.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:45 smithi019 ceph-mon[173579]: pgmap v644: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:47.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:47 smithi062 ceph-mon[106861]: pgmap v645: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:47.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:47 smithi019 ceph-mon[173579]: pgmap v645: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:49.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:49 smithi062 ceph-mon[106861]: pgmap v646: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:49.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:49.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:49 smithi019 ceph-mon[173579]: pgmap v646: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:49.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:14:51.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:51 smithi019 ceph-mon[173579]: pgmap v647: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:51.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:51 smithi062 ceph-mon[106861]: pgmap v647: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:53.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:53 smithi062 ceph-mon[106861]: pgmap v648: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:53.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:53 smithi019 ceph-mon[173579]: pgmap v648: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:55.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:55 smithi062 ceph-mon[106861]: pgmap v649: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:55.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:55 smithi019 ceph-mon[173579]: pgmap v649: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:56.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:14:56.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.fesnwx"}]': finished 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.lkzblb"}]': finished 2024-02-22T16:14:56.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T16:14:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.eyhruo"}]': finished 2024-02-22T16:14:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T16:14:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.oyfagg"}]': finished 2024-02-22T16:14:56.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:56 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-22T16:14:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 -- 172.21.15.19:0/1125909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 msgr2=0x7ff598103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 --2- 172.21.15.19:0/1125909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598103510 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff588004940 tx=0x7ff588009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 -- 172.21.15.19:0/1125909391 shutdown_connections 2024-02-22T16:14:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 --2- 172.21.15.19:0/1125909391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5980fb570 0x7ff5980fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 --2- 172.21.15.19:0/1125909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598103510 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 -- 172.21.15.19:0/1125909391 >> 172.21.15.19:0/1125909391 conn(0x7ff598071560 msgr2=0x7ff598071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 -- 172.21.15.19:0/1125909391 shutdown_connections 2024-02-22T16:14:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.490+0000 7ff59c96e700 1 -- 172.21.15.19:0/1125909391 wait complete. 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff59c96e700 1 Processor -- start 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff59c96e700 1 -- start start 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff59c96e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5980fb570 0x7ff598113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff59c96e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff59c96e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5981145d0 con 0x7ff598103140 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff59c96e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59810d900 con 0x7ff5980fb570 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff595d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff595d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34506/0 (socket says 172.21.15.19:34506) 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.491+0000 7ff595d9b700 1 -- 172.21.15.19:0/748442612 learned_addr learned my addr 172.21.15.19:0/748442612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff59659c700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5980fb570 0x7ff598113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff595d9b700 1 -- 172.21.15.19:0/748442612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5980fb570 msgr2=0x7ff598113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff595d9b700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5980fb570 0x7ff598113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff595d9b700 1 -- 172.21.15.19:0/748442612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff588005040 con 0x7ff598103140 2024-02-22T16:14:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff595d9b700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598113f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00eb40 tx=0x7ff58c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58c007cc0 con 0x7ff598103140 2024-02-22T16:14:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff58c00f040 con 0x7ff598103140 2024-02-22T16:14:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58c009570 con 0x7ff598103140 2024-02-22T16:14:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59810db80 con 0x7ff598103140 2024-02-22T16:14:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.492+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59810e070 con 0x7ff598103140 2024-02-22T16:14:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.494+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff598100310 con 0x7ff598103140 2024-02-22T16:14:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.495+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff58c0096d0 con 0x7ff598103140 2024-02-22T16:14:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.495+0000 7ff586ffd700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff58006cc70 0x7ff58006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.495+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff58c08c020 con 0x7ff598103140 2024-02-22T16:14:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.495+0000 7ff59659c700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff58006cc70 0x7ff58006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.496+0000 7ff59659c700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff58006cc70 0x7ff58006f120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff58800b0d0 tx=0x7ff588009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:56.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.503+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff58c059af0 con 0x7ff598103140 2024-02-22T16:14:56.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:14:56.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.fesnwx"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.fesnwx"}]': finished 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.lkzblb"}]: dispatch 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.lkzblb"}]': finished 2024-02-22T16:14:56.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.eyhruo"}]: dispatch 2024-02-22T16:14:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.eyhruo"}]': finished 2024-02-22T16:14:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.oyfagg"}]: dispatch 2024-02-22T16:14:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi062.oyfagg"}]': finished 2024-02-22T16:14:56.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:56 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-22T16:14:56.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.732+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff59810f090 con 0x7ff58006cc70 2024-02-22T16:14:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.734+0000 7ff586ffd700 1 -- 172.21.15.19:0/748442612 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff59810f090 con 0x7ff58006cc70 2024-02-22T16:14:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.736+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff58006cc70 msgr2=0x7ff58006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.736+0000 7ff59c96e700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff58006cc70 0x7ff58006f120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff58800b0d0 tx=0x7ff588009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.736+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 msgr2=0x7ff598113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.736+0000 7ff59c96e700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598113f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00eb40 tx=0x7ff58c00b040 comp rx=0 tx=0).stop 2024-02-22T16:14:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 shutdown_connections 2024-02-22T16:14:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff58006cc70 0x7ff58006f120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5980fb570 0x7ff598113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 --2- 172.21.15.19:0/748442612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff598103140 0x7ff598113f80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 >> 172.21.15.19:0/748442612 conn(0x7ff598071560 msgr2=0x7ff5980fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 shutdown_connections 2024-02-22T16:14:56.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.737+0000 7ff59c96e700 1 -- 172.21.15.19:0/748442612 wait complete. 2024-02-22T16:14:56.722 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:14:56.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 -- 172.21.15.19:0/1359725354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 msgr2=0x7fcdb4106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/1359725354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 0x7fcdb4106f40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fcda8004940 tx=0x7fcda8009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 -- 172.21.15.19:0/1359725354 shutdown_connections 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/1359725354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 0x7fcdb4106f40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/1359725354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb40fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 -- 172.21.15.19:0/1359725354 >> 172.21.15.19:0/1359725354 conn(0x7fcdb40f7a10 msgr2=0x7fcdb40f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.876+0000 7fcdb95f5700 1 -- 172.21.15.19:0/1359725354 shutdown_connections 2024-02-22T16:14:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.877+0000 7fcdb95f5700 1 -- 172.21.15.19:0/1359725354 wait complete. 2024-02-22T16:14:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.877+0000 7fcdb95f5700 1 Processor -- start 2024-02-22T16:14:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb95f5700 1 -- start start 2024-02-22T16:14:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb95f5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb41127a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb95f5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 0x7fcdb410b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb95f5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb410bb00 con 0x7fcdb40fe900 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb95f5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb410bc40 con 0x7fcdb40fdf60 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb41127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb41127a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:51226/0 (socket says 172.21.15.19:51226) 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.878+0000 7fcdb2ffd700 1 -- 172.21.15.19:0/629771943 learned_addr learned my addr 172.21.15.19:0/629771943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcdb2ffd700 1 -- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 msgr2=0x7fcdb410b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcdb2ffd700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 0x7fcdb410b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcdb2ffd700 1 -- 172.21.15.19:0/629771943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8005040 con 0x7fcdb40fdf60 2024-02-22T16:14:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcdb2ffd700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb41127a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcda400eb70 tx=0x7fcda400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda4007cc0 con 0x7fcdb40fdf60 2024-02-22T16:14:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb410bec0 con 0x7fcdb40fdf60 2024-02-22T16:14:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcda400f040 con 0x7fcdb40fdf60 2024-02-22T16:14:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda40095f0 con 0x7fcdb40fdf60 2024-02-22T16:14:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.879+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb410c400 con 0x7fcdb40fdf60 2024-02-22T16:14:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.881+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb40fb610 con 0x7fcdb40fdf60 2024-02-22T16:14:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.881+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcda4009750 con 0x7fcdb40fdf60 2024-02-22T16:14:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.882+0000 7fcd9b7fe700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcd9c06cc20 0x7fcd9c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.882+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcda408bce0 con 0x7fcdb40fdf60 2024-02-22T16:14:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.882+0000 7fcdb27fc700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcd9c06cc20 0x7fcd9c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.883+0000 7fcdb27fc700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcd9c06cc20 0x7fcd9c06f0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b0d0 tx=0x7fcda8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:56.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:56.887+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fcda4059700 con 0x7fcdb40fdf60 2024-02-22T16:14:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.113+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdb410ce10 con 0x7fcd9c06cc20 2024-02-22T16:14:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.115+0000 7fcd9b7fe700 1 -- 172.21.15.19:0/629771943 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcdb410ce10 con 0x7fcd9c06cc20 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcd9c06cc20 msgr2=0x7fcd9c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcd9c06cc20 0x7fcd9c06f0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b0d0 tx=0x7fcda8009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 msgr2=0x7fcdb41127a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb41127a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcda400eb70 tx=0x7fcda400b040 comp rx=0 tx=0).stop 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 shutdown_connections 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fcd9c06cc20 0x7fcd9c06f0d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdb40fdf60 0x7fcdb41127a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 --2- 172.21.15.19:0/629771943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdb40fe900 0x7fcdb410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 >> 172.21.15.19:0/629771943 conn(0x7fcdb40f7a10 msgr2=0x7fcdb40fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:57.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 shutdown_connections 2024-02-22T16:14:57.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.117+0000 7fcdb95f5700 1 -- 172.21.15.19:0/629771943 wait complete. 2024-02-22T16:14:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.257+0000 7f621c22a700 1 -- 172.21.15.19:0/2040000504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 msgr2=0x7f62140fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.257+0000 7f621c22a700 1 --2- 172.21.15.19:0/2040000504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fb9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6204004940 tx=0x7f6204009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.257+0000 7f621c22a700 1 -- 172.21.15.19:0/2040000504 shutdown_connections 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.257+0000 7f621c22a700 1 --2- 172.21.15.19:0/2040000504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fb9c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.257+0000 7f621c22a700 1 --2- 172.21.15.19:0/2040000504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 0x7f6214103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.258+0000 7f621c22a700 1 -- 172.21.15.19:0/2040000504 >> 172.21.15.19:0/2040000504 conn(0x7f6214071560 msgr2=0x7f6214071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.258+0000 7f621c22a700 1 -- 172.21.15.19:0/2040000504 shutdown_connections 2024-02-22T16:14:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.258+0000 7f621c22a700 1 -- 172.21.15.19:0/2040000504 wait complete. 2024-02-22T16:14:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.258+0000 7f621c22a700 1 Processor -- start 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.258+0000 7f621c22a700 1 -- start start 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f621c22a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f621c22a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 0x7f62140fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f621c22a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62141119a0 con 0x7f62140fb570 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f621c22a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6214111b10 con 0x7f6214103140 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f6219fc6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f6219fc6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34538/0 (socket says 172.21.15.19:34538) 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f6219fc6700 1 -- 172.21.15.19:0/3798620129 learned_addr learned my addr 172.21.15.19:0/3798620129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f62197c5700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 0x7f62140fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f6219fc6700 1 -- 172.21.15.19:0/3798620129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 msgr2=0x7f62140fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f6219fc6700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 0x7f62140fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f6219fc6700 1 -- 172.21.15.19:0/3798620129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204005040 con 0x7f62140fb570 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.259+0000 7f62197c5700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 0x7f62140fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-22T16:14:57.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.260+0000 7f6219fc6700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fc750 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f621000eb40 tx=0x7f621000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.260+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6210007cc0 con 0x7f62140fb570 2024-02-22T16:14:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.260+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f621000f040 con 0x7f62140fb570 2024-02-22T16:14:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.260+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f621000c930 con 0x7f62140fb570 2024-02-22T16:14:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.261+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62140fd2c0 con 0x7f62140fb570 2024-02-22T16:14:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.261+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621411e6a0 con 0x7f62140fb570 2024-02-22T16:14:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.262+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62141054d0 con 0x7f62140fb570 2024-02-22T16:14:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.262+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6210009120 con 0x7f62140fb570 2024-02-22T16:14:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.263+0000 7f620a7fc700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f620006cc20 0x7f620006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.263+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f621008b7c0 con 0x7f62140fb570 2024-02-22T16:14:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.263+0000 7f62197c5700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f620006cc20 0x7f620006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.264+0000 7f62197c5700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f620006cc20 0x7f620006f0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f62140fe860 tx=0x7f6204009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:57.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.268+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f62100557f0 con 0x7f62140fb570 2024-02-22T16:14:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: pgmap v650: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all mds 2024-02-22T16:14:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: Upgrade: Setting filesystem cephfs Joinable 2024-02-22T16:14:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-02-22T16:14:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-22T16:14:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: Standby daemon mds.cephfs.smithi019.lkzblb assigned to filesystem cephfs as rank 0 2024-02-22T16:14:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: Standby daemon mds.cephfs.smithi019.fesnwx assigned to filesystem cephfs as rank 1 2024-02-22T16:14:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-22T16:14:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T16:14:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:replay,1=cephfs.smithi019.fesnwx=up:replay} 2 up:standby 2024-02-22T16:14:57.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:57 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-02-22T16:14:57.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: pgmap v650: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all mds 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: Upgrade: Setting filesystem cephfs Joinable 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: Standby daemon mds.cephfs.smithi019.lkzblb assigned to filesystem cephfs as rank 0 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: Standby daemon mds.cephfs.smithi019.fesnwx assigned to filesystem cephfs as rank 1 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:replay,1=cephfs.smithi019.fesnwx=up:replay} 2 up:standby 2024-02-22T16:14:57.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:57 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-02-22T16:14:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.697+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f621404bed0 con 0x7f620006cc20 2024-02-22T16:14:57.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.718+0000 7f620a7fc700 1 -- 172.21.15.19:0/3798620129 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f621404bed0 con 0x7f620006cc20 2024-02-22T16:14:57.692 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:14:57.692 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (22m) 15m ago 33m 27.0M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:14:57.692 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (19m) 15m ago 33m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (19m) 14m ago 32m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (21m) 15m ago 32m 88.1M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (15m) 15m ago 27m 13.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (15m) 15m ago 27m 12.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (15m) 14m ago 27m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (15m) 14m ago 27m 12.4M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (24m) 15m ago 34m 504M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (23m) 14m ago 32m 450M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (20m) 15m ago 34m 54.2M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (20m) 14m ago 32m 45.4M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (23m) 15m ago 33m 18.5M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:14:57.693 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (22m) 14m ago 32m 17.3M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (19m) 15m ago 31m 92.1M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (18m) 15m ago 30m 95.9M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (17m) 15m ago 30m 83.2M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (17m) 14m ago 29m 99.8M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (16m) 14m ago 29m 92.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (16m) 14m ago 29m 86.1M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:14:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (22m) 15m ago 32m 61.8M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:14:57.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f620006cc20 msgr2=0x7f620006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f620006cc20 0x7f620006f0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f62140fe860 tx=0x7f6204009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:57.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 msgr2=0x7f62140fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fc750 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f621000eb40 tx=0x7f621000b040 comp rx=0 tx=0).stop 2024-02-22T16:14:57.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 shutdown_connections 2024-02-22T16:14:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f620006cc20 0x7f620006f0d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62140fb570 0x7f62140fc750 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 --2- 172.21.15.19:0/3798620129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6214103140 0x7f62140fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.721+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 >> 172.21.15.19:0/3798620129 conn(0x7f6214071560 msgr2=0x7f62140f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:57.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.727+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 shutdown_connections 2024-02-22T16:14:57.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.727+0000 7f621c22a700 1 -- 172.21.15.19:0/3798620129 wait complete. 2024-02-22T16:14:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.899+0000 7f5574b3c700 1 -- 172.21.15.19:0/2449594956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fbdd0 msgr2=0x7f55700fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.899+0000 7f5574b3c700 1 --2- 172.21.15.19:0/2449594956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fbdd0 0x7f55700fc1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5564004940 tx=0x7f5564009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 -- 172.21.15.19:0/2449594956 shutdown_connections 2024-02-22T16:14:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 --2- 172.21.15.19:0/2449594956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55700fc6e0 0x7f5570100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 --2- 172.21.15.19:0/2449594956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fbdd0 0x7f55700fc1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 -- 172.21.15.19:0/2449594956 >> 172.21.15.19:0/2449594956 conn(0x7f5570071d90 msgr2=0x7f5570072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 -- 172.21.15.19:0/2449594956 shutdown_connections 2024-02-22T16:14:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 -- 172.21.15.19:0/2449594956 wait complete. 2024-02-22T16:14:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 Processor -- start 2024-02-22T16:14:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.900+0000 7f5574b3c700 1 -- start start 2024-02-22T16:14:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f5574b3c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55700fbdd0 0x7f557006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f5574b3c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 0x7f557006b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f5574b3c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557006fda0 con 0x7f55700fbdd0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f5574b3c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557006fee0 con 0x7f55700fc6e0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 0x7f557006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 0x7f557006b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:46634/0 (socket says 172.21.15.19:46634) 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 -- 172.21.15.19:0/3048685946 learned_addr learned my addr 172.21.15.19:0/3048685946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 -- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55700fbdd0 msgr2=0x7f557006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55700fbdd0 0x7f557006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 -- 172.21.15.19:0/3048685946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5564005040 con 0x7f55700fc6e0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.901+0000 7f556dd9b700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 0x7f557006b080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5560002a50 tx=0x7f55600046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.902+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5560007cc0 con 0x7f55700fc6e0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.902+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f556000f040 con 0x7f55700fc6e0 2024-02-22T16:14:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.902+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5560009540 con 0x7f55700fc6e0 2024-02-22T16:14:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.902+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557006b5c0 con 0x7f55700fc6e0 2024-02-22T16:14:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.902+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5570126d10 con 0x7f55700fc6e0 2024-02-22T16:14:57.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.903+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55700fe540 con 0x7f55700fc6e0 2024-02-22T16:14:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.904+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f55600096a0 con 0x7f55700fc6e0 2024-02-22T16:14:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.905+0000 7f555effd700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f555806cc30 0x7f555806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.905+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f556008b800 con 0x7f55700fc6e0 2024-02-22T16:14:57.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.906+0000 7f556e59c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f555806cc30 0x7f555806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.906+0000 7f556e59c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f555806cc30 0x7f555806f0e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5564004f90 tx=0x7f5564009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:57.909+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f55600557b0 con 0x7f55700fc6e0 2024-02-22T16:14:58.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.283+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f55700738b0 con 0x7f55700fc6e0 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:14:58.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.284+0000 7f555effd700 1 -- 172.21.15.19:0/3048685946 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f5560058dd0 con 0x7f55700fc6e0 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f555806cc30 msgr2=0x7f555806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f555806cc30 0x7f555806f0e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5564004f90 tx=0x7f5564009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 msgr2=0x7f557006b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 0x7f557006b080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5560002a50 tx=0x7f55600046b0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 shutdown_connections 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f555806cc30 0x7f555806f0e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55700fbdd0 0x7f557006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 --2- 172.21.15.19:0/3048685946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55700fc6e0 0x7f557006b080 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.286+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 >> 172.21.15.19:0/3048685946 conn(0x7f5570071d90 msgr2=0x7f557010d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:58.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.291+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 shutdown_connections 2024-02-22T16:14:58.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.291+0000 7f5574b3c700 1 -- 172.21.15.19:0/3048685946 wait complete. 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 -- 172.21.15.19:0/2334794319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 msgr2=0x7f708c0fbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 --2- 172.21.15.19:0/2334794319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 0x7f708c0fbb90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f707c004970 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 -- 172.21.15.19:0/2334794319 shutdown_connections 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 --2- 172.21.15.19:0/2334794319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c10d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 --2- 172.21.15.19:0/2334794319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 0x7f708c0fbb90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 -- 172.21.15.19:0/2334794319 >> 172.21.15.19:0/2334794319 conn(0x7f708c071560 msgr2=0x7f708c071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 -- 172.21.15.19:0/2334794319 shutdown_connections 2024-02-22T16:14:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.422+0000 7f7093486700 1 -- 172.21.15.19:0/2334794319 wait complete. 2024-02-22T16:14:58.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7093486700 1 Processor -- start 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7093486700 1 -- start start 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7093486700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 0x7f708c11a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7093486700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c11a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7093486700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c11abd0 con 0x7f708c0fc0d0 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7093486700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c113ee0 con 0x7f708c0fb7c0 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7090a21700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c11a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7090a21700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c11a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49850/0 (socket says 172.21.15.19:49850) 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.423+0000 7f7090a21700 1 -- 172.21.15.19:0/811514613 learned_addr learned my addr 172.21.15.19:0/811514613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.424+0000 7f7091222700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 0x7f708c11a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.424+0000 7f7090a21700 1 -- 172.21.15.19:0/811514613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 msgr2=0x7f708c11a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.424+0000 7f7090a21700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 0x7f708c11a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.424+0000 7f7090a21700 1 -- 172.21.15.19:0/811514613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c005040 con 0x7f708c0fc0d0 2024-02-22T16:14:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.424+0000 7f7090a21700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c11a580 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7088002a50 tx=0x7f7088004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.425+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7088007cc0 con 0x7f708c0fc0d0 2024-02-22T16:14:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.425+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f708800f040 con 0x7f708c0fc0d0 2024-02-22T16:14:58.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.425+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1141c0 con 0x7f708c0fc0d0 2024-02-22T16:14:58.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.425+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f708800d8b0 con 0x7f708c0fc0d0 2024-02-22T16:14:58.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.425+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c114700 con 0x7f708c0fc0d0 2024-02-22T16:14:58.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.427+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c0fe0e0 con 0x7f708c0fc0d0 2024-02-22T16:14:58.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.431+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f708800da10 con 0x7f708c0fc0d0 2024-02-22T16:14:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.431+0000 7f7081ffb700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f707806ccf0 0x7f707806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.432+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f708808bbd0 con 0x7f708c0fc0d0 2024-02-22T16:14:58.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.434+0000 7f7091222700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f707806ccf0 0x7f707806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:58.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.435+0000 7f7091222700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f707806ccf0 0x7f707806f1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f707c004f50 tx=0x7f707c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:58.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.436+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7088059670 con 0x7f708c0fc0d0 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='client.44997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:replay,1=cephfs.smithi019.fesnwx=up:replay} 2 up:standby 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.451 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-22T16:14:58.452 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.453 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.454 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:58 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='client.44997 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:replay,1=cephfs.smithi019.fesnwx=up:replay} 2 up:standby 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-22T16:14:58.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-22T16:14:58.490 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.491 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.492 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:58 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.750+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f708c073080 con 0x7f708c0fc0d0 2024-02-22T16:14:58.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.750+0000 7f7081ffb700 1 -- 172.21.15.19:0/811514613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f7088059200 con 0x7f708c0fc0d0 2024-02-22T16:14:58.724 INFO:teuthology.orchestra.run.smithi019.stdout:e38 2024-02-22T16:14:58.724 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 38 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T16:14:57.383267+0000 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:14:58.725 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=34392,1=34424} 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:14:58.726 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:34392} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{1:34424} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{-1:44333} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f707806ccf0 msgr2=0x7f707806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f707806ccf0 0x7f707806f1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f707c004f50 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 msgr2=0x7f708c11a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c11a580 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7088002a50 tx=0x7f7088004600 comp rx=0 tx=0).stop 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 shutdown_connections 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f707806ccf0 0x7f707806f1a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708c0fb7c0 0x7f708c11a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 --2- 172.21.15.19:0/811514613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f708c0fc0d0 0x7f708c11a580 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 >> 172.21.15.19:0/811514613 conn(0x7f708c071560 msgr2=0x7f708c10bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 shutdown_connections 2024-02-22T16:14:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.754+0000 7f7093486700 1 -- 172.21.15.19:0/811514613 wait complete. 2024-02-22T16:14:58.729 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 38 2024-02-22T16:14:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 -- 172.21.15.19:0/4078281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 msgr2=0x7f68e00fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 --2- 172.21.15.19:0/4078281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e00fc110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f68c8004940 tx=0x7f68c8009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 -- 172.21.15.19:0/4078281804 shutdown_connections 2024-02-22T16:14:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 --2- 172.21.15.19:0/4078281804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 0x7f68e0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 --2- 172.21.15.19:0/4078281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e00fc110 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 -- 172.21.15.19:0/4078281804 >> 172.21.15.19:0/4078281804 conn(0x7f68e00f79e0 msgr2=0x7f68e00f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 -- 172.21.15.19:0/4078281804 shutdown_connections 2024-02-22T16:14:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.922+0000 7f68e6286700 1 -- 172.21.15.19:0/4078281804 wait complete. 2024-02-22T16:14:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.923+0000 7f68e6286700 1 Processor -- start 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.923+0000 7f68e6286700 1 -- start start 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68e6286700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68e6286700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 0x7f68e006ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68e6286700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e006b110 con 0x7f68e00fbd40 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68e6286700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e006b280 con 0x7f68e00fc6e0 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68df7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 0x7f68e006ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e006fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49864/0 (socket says 172.21.15.19:49864) 2024-02-22T16:14:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68dffff700 1 -- 172.21.15.19:0/481732334 learned_addr learned my addr 172.21.15.19:0/481732334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68df7fe700 1 -- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 msgr2=0x7f68e006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68df7fe700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68df7fe700 1 -- 172.21.15.19:0/481732334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8005040 con 0x7f68e00fc6e0 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.924+0000 7f68df7fe700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 0x7f68e006ab10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009fd0 tx=0x7f68d000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.925+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68d0004e70 con 0x7f68e00fc6e0 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.925+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68d000f040 con 0x7f68e00fc6e0 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.925+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68d000c930 con 0x7f68e00fc6e0 2024-02-22T16:14:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.925+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e006b500 con 0x7f68e00fc6e0 2024-02-22T16:14:58.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.925+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e0114d20 con 0x7f68e00fc6e0 2024-02-22T16:14:58.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.926+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e00fe420 con 0x7f68e00fc6e0 2024-02-22T16:14:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.927+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f68d00090d0 con 0x7f68e00fc6e0 2024-02-22T16:14:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.927+0000 7f68dcff9700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68cc06cc20 0x7f68cc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.927+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f68d008bbb0 con 0x7f68e00fc6e0 2024-02-22T16:14:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.928+0000 7f68dffff700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68cc06cc20 0x7f68cc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.928+0000 7f68dffff700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68cc06cc20 0x7f68cc06f0d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b0d0 tx=0x7f68c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:58.932+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f68d0092080 con 0x7f68e00fc6e0 2024-02-22T16:14:59.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: pgmap v651: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:59.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all rgw 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all rbd-mirror 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all ceph-exporter 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all iscsi 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all nfs 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Setting container_image for all nvmeof 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Finalizing container_image settings 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: Upgrade: Complete! 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3048685946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:59.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:14:59 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/811514613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:14:59.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.296+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68e004bf70 con 0x7f68cc06cc20 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.297+0000 7f68dcff9700 1 -- 172.21.15.19:0/481732334 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f68c8007de0 con 0x7f68cc06cc20 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": null, 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": false, 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "", 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": null, 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:14:59.271 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:14:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68cc06cc20 msgr2=0x7f68cc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68cc06cc20 0x7f68cc06f0d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b0d0 tx=0x7f68c8009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 msgr2=0x7f68e006ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 0x7f68e006ab10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009fd0 tx=0x7f68d000b040 comp rx=0 tx=0).stop 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 shutdown_connections 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68cc06cc20 0x7f68cc06f0d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68e00fbd40 0x7f68e006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 --2- 172.21.15.19:0/481732334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68e00fc6e0 0x7f68e006ab10 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009fd0 tx=0x7f68d000b040 comp rx=0 tx=0).stop 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.299+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 >> 172.21.15.19:0/481732334 conn(0x7f68e00f79e0 msgr2=0x7f68e010d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.300+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 shutdown_connections 2024-02-22T16:14:59.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.300+0000 7f68e6286700 1 -- 172.21.15.19:0/481732334 wait complete. 2024-02-22T16:14:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.464+0000 7f68e23e0700 1 -- 172.21.15.19:0/734164515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc103dc0 msgr2=0x7f68dc104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.464+0000 7f68e23e0700 1 --2- 172.21.15.19:0/734164515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc103dc0 0x7f68dc104230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f68cc004940 tx=0x7f68cc009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 -- 172.21.15.19:0/734164515 shutdown_connections 2024-02-22T16:14:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 --2- 172.21.15.19:0/734164515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc103dc0 0x7f68dc104230 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 --2- 172.21.15.19:0/734164515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc1095b0 0x7f68dc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 -- 172.21.15.19:0/734164515 >> 172.21.15.19:0/734164515 conn(0x7f68dc072d70 msgr2=0x7f68dc075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 -- 172.21.15.19:0/734164515 shutdown_connections 2024-02-22T16:14:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 -- 172.21.15.19:0/734164515 wait complete. 2024-02-22T16:14:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.466+0000 7f68e23e0700 1 Processor -- start 2024-02-22T16:14:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68e23e0700 1 -- start start 2024-02-22T16:14:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68e23e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 0x7f68dc06fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68e23e0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc1095b0 0x7f68dc06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68e23e0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc06b110 con 0x7f68dc1095b0 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68e23e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc06b280 con 0x7f68dc103dc0 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 0x7f68dc06fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68dbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 0x7f68dc06fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.19:46692/0 (socket says 172.21.15.19:46692) 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.467+0000 7f68dbfff700 1 -- 172.21.15.19:0/263514926 learned_addr learned my addr 172.21.15.19:0/263514926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68db7fe700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc1095b0 0x7f68dc06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68dbfff700 1 -- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc1095b0 msgr2=0x7f68dc06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68dbfff700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc1095b0 0x7f68dc06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68dbfff700 1 -- 172.21.15.19:0/263514926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc005040 con 0x7f68dc103dc0 2024-02-22T16:14:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68db7fe700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc1095b0 0x7f68dc06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68dbfff700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 0x7f68dc06fc60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f68d0002a80 tx=0x7f68d0004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68d0007cc0 con 0x7f68dc103dc0 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc06b560 con 0x7f68dc103dc0 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc0767e0 con 0x7f68dc103dc0 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68d000f040 con 0x7f68dc103dc0 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.468+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68d000da20 con 0x7f68dc103dc0 2024-02-22T16:14:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.469+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80052e0 con 0x7f68dc103dc0 2024-02-22T16:14:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.470+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f68d000db80 con 0x7f68dc103dc0 2024-02-22T16:14:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.471+0000 7f68d8ff9700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68c406cc30 0x7f68c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:14:59.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.471+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f68d008be70 con 0x7f68dc103dc0 2024-02-22T16:14:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.471+0000 7f68db7fe700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68c406cc30 0x7f68c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:14:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.471+0000 7f68db7fe700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68c406cc30 0x7f68c406f0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b3b0 tx=0x7f68cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:14:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:14:59.475+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f68d0055e70 con 0x7f68dc103dc0 2024-02-22T16:14:59.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: pgmap v651: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-02-22T16:14:59.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all rgw 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all rbd-mirror 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all ceph-exporter 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all iscsi 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all nfs 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Setting container_image for all nvmeof 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Finalizing container_image settings 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: Upgrade: Complete! 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3048685946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:14:59.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:14:59 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/811514613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:14:59.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.021+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f68c8005590 con 0x7f68dc103dc0 2024-02-22T16:14:59.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.022+0000 7f68d8ff9700 1 -- 172.21.15.19:0/263514926 <== mon.1 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3457 (secure 0 0 0) 0x7f68d0059490 con 0x7f68dc103dc0 2024-02-22T16:14:59.995 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: host smithi062 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi062/config 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 -e NODE_NAME=smithi062 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715:/var/run/ceph:z -v /var/log/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpfb04icqv:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 inventory --format=json-pretty --filter-for-batch 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout:Traceback (most recent call last): 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: "__main__", mod_spec) 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: exec(code, run_globals) 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 10863, in 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 10851, in main 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 2568, in _infer_config 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 2484, in _infer_fsid 2024-02-22T16:14:59.996 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 2596, in _infer_image 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 2471, in _validate_fsid 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 7129, in command_ceph_volume 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stdout: File "/tmp/tmph4d12z8_.cephadm.build/__main__.py", line 2259, in call_throws 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 -e NODE_NAME=smithi062 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715:/var/run/ceph:z -v /var/log/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpfb04icqv:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:75ec18f7067b26b32c6f90cd6827cc96a276ce86be68e998f0bd36838cbd5bd0 inventory --format=json-pretty --filter-for-batch 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stdout: fs cephfs is degraded 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68c406cc30 msgr2=0x7f68c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68c406cc30 0x7f68c406f0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b3b0 tx=0x7f68cc009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 msgr2=0x7f68dc06fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 0x7f68dc06fc60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f68d0002a80 tx=0x7f68d0004600 comp rx=0 tx=0).stop 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 shutdown_connections 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f68c406cc30 0x7f68c406f0e0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b3b0 tx=0x7f68cc009e30 comp rx=0 tx=0).stop 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68dc103dc0 0x7f68dc06fc60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 --2- 172.21.15.19:0/263514926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68dc1095b0 0x7f68dc06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:14:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.024+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 >> 172.21.15.19:0/263514926 conn(0x7f68dc072d70 msgr2=0x7f68dc0749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:14:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.025+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 shutdown_connections 2024-02-22T16:14:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:00.025+0000 7f68e23e0700 1 -- 172.21.15.19:0/263514926 wait complete. 2024-02-22T16:15:00.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:00 smithi019 ceph-mon[173579]: pgmap v652: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-02-22T16:15:00.162 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:00 smithi019 ceph-mon[173579]: from='client.45013 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:00.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:00 smithi062 ceph-mon[106861]: pgmap v652: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-02-22T16:15:00.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:00 smithi062 ceph-mon[106861]: from='client.45013 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:00.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:00 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/263514926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:15:00.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:00 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:00.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:00 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:00.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:00 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/263514926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:15:00.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:00 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:00.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:00 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:02.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:02 smithi019 ceph-mon[173579]: pgmap v653: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-02-22T16:15:02.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:02 smithi062 ceph-mon[106861]: pgmap v653: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-02-22T16:15:04.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: pgmap v654: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:resolve 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:replay,1=cephfs.smithi019.fesnwx=up:resolve} 2 up:standby 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: pgmap v655: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-02-22T16:15:04.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:04 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: pgmap v654: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-02-22T16:15:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:resolve 2024-02-22T16:15:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:replay,1=cephfs.smithi019.fesnwx=up:resolve} 2 up:standby 2024-02-22T16:15:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:15:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:15:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: pgmap v655: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-02-22T16:15:04.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:04 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:05 smithi062 ceph-mon[106861]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-02-22T16:15:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:15:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:15:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:15:05.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:05 smithi019 ceph-mon[173579]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-02-22T16:15:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:15:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:15:05.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:15:05.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:06.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:06 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:resolve 2024-02-22T16:15:06.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:06 smithi062 ceph-mon[106861]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:resolve,1=cephfs.smithi019.fesnwx=up:resolve} 2 up:standby 2024-02-22T16:15:06.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:06 smithi062 ceph-mon[106861]: pgmap v656: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-22T16:15:06.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:06 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:resolve 2024-02-22T16:15:06.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:06 smithi019 ceph-mon[173579]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:resolve,1=cephfs.smithi019.fesnwx=up:resolve} 2 up:standby 2024-02-22T16:15:06.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:06 smithi019 ceph-mon[173579]: pgmap v656: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-22T16:15:07.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: reconnect by client.14528 192.168.0.1:0/2316115328 after 0.00099999 2024-02-22T16:15:07.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: reconnect by client.14528 192.168.0.1:0/2316115328 after 0.00099999 2024-02-22T16:15:07.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: reconnect by client.24311 192.168.0.1:0/1047278498 after 0.00099999 2024-02-22T16:15:07.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: reconnect by client.24311 192.168.0.1:0/1047278498 after 0.00099999 2024-02-22T16:15:07.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:reconnect 2024-02-22T16:15:07.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:reconnect 2024-02-22T16:15:07.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:07 smithi062 ceph-mon[106861]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:reconnect,1=cephfs.smithi019.fesnwx=up:reconnect} 2 up:standby 2024-02-22T16:15:07.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: reconnect by client.14528 192.168.0.1:0/2316115328 after 0.00099999 2024-02-22T16:15:07.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: reconnect by client.14528 192.168.0.1:0/2316115328 after 0.00099999 2024-02-22T16:15:07.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: reconnect by client.24311 192.168.0.1:0/1047278498 after 0.00099999 2024-02-22T16:15:07.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: reconnect by client.24311 192.168.0.1:0/1047278498 after 0.00099999 2024-02-22T16:15:07.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:reconnect 2024-02-22T16:15:07.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:reconnect 2024-02-22T16:15:07.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:07 smithi019 ceph-mon[173579]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:reconnect,1=cephfs.smithi019.fesnwx=up:reconnect} 2 up:standby 2024-02-22T16:15:08.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:rejoin 2024-02-22T16:15:08.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:rejoin 2024-02-22T16:15:08.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:rejoin,1=cephfs.smithi019.fesnwx=up:rejoin} 2 up:standby 2024-02-22T16:15:08.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: daemon mds.cephfs.smithi019.fesnwx is now active in filesystem cephfs as rank 1 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: daemon mds.cephfs.smithi019.lkzblb is now active in filesystem cephfs as rank 0 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: pgmap v657: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: Cluster is now healthy 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:active 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:active 2024-02-22T16:15:08.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:08 smithi062 ceph-mon[106861]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi019.fesnwx=up:active} 2 up:standby-replay 2024-02-22T16:15:08.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:rejoin 2024-02-22T16:15:08.737 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:rejoin 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: fsmap cephfs:2/2 {0=cephfs.smithi019.lkzblb=up:rejoin,1=cephfs.smithi019.fesnwx=up:rejoin} 2 up:standby 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: daemon mds.cephfs.smithi019.fesnwx is now active in filesystem cephfs as rank 1 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: daemon mds.cephfs.smithi019.lkzblb is now active in filesystem cephfs as rank 0 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: pgmap v657: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: Cluster is now healthy 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] up:active 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: mds.? [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] up:active 2024-02-22T16:15:08.738 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:08 smithi019 ceph-mon[173579]: fsmap cephfs:2 {0=cephfs.smithi019.lkzblb=up:active,1=cephfs.smithi019.fesnwx=up:active} 2 up:standby-replay 2024-02-22T16:15:10.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:10 smithi019 ceph-mon[173579]: pgmap v658: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-22T16:15:10.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:10 smithi062 ceph-mon[106861]: pgmap v658: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-02-22T16:15:12.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:12 smithi019 ceph-mon[173579]: pgmap v659: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-02-22T16:15:12.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:12 smithi062 ceph-mon[106861]: pgmap v659: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-02-22T16:15:15.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:15 smithi062 ceph-mon[106861]: pgmap v660: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-02-22T16:15:15.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:15 smithi019 ceph-mon[173579]: pgmap v660: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-02-22T16:15:17.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:17 smithi062 ceph-mon[106861]: pgmap v661: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8.5 KiB/s wr, 17 op/s 2024-02-22T16:15:17.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:17 smithi019 ceph-mon[173579]: pgmap v661: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8.5 KiB/s wr, 17 op/s 2024-02-22T16:15:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:19 smithi062 ceph-mon[106861]: pgmap v662: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.5 KiB/s wr, 17 op/s 2024-02-22T16:15:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:19 smithi019 ceph-mon[173579]: pgmap v662: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.5 KiB/s wr, 17 op/s 2024-02-22T16:15:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:15:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:20.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:20 smithi062 ceph-mon[106861]: pgmap v663: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.5 KiB/s wr, 17 op/s 2024-02-22T16:15:20.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:20 smithi019 ceph-mon[173579]: pgmap v663: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.5 KiB/s wr, 17 op/s 2024-02-22T16:15:22.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:22 smithi019 ceph-mon[173579]: pgmap v664: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 8.5 KiB/s wr, 18 op/s 2024-02-22T16:15:22.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:22 smithi062 ceph-mon[106861]: pgmap v664: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 8.5 KiB/s wr, 18 op/s 2024-02-22T16:15:25.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:25 smithi062 ceph-mon[106861]: pgmap v665: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-02-22T16:15:25.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:25 smithi019 ceph-mon[173579]: pgmap v665: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-02-22T16:15:27.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:27 smithi062 ceph-mon[106861]: pgmap v666: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-02-22T16:15:27.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:27 smithi019 ceph-mon[173579]: pgmap v666: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-02-22T16:15:29.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:29 smithi062 ceph-mon[106861]: pgmap v667: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-22T16:15:29.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:29 smithi019 ceph-mon[173579]: pgmap v667: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-22T16:15:30.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 -- 172.21.15.19:0/1343724477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 msgr2=0x7f71880fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:30.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 --2- 172.21.15.19:0/1343724477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f71880fcf50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f717c004940 tx=0x7f717c009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:30.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 -- 172.21.15.19:0/1343724477 shutdown_connections 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 --2- 172.21.15.19:0/1343724477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f71880fcf50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 --2- 172.21.15.19:0/1343724477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7188102f40 0x7f7188103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 -- 172.21.15.19:0/1343724477 >> 172.21.15.19:0/1343724477 conn(0x7f71880f8af0 msgr2=0x7f71880faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 -- 172.21.15.19:0/1343724477 shutdown_connections 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.146+0000 7f718d028700 1 -- 172.21.15.19:0/1343724477 wait complete. 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718d028700 1 Processor -- start 2024-02-22T16:15:30.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718d028700 1 -- start start 2024-02-22T16:15:30.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718d028700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7188102f40 0x7f7188113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718d028700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f7188113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718d028700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71881145d0 con 0x7f71881038e0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718d028700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718810d900 con 0x7f7188102f40 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f7188113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f7188113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59754/0 (socket says 172.21.15.19:59754) 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f718659c700 1 -- 172.21.15.19:0/2352998404 learned_addr learned my addr 172.21.15.19:0/2352998404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.147+0000 7f7186d9d700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7188102f40 0x7f7188113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f718659c700 1 -- 172.21.15.19:0/2352998404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7188102f40 msgr2=0x7f7188113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f718659c700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7188102f40 0x7f7188113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f718659c700 1 -- 172.21.15.19:0/2352998404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c005040 con 0x7f71881038e0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f718659c700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f7188113f80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f717c009fd0 tx=0x7f717c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f717c00e040 con 0x7f71881038e0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f717c013420 con 0x7f71881038e0 2024-02-22T16:15:30.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718810db80 con 0x7f71881038e0 2024-02-22T16:15:30.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f717c01c630 con 0x7f71881038e0 2024-02-22T16:15:30.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.148+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718810e0c0 con 0x7f71881038e0 2024-02-22T16:15:30.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.149+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7188105410 con 0x7f71881038e0 2024-02-22T16:15:30.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.150+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f717c013a00 con 0x7f71881038e0 2024-02-22T16:15:30.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.150+0000 7f716f7fe700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f717006cc20 0x7f717006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:30.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.150+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f717c08d190 con 0x7f71881038e0 2024-02-22T16:15:30.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.150+0000 7f7186d9d700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f717006cc20 0x7f717006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:30.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.151+0000 7f7186d9d700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f717006cc20 0x7f717006f0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71780041d0 tx=0x7f71780043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:30.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.155+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f717c05acf0 con 0x7f71881038e0 2024-02-22T16:15:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.376+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7188109ee0 con 0x7f717006cc20 2024-02-22T16:15:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.377+0000 7f716f7fe700 1 -- 172.21.15.19:0/2352998404 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f7188109ee0 con 0x7f717006cc20 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f717006cc20 msgr2=0x7f717006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f717006cc20 0x7f717006f0d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71780041d0 tx=0x7f71780043c0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 msgr2=0x7f7188113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f7188113f80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f717c009fd0 tx=0x7f717c009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 shutdown_connections 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f717006cc20 0x7f717006f0d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7188102f40 0x7f7188113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 --2- 172.21.15.19:0/2352998404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71881038e0 0x7f7188113f80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 >> 172.21.15.19:0/2352998404 conn(0x7f71880f8af0 msgr2=0x7f71880fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:30.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 shutdown_connections 2024-02-22T16:15:30.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:30.379+0000 7f718d028700 1 -- 172.21.15.19:0/2352998404 wait complete. 2024-02-22T16:15:31.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch ps' 2024-02-22T16:15:31.389 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:15:31.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:31 smithi062 ceph-mon[106861]: pgmap v668: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-22T16:15:31.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:31 smithi019 ceph-mon[173579]: pgmap v668: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-22T16:15:32.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:32 smithi062 ceph-mon[106861]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:32.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:32 smithi019 ceph-mon[173579]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 -- 172.21.15.19:0/3875011002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c80fb570 msgr2=0x7f11c80fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/3875011002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c80fb570 0x7f11c80fb9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f11b8004940 tx=0x7f11b8009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 -- 172.21.15.19:0/3875011002 shutdown_connections 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/3875011002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c80fb570 0x7f11c80fb9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/3875011002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c8103140 0x7f11c8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 -- 172.21.15.19:0/3875011002 >> 172.21.15.19:0/3875011002 conn(0x7f11c8071560 msgr2=0x7f11c8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 -- 172.21.15.19:0/3875011002 shutdown_connections 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 -- 172.21.15.19:0/3875011002 wait complete. 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 Processor -- start 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.347+0000 7f11cfb6a700 1 -- start start 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cfb6a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c80fb570 0x7f11c8115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cfb6a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 0x7f11c8116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cfb6a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81167a0 con 0x7f11c8103140 2024-02-22T16:15:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cfb6a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c810faf0 con 0x7f11c80fb570 2024-02-22T16:15:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 0x7f11c8116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 0x7f11c8116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59778/0 (socket says 172.21.15.19:59778) 2024-02-22T16:15:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 -- 172.21.15.19:0/1354344859 learned_addr learned my addr 172.21.15.19:0/1354344859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:15:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd906700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c80fb570 0x7f11c8115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 -- 172.21.15.19:0/1354344859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c80fb570 msgr2=0x7f11c8115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c80fb570 0x7f11c8115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 -- 172.21.15.19:0/1354344859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8005040 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd906700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c80fb570 0x7f11c8115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cd105700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 0x7f11c8116150 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009360 tx=0x7f11b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b800e040 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11b800bc50 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c810fd70 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.348+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b801c630 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.349+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c81102b0 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.350+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8073080 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.350+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f11b8013420 con 0x7f11c8103140 2024-02-22T16:15:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.351+0000 7f11be7fc700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f11b406cc30 0x7f11b406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.351+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f11b808d350 con 0x7f11c8103140 2024-02-22T16:15:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.351+0000 7f11cd906700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f11b406cc30 0x7f11b406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.351+0000 7f11cd906700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f11b406cc30 0x7f11b406f0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009fd0 tx=0x7f11c400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.355+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f11b8057380 con 0x7f11c8103140 2024-02-22T16:15:33.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:33 smithi062 ceph-mon[106861]: pgmap v669: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-22T16:15:33.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:33 smithi019 ceph-mon[173579]: pgmap v669: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-22T16:15:33.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.569+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11c81111f0 con 0x7f11b406cc30 2024-02-22T16:15:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.579+0000 7f11be7fc700 1 -- 172.21.15.19:0/1354344859 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f11c81111f0 con 0x7f11b406cc30 2024-02-22T16:15:33.553 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:15:33.553 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (22m) 29s ago 33m 26.9M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (20m) 29s ago 34m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (20m) 33s ago 32m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (22m) 29s ago 33m 89.2M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (16m) 29s ago 28m 118M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (16m) 29s ago 28m 94.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (15m) 33s ago 27m 17.9M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (15m) 33s ago 28m 18.0M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (24m) 29s ago 35m 510M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (23m) 33s ago 32m 456M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (21m) 29s ago 35m 71.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (20m) 33s ago 32m 54.3M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (23m) 29s ago 34m 18.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (23m) 33s ago 32m 17.5M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:15:33.554 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (19m) 29s ago 31m 97.7M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:15:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (19m) 29s ago 31m 106M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:15:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (18m) 29s ago 30m 88.6M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:15:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (17m) 33s ago 30m 110M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:15:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (17m) 33s ago 30m 97.9M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:15:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (16m) 33s ago 29m 96.5M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:15:33.555 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (23m) 29s ago 33m 77.4M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f11b406cc30 msgr2=0x7f11b406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f11b406cc30 0x7f11b406f0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009fd0 tx=0x7f11c400b040 comp rx=0 tx=0).stop 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 msgr2=0x7f11c8116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 0x7f11c8116150 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009360 tx=0x7f11b8009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 shutdown_connections 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f11b406cc30 0x7f11b406f0e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11c80fb570 0x7f11c8115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.581+0000 7f11cfb6a700 1 --2- 172.21.15.19:0/1354344859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11c8103140 0x7f11c8116150 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.582+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 >> 172.21.15.19:0/1354344859 conn(0x7f11c8071560 msgr2=0x7f11c80f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.582+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 shutdown_connections 2024-02-22T16:15:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:33.582+0000 7f11cfb6a700 1 -- 172.21.15.19:0/1354344859 wait complete. 2024-02-22T16:15:34.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:34.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:34.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch upgrade status' 2024-02-22T16:15:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:15:35.273 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:35 smithi019 ceph-mon[173579]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:35.273 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:35 smithi019 ceph-mon[173579]: pgmap v670: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:35.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:35 smithi062 ceph-mon[106861]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:35.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:35 smithi062 ceph-mon[106861]: pgmap v670: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:36.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 -- 172.21.15.19:0/1552079758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec4102f50 msgr2=0x7f5ec4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 --2- 172.21.15.19:0/1552079758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec4102f50 0x7f5ec4103320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4004940 tx=0x7f5eb4009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 -- 172.21.15.19:0/1552079758 shutdown_connections 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 --2- 172.21.15.19:0/1552079758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ec41038f0 0x7f5ec40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 --2- 172.21.15.19:0/1552079758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec4102f50 0x7f5ec4103320 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 -- 172.21.15.19:0/1552079758 >> 172.21.15.19:0/1552079758 conn(0x7f5ec40f8ae0 msgr2=0x7f5ec40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 -- 172.21.15.19:0/1552079758 shutdown_connections 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.762+0000 7f5ec933f700 1 -- 172.21.15.19:0/1552079758 wait complete. 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec933f700 1 Processor -- start 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec933f700 1 -- start start 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec933f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ec4102f50 0x7f5ec4070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:36.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec933f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 0x7f5ec40707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec933f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec4070dd0 con 0x7f5ec41038f0 2024-02-22T16:15:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec933f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec4070f40 con 0x7f5ec4102f50 2024-02-22T16:15:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 0x7f5ec40707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec27fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 0x7f5ec40707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59796/0 (socket says 172.21.15.19:59796) 2024-02-22T16:15:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec27fc700 1 -- 172.21.15.19:0/2324323062 learned_addr learned my addr 172.21.15.19:0/2324323062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:15:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec2ffd700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ec4102f50 0x7f5ec4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec27fc700 1 -- 172.21.15.19:0/2324323062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ec4102f50 msgr2=0x7f5ec4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec27fc700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ec4102f50 0x7f5ec4070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.763+0000 7f5ec27fc700 1 -- 172.21.15.19:0/2324323062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb4005040 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.764+0000 7f5ec27fc700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 0x7f5ec40707d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5eac002a50 tx=0x7f5eac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.764+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5eac007cc0 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.764+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5eac00f040 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.764+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec4119bb0 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.764+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5eac00d920 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.764+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec411a090 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.765+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec4100260 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.765+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5eac009680 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.766+0000 7f5ebb7fe700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5eb006cc20 0x7f5eb006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.766+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5eac08bdc0 con 0x7f5ec41038f0 2024-02-22T16:15:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.766+0000 7f5ec2ffd700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5eb006cc20 0x7f5eb006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.766+0000 7f5ec2ffd700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5eb006cc20 0x7f5eb006f0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4004f90 tx=0x7f5eb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.770+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f5eac055d70 con 0x7f5ec41038f0 2024-02-22T16:15:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.994+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ec40719d0 con 0x7f5eb006cc20 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.995+0000 7f5ebb7fe700 1 -- 172.21.15.19:0/2324323062 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f5ec40719d0 con 0x7f5eb006cc20 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "target_image": null, 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "in_progress": false, 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "which": "", 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "services_complete": [], 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "progress": null, 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-02-22T16:15:36.969 INFO:teuthology.orchestra.run.smithi019.stdout: "is_paused": false 2024-02-22T16:15:36.970 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5eb006cc20 msgr2=0x7f5eb006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5eb006cc20 0x7f5eb006f0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4004f90 tx=0x7f5eb4009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 msgr2=0x7f5ec40707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 0x7f5ec40707d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5eac002a50 tx=0x7f5eac004600 comp rx=0 tx=0).stop 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 shutdown_connections 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f5eb006cc20 0x7f5eb006f0d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ec4102f50 0x7f5ec4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 --2- 172.21.15.19:0/2324323062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ec41038f0 0x7f5ec40707d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.997+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 >> 172.21.15.19:0/2324323062 conn(0x7f5ec40f8ae0 msgr2=0x7f5ec40fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.998+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 shutdown_connections 2024-02-22T16:15:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:36.998+0000 7f5ec933f700 1 -- 172.21.15.19:0/2324323062 wait complete. 2024-02-22T16:15:37.326 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:37 smithi019 ceph-mon[173579]: pgmap v671: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:37.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:37 smithi062 ceph-mon[106861]: pgmap v671: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:37.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph health detail' 2024-02-22T16:15:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:15:38.420 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:38 smithi019 ceph-mon[173579]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:38.420 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:38 smithi019 ceph-mon[173579]: pgmap v672: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:38.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:38 smithi062 ceph-mon[106861]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:15:38.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:38 smithi062 ceph-mon[106861]: pgmap v672: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 -- 172.21.15.19:0/4033596528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4102880 msgr2=0x7f64b40fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/4033596528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4102880 0x7f64b40fc680 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f64a8004940 tx=0x7f64a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 -- 172.21.15.19:0/4033596528 shutdown_connections 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/4033596528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4102880 0x7f64b40fc680 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/4033596528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64b4101ee0 0x7f64b41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 -- 172.21.15.19:0/4033596528 >> 172.21.15.19:0/4033596528 conn(0x7f64b4071560 msgr2=0x7f64b4071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 -- 172.21.15.19:0/4033596528 shutdown_connections 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 -- 172.21.15.19:0/4033596528 wait complete. 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 Processor -- start 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.721+0000 7f64b8dc8700 1 -- start start 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b8dc8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 0x7f64b41128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b8dc8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64b4102880 0x7f64b4112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b8dc8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b4113470 con 0x7f64b4101ee0 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b8dc8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b410c7a0 con 0x7f64b4102880 2024-02-22T16:15:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 0x7f64b41128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 0x7f64b41128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46120/0 (socket says 172.21.15.19:46120) 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 -- 172.21.15.19:0/117085553 learned_addr learned my addr 172.21.15.19:0/117085553 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b1d9b700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64b4102880 0x7f64b4112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 -- 172.21.15.19:0/117085553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64b4102880 msgr2=0x7f64b4112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64b4102880 0x7f64b4112e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 -- 172.21.15.19:0/117085553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8005040 con 0x7f64b4101ee0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b259c700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 0x7f64b41128e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c830 tx=0x7f64a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a40106f0 con 0x7f64b4101ee0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64a4010d00 con 0x7f64b4101ee0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b410ca80 con 0x7f64b4101ee0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.722+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a4012e90 con 0x7f64b4101ee0 2024-02-22T16:15:39.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.723+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b410cf90 con 0x7f64b4101ee0 2024-02-22T16:15:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.723+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b41043b0 con 0x7f64b4101ee0 2024-02-22T16:15:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.724+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f64a4010850 con 0x7f64b4101ee0 2024-02-22T16:15:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.724+0000 7f64a2ffd700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f649c06cc20 0x7f649c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.724+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f64a408bf90 con 0x7f64b4101ee0 2024-02-22T16:15:39.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.724+0000 7f64b1d9b700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f649c06cc20 0x7f649c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:39.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.725+0000 7f64b1d9b700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f649c06cc20 0x7f649c06f0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f64a800b3b0 tx=0x7f64a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:39.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:39.729+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f64a4059af0 con 0x7f64b4101ee0 2024-02-22T16:15:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.018+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f64b404bed0 con 0x7f64b4101ee0 2024-02-22T16:15:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.018+0000 7f64a2ffd700 1 -- 172.21.15.19:0/117085553 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f64a4059680 con 0x7f64b4101ee0 2024-02-22T16:15:39.992 INFO:teuthology.orchestra.run.smithi019.stdout:HEALTH_OK 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f649c06cc20 msgr2=0x7f649c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f649c06cc20 0x7f649c06f0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f64a800b3b0 tx=0x7f64a8009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 msgr2=0x7f64b41128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 0x7f64b41128e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c830 tx=0x7f64a4004470 comp rx=0 tx=0).stop 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 shutdown_connections 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f649c06cc20 0x7f649c06f0d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b4101ee0 0x7f64b41128e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 --2- 172.21.15.19:0/117085553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64b4102880 0x7f64b4112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 >> 172.21.15.19:0/117085553 conn(0x7f64b4071560 msgr2=0x7f64b40fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 shutdown_connections 2024-02-22T16:15:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:40.020+0000 7f64b8dc8700 1 -- 172.21.15.19:0/117085553 wait complete. 2024-02-22T16:15:40.415 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:40 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/117085553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:15:40.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:40 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/117085553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-22T16:15:40.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions' 2024-02-22T16:15:40.982 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:15:41.313 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:41 smithi019 ceph-mon[173579]: pgmap v673: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:41.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:41 smithi062 ceph-mon[106861]: pgmap v673: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:42.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:42 smithi019 ceph-mon[173579]: pgmap v674: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:42.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:42 smithi062 ceph-mon[106861]: pgmap v674: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 -- 172.21.15.19:0/1812080687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb4102f50 msgr2=0x7f0cb4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 --2- 172.21.15.19:0/1812080687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb4102f50 0x7f0cb4103320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0004940 tx=0x7f0cb0009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 -- 172.21.15.19:0/1812080687 shutdown_connections 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 --2- 172.21.15.19:0/1812080687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cb41038f0 0x7f0cb40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 --2- 172.21.15.19:0/1812080687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb4102f50 0x7f0cb4103320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 -- 172.21.15.19:0/1812080687 >> 172.21.15.19:0/1812080687 conn(0x7f0cb40f8ae0 msgr2=0x7f0cb40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 -- 172.21.15.19:0/1812080687 shutdown_connections 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.768+0000 7f0cbc052700 1 -- 172.21.15.19:0/1812080687 wait complete. 2024-02-22T16:15:42.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.769+0000 7f0cbc052700 1 Processor -- start 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.769+0000 7f0cbc052700 1 -- start start 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.769+0000 7f0cbc052700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cb4102f50 0x7f0cb4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.769+0000 7f0cbc052700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 0x7f0cb4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.769+0000 7f0cbc052700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4114500 con 0x7f0cb41038f0 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.769+0000 7f0cbc052700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb410d830 con 0x7f0cb4102f50 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 0x7f0cb4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 0x7f0cb4113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46128/0 (socket says 172.21.15.19:46128) 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 -- 172.21.15.19:0/2168856113 learned_addr learned my addr 172.21.15.19:0/2168856113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb9dee700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cb4102f50 0x7f0cb4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 -- 172.21.15.19:0/2168856113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cb4102f50 msgr2=0x7f0cb4113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cb4102f50 0x7f0cb4113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:42.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 -- 172.21.15.19:0/2168856113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0005040 con 0x7f0cb41038f0 2024-02-22T16:15:42.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cb95ed700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 0x7f0cb4113eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8002a50 tx=0x7f0ca8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:42.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ca8007cc0 con 0x7f0cb41038f0 2024-02-22T16:15:42.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb410db10 con 0x7f0cb41038f0 2024-02-22T16:15:42.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ca800f040 con 0x7f0cb41038f0 2024-02-22T16:15:42.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ca800d920 con 0x7f0cb41038f0 2024-02-22T16:15:42.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.770+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb410e050 con 0x7f0cb41038f0 2024-02-22T16:15:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.771+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb4100260 con 0x7f0cb41038f0 2024-02-22T16:15:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.772+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ca8009680 con 0x7f0cb41038f0 2024-02-22T16:15:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.772+0000 7f0ca67fc700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ca006cc20 0x7f0ca006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:15:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.772+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0ca808bdf0 con 0x7f0cb41038f0 2024-02-22T16:15:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.772+0000 7f0cb9dee700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ca006cc20 0x7f0ca006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:15:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.773+0000 7f0cb9dee700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ca006cc20 0x7f0ca006f0d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0004f90 tx=0x7f0cb0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:15:42.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:42.777+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0ca8059830 con 0x7f0cb41038f0 2024-02-22T16:15:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.063+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0cb404bed0 con 0x7f0cb41038f0 2024-02-22T16:15:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.064+0000 7f0ca67fc700 1 -- 172.21.15.19:0/2168856113 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f0ca80593c0 con 0x7f0cb41038f0 2024-02-22T16:15:43.037 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:15:43.037 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:15:43.037 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:15:43.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:15:43.039 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:15:43.039 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:15:43.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:15:43.039 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ca006cc20 msgr2=0x7f0ca006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ca006cc20 0x7f0ca006f0d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0004f90 tx=0x7f0cb0009e30 comp rx=0 tx=0).stop 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 msgr2=0x7f0cb4113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 0x7f0cb4113eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8002a50 tx=0x7f0ca8004600 comp rx=0 tx=0).stop 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 shutdown_connections 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f0ca006cc20 0x7f0ca006f0d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cb4102f50 0x7f0cb4113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 --2- 172.21.15.19:0/2168856113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0cb41038f0 0x7f0cb4113eb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 >> 172.21.15.19:0/2168856113 conn(0x7f0cb40f8ae0 msgr2=0x7f0cb40fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 shutdown_connections 2024-02-22T16:15:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:15:43.066+0000 7f0cbc052700 1 -- 172.21.15.19:0/2168856113 wait complete. 2024-02-22T16:15:43.412 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:43 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2168856113' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:15:43.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:43 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2168856113' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:15:43.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-22T16:15:44.059 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:15:44.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:44 smithi019 ceph-mon[173579]: pgmap v675: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:44.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:44 smithi062 ceph-mon[106861]: pgmap v675: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:45.832 INFO:teuthology.orchestra.run.smithi019.stdout:wait for servicemap items w/ changing names to refresh 2024-02-22T16:15:46.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:46 smithi019 ceph-mon[173579]: pgmap v676: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:46.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'sleep 60' 2024-02-22T16:15:46.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:46 smithi062 ceph-mon[106861]: pgmap v676: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:46.799 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:15:49.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:49 smithi062 ceph-mon[106861]: pgmap v677: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:49.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:49 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:49.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:49 smithi019 ceph-mon[173579]: pgmap v677: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:49.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:49 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:15:51.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:51 smithi019 ceph-mon[173579]: pgmap v678: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:51.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:51 smithi062 ceph-mon[106861]: pgmap v678: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:53.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:53 smithi062 ceph-mon[106861]: pgmap v679: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:53.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:53 smithi019 ceph-mon[173579]: pgmap v679: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:55.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:55 smithi062 ceph-mon[106861]: pgmap v680: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:55.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:55 smithi019 ceph-mon[173579]: pgmap v680: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:57 smithi062 ceph-mon[106861]: pgmap v681: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:57.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:57 smithi019 ceph-mon[173579]: pgmap v681: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:15:59.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:15:59 smithi062 ceph-mon[106861]: pgmap v682: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:15:59.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:15:59 smithi019 ceph-mon[173579]: pgmap v682: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:01.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:01 smithi019 ceph-mon[173579]: pgmap v683: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:01.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:01 smithi062 ceph-mon[106861]: pgmap v683: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:03.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:03 smithi062 ceph-mon[106861]: pgmap v684: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:03.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:03.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:03 smithi019 ceph-mon[173579]: pgmap v684: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:03.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:04.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:04 smithi062 ceph-mon[106861]: pgmap v685: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:04 smithi019 ceph-mon[173579]: pgmap v685: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:16:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:16:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:16:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:16:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:16:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:16:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:16:05.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:16:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:06 smithi062 ceph-mon[106861]: pgmap v686: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:06.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:06 smithi019 ceph-mon[173579]: pgmap v686: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:09.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:09 smithi062 ceph-mon[106861]: pgmap v687: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:09.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:09 smithi019 ceph-mon[173579]: pgmap v687: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:11.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:11 smithi019 ceph-mon[173579]: pgmap v688: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:11.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:11 smithi062 ceph-mon[106861]: pgmap v688: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:13.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:13 smithi062 ceph-mon[106861]: pgmap v689: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:13.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:13 smithi019 ceph-mon[173579]: pgmap v689: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:15.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:15 smithi062 ceph-mon[106861]: pgmap v690: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:15.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:15 smithi019 ceph-mon[173579]: pgmap v690: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:17.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:17 smithi062 ceph-mon[106861]: pgmap v691: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:17.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:17 smithi019 ceph-mon[173579]: pgmap v691: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:19 smithi062 ceph-mon[106861]: pgmap v692: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:19.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:19 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:19 smithi019 ceph-mon[173579]: pgmap v692: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:19.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:19 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:21.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:21 smithi019 ceph-mon[173579]: pgmap v693: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:21.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:21 smithi062 ceph-mon[106861]: pgmap v693: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:23.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:23 smithi062 ceph-mon[106861]: pgmap v694: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:23.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:23 smithi019 ceph-mon[173579]: pgmap v694: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:25.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:25 smithi062 ceph-mon[106861]: pgmap v695: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:25.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:25 smithi019 ceph-mon[173579]: pgmap v695: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:27.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:27 smithi062 ceph-mon[106861]: pgmap v696: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:27.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:27 smithi019 ceph-mon[173579]: pgmap v696: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:29.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:29 smithi062 ceph-mon[106861]: pgmap v697: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:29.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:29 smithi019 ceph-mon[173579]: pgmap v697: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:30.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:30 smithi019 ceph-mon[173579]: pgmap v698: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:30.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:30 smithi062 ceph-mon[106861]: pgmap v698: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:32.488 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:32 smithi019 ceph-mon[173579]: pgmap v699: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:32.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:32 smithi062 ceph-mon[106861]: pgmap v699: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:34.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:34 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:34.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:34 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:35.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:35 smithi062 ceph-mon[106861]: pgmap v700: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:35.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:35 smithi019 ceph-mon[173579]: pgmap v700: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:37.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:37 smithi062 ceph-mon[106861]: pgmap v701: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:37.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:37 smithi019 ceph-mon[173579]: pgmap v701: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:39.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:39 smithi062 ceph-mon[106861]: pgmap v702: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:39.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:39 smithi019 ceph-mon[173579]: pgmap v702: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:41.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:41 smithi019 ceph-mon[173579]: pgmap v703: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:41.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:41 smithi062 ceph-mon[106861]: pgmap v703: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:42.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:42 smithi019 ceph-mon[173579]: pgmap v704: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:42.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:42 smithi062 ceph-mon[106861]: pgmap v704: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:45.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:45 smithi062 ceph-mon[106861]: pgmap v705: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:45.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:45 smithi019 ceph-mon[173579]: pgmap v705: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:47.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:47 smithi062 ceph-mon[106861]: pgmap v706: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:47.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:47 smithi019 ceph-mon[173579]: pgmap v706: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:48.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:48.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:16:49.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph orch ps' 2024-02-22T16:16:49.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:49 smithi062 ceph-mon[106861]: pgmap v707: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:49.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:49 smithi019 ceph-mon[173579]: pgmap v707: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:16:50.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:50 smithi019 ceph-mon[173579]: pgmap v708: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:50.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:50 smithi062 ceph-mon[106861]: pgmap v708: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 -- 172.21.15.19:0/910179044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fdfe0 msgr2=0x7f86040fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 --2- 172.21.15.19:0/910179044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fdfe0 0x7f86040fe3b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8600004970 tx=0x7f8600009e30 comp rx=0 tx=0).stop 2024-02-22T16:16:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 -- 172.21.15.19:0/910179044 shutdown_connections 2024-02-22T16:16:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 --2- 172.21.15.19:0/910179044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86040fe8f0 0x7f8604107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 --2- 172.21.15.19:0/910179044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fdfe0 0x7f86040fe3b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 -- 172.21.15.19:0/910179044 >> 172.21.15.19:0/910179044 conn(0x7f8604071d90 msgr2=0x7f8604072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 -- 172.21.15.19:0/910179044 shutdown_connections 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.299+0000 7f860c128700 1 -- 172.21.15.19:0/910179044 wait complete. 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f860c128700 1 Processor -- start 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f860c128700 1 -- start start 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f860c128700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86040fdfe0 0x7f86040fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f860c128700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 0x7f86040fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f860c128700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040fca70 con 0x7f86040fe8f0 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f860c128700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040fcbe0 con 0x7f86040fdfe0 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f86096c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 0x7f86040fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f86096c3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 0x7f86040fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56836/0 (socket says 172.21.15.19:56836) 2024-02-22T16:16:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f86096c3700 1 -- 172.21.15.19:0/4287106283 learned_addr learned my addr 172.21.15.19:0/4287106283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f8609ec4700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86040fdfe0 0x7f86040fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f86096c3700 1 -- 172.21.15.19:0/4287106283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86040fdfe0 msgr2=0x7f86040fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f86096c3700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86040fdfe0 0x7f86040fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.300+0000 7f86096c3700 1 -- 172.21.15.19:0/4287106283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8600005040 con 0x7f86040fe8f0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.301+0000 7f86096c3700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 0x7f86040fc3e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f85f400b790 tx=0x7f85f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.301+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f400f6f0 con 0x7f86040fe8f0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.301+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85f400fd00 con 0x7f86040fe8f0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.301+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8604119c70 con 0x7f86040fe8f0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.301+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f40103a0 con 0x7f86040fe8f0 2024-02-22T16:16:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.301+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860411a160 con 0x7f86040fe8f0 2024-02-22T16:16:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.302+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86041046c0 con 0x7f86040fe8f0 2024-02-22T16:16:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.302+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f85f400f850 con 0x7f86040fe8f0 2024-02-22T16:16:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.303+0000 7f85fa7fc700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f85f006cc20 0x7f85f006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.303+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f85f408bde0 con 0x7f86040fe8f0 2024-02-22T16:16:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.303+0000 7f8609ec4700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f85f006cc20 0x7f85f006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.303+0000 7f8609ec4700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f85f006cc20 0x7f85f006f0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8600004f50 tx=0x7f8600009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:16:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.307+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f85f4055eb0 con 0x7f86040fe8f0 2024-02-22T16:16:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.519+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 --> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f86040fd5e0 con 0x7f85f006cc20 2024-02-22T16:16:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.529+0000 7f85fa7fc700 1 -- 172.21.15.19:0/4287106283 <== mgr.34104 v2:172.21.15.19:6800/1006240378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f86040fd5e0 con 0x7f85f006cc20 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:alertmanager.smithi019 smithi019 *:9093,9094 running (24m) 107s ago 35m 26.9M - 0.25.0 c8568f914cd2 794adc4d233c 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi019 smithi019 running (21m) 107s ago 35m 6647k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 dd5d76815b29 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:crash.smithi062 smithi062 running (21m) 111s ago 34m 6651k - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 c567930ebd77 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:grafana.smithi019 smithi019 *:3000 running (23m) 107s ago 34m 89.2M - 9.4.7 954c08fa6188 a067e7dc2ef5 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.fesnwx smithi019 running (17m) 107s ago 29m 118M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 03874480adf1 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi019.lkzblb smithi019 running (17m) 107s ago 29m 94.1M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f575bdbe37bd 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.eyhruo smithi062 running (16m) 111s ago 29m 17.9M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 538dee3ff278 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:mds.cephfs.smithi062.oyfagg smithi062 running (17m) 111s ago 29m 18.0M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce3412a46de1 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi019.yvjrkp smithi019 *:8443,9283,8765 running (25m) 107s ago 36m 510M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 3981494d6ccc 2024-02-22T16:16:51.503 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.smithi062.ecubmv smithi062 *:8443,9283,8765 running (25m) 111s ago 34m 456M - 18.2.1-638-gc1dc18f4 f643fdc7b9a3 f9af39f3611b 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi019 smithi019 running (22m) 107s ago 36m 71.1M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 a352b4940a74 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:mon.smithi062 smithi062 running (22m) 111s ago 34m 54.3M 2048M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 ce641474408a 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi019 smithi019 *:9100 running (24m) 107s ago 35m 18.2M - 1.5.0 0da6a335fe13 8b091a10e24c 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:node-exporter.smithi062 smithi062 *:9100 running (24m) 111s ago 34m 17.5M - 1.5.0 0da6a335fe13 7a1c607b049f 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (20m) 107s ago 33m 97.7M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 b15c807d7724 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi019 running (20m) 107s ago 32m 106M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bf25d550ab0c 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi019 running (19m) 107s ago 32m 88.6M 1570M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 6d01d37aaf6c 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi062 running (19m) 111s ago 31m 110M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 d462bbda7639 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi062 running (18m) 111s ago 31m 97.9M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 162bbdb5756d 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi062 running (18m) 111s ago 30m 96.5M 2592M 18.2.1-638-gc1dc18f4 f643fdc7b9a3 bd8ad90a0c7a 2024-02-22T16:16:51.504 INFO:teuthology.orchestra.run.smithi019.stdout:prometheus.smithi019 smithi019 *:9095 running (24m) 107s ago 34m 77.4M - 2.43.0 a07b618ecd1d 05b7968720bd 2024-02-22T16:16:51.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f85f006cc20 msgr2=0x7f85f006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:51.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f85f006cc20 0x7f85f006f0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8600004f50 tx=0x7f8600009e30 comp rx=0 tx=0).stop 2024-02-22T16:16:51.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 msgr2=0x7f86040fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:51.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 0x7f86040fc3e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f85f400b790 tx=0x7f85f4009250 comp rx=0 tx=0).stop 2024-02-22T16:16:51.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 shutdown_connections 2024-02-22T16:16:51.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f85f006cc20 0x7f85f006f0d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86040fdfe0 0x7f86040fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 --2- 172.21.15.19:0/4287106283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86040fe8f0 0x7f86040fc3e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 >> 172.21.15.19:0/4287106283 conn(0x7f8604071d90 msgr2=0x7f8604101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:16:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 shutdown_connections 2024-02-22T16:16:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:51.531+0000 7f860c128700 1 -- 172.21.15.19:0/4287106283 wait complete. 2024-02-22T16:16:52.277 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions' 2024-02-22T16:16:52.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:52 smithi019 ceph-mon[173579]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:16:52.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:52 smithi019 ceph-mon[173579]: pgmap v709: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:52.564 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:16:52.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:52 smithi062 ceph-mon[106861]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-22T16:16:52.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:52 smithi062 ceph-mon[106861]: pgmap v709: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 -- 172.21.15.19:0/2090602809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 msgr2=0x7f3ef8107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 --2- 172.21.15.19:0/2090602809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8107050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3eec004970 tx=0x7f3eec009e30 comp rx=0 tx=0).stop 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 -- 172.21.15.19:0/2090602809 shutdown_connections 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 --2- 172.21.15.19:0/2090602809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8107050 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 --2- 172.21.15.19:0/2090602809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 0x7f3ef80fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 -- 172.21.15.19:0/2090602809 >> 172.21.15.19:0/2090602809 conn(0x7f3ef8071d90 msgr2=0x7f3ef8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.622+0000 7f3effb04700 1 -- 172.21.15.19:0/2090602809 shutdown_connections 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 -- 172.21.15.19:0/2090602809 wait complete. 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 Processor -- start 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 -- start start 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 0x7f3ef8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef8114500 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.623+0000 7f3effb04700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef810d830 con 0x7f3ef80fdfe0 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56866/0 (socket says 172.21.15.19:56866) 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 -- 172.21.15.19:0/1890746487 learned_addr learned my addr 172.21.15.19:0/1890746487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd8a0700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 0x7f3ef8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 -- 172.21.15.19:0/1890746487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 msgr2=0x7f3ef8113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 0x7f3ef8113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 -- 172.21.15.19:0/1890746487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec005040 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd8a0700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 0x7f3ef8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3efd09f700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8113eb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009380 tx=0x7f3eec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:16:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eec00e040 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3eec004290 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eec01c630 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef810dab0 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.624+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef810dff0 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.625+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef81046c0 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.626+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3eec00bc10 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.626+0000 7f3eea7fc700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ee406cc20 0x7f3ee406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.626+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3eec08d3f0 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.626+0000 7f3efd8a0700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ee406cc20 0x7f3ee406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.627+0000 7f3efd8a0700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ee406cc20 0x7f3ee406f0d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009fd0 tx=0x7f3ef400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:16:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.631+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3eec05aeb0 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.916+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ef8109f10 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.917+0000 7f3eea7fc700 1 -- 172.21.15.19:0/1890746487 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f3eec05aa40 con 0x7f3ef80fe8f0 2024-02-22T16:16:54.890 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-02-22T16:16:54.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": { 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": { 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 2 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": { 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 6 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 4 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "overall": { 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)": 14 2024-02-22T16:16:54.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-02-22T16:16:54.892 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-02-22T16:16:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ee406cc20 msgr2=0x7f3ee406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ee406cc20 0x7f3ee406f0d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009fd0 tx=0x7f3ef400b040 comp rx=0 tx=0).stop 2024-02-22T16:16:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 msgr2=0x7f3ef8113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8113eb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009380 tx=0x7f3eec009e30 comp rx=0 tx=0).stop 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 shutdown_connections 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f3ee406cc20 0x7f3ee406f0d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ef80fdfe0 0x7f3ef8113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 --2- 172.21.15.19:0/1890746487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ef80fe8f0 0x7f3ef8113eb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 >> 172.21.15.19:0/1890746487 conn(0x7f3ef8071d90 msgr2=0x7f3ef8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 shutdown_connections 2024-02-22T16:16:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:54.919+0000 7f3effb04700 1 -- 172.21.15.19:0/1890746487 wait complete. 2024-02-22T16:16:55.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:55 smithi019 ceph-mon[173579]: pgmap v710: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:55.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:55 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1890746487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:16:55.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:55 smithi062 ceph-mon[106861]: pgmap v710: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:16:55.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:55 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1890746487' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:16:55.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-22T16:16:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:16:57.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:57 smithi019 ceph-mon[173579]: pgmap v711: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T16:16:57.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:57 smithi062 ceph-mon[106861]: pgmap v711: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 -- 172.21.15.19:0/884478631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fe900 msgr2=0x7f1b58106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 --2- 172.21.15.19:0/884478631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fe900 0x7f1b58106f40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c004940 tx=0x7f1b4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 -- 172.21.15.19:0/884478631 shutdown_connections 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 --2- 172.21.15.19:0/884478631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fe900 0x7f1b58106f40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 --2- 172.21.15.19:0/884478631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b580fdf60 0x7f1b580fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 -- 172.21.15.19:0/884478631 >> 172.21.15.19:0/884478631 conn(0x7f1b580f7a10 msgr2=0x7f1b580f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 -- 172.21.15.19:0/884478631 shutdown_connections 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.877+0000 7f1b5d819700 1 -- 172.21.15.19:0/884478631 wait complete. 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b5d819700 1 Processor -- start 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b5d819700 1 -- start start 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b5d819700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 0x7f1b580fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b5d819700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b580fe900 0x7f1b580fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b5d819700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580fc8d0 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b5d819700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580fca40 con 0x7f1b580fe900 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b56ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 0x7f1b580fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b56ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 0x7f1b580fbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50052/0 (socket says 172.21.15.19:50052) 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b56ffd700 1 -- 172.21.15.19:0/897100504 learned_addr learned my addr 172.21.15.19:0/897100504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b56ffd700 1 -- 172.21.15.19:0/897100504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b580fe900 msgr2=0x7f1b580fc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b56ffd700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b580fe900 0x7f1b580fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.878+0000 7f1b56ffd700 1 -- 172.21.15.19:0/897100504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c005040 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.879+0000 7f1b56ffd700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 0x7f1b580fbd90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800eb40 tx=0x7f1b4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.879+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b48007cc0 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.879+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b58119c10 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.879+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b4800f040 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.879+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b48009570 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.879+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5811a070 con 0x7f1b580fdf60 2024-02-22T16:16:57.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.880+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b58104550 con 0x7f1b580fdf60 2024-02-22T16:16:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.880+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1b480096d0 con 0x7f1b580fdf60 2024-02-22T16:16:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.881+0000 7f1b3f7fe700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1b4006cc20 0x7f1b4006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:16:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.881+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1b4808bd40 con 0x7f1b580fdf60 2024-02-22T16:16:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.881+0000 7f1b567fc700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1b4006cc20 0x7f1b4006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:16:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.881+0000 7f1b567fc700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1b4006cc20 0x7f1b4006f0d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c000c00 tx=0x7f1b4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:16:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:57.886+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1b48055da0 con 0x7f1b580fdf60 2024-02-22T16:16:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.173+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b580fd4d0 con 0x7f1b580fdf60 2024-02-22T16:16:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.173+0000 7f1b3f7fe700 1 -- 172.21.15.19:0/897100504 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f1b480593c0 con 0x7f1b580fdf60 2024-02-22T16:16:58.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1b4006cc20 msgr2=0x7f1b4006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1b4006cc20 0x7f1b4006f0d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c000c00 tx=0x7f1b4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 msgr2=0x7f1b580fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 0x7f1b580fbd90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800eb40 tx=0x7f1b4800b040 comp rx=0 tx=0).stop 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 shutdown_connections 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1b4006cc20 0x7f1b4006f0d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b580fdf60 0x7f1b580fbd90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 --2- 172.21.15.19:0/897100504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b580fe900 0x7f1b580fc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 >> 172.21.15.19:0/897100504 conn(0x7f1b580f7a10 msgr2=0x7f1b581016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 shutdown_connections 2024-02-22T16:16:58.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:16:58.175+0000 7f1b5d819700 1 -- 172.21.15.19:0/897100504 wait complete. 2024-02-22T16:16:58.160 INFO:teuthology.orchestra.run.smithi019.stdout:true 2024-02-22T16:16:58.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:58 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/897100504' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:16:58.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:58 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/897100504' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:16:58.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -e sha1=c1dc18f444603397768fcb7c34a8488b10284df6 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-22T16:16:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:16:59.446 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:16:59 smithi019 ceph-mon[173579]: pgmap v712: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T16:16:59.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:16:59 smithi062 ceph-mon[106861]: pgmap v712: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 -- 172.21.15.19:0/3408443850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 msgr2=0x7fdf900fbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/3408443850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf900fbea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdf78004940 tx=0x7fdf78009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 -- 172.21.15.19:0/3408443850 shutdown_connections 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/3408443850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf900fbea0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/3408443850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf90101e40 0x7fdf90102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 -- 172.21.15.19:0/3408443850 >> 172.21.15.19:0/3408443850 conn(0x7fdf90071d90 msgr2=0x7fdf90072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 -- 172.21.15.19:0/3408443850 shutdown_connections 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 -- 172.21.15.19:0/3408443850 wait complete. 2024-02-22T16:17:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.004+0000 7fdf95ab9700 1 Processor -- start 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf95ab9700 1 -- start start 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf95ab9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf90101e40 0x7fdf901139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf95ab9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf90113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf95ab9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90114540 con 0x7fdf901027e0 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf95ab9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9010d870 con 0x7fdf90101e40 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf90113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf90113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50066/0 (socket says 172.21.15.19:50066) 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 -- 172.21.15.19:0/2949019789 learned_addr learned my addr 172.21.15.19:0/2949019789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8f7fe700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf90101e40 0x7fdf901139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 -- 172.21.15.19:0/2949019789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf90101e40 msgr2=0x7fdf901139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf90101e40 0x7fdf901139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 -- 172.21.15.19:0/2949019789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78005040 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.005+0000 7fdf8effd700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf90113ef0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009fd0 tx=0x7fdf78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.006+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7800e040 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.006+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf78013420 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.006+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9010daf0 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.006+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7801c630 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.006+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9010e030 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.006+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf900738b0 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.007+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdf78013a00 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.008+0000 7fdf87fff700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf7c06cc20 0x7fdf7c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.008+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdf7808ddf0 con 0x7fdf901027e0 2024-02-22T16:17:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.008+0000 7fdf8f7fe700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf7c06cc20 0x7fdf7c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:00.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.008+0000 7fdf8f7fe700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf7c06cc20 0x7fdf7c06f0d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf800041d0 tx=0x7fdf800043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:00.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.012+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fdf7805b8d0 con 0x7fdf901027e0 2024-02-22T16:17:01.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.302+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf9010ef90 con 0x7fdf901027e0 2024-02-22T16:17:01.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.303+0000 7fdf87fff700 1 -- 172.21.15.19:0/2949019789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fdf7805b460 con 0x7fdf901027e0 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf7c06cc20 msgr2=0x7fdf7c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf7c06cc20 0x7fdf7c06f0d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf800041d0 tx=0x7fdf800043c0 comp rx=0 tx=0).stop 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 msgr2=0x7fdf90113ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf90113ef0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009fd0 tx=0x7fdf78009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 shutdown_connections 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf7c06cc20 0x7fdf7c06f0d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf90101e40 0x7fdf901139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:01.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 --2- 172.21.15.19:0/2949019789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf901027e0 0x7fdf90113ef0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 >> 172.21.15.19:0/2949019789 conn(0x7fdf90071d90 msgr2=0x7fdf900657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 shutdown_connections 2024-02-22T16:17:01.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:01.305+0000 7fdf95ab9700 1 -- 172.21.15.19:0/2949019789 wait complete. 2024-02-22T16:17:01.289 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph version 18.2.1-638-gc1dc18f4 (c1dc18f444603397768fcb7c34a8488b10284df6) reef (stable)" 2024-02-22T16:17:01.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:01 smithi062 ceph-mon[106861]: pgmap v713: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T16:17:01.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:01 smithi019 ceph-mon[173579]: pgmap v713: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T16:17:02.112 DEBUG:teuthology.parallel:result is None 2024-02-22T16:17:02.112 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-22T16:17:02.124 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T16:17:02.124 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-22T16:17:02.412 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/mon.smithi019/config 2024-02-22T16:17:02.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:02 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2949019789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:17:02.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:02 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2949019789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-22T16:17:03.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:03 smithi062 ceph-mon[106861]: pgmap v714: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T16:17:03.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:03.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:03 smithi019 ceph-mon[173579]: pgmap v714: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T16:17:03.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 -- 172.21.15.19:0/2543151951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 msgr2=0x7fb874102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 --2- 172.21.15.19:0/2543151951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb874102210 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb870004940 tx=0x7fb870009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 -- 172.21.15.19:0/2543151951 shutdown_connections 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 --2- 172.21.15.19:0/2543151951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 0x7fb8740fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 --2- 172.21.15.19:0/2543151951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb874102210 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 -- 172.21.15.19:0/2543151951 >> 172.21.15.19:0/2543151951 conn(0x7fb874071d90 msgr2=0x7fb874072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 -- 172.21.15.19:0/2543151951 shutdown_connections 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.313+0000 7fb87bb43700 1 -- 172.21.15.19:0/2543151951 wait complete. 2024-02-22T16:17:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb87bb43700 1 Processor -- start 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb87bb43700 1 -- start start 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb87bb43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb8741139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb87bb43700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 0x7fb874113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb87bb43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb874114530 con 0x7fb874101e40 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb87bb43700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87410d860 con 0x7fb8741027e0 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb8741139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb8741139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50086/0 (socket says 172.21.15.19:50086) 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 -- 172.21.15.19:0/1605905128 learned_addr learned my addr 172.21.15.19:0/1605905128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 -- 172.21.15.19:0/1605905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 msgr2=0x7fb874113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8790de700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 0x7fb874113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 0x7fb874113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 -- 172.21.15.19:0/1605905128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870005040 con 0x7fb874101e40 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8790de700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 0x7fb874113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.314+0000 7fb8798df700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb8741139a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b3b0 tx=0x7fb870009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.315+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87000e040 con 0x7fb874101e40 2024-02-22T16:17:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.315+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87410dae0 con 0x7fb874101e40 2024-02-22T16:17:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.315+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb87000bc50 con 0x7fb874101e40 2024-02-22T16:17:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.315+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb870013420 con 0x7fb874101e40 2024-02-22T16:17:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.315+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87410e020 con 0x7fb874101e40 2024-02-22T16:17:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.316+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8740738b0 con 0x7fb874101e40 2024-02-22T16:17:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.316+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb87001c430 con 0x7fb874101e40 2024-02-22T16:17:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.316+0000 7fb86a7fc700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb86006cc20 0x7fb86006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.317+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb87008ce70 con 0x7fb874101e40 2024-02-22T16:17:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.317+0000 7fb8790de700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb86006cc20 0x7fb86006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.317+0000 7fb8790de700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb86006cc20 0x7fb86006f0d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb864007fd0 tx=0x7fb864009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.321+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb87005a9e0 con 0x7fb874101e40 2024-02-22T16:17:04.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:04 smithi062 ceph-mon[106861]: pgmap v715: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T16:17:04.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:04 smithi019 ceph-mon[173579]: pgmap v715: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-02-22T16:17:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.569+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb87410efb0 con 0x7fb874101e40 2024-02-22T16:17:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.569+0000 7fb86a7fc700 1 -- 172.21.15.19:0/1605905128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 43 v43) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fb87005a570 con 0x7fb874101e40 2024-02-22T16:17:04.543 INFO:teuthology.orchestra.run.smithi019.stdout:e43 2024-02-22T16:17:04.543 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-22T16:17:04.543 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:17:04.543 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-02-22T16:17:04.544 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:04.544 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-02-22T16:17:04.544 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-02-22T16:17:04.544 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 43 2024-02-22T16:17:04.544 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-22T16:17:04.544 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-02-22T15:47:22.185528+0000 2024-02-22T16:17:04.545 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-02-22T16:15:08.151484+0000 2024-02-22T16:17:04.545 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-02-22T16:17:04.545 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-02-22T16:17:04.545 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 60 2024-02-22T16:17:04.545 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 82 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-22T16:17:04.546 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 2 2024-02-22T16:17:04.547 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1 2024-02-22T16:17:04.547 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=34392,1=34424} 2024-02-22T16:17:04.547 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-02-22T16:17:04.547 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-02-22T16:17:04.547 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-02-22T16:17:04.547 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-02-22T16:17:04.548 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-02-22T16:17:04.548 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-02-22T16:17:04.548 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-02-22T16:17:04.548 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-02-22T16:17:04.548 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 1 2024-02-22T16:17:04.548 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.lkzblb{0:34392} state up:active seq 245 join_fscid=1 addr [v2:172.21.15.19:6826/2225716389,v1:172.21.15.19:6827/2225716389] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:17:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.oyfagg{0:44333} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.62:6824/2401270415,v1:172.21.15.62:6825/2401270415] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:17:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi019.fesnwx{1:34424} state up:active seq 239 join_fscid=1 addr [v2:172.21.15.19:6828/3128614024,v1:172.21.15.19:6829/3128614024] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:17:04.549 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.cephfs.smithi062.eyhruo{1:44365} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.62:6826/1657799059,v1:172.21.15.62:6827/1657799059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-22T16:17:04.549 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:04.549 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb86006cc20 msgr2=0x7fb86006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb86006cc20 0x7fb86006f0d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb864007fd0 tx=0x7fb864009040 comp rx=0 tx=0).stop 2024-02-22T16:17:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 msgr2=0x7fb8741139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb8741139a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b3b0 tx=0x7fb870009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 shutdown_connections 2024-02-22T16:17:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb86006cc20 0x7fb86006f0d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874101e40 0x7fb8741139a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 --2- 172.21.15.19:0/1605905128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8741027e0 0x7fb874113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 >> 172.21.15.19:0/1605905128 conn(0x7fb874071d90 msgr2=0x7fb8740657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 shutdown_connections 2024-02-22T16:17:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:04.571+0000 7fb87bb43700 1 -- 172.21.15.19:0/1605905128 wait complete. 2024-02-22T16:17:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-02-22T16:17:05.349 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-22T16:17:05.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-22T16:17:05.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:05 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1605905128' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:17:05.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:05 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:17:05.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:05 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1605905128' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-22T16:17:05.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:05 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 -- 172.21.15.19:0/2527676543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 msgr2=0x7fc728106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 --2- 172.21.15.19:0/2527676543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728106fd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc718004940 tx=0x7fc718009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 -- 172.21.15.19:0/2527676543 shutdown_connections 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 --2- 172.21.15.19:0/2527676543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728106fd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 --2- 172.21.15.19:0/2527676543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7280fdff0 0x7fc7280fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 -- 172.21.15.19:0/2527676543 >> 172.21.15.19:0/2527676543 conn(0x7fc7280f7a40 msgr2=0x7fc7280f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 -- 172.21.15.19:0/2527676543 shutdown_connections 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.519+0000 7fc72faf8700 1 -- 172.21.15.19:0/2527676543 wait complete. 2024-02-22T16:17:05.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72faf8700 1 Processor -- start 2024-02-22T16:17:05.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72faf8700 1 -- start start 2024-02-22T16:17:05.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72faf8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7280fdff0 0x7fc728113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:05.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72faf8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728113d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:05.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72faf8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281143c0 con 0x7fc7280fe990 2024-02-22T16:17:05.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72faf8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72810d900 con 0x7fc7280fdff0 2024-02-22T16:17:05.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.520+0000 7fc72d093700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728113d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d093700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728113d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50100/0 (socket says 172.21.15.19:50100) 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d093700 1 -- 172.21.15.19:0/3031230708 learned_addr learned my addr 172.21.15.19:0/3031230708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d894700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7280fdff0 0x7fc728113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d093700 1 -- 172.21.15.19:0/3031230708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7280fdff0 msgr2=0x7fc728113830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d093700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7280fdff0 0x7fc728113830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d093700 1 -- 172.21.15.19:0/3031230708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718005040 con 0x7fc7280fe990 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72d093700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728113d70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc718009390 tx=0x7fc718009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71800e040 con 0x7fc7280fe990 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc718004290 con 0x7fc7280fe990 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72810db80 con 0x7fc7280fe990 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71801c630 con 0x7fc7280fe990 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.521+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72810e0c0 con 0x7fc7280fe990 2024-02-22T16:17:05.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.522+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7281045e0 con 0x7fc7280fe990 2024-02-22T16:17:05.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.523+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc71801c8f0 con 0x7fc7280fe990 2024-02-22T16:17:05.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.523+0000 7fc71e7fc700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc71406cc30 0x7fc71406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:05.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.523+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc71808d400 con 0x7fc7280fe990 2024-02-22T16:17:05.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.523+0000 7fc72d894700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc71406cc30 0x7fc71406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:05.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.524+0000 7fc72d894700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc71406cc30 0x7fc71406f0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc724004990 tx=0x7fc724009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:05.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.528+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc71805af30 con 0x7fc7280fe990 2024-02-22T16:17:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.735+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc728109f40 con 0x7fc7280fe990 2024-02-22T16:17:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.737+0000 7fc71e7fc700 1 -- 172.21.15.19:0/3031230708 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16490 (secure 0 0 0) 0x7fc71805aac0 con 0x7fc7280fe990 2024-02-22T16:17:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:05.710 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:59:50.275353+0000","last_up_change":"2024-02-22T15:58:51.952345+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-22T15:47:20.868526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-22T15:47:21.608692+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6803","nonce":2998470349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6805","nonce":2998470349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6809","nonce":2998470349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6807","nonce":2998470349}]},"public_addr":"172.21.15.19:6803/2998470349","cluster_addr":"172.21.15.19:6805/2998470349","heartbeat_back_addr":"172.21.15.19:6809/2998470349","heartbeat_front_addr":"172.21.15.19:6807/2998470349","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6811","nonce":1876741852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6813","nonce":1876741852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6817","nonce":1876741852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6815","nonce":1876741852}]},"public_addr":"172.21.15.19:6811/1876741852","cluster_addr":"172.21.15.19:6813/1876741852","heartbeat_back_addr":"172.21.15.19:6817/1876741852","heartbeat_front_addr":"172.21.15.19:6815/1876741852","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6819","nonce":947220918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6821","nonce":947220918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6825","nonce":947220918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6823","nonce":947220918}]},"public_addr":"172.21.15.19:6819/947220918","cluster_addr":"172.21.15.19:6821/947220918","heartbeat_back_addr":"172.21.15.19:6825/947220918","heartbeat_front_addr":"172.21.15.19:6823/947220918","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6801","nonce":1420983624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6803","nonce":1420983624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6807","nonce":1420983624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6805","nonce":1420983624}]},"public_addr":"172.21.15.62:6801/1420983624","cluster_addr":"172.21.15.62:6803/1420983624","heartbeat_back_addr":"172.21.15.62:6807/1420983624","heartbeat_front_addr":"172.21.15.62:6805/1420983624","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6809","nonce":227974398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6811","nonce":227974398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6815","nonce":227974398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6813","nonce":227974398}]},"public_addr":"172.21.15.62:6809/227974398","cluster_addr":"172.21.15.62:6811/227974398","heartbeat_back_addr":"172.21.15.62:6815/227974398","heartbeat_front_addr":"172.21.15.62:6813/227974398","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6817","nonce":823340437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6819","nonce":823340437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6823","nonce":823340437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6821","nonce":823340437}]},"public_addr":"172.21.15.62:6817/823340437","cluster_addr":"172.21.15.62:6819/823340437","heartbeat_back_addr":"172.21.15.62:6823/823340437","heartbeat_front_addr":"172.21.15.62:6821/823340437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6829/3100392983":"2024-02-23T15:58:54.154098+0000","172.21.15.19:6828/3100392983":"2024-02-23T15:58:54.154098+0000","172.21.15.62:0/3742623644":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6800/236956964":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6800/3932892700":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.62:0/2157627582":"2024-02-23T15:51:10.906421+0000","172.21.15.62:6825/1888029196":"2024-02-23T15:58:54.154118+0000","172.21.15.19:6801/3932892700":"2024-02-23T15:54:33.072657+0000","172.21.15.19:6827/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.19:6801/236956964":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6826/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.62:6829/2665490935":"2024-02-23T15:51:10.906421+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/576195047":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/163225331":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.62:0/1007979954":"2024-02-23T15:51:10.906421+0000","172.21.15.62:0/1428056285":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/2452913904":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1542879988":"2024-02-23T15:49:33.669856+0000","172.21.15.62:6828/2665490935":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/1375214472":"2024-02-23T15:49:33.669856+0000","172.21.15.19:0/3432676624":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/1503676495":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/809216990":"2024-02-23T15:54:33.072657+0000","172.21.15.62:6824/1888029196":"2024-02-23T15:58:54.154118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T16:17:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc71406cc30 msgr2=0x7fc71406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc71406cc30 0x7fc71406f0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc724004990 tx=0x7fc724009ee0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 msgr2=0x7fc728113d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728113d70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc718009390 tx=0x7fc718009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 shutdown_connections 2024-02-22T16:17:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc71406cc30 0x7fc71406f0e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7280fdff0 0x7fc728113830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 --2- 172.21.15.19:0/3031230708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7280fe990 0x7fc728113d70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 >> 172.21.15.19:0/3031230708 conn(0x7fc7280f7a40 msgr2=0x7fc728101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 shutdown_connections 2024-02-22T16:17:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.739+0000 7fc72faf8700 1 -- 172.21.15.19:0/3031230708 wait complete. 2024-02-22T16:17:05.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-22T15:44:43.417715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-22T15:47:20.868526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-22T15:47:21.608692+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-22T16:17:05.725 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-22T16:17:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.877+0000 7f7e6ddca700 1 -- 172.21.15.19:0/4007994064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 msgr2=0x7f7e680fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.877+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/4007994064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e680fe2c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7e58004940 tx=0x7f7e58009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 -- 172.21.15.19:0/4007994064 shutdown_connections 2024-02-22T16:17:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/4007994064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 0x7f7e68106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/4007994064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e680fe2c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 -- 172.21.15.19:0/4007994064 >> 172.21.15.19:0/4007994064 conn(0x7f7e680f7940 msgr2=0x7f7e680f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 -- 172.21.15.19:0/4007994064 shutdown_connections 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 -- 172.21.15.19:0/4007994064 wait complete. 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 Processor -- start 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 -- start start 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e68113590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.878+0000 7f7e6ddca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 0x7f7e68113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e6ddca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68114120 con 0x7f7e680fdef0 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e6ddca700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6810d660 con 0x7f7e680fe890 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e68113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e68113590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50114/0 (socket says 172.21.15.19:50114) 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 -- 172.21.15.19:0/1707815090 learned_addr learned my addr 172.21.15.19:0/1707815090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e66ffd700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 0x7f7e68113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 -- 172.21.15.19:0/1707815090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 msgr2=0x7f7e68113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 0x7f7e68113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 -- 172.21.15.19:0/1707815090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c005040 con 0x7f7e680fdef0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e66ffd700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 0x7f7e68113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e677fe700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e68113590 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c00 tx=0x7f7e58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e58021070 con 0x7f7e680fdef0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e5801c400 con 0x7f7e680fdef0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e58005040 con 0x7f7e680fdef0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e5800e070 con 0x7f7e680fdef0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.879+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6810dc90 con 0x7f7e680fdef0 2024-02-22T16:17:05.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.880+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e681044e0 con 0x7f7e680fdef0 2024-02-22T16:17:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.881+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7e5801c9c0 con 0x7f7e680fdef0 2024-02-22T16:17:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.881+0000 7f7e4ffff700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e5006cc20 0x7f7e5006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.881+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7e5808d170 con 0x7f7e680fdef0 2024-02-22T16:17:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.881+0000 7f7e66ffd700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e5006cc20 0x7f7e5006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:05.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.882+0000 7f7e66ffd700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e5006cc20 0x7f7e5006f0d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c007fd0 tx=0x7f7e5c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:05.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:05.886+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7e5805acd0 con 0x7f7e680fdef0 2024-02-22T16:17:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.087+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e6810d820 con 0x7f7e680fdef0 2024-02-22T16:17:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.088+0000 7f7e4ffff700 1 -- 172.21.15.19:0/1707815090 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e6810d820 con 0x7f7e680fdef0 2024-02-22T16:17:06.061 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-02-22T16:17:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e5006cc20 msgr2=0x7f7e5006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e5006cc20 0x7f7e5006f0d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c007fd0 tx=0x7f7e5c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 msgr2=0x7f7e68113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e68113590 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c00 tx=0x7f7e58009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 shutdown_connections 2024-02-22T16:17:06.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7e5006cc20 0x7f7e5006f0d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e680fdef0 0x7f7e68113590 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 --2- 172.21.15.19:0/1707815090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e680fe890 0x7f7e68113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 >> 172.21.15.19:0/1707815090 conn(0x7f7e680f7940 msgr2=0x7f7e68101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:06.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 shutdown_connections 2024-02-22T16:17:06.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.090+0000 7f7e6ddca700 1 -- 172.21.15.19:0/1707815090 wait complete. 2024-02-22T16:17:06.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 -- 172.21.15.19:0/3931434302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 msgr2=0x7fda44102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 --2- 172.21.15.19:0/3931434302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda44102370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fda34004940 tx=0x7fda34009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 -- 172.21.15.19:0/3931434302 shutdown_connections 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 --2- 172.21.15.19:0/3931434302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda44102940 0x7fda440fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 --2- 172.21.15.19:0/3931434302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda44102370 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 -- 172.21.15.19:0/3931434302 >> 172.21.15.19:0/3931434302 conn(0x7fda440f7b10 msgr2=0x7fda440f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 -- 172.21.15.19:0/3931434302 shutdown_connections 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 -- 172.21.15.19:0/3931434302 wait complete. 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 Processor -- start 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.228+0000 7fda49879700 1 -- start start 2024-02-22T16:17:06.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda49879700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda4410e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda49879700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda44102940 0x7fda4410ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda49879700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4410f380 con 0x7fda44101fa0 2024-02-22T16:17:06.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda49879700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4410f4c0 con 0x7fda44102940 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda4410e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda4410e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50130/0 (socket says 172.21.15.19:50130) 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 -- 172.21.15.19:0/396209872 learned_addr learned my addr 172.21.15.19:0/396209872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda427fc700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda44102940 0x7fda4410ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 -- 172.21.15.19:0/396209872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda44102940 msgr2=0x7fda4410ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda44102940 0x7fda4410ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 -- 172.21.15.19:0/396209872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34005040 con 0x7fda44101fa0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda42ffd700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda4410e900 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b3b0 tx=0x7fda34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda3400e040 con 0x7fda44101fa0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda3400bc50 con 0x7fda44101fa0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4410f740 con 0x7fda44101fa0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda34013420 con 0x7fda44101fa0 2024-02-22T16:17:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.229+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda44117d70 con 0x7fda44101fa0 2024-02-22T16:17:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.230+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44064ed0 con 0x7fda44101fa0 2024-02-22T16:17:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.231+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fda3401c430 con 0x7fda44101fa0 2024-02-22T16:17:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.231+0000 7fda2b7fe700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fda2c06cc20 0x7fda2c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.231+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fda3408cd40 con 0x7fda44101fa0 2024-02-22T16:17:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.231+0000 7fda427fc700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fda2c06cc20 0x7fda2c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.232+0000 7fda427fc700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fda2c06cc20 0x7fda2c06f0d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda38009fd0 tx=0x7fda38009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:06.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.235+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fda34056ec0 con 0x7fda44101fa0 2024-02-22T16:17:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.437+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fda44108f60 con 0x7fda44101fa0 2024-02-22T16:17:06.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.437+0000 7fda2b7fe700 1 -- 172.21.15.19:0/396209872 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fda3405a4e0 con 0x7fda44101fa0 2024-02-22T16:17:06.410 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T16:17:06.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fda2c06cc20 msgr2=0x7fda2c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fda2c06cc20 0x7fda2c06f0d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda38009fd0 tx=0x7fda38009f60 comp rx=0 tx=0).stop 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 msgr2=0x7fda4410e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda4410e900 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b3b0 tx=0x7fda34009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 shutdown_connections 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fda2c06cc20 0x7fda2c06f0d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda44101fa0 0x7fda4410e900 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 --2- 172.21.15.19:0/396209872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda44102940 0x7fda4410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 >> 172.21.15.19:0/396209872 conn(0x7fda440f7b10 msgr2=0x7fda440f9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 shutdown_connections 2024-02-22T16:17:06.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.439+0000 7fda49879700 1 -- 172.21.15.19:0/396209872 wait complete. 2024-02-22T16:17:06.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:17:06.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:17:06.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:17:06.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:06 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3031230708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T16:17:06.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:06 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1707815090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T16:17:06.424 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:06 smithi019 ceph-mon[173579]: pgmap v716: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T16:17:06.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-22T16:17:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:17:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:17:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:17:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:06 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3031230708' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T16:17:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:06 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1707815090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T16:17:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:06 smithi062 ceph-mon[106861]: pgmap v716: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-02-22T16:17:06.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 -- 172.21.15.19:0/668401493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 msgr2=0x7f2cf4103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 --2- 172.21.15.19:0/668401493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4103250 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8004940 tx=0x7f2ce8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 -- 172.21.15.19:0/668401493 shutdown_connections 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 --2- 172.21.15.19:0/668401493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf4103820 0x7f2cf40fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 --2- 172.21.15.19:0/668401493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4103250 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 -- 172.21.15.19:0/668401493 >> 172.21.15.19:0/668401493 conn(0x7f2cf40f89d0 msgr2=0x7f2cf40fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 -- 172.21.15.19:0/668401493 shutdown_connections 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 -- 172.21.15.19:0/668401493 wait complete. 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 Processor -- start 2024-02-22T16:17:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.577+0000 7f2cfb028700 1 -- start start 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cfb028700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cfb028700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf4103820 0x7f2cf40707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cfb028700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4070d10 con 0x7f2cf4102e80 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cfb028700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4070e50 con 0x7f2cf4103820 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50148/0 (socket says 172.21.15.19:50148) 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 -- 172.21.15.19:0/2389879217 learned_addr learned my addr 172.21.15.19:0/2389879217 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 -- 172.21.15.19:0/2389879217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf4103820 msgr2=0x7f2cf40707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf3fff700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf4103820 0x7f2cf40707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf4103820 0x7f2cf40707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 -- 172.21.15.19:0/2389879217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce8005040 con 0x7f2cf4102e80 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.578+0000 7f2cf8dc4700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4070290 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8008d80 tx=0x7f2ce8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.579+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce800e030 con 0x7f2cf4102e80 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.579+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ce8004290 con 0x7f2cf4102e80 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.579+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf40710d0 con 0x7f2cf4102e80 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.579+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ce801ce60 con 0x7f2cf4102e80 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.579+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf4119d20 con 0x7f2cf4102e80 2024-02-22T16:17:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.580+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf40641f0 con 0x7f2cf4102e80 2024-02-22T16:17:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.580+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2ce801c690 con 0x7f2cf4102e80 2024-02-22T16:17:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.581+0000 7f2cf17fa700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce406cc30 0x7f2ce406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.581+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2ce808d900 con 0x7f2cf4102e80 2024-02-22T16:17:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.581+0000 7f2cf3fff700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce406cc30 0x7f2ce406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.581+0000 7f2cf3fff700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce406cc30 0x7f2ce406f0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004600 tx=0x7f2ce000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.586+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2ce80579d0 con 0x7f2cf4102e80 2024-02-22T16:17:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.787+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2cf4109e40 con 0x7f2cf4102e80 2024-02-22T16:17:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.787+0000 7f2cf17fa700 1 -- 172.21.15.19:0/2389879217 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2ce805aff0 con 0x7f2cf4102e80 2024-02-22T16:17:06.760 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T16:17:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce406cc30 msgr2=0x7f2ce406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce406cc30 0x7f2ce406f0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004600 tx=0x7f2ce000b040 comp rx=0 tx=0).stop 2024-02-22T16:17:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 msgr2=0x7f2cf4070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4070290 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8008d80 tx=0x7f2ce8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 shutdown_connections 2024-02-22T16:17:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2ce406cc30 0x7f2ce406f0e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cf4102e80 0x7f2cf4070290 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 --2- 172.21.15.19:0/2389879217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cf4103820 0x7f2cf40707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 >> 172.21.15.19:0/2389879217 conn(0x7f2cf40f89d0 msgr2=0x7f2cf40fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 shutdown_connections 2024-02-22T16:17:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.789+0000 7f2cfb028700 1 -- 172.21.15.19:0/2389879217 wait complete. 2024-02-22T16:17:06.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-22T16:17:06.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 -- 172.21.15.19:0/3162918760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498102dc0 msgr2=0x7f3498103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 --2- 172.21.15.19:0/3162918760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498102dc0 0x7f3498103190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f348c004940 tx=0x7f348c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 -- 172.21.15.19:0/3162918760 shutdown_connections 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 --2- 172.21.15.19:0/3162918760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498103760 0x7f34980fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 --2- 172.21.15.19:0/3162918760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498102dc0 0x7f3498103190 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 -- 172.21.15.19:0/3162918760 >> 172.21.15.19:0/3162918760 conn(0x7f34980f8930 msgr2=0x7f34980fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 -- 172.21.15.19:0/3162918760 shutdown_connections 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 -- 172.21.15.19:0/3162918760 wait complete. 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 Processor -- start 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.925+0000 7f349cf20700 1 -- start start 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349cf20700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498102dc0 0x7f34981135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349cf20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 0x7f3498113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349cf20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34981141f0 con 0x7f3498103760 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349cf20700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349810d6a0 con 0x7f3498102dc0 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 0x7f3498113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 0x7f3498113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50162/0 (socket says 172.21.15.19:50162) 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 -- 172.21.15.19:0/414213188 learned_addr learned my addr 172.21.15.19:0/414213188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349659c700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498102dc0 0x7f34981135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 -- 172.21.15.19:0/414213188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498102dc0 msgr2=0x7f34981135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498102dc0 0x7f34981135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 -- 172.21.15.19:0/414213188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c005040 con 0x7f3498103760 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349659c700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498102dc0 0x7f34981135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3495d9b700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 0x7f3498113b10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f348800b790 tx=0x7f3488009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348800f6f0 con 0x7f3498103760 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349810d980 con 0x7f3498103760 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f348800fd00 con 0x7f3498103760 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34880103a0 con 0x7f3498103760 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.926+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349810df40 con 0x7f3498103760 2024-02-22T16:17:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.927+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34981000d0 con 0x7f3498103760 2024-02-22T16:17:06.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.928+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f348800f850 con 0x7f3498103760 2024-02-22T16:17:06.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.928+0000 7f3486ffd700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f348006cc20 0x7f348006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.928+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f348808bf00 con 0x7f3498103760 2024-02-22T16:17:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.929+0000 7f349659c700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f348006cc20 0x7f348006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.929+0000 7f349659c700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f348006cc20 0x7f348006f0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f348c004560 tx=0x7f348c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:06.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:06.933+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f3488059940 con 0x7f3498103760 2024-02-22T16:17:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.184+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34980641f0 con 0x7f3498103760 2024-02-22T16:17:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.184+0000 7f3486ffd700 1 -- 172.21.15.19:0/414213188 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4861 (secure 0 0 0) 0x7f34880594d0 con 0x7f3498103760 2024-02-22T16:17:07.157 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:07.158 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:15:08.151484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:active","state_seq":245,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:active","state_seq":239,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44333":{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44365":{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:07.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f348006cc20 msgr2=0x7f348006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f348006cc20 0x7f348006f0d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f348c004560 tx=0x7f348c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:07.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 msgr2=0x7f3498113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 0x7f3498113b10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f348800b790 tx=0x7f3488009220 comp rx=0 tx=0).stop 2024-02-22T16:17:07.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 shutdown_connections 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f348006cc20 0x7f348006f0d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3498102dc0 0x7f34981135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 --2- 172.21.15.19:0/414213188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3498103760 0x7f3498113b10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 >> 172.21.15.19:0/414213188 conn(0x7f34980f8930 msgr2=0x7f34980fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 shutdown_connections 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.187+0000 7f349cf20700 1 -- 172.21.15.19:0/414213188 wait complete. 2024-02-22T16:17:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 43 2024-02-22T16:17:07.172 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 20, 'max_mds': 2, 'flags': 50} 2024-02-22T16:17:07.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 -- 172.21.15.19:0/484507339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 msgr2=0x7fbd58106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/484507339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd58106ed0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c004940 tx=0x7fbd4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 -- 172.21.15.19:0/484507339 shutdown_connections 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/484507339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd58106ed0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/484507339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd580fdef0 0x7fbd580fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 -- 172.21.15.19:0/484507339 >> 172.21.15.19:0/484507339 conn(0x7fbd580f7940 msgr2=0x7fbd580f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:07.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.322+0000 7fbd5e49f700 1 -- 172.21.15.19:0/484507339 shutdown_connections 2024-02-22T16:17:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 -- 172.21.15.19:0/484507339 wait complete. 2024-02-22T16:17:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 Processor -- start 2024-02-22T16:17:07.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 -- start start 2024-02-22T16:17:07.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd580fdef0 0x7fbd580fbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd580fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580fc7a0 con 0x7fbd580fe890 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.323+0000 7fbd5e49f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580fc8e0 con 0x7fbd580fdef0 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd580fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd580fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50176/0 (socket says 172.21.15.19:50176) 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 -- 172.21.15.19:0/2556590649 learned_addr learned my addr 172.21.15.19:0/2556590649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd57fff700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd580fdef0 0x7fbd580fbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 -- 172.21.15.19:0/2556590649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd580fdef0 msgr2=0x7fbd580fbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd580fdef0 0x7fbd580fbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 -- 172.21.15.19:0/2556590649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c005040 con 0x7fbd580fe890 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd577fe700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd580fc260 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009390 tx=0x7fbd4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4c00e040 con 0x7fbd580fe890 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58119a70 con 0x7fbd580fe890 2024-02-22T16:17:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd4c013420 con 0x7fbd580fe890 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4c01c630 con 0x7fbd580fe890 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.324+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58119dc0 con 0x7fbd580fe890 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.325+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd581044e0 con 0x7fbd580fe890 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.326+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbd4c013590 con 0x7fbd580fe890 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.326+0000 7fbd54ff9700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbd4006cc30 0x7fbd4006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.326+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbd4c08d4b0 con 0x7fbd580fe890 2024-02-22T16:17:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.326+0000 7fbd57fff700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbd4006cc30 0x7fbd4006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:07.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.327+0000 7fbd57fff700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbd4006cc30 0x7fbd4006f0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004f40 tx=0x7fbd4800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.331+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fbd4c0574e0 con 0x7fbd580fe890 2024-02-22T16:17:07.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:07 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/396209872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T16:17:07.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:07 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2389879217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T16:17:07.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:07 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/414213188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-22T16:17:07.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:07 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/396209872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T16:17:07.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:07 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2389879217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T16:17:07.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:07 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/414213188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-22T16:17:07.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.580+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fbd580641f0 con 0x7fbd580fe890 2024-02-22T16:17:07.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.581+0000 7fbd54ff9700 1 -- 172.21.15.19:0/2556590649 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v43) v1 ==== 107+0+4854 (secure 0 0 0) 0x7fbd4c013840 con 0x7fbd580fe890 2024-02-22T16:17:07.554 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:07.554 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:49:38.056116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi019.fesnwx","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6829/3100392983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3100392983},{"type":"v1","addr":"172.21.15.19:6829","nonce":3100392983}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6827/588513461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":588513461},{"type":"v1","addr":"172.21.15.19:6827","nonce":588513461}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi062.oyfagg","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6825/1888029196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1888029196},{"type":"v1","addr":"172.21.15.62:6825","nonce":1888029196}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi062.eyhruo","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.62:6827/3228927024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3228927024},{"type":"v1","addr":"172.21.15.62:6827","nonce":3228927024}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbd4006cc30 msgr2=0x7fbd4006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbd4006cc30 0x7fbd4006f0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004f40 tx=0x7fbd4800b040 comp rx=0 tx=0).stop 2024-02-22T16:17:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 msgr2=0x7fbd580fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd580fc260 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009390 tx=0x7fbd4c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 shutdown_connections 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fbd4006cc30 0x7fbd4006f0e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd580fdef0 0x7fbd580fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 --2- 172.21.15.19:0/2556590649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd580fe890 0x7fbd580fc260 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 >> 172.21.15.19:0/2556590649 conn(0x7fbd580f7940 msgr2=0x7fbd58101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 shutdown_connections 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.583+0000 7fbd5e49f700 1 -- 172.21.15.19:0/2556590649 wait complete. 2024-02-22T16:17:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 21 2024-02-22T16:17:07.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-22T16:17:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.719+0000 7fae1dc3e700 1 -- 172.21.15.19:0/3215628203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fde90 msgr2=0x7fae180fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.719+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/3215628203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fde90 0x7fae180fe260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fae08004940 tx=0x7fae08009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.719+0000 7fae1dc3e700 1 -- 172.21.15.19:0/3215628203 shutdown_connections 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.719+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/3215628203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae180fe830 0x7fae18106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.719+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/3215628203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fde90 0x7fae180fe260 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.719+0000 7fae1dc3e700 1 -- 172.21.15.19:0/3215628203 >> 172.21.15.19:0/3215628203 conn(0x7fae180f7950 msgr2=0x7fae180f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 -- 172.21.15.19:0/3215628203 shutdown_connections 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 -- 172.21.15.19:0/3215628203 wait complete. 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 Processor -- start 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 -- start start 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae180fde90 0x7fae18113610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 0x7fae18113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181141e0 con 0x7fae180fe830 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.720+0000 7fae1dc3e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1810d750 con 0x7fae180fde90 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 0x7fae18113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 0x7fae18113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45964/0 (socket says 172.21.15.19:45964) 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 -- 172.21.15.19:0/2357965957 learned_addr learned my addr 172.21.15.19:0/2357965957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae177fe700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae180fde90 0x7fae18113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 -- 172.21.15.19:0/2357965957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae180fde90 msgr2=0x7fae18113610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae180fde90 0x7fae18113610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 -- 172.21.15.19:0/2357965957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08005040 con 0x7fae180fe830 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae16ffd700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 0x7fae18113b90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fae0c002a50 tx=0x7fae0c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae0c007cc0 con 0x7fae180fe830 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1810d9e0 con 0x7fae180fe830 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1810df20 con 0x7fae180fe830 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae0c00f040 con 0x7fae180fe830 2024-02-22T16:17:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.721+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae0c00d8d0 con 0x7fae180fe830 2024-02-22T16:17:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.722+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae040052e0 con 0x7fae180fe830 2024-02-22T16:17:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.723+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fae0c009070 con 0x7fae180fe830 2024-02-22T16:17:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.723+0000 7fadfffff700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae0006cc20 0x7fae0006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.723+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fae0c08bcc0 con 0x7fae180fe830 2024-02-22T16:17:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.723+0000 7fae177fe700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae0006cc20 0x7fae0006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:07.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.724+0000 7fae177fe700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae0006cc20 0x7fae0006f0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fae08009fd0 tx=0x7fae08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.728+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fae0c059700 con 0x7fae180fe830 2024-02-22T16:17:07.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.981+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fae04005590 con 0x7fae180fe830 2024-02-22T16:17:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.982+0000 7fadfffff700 1 -- 172.21.15.19:0/2357965957 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v43) v1 ==== 107+0+4855 (secure 0 0 0) 0x7fae0c059290 con 0x7fae180fe830 2024-02-22T16:17:07.955 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:07.955 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:50:20.035267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi019.fesnwx","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6829/3100392983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3100392983},{"type":"v1","addr":"172.21.15.19:6829","nonce":3100392983}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6827/588513461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":588513461},{"type":"v1","addr":"172.21.15.19:6827","nonce":588513461}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi062.oyfagg","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6825/1888029196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1888029196},{"type":"v1","addr":"172.21.15.62:6825","nonce":1888029196}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi062.eyhruo","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.62:6827/3228927024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3228927024},{"type":"v1","addr":"172.21.15.62:6827","nonce":3228927024}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae0006cc20 msgr2=0x7fae0006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae0006cc20 0x7fae0006f0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fae08009fd0 tx=0x7fae08009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 msgr2=0x7fae18113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 0x7fae18113b90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fae0c002a50 tx=0x7fae0c0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 shutdown_connections 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae0006cc20 0x7fae0006f0d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae180fde90 0x7fae18113610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 --2- 172.21.15.19:0/2357965957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae180fe830 0x7fae18113b90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 >> 172.21.15.19:0/2357965957 conn(0x7fae180f7950 msgr2=0x7fae181016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 shutdown_connections 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:07.984+0000 7fae1dc3e700 1 -- 172.21.15.19:0/2357965957 wait complete. 2024-02-22T16:17:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 22 2024-02-22T16:17:07.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 -- 172.21.15.19:0/3071215485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 msgr2=0x7f1a080fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 --2- 172.21.15.19:0/3071215485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a080fbd90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f19fc004940 tx=0x7f19fc009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 -- 172.21.15.19:0/3071215485 shutdown_connections 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 --2- 172.21.15.19:0/3071215485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a080fbd90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 --2- 172.21.15.19:0/3071215485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 0x7f1a08102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 -- 172.21.15.19:0/3071215485 >> 172.21.15.19:0/3071215485 conn(0x7f1a08072930 msgr2=0x7f1a080f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 -- 172.21.15.19:0/3071215485 shutdown_connections 2024-02-22T16:17:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.118+0000 7f1a0de79700 1 -- 172.21.15.19:0/3071215485 wait complete. 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a0de79700 1 Processor -- start 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a0de79700 1 -- start start 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a0de79700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 0x7f1a081136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a0de79700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a08113c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a0de79700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08114280 con 0x7f1a08102720 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a0de79700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0810d780 con 0x7f1a08101d80 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.119+0000 7f1a06ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a08113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a06ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a08113c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45990/0 (socket says 172.21.15.19:45990) 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a06ffd700 1 -- 172.21.15.19:0/14036751 learned_addr learned my addr 172.21.15.19:0/14036751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a06ffd700 1 -- 172.21.15.19:0/14036751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 msgr2=0x7f1a081136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a077fe700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 0x7f1a081136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a06ffd700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 0x7f1a081136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a06ffd700 1 -- 172.21.15.19:0/14036751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc005040 con 0x7f1a08102720 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a077fe700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 0x7f1a081136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a06ffd700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a08113c30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009360 tx=0x7f19fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19fc00e040 con 0x7f1a08102720 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19fc00bc50 con 0x7f1a08102720 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0810d9b0 con 0x7f1a08102720 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19fc01c630 con 0x7f1a08102720 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.120+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0810def0 con 0x7f1a08102720 2024-02-22T16:17:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.121+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a08064a80 con 0x7f1a08102720 2024-02-22T16:17:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.122+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f19fc013420 con 0x7f1a08102720 2024-02-22T16:17:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.122+0000 7f19effff700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19f006cc30 0x7f19f006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.122+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19fc08d410 con 0x7f1a08102720 2024-02-22T16:17:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.122+0000 7f1a077fe700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19f006cc30 0x7f19f006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.123+0000 7f1a077fe700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19f006cc30 0x7f19f006f0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f19f80041d0 tx=0x7f19f8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.127+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f19fc05af70 con 0x7f1a08102720 2024-02-22T16:17:08.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:08 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2556590649' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-22T16:17:08.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:08 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2357965957' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-22T16:17:08.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:08 smithi019 ceph-mon[173579]: pgmap v717: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:08.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.378+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f1a08108d20 con 0x7f1a08102720 2024-02-22T16:17:08.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.379+0000 7f19effff700 1 -- 172.21.15.19:0/14036751 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v43) v1 ==== 107+0+4854 (secure 0 0 0) 0x7f19fc05ab00 con 0x7f1a08102720 2024-02-22T16:17:08.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:08.352 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:53:53.009738+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi019.fesnwx","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6829/3100392983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3100392983},{"type":"v1","addr":"172.21.15.19:6829","nonce":3100392983}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6827/588513461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":588513461},{"type":"v1","addr":"172.21.15.19:6827","nonce":588513461}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi062.oyfagg","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6825/1888029196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1888029196},{"type":"v1","addr":"172.21.15.62:6825","nonce":1888029196}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi062.eyhruo","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.62:6827/3228927024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3228927024},{"type":"v1","addr":"172.21.15.62:6827","nonce":3228927024}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19f006cc30 msgr2=0x7f19f006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19f006cc30 0x7f19f006f0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f19f80041d0 tx=0x7f19f8009040 comp rx=0 tx=0).stop 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 msgr2=0x7f1a08113c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a08113c30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009360 tx=0x7f19fc009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 shutdown_connections 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19f006cc30 0x7f19f006f0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a08101d80 0x7f1a081136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 --2- 172.21.15.19:0/14036751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a08102720 0x7f1a08113c30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.381+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 >> 172.21.15.19:0/14036751 conn(0x7f1a08072930 msgr2=0x7f1a08064f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.382+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 shutdown_connections 2024-02-22T16:17:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.382+0000 7f1a0de79700 1 -- 172.21.15.19:0/14036751 wait complete. 2024-02-22T16:17:08.356 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 23 2024-02-22T16:17:08.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-22T16:17:08.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:08 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2556590649' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-22T16:17:08.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:08 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2357965957' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-22T16:17:08.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:08 smithi062 ceph-mon[106861]: pgmap v717: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:08.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 -- 172.21.15.19:0/2062043038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fe880 msgr2=0x7fee14106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 --2- 172.21.15.19:0/2062043038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fe880 0x7fee14106ec0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fedfc004940 tx=0x7fedfc009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 -- 172.21.15.19:0/2062043038 shutdown_connections 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 --2- 172.21.15.19:0/2062043038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fe880 0x7fee14106ec0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 --2- 172.21.15.19:0/2062043038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fdee0 0x7fee140fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 -- 172.21.15.19:0/2062043038 >> 172.21.15.19:0/2062043038 conn(0x7fee14072930 msgr2=0x7fee140f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 -- 172.21.15.19:0/2062043038 shutdown_connections 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.517+0000 7fee19115700 1 -- 172.21.15.19:0/2062043038 wait complete. 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee19115700 1 Processor -- start 2024-02-22T16:17:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee19115700 1 -- start start 2024-02-22T16:17:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee19115700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 0x7fee14113580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee19115700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fe880 0x7fee14113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee19115700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee14114110 con 0x7fee140fdee0 2024-02-22T16:17:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee19115700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1410d650 con 0x7fee140fe880 2024-02-22T16:17:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 0x7fee14113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 0x7fee14113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46006/0 (socket says 172.21.15.19:46006) 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 -- 172.21.15.19:0/1693823697 learned_addr learned my addr 172.21.15.19:0/1693823697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee1259c700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fe880 0x7fee14113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 -- 172.21.15.19:0/1693823697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fe880 msgr2=0x7fee14113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fe880 0x7fee14113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 -- 172.21.15.19:0/1693823697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc005040 con 0x7fee140fdee0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee1259c700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fe880 0x7fee14113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.518+0000 7fee12d9d700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 0x7fee14113580 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fee0400eb70 tx=0x7fee0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.519+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee04007cc0 con 0x7fee140fdee0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.519+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee0400f040 con 0x7fee140fdee0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.519+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1410d8d0 con 0x7fee140fdee0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.519+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee040095f0 con 0x7fee140fdee0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.519+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1410de10 con 0x7fee140fdee0 2024-02-22T16:17:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.520+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee141044d0 con 0x7fee140fdee0 2024-02-22T16:17:08.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.520+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fee04009750 con 0x7fee140fdee0 2024-02-22T16:17:08.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.520+0000 7fee0b7fe700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fee0006cc20 0x7fee0006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.521+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fee0408bce0 con 0x7fee140fdee0 2024-02-22T16:17:08.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.521+0000 7fee1259c700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fee0006cc20 0x7fee0006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.521+0000 7fee1259c700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fee0006cc20 0x7fee0006f0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00b0d0 tx=0x7fedfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:08.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.525+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fee04059720 con 0x7fee140fdee0 2024-02-22T16:17:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.775+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fee1410eb60 con 0x7fee140fdee0 2024-02-22T16:17:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.775+0000 7fee0b7fe700 1 -- 172.21.15.19:0/1693823697 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v43) v1 ==== 107+0+4853 (secure 0 0 0) 0x7fee040592b0 con 0x7fee140fdee0 2024-02-22T16:17:08.748 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:08.749 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:54:05.091029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi019.fesnwx","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6829/3100392983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3100392983},{"type":"v1","addr":"172.21.15.19:6829","nonce":3100392983}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14484":{"gid":14484,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6827/588513461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":588513461},{"type":"v1","addr":"172.21.15.19:6827","nonce":588513461}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi062.oyfagg","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6825/1888029196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1888029196},{"type":"v1","addr":"172.21.15.62:6825","nonce":1888029196}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi062.eyhruo","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.62:6827/3228927024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":3228927024},{"type":"v1","addr":"172.21.15.62:6827","nonce":3228927024}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:08.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fee0006cc20 msgr2=0x7fee0006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fee0006cc20 0x7fee0006f0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00b0d0 tx=0x7fedfc009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 msgr2=0x7fee14113580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 0x7fee14113580 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fee0400eb70 tx=0x7fee0400b040 comp rx=0 tx=0).stop 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 shutdown_connections 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fee0006cc20 0x7fee0006f0d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee140fdee0 0x7fee14113580 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 --2- 172.21.15.19:0/1693823697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee140fe880 0x7fee14113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 >> 172.21.15.19:0/1693823697 conn(0x7fee14072930 msgr2=0x7fee14101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.777+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 shutdown_connections 2024-02-22T16:17:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.778+0000 7fee19115700 1 -- 172.21.15.19:0/1693823697 wait complete. 2024-02-22T16:17:08.752 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 24 2024-02-22T16:17:08.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-22T16:17:08.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 -- 172.21.15.19:0/3939972027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 msgr2=0x7ffad8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 --2- 172.21.15.19:0/3939972027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8102150 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffac8004940 tx=0x7ffac8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:08.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 -- 172.21.15.19:0/3939972027 shutdown_connections 2024-02-22T16:17:08.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 --2- 172.21.15.19:0/3939972027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 0x7ffad80fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 --2- 172.21.15.19:0/3939972027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8102150 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 -- 172.21.15.19:0/3939972027 >> 172.21.15.19:0/3939972027 conn(0x7ffad8072930 msgr2=0x7ffad80f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 -- 172.21.15.19:0/3939972027 shutdown_connections 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 -- 172.21.15.19:0/3939972027 wait complete. 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 Processor -- start 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.913+0000 7ffae046d700 1 -- start start 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffae046d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8112530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffae046d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 0x7ffad8112a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffae046d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81130c0 con 0x7ffad8101d80 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffae046d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad810c600 con 0x7ffad8102720 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8112530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8112530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46024/0 (socket says 172.21.15.19:46024) 2024-02-22T16:17:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 -- 172.21.15.19:0/3412554698 learned_addr learned my addr 172.21.15.19:0/3412554698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffadda08700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 0x7ffad8112a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 -- 172.21.15.19:0/3412554698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 msgr2=0x7ffad8112a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 0x7ffad8112a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 -- 172.21.15.19:0/3412554698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8005040 con 0x7ffad8101d80 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffadda08700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 0x7ffad8112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffade209700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8112530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffac800b3b0 tx=0x7ffac8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac800e040 con 0x7ffad8101d80 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffac800bc50 con 0x7ffad8101d80 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.914+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac8013420 con 0x7ffad8101d80 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.915+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad810c880 con 0x7ffad8101d80 2024-02-22T16:17:08.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.915+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad810ce40 con 0x7ffad8101d80 2024-02-22T16:17:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.915+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8064a80 con 0x7ffad8101d80 2024-02-22T16:17:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.916+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ffac801c430 con 0x7ffad8101d80 2024-02-22T16:17:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.916+0000 7ffaceffd700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffac406cc20 0x7ffac406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.916+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ffac808ce70 con 0x7ffad8101d80 2024-02-22T16:17:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.917+0000 7ffadda08700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffac406cc20 0x7ffac406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:08.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.917+0000 7ffadda08700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffac406cc20 0x7ffac406f0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffad4007fd0 tx=0x7ffad4009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:08.921+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ffac805a9e0 con 0x7ffad8101d80 2024-02-22T16:17:09.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.174+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7ffad810d060 con 0x7ffad8101d80 2024-02-22T16:17:09.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.175+0000 7ffaceffd700 1 -- 172.21.15.19:0/3412554698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v43) v1 ==== 107+0+3284 (secure 0 0 0) 0x7ffac805a570 con 0x7ffad8101d80 2024-02-22T16:17:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:09.148 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:53.989078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi019.fesnwx","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6829/3100392983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3100392983},{"type":"v1","addr":"172.21.15.19:6829","nonce":3100392983}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi062.oyfagg","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6825/1888029196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1888029196},{"type":"v1","addr":"172.21.15.62:6825","nonce":1888029196}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffac406cc20 msgr2=0x7ffac406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffac406cc20 0x7ffac406f0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffad4007fd0 tx=0x7ffad4009f60 comp rx=0 tx=0).stop 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 msgr2=0x7ffad8112530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8112530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffac800b3b0 tx=0x7ffac8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 shutdown_connections 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ffac406cc20 0x7ffac406f0d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad8101d80 0x7ffad8112530 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 --2- 172.21.15.19:0/3412554698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffad8102720 0x7ffad8112a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 >> 172.21.15.19:0/3412554698 conn(0x7ffad8072930 msgr2=0x7ffad8073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 shutdown_connections 2024-02-22T16:17:09.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.177+0000 7ffae046d700 1 -- 172.21.15.19:0/3412554698 wait complete. 2024-02-22T16:17:09.152 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-02-22T16:17:09.162 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-02-22T16:17:09.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 -- 172.21.15.19:0/4067882688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398102e90 msgr2=0x7fe398103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 --2- 172.21.15.19:0/4067882688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398102e90 0x7fe398103260 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe388004940 tx=0x7fe388009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 -- 172.21.15.19:0/4067882688 shutdown_connections 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 --2- 172.21.15.19:0/4067882688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398103830 0x7fe3980fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 --2- 172.21.15.19:0/4067882688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398102e90 0x7fe398103260 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 -- 172.21.15.19:0/4067882688 >> 172.21.15.19:0/4067882688 conn(0x7fe3980f8a20 msgr2=0x7fe3980fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:09.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 -- 172.21.15.19:0/4067882688 shutdown_connections 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.314+0000 7fe39fe88700 1 -- 172.21.15.19:0/4067882688 wait complete. 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39fe88700 1 Processor -- start 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39fe88700 1 -- start start 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39fe88700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102e90 0x7fe3981136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39fe88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 0x7fe398113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39fe88700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398114270 con 0x7fe398103830 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39fe88700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39810d770 con 0x7fe398102e90 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39d423700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 0x7fe398113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39d423700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 0x7fe398113c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46036/0 (socket says 172.21.15.19:46036) 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39d423700 1 -- 172.21.15.19:0/2256090398 learned_addr learned my addr 172.21.15.19:0/2256090398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39dc24700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102e90 0x7fe3981136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39d423700 1 -- 172.21.15.19:0/2256090398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102e90 msgr2=0x7fe3981136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39d423700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102e90 0x7fe3981136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.315+0000 7fe39d423700 1 -- 172.21.15.19:0/2256090398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe388005040 con 0x7fe398103830 2024-02-22T16:17:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.316+0000 7fe39d423700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 0x7fe398113c20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c800 tx=0x7fe394004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.316+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3940106f0 con 0x7fe398103830 2024-02-22T16:17:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.316+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39810da00 con 0x7fe398103830 2024-02-22T16:17:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.316+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe394010d00 con 0x7fe398103830 2024-02-22T16:17:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.316+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3940113a0 con 0x7fe398103830 2024-02-22T16:17:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.316+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39810df40 con 0x7fe398103830 2024-02-22T16:17:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.317+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398064a80 con 0x7fe398103830 2024-02-22T16:17:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.317+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe394012400 con 0x7fe398103830 2024-02-22T16:17:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.318+0000 7fe38e7fc700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe38406cc30 0x7fe38406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.318+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe39408bff0 con 0x7fe398103830 2024-02-22T16:17:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.318+0000 7fe39dc24700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe38406cc30 0x7fe38406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:09.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.318+0000 7fe39dc24700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe38406cc30 0x7fe38406f0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe38800b840 tx=0x7fe388009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:09.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.322+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe394059ab0 con 0x7fe398103830 2024-02-22T16:17:09.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:09 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/14036751' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-22T16:17:09.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:09 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1693823697' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-22T16:17:09.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:09 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3412554698' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-22T16:17:09.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:09 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/14036751' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-22T16:17:09.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:09 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1693823697' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-22T16:17:09.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:09 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3412554698' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-22T16:17:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.575+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fe39810e1f0 con 0x7fe398103830 2024-02-22T16:17:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.575+0000 7fe38e7fc700 1 -- 172.21.15.19:0/2256090398 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v43) v1 ==== 107+0+3260 (secure 0 0 0) 0x7fe394059640 con 0x7fe398103830 2024-02-22T16:17:09.549 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44295,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/720124526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":720124526},{"type":"v1","addr":"172.21.15.19:6827","nonce":720124526}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe38406cc30 msgr2=0x7fe38406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe38406cc30 0x7fe38406f0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe38800b840 tx=0x7fe388009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 msgr2=0x7fe398113c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 0x7fe398113c20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c800 tx=0x7fe394004470 comp rx=0 tx=0).stop 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 shutdown_connections 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe38406cc30 0x7fe38406f0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe398102e90 0x7fe3981136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 --2- 172.21.15.19:0/2256090398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe398103830 0x7fe398113c20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 >> 172.21.15.19:0/2256090398 conn(0x7fe3980f8a20 msgr2=0x7fe3980fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 shutdown_connections 2024-02-22T16:17:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.578+0000 7fe39fe88700 1 -- 172.21.15.19:0/2256090398 wait complete. 2024-02-22T16:17:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-02-22T16:17:09.563 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:09.563 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-02-22T16:17:09.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-22T16:17:09.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 -- 172.21.15.19:0/4173028708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 msgr2=0x7fc6600fb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 --2- 172.21.15.19:0/4173028708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fb8d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc654004940 tx=0x7fc654009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 -- 172.21.15.19:0/4173028708 shutdown_connections 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 --2- 172.21.15.19:0/4173028708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fb8d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 --2- 172.21.15.19:0/4173028708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 0x7fc6601033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 -- 172.21.15.19:0/4173028708 >> 172.21.15.19:0/4173028708 conn(0x7fc6600723b0 msgr2=0x7fc6600727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 -- 172.21.15.19:0/4173028708 shutdown_connections 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.714+0000 7fc6669f0700 1 -- 172.21.15.19:0/4173028708 wait complete. 2024-02-22T16:17:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc6669f0700 1 Processor -- start 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc6669f0700 1 -- start start 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc6669f0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc6669f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 0x7fc6600fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc6669f0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600fd0e0 con 0x7fc6600fb480 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc6669f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600fd220 con 0x7fc660102ff0 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46056/0 (socket says 172.21.15.19:46056) 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65ffff700 1 -- 172.21.15.19:0/2990630089 learned_addr learned my addr 172.21.15.19:0/2990630089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65f7fe700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 0x7fc6600fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65ffff700 1 -- 172.21.15.19:0/2990630089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 msgr2=0x7fc6600fcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65ffff700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 0x7fc6600fcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65ffff700 1 -- 172.21.15.19:0/2990630089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654005040 con 0x7fc6600fb480 2024-02-22T16:17:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.715+0000 7fc65f7fe700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 0x7fc6600fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.716+0000 7fc65ffff700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fc660 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc650002a50 tx=0x7fc6500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.716+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc650007cc0 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.716+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc660119ad0 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.716+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc65000f040 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.716+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc650009540 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.716+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc660119e20 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.717+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6601053d0 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.717+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc6500096a0 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.717+0000 7fc65cff9700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc64806cc30 0x7fc64806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.718+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc65008be80 con 0x7fc6600fb480 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.718+0000 7fc65f7fe700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc64806cc30 0x7fc64806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:09.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.718+0000 7fc65f7fe700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc64806cc30 0x7fc64806f0e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc6600fde50 tx=0x7fc654009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.722+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fc650055e30 con 0x7fc6600fb480 2024-02-22T16:17:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.976+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fc6600fdbc0 con 0x7fc6600fb480 2024-02-22T16:17:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.977+0000 7fc65cff9700 1 -- 172.21.15.19:0/2990630089 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v43) v1 ==== 107+0+4818 (secure 0 0 0) 0x7fc650059450 con 0x7fc6600fb480 2024-02-22T16:17:09.950 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:09.950 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3538613108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3538613108},{"type":"v1","addr":"172.21.15.19:6829","nonce":3538613108}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44295,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/720124526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":720124526},{"type":"v1","addr":"172.21.15.19:6827","nonce":720124526}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44299,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/4079594902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4079594902},{"type":"v1","addr":"172.21.15.62:6825","nonce":4079594902}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc64806cc30 msgr2=0x7fc64806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc64806cc30 0x7fc64806f0e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc6600fde50 tx=0x7fc654009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 msgr2=0x7fc6600fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fc660 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc650002a50 tx=0x7fc6500046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 shutdown_connections 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fc64806cc30 0x7fc64806f0e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6600fb480 0x7fc6600fc660 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 --2- 172.21.15.19:0/2990630089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc660102ff0 0x7fc6600fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 >> 172.21.15.19:0/2990630089 conn(0x7fc6600723b0 msgr2=0x7fc6600fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 shutdown_connections 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:09.979+0000 7fc6669f0700 1 -- 172.21.15.19:0/2990630089 wait complete. 2024-02-22T16:17:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-02-22T16:17:09.964 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:09.964 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-02-22T16:17:09.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.114+0000 7fdf2d383700 1 -- 172.21.15.19:0/3943478455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28101f70 msgr2=0x7fdf28102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.114+0000 7fdf2d383700 1 --2- 172.21.15.19:0/3943478455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28101f70 0x7fdf28102340 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c004940 tx=0x7fdf1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.114+0000 7fdf2d383700 1 -- 172.21.15.19:0/3943478455 shutdown_connections 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.114+0000 7fdf2d383700 1 --2- 172.21.15.19:0/3943478455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28102910 0x7fdf280fbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 --2- 172.21.15.19:0/3943478455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28101f70 0x7fdf28102340 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 -- 172.21.15.19:0/3943478455 >> 172.21.15.19:0/3943478455 conn(0x7fdf280f7aa0 msgr2=0x7fdf280f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 -- 172.21.15.19:0/3943478455 shutdown_connections 2024-02-22T16:17:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 -- 172.21.15.19:0/3943478455 wait complete. 2024-02-22T16:17:10.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 Processor -- start 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 -- start start 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28101f70 0x7fdf281127e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 0x7fdf28112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28113370 con 0x7fdf28102910 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.115+0000 7fdf2d383700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2810c8b0 con 0x7fdf28101f70 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 0x7fdf28112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 0x7fdf28112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46068/0 (socket says 172.21.15.19:46068) 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 -- 172.21.15.19:0/532121321 learned_addr learned my addr 172.21.15.19:0/532121321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf26ffd700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28101f70 0x7fdf281127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 -- 172.21.15.19:0/532121321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28101f70 msgr2=0x7fdf281127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28101f70 0x7fdf281127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 -- 172.21.15.19:0/532121321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c005040 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf26ffd700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28101f70 0x7fdf281127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf267fc700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 0x7fdf28112d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c890 tx=0x7fdf180044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf1800a7d0 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf1800ade0 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2810cb90 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf180113a0 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.116+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2810d0d0 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.117+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf280ff280 con 0x7fdf28102910 2024-02-22T16:17:10.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.118+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdf18012400 con 0x7fdf28102910 2024-02-22T16:17:10.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.118+0000 7fdf0f7fe700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf1006cc20 0x7fdf1006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.118+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdf1808c080 con 0x7fdf28102910 2024-02-22T16:17:10.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.118+0000 7fdf26ffd700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf1006cc20 0x7fdf1006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.119+0000 7fdf26ffd700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf1006cc20 0x7fdf1006f0d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c004560 tx=0x7fdf1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:10.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.123+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fdf18059be0 con 0x7fdf28102910 2024-02-22T16:17:10.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:10 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2256090398' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-22T16:17:10.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:10 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2990630089' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-22T16:17:10.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:10 smithi019 ceph-mon[173579]: pgmap v718: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.373+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fdf2810ddf0 con 0x7fdf28102910 2024-02-22T16:17:10.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.373+0000 7fdf0f7fe700 1 -- 172.21.15.19:0/532121321 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v43) v1 ==== 107+0+4042 (secure 0 0 0) 0x7fdf18059770 con 0x7fdf28102910 2024-02-22T16:17:10.347 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:10.347 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3538613108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3538613108},{"type":"v1","addr":"172.21.15.19:6829","nonce":3538613108}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44299,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/4079594902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4079594902},{"type":"v1","addr":"172.21.15.62:6825","nonce":4079594902}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.375+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf1006cc20 msgr2=0x7fdf1006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.375+0000 7fdf2d383700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf1006cc20 0x7fdf1006f0d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c004560 tx=0x7fdf1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.375+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 msgr2=0x7fdf28112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.375+0000 7fdf2d383700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 0x7fdf28112d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c890 tx=0x7fdf180044c0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 shutdown_connections 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fdf1006cc20 0x7fdf1006f0d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf28101f70 0x7fdf281127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 --2- 172.21.15.19:0/532121321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf28102910 0x7fdf28112d20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 >> 172.21.15.19:0/532121321 conn(0x7fdf280f7aa0 msgr2=0x7fdf280f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 shutdown_connections 2024-02-22T16:17:10.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.376+0000 7fdf2d383700 1 -- 172.21.15.19:0/532121321 wait complete. 2024-02-22T16:17:10.350 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-02-22T16:17:10.361 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:10.361 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-02-22T16:17:10.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-22T16:17:10.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:10 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2256090398' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-22T16:17:10.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:10 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2990630089' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-22T16:17:10.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:10 smithi062 ceph-mon[106861]: pgmap v718: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 -- 172.21.15.19:0/40614182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4101d80 msgr2=0x7f2ce4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 --2- 172.21.15.19:0/40614182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4101d80 0x7f2ce4102150 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004940 tx=0x7f2ce0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 -- 172.21.15.19:0/40614182 shutdown_connections 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 --2- 172.21.15.19:0/40614182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ce4102720 0x7f2ce40fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 --2- 172.21.15.19:0/40614182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4101d80 0x7f2ce4102150 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 -- 172.21.15.19:0/40614182 >> 172.21.15.19:0/40614182 conn(0x7f2ce4072930 msgr2=0x7f2ce40f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 -- 172.21.15.19:0/40614182 shutdown_connections 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 -- 172.21.15.19:0/40614182 wait complete. 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.511+0000 7f2cec21d700 1 Processor -- start 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 -- start start 2024-02-22T16:17:10.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ce4101d80 0x7f2ce4113670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 0x7f2ce4113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4114240 con 0x7f2ce4102720 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce410d740 con 0x7f2ce4101d80 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 0x7f2ce4113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 0x7f2ce4113bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46086/0 (socket says 172.21.15.19:46086) 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 -- 172.21.15.19:0/1212441513 learned_addr learned my addr 172.21.15.19:0/1212441513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 -- 172.21.15.19:0/1212441513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ce4101d80 msgr2=0x7f2ce4113670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ce4101d80 0x7f2ce4113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 -- 172.21.15.19:0/1212441513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0005040 con 0x7f2ce4102720 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2ce97b8700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 0x7f2ce4113bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4002a50 tx=0x7f2cd4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd4007cc0 con 0x7f2ce4102720 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cd400f040 con 0x7f2ce4102720 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd400d9a0 con 0x7f2ce4102720 2024-02-22T16:17:10.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce410d9d0 con 0x7f2ce4102720 2024-02-22T16:17:10.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.512+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce410df10 con 0x7f2ce4102720 2024-02-22T16:17:10.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.513+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce4064a80 con 0x7f2ce4102720 2024-02-22T16:17:10.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.514+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2cd400dc10 con 0x7f2ce4102720 2024-02-22T16:17:10.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.514+0000 7f2cda7fc700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd006cc20 0x7f2cd006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.514+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2cd408bdd0 con 0x7f2ce4102720 2024-02-22T16:17:10.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.515+0000 7f2ce9fb9700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd006cc20 0x7f2cd006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.515+0000 7f2ce9fb9700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd006cc20 0x7f2cd006f0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004f90 tx=0x7f2ce0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:10.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.519+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f2cd4059890 con 0x7f2ce4102720 2024-02-22T16:17:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.770+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f2ce410e280 con 0x7f2ce4102720 2024-02-22T16:17:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.771+0000 7f2cda7fc700 1 -- 172.21.15.19:0/1212441513 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v43) v1 ==== 107+0+4821 (secure 0 0 0) 0x7f2cd4059420 con 0x7f2ce4102720 2024-02-22T16:17:10.744 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:10.744 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3538613108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3538613108},{"type":"v1","addr":"172.21.15.19:6829","nonce":3538613108}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44299,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/4079594902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4079594902},{"type":"v1","addr":"172.21.15.62:6825","nonce":4079594902}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd006cc20 msgr2=0x7f2cd006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd006cc20 0x7f2cd006f0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0004f90 tx=0x7f2ce0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 msgr2=0x7f2ce4113bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 0x7f2ce4113bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4002a50 tx=0x7f2cd4004600 comp rx=0 tx=0).stop 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 shutdown_connections 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f2cd006cc20 0x7f2cd006f0d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ce4101d80 0x7f2ce4113670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 --2- 172.21.15.19:0/1212441513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4102720 0x7f2ce4113bf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 >> 172.21.15.19:0/1212441513 conn(0x7f2ce4072930 msgr2=0x7f2ce4105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 shutdown_connections 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.773+0000 7f2cec21d700 1 -- 172.21.15.19:0/1212441513 wait complete. 2024-02-22T16:17:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 29 2024-02-22T16:17:10.758 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:10.759 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-02-22T16:17:10.759 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-22T16:17:10.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 -- 172.21.15.19:0/1184876466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38103850 msgr2=0x7fea380fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 --2- 172.21.15.19:0/1184876466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38103850 0x7fea380fcec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fea2c004940 tx=0x7fea2c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:10.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 -- 172.21.15.19:0/1184876466 shutdown_connections 2024-02-22T16:17:10.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 --2- 172.21.15.19:0/1184876466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38103850 0x7fea380fcec0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 --2- 172.21.15.19:0/1184876466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38102eb0 0x7fea38103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 -- 172.21.15.19:0/1184876466 >> 172.21.15.19:0/1184876466 conn(0x7fea380f8a60 msgr2=0x7fea380fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 -- 172.21.15.19:0/1184876466 shutdown_connections 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.909+0000 7fea3db60700 1 -- 172.21.15.19:0/1184876466 wait complete. 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea3db60700 1 Processor -- start 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea3db60700 1 -- start start 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea3db60700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 0x7fea38113690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea3db60700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38103850 0x7fea38113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea3db60700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38114260 con 0x7fea38102eb0 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea3db60700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3810d760 con 0x7fea38103850 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 0x7fea38113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 0x7fea38113690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46108/0 (socket says 172.21.15.19:46108) 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.910+0000 7fea377fe700 1 -- 172.21.15.19:0/65225833 learned_addr learned my addr 172.21.15.19:0/65225833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:10.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea36ffd700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38103850 0x7fea38113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea377fe700 1 -- 172.21.15.19:0/65225833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38103850 msgr2=0x7fea38113c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea377fe700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38103850 0x7fea38113c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea377fe700 1 -- 172.21.15.19:0/65225833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c005040 con 0x7fea38102eb0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea36ffd700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38103850 0x7fea38113c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea377fe700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 0x7fea38113690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fea28002a50 tx=0x7fea280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea28007cc0 con 0x7fea38102eb0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea2800f040 con 0x7fea38102eb0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3810d9f0 con 0x7fea38102eb0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea28009540 con 0x7fea38102eb0 2024-02-22T16:17:10.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.911+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3810df30 con 0x7fea38102eb0 2024-02-22T16:17:10.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.912+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38105380 con 0x7fea38102eb0 2024-02-22T16:17:10.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.912+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fea280096a0 con 0x7fea38102eb0 2024-02-22T16:17:10.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.913+0000 7fea1ffff700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea2006cc30 0x7fea2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:10.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.913+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fea2808be80 con 0x7fea38102eb0 2024-02-22T16:17:10.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.913+0000 7fea36ffd700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea2006cc30 0x7fea2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:10.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.913+0000 7fea36ffd700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea2006cc30 0x7fea2006f0e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fea3810ef30 tx=0x7fea2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:10.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:10.917+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fea280598c0 con 0x7fea38102eb0 2024-02-22T16:17:11.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.170+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fea3810ec70 con 0x7fea38102eb0 2024-02-22T16:17:11.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.171+0000 7fea1ffff700 1 -- 172.21.15.19:0/65225833 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v43) v1 ==== 107+0+4042 (secure 0 0 0) 0x7fea28059450 con 0x7fea38102eb0 2024-02-22T16:17:11.144 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:11.144 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44299,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/4079594902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4079594902},{"type":"v1","addr":"172.21.15.62:6825","nonce":4079594902}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea2006cc30 msgr2=0x7fea2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea2006cc30 0x7fea2006f0e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fea3810ef30 tx=0x7fea2c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 msgr2=0x7fea38113690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 0x7fea38113690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fea28002a50 tx=0x7fea280046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 shutdown_connections 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fea2006cc30 0x7fea2006f0e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea38102eb0 0x7fea38113690 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 --2- 172.21.15.19:0/65225833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea38103850 0x7fea38113c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 >> 172.21.15.19:0/65225833 conn(0x7fea380f8a60 msgr2=0x7fea380fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.173+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 shutdown_connections 2024-02-22T16:17:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.174+0000 7fea3db60700 1 -- 172.21.15.19:0/65225833 wait complete. 2024-02-22T16:17:11.148 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-02-22T16:17:11.159 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:11.162 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-02-22T16:17:11.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-22T16:17:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 -- 172.21.15.19:0/997560762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044102720 msgr2=0x7fb0440fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 --2- 172.21.15.19:0/997560762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044102720 0x7fb0440fbd90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb034004940 tx=0x7fb034009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 -- 172.21.15.19:0/997560762 shutdown_connections 2024-02-22T16:17:11.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 --2- 172.21.15.19:0/997560762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044102720 0x7fb0440fbd90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 --2- 172.21.15.19:0/997560762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044101d80 0x7fb044102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 -- 172.21.15.19:0/997560762 >> 172.21.15.19:0/997560762 conn(0x7fb044072930 msgr2=0x7fb0440f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:11.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 -- 172.21.15.19:0/997560762 shutdown_connections 2024-02-22T16:17:11.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 -- 172.21.15.19:0/997560762 wait complete. 2024-02-22T16:17:11.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 Processor -- start 2024-02-22T16:17:11.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 -- start start 2024-02-22T16:17:11.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.313+0000 7fb049100700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 0x7fb044112570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:11.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb049100700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044102720 0x7fb044112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:11.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb049100700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb044113100 con 0x7fb044101d80 2024-02-22T16:17:11.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb049100700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410c640 con 0x7fb044102720 2024-02-22T16:17:11.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 0x7fb044112570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 0x7fb044112570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46120/0 (socket says 172.21.15.19:46120) 2024-02-22T16:17:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 -- 172.21.15.19:0/3610842485 learned_addr learned my addr 172.21.15.19:0/3610842485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb04259c700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044102720 0x7fb044112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 -- 172.21.15.19:0/3610842485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044102720 msgr2=0x7fb044112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044102720 0x7fb044112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 -- 172.21.15.19:0/3610842485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034005040 con 0x7fb044101d80 2024-02-22T16:17:11.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb04259c700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044102720 0x7fb044112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:17:11.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb042d9d700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 0x7fb044112570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb02c002a50 tx=0x7fb02c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:11.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02c007cc0 con 0x7fb044101d80 2024-02-22T16:17:11.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04410c920 con 0x7fb044101d80 2024-02-22T16:17:11.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb02c00f040 con 0x7fb044101d80 2024-02-22T16:17:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02c009540 con 0x7fb044101d80 2024-02-22T16:17:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.314+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04410ceb0 con 0x7fb044101d80 2024-02-22T16:17:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.315+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044104250 con 0x7fb044101d80 2024-02-22T16:17:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.316+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb02c0096a0 con 0x7fb044101d80 2024-02-22T16:17:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.316+0000 7fb03b7fe700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb03006cc20 0x7fb03006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.316+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb02c08be80 con 0x7fb044101d80 2024-02-22T16:17:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.316+0000 7fb04259c700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb03006cc20 0x7fb03006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.317+0000 7fb04259c700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb03006cc20 0x7fb03006f0d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb04410d970 tx=0x7fb034009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.321+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb02c0598c0 con 0x7fb044101d80 2024-02-22T16:17:11.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:11 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/532121321' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-22T16:17:11.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:11 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1212441513' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-22T16:17:11.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:11 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/65225833' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-22T16:17:11.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:11 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/532121321' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-22T16:17:11.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:11 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1212441513' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-22T16:17:11.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:11 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/65225833' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-22T16:17:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.571+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fb044064a80 con 0x7fb044101d80 2024-02-22T16:17:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.571+0000 7fb03b7fe700 1 -- 172.21.15.19:0/3610842485 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v43) v1 ==== 107+0+4821 (secure 0 0 0) 0x7fb02c059450 con 0x7fb044101d80 2024-02-22T16:17:11.544 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:11.545 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44299,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/4079594902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4079594902},{"type":"v1","addr":"172.21.15.62:6825","nonce":4079594902}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb03006cc20 msgr2=0x7fb03006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb03006cc20 0x7fb03006f0d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb04410d970 tx=0x7fb034009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 msgr2=0x7fb044112570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 0x7fb044112570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb02c002a50 tx=0x7fb02c0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 shutdown_connections 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb03006cc20 0x7fb03006f0d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044101d80 0x7fb044112570 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 --2- 172.21.15.19:0/3610842485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb044102720 0x7fb044112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 >> 172.21.15.19:0/3610842485 conn(0x7fb044072930 msgr2=0x7fb044073d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.573+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 shutdown_connections 2024-02-22T16:17:11.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.574+0000 7fb049100700 1 -- 172.21.15.19:0/3610842485 wait complete. 2024-02-22T16:17:11.548 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-02-22T16:17:11.559 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:11.559 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-02-22T16:17:11.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 -- 172.21.15.19:0/4164751350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c102720 msgr2=0x7fae2c0fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 --2- 172.21.15.19:0/4164751350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c102720 0x7fae2c0fbd90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fae1c004940 tx=0x7fae1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 -- 172.21.15.19:0/4164751350 shutdown_connections 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 --2- 172.21.15.19:0/4164751350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c102720 0x7fae2c0fbd90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 --2- 172.21.15.19:0/4164751350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2c101d80 0x7fae2c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 -- 172.21.15.19:0/4164751350 >> 172.21.15.19:0/4164751350 conn(0x7fae2c072930 msgr2=0x7fae2c0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 -- 172.21.15.19:0/4164751350 shutdown_connections 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 -- 172.21.15.19:0/4164751350 wait complete. 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 Processor -- start 2024-02-22T16:17:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.710+0000 7fae3407d700 1 -- start start 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae3407d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 0x7fae2c0fda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae3407d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2c102720 0x7fae2c0fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae3407d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0fe480 con 0x7fae2c101d80 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae3407d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0fe5c0 con 0x7fae2c102720 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 0x7fae2c0fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 0x7fae2c0fda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46144/0 (socket says 172.21.15.19:46144) 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 -- 172.21.15.19:0/3509218102 learned_addr learned my addr 172.21.15.19:0/3509218102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31618700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2c102720 0x7fae2c0fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 -- 172.21.15.19:0/3509218102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2c102720 msgr2=0x7fae2c0fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2c102720 0x7fae2c0fdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 -- 172.21.15.19:0/3509218102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c005040 con 0x7fae2c101d80 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae31e19700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 0x7fae2c0fda00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fae28002a50 tx=0x7fae280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae28007cc0 con 0x7fae2c101d80 2024-02-22T16:17:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae2800f040 con 0x7fae2c101d80 2024-02-22T16:17:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c119b00 con 0x7fae2c101d80 2024-02-22T16:17:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae28009540 con 0x7fae2c101d80 2024-02-22T16:17:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.711+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c119e50 con 0x7fae2c101d80 2024-02-22T16:17:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.712+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c064a80 con 0x7fae2c101d80 2024-02-22T16:17:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.713+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fae280096a0 con 0x7fae2c101d80 2024-02-22T16:17:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.713+0000 7fae227fc700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae1806cc20 0x7fae1806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.713+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fae2808bd80 con 0x7fae2c101d80 2024-02-22T16:17:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.713+0000 7fae31618700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae1806cc20 0x7fae1806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.714+0000 7fae31618700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae1806cc20 0x7fae1806f0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009fd0 tx=0x7fae1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.718+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fae28055db0 con 0x7fae2c101d80 2024-02-22T16:17:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.967+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fae2c11a0a0 con 0x7fae2c101d80 2024-02-22T16:17:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.968+0000 7fae227fc700 1 -- 172.21.15.19:0/3509218102 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v43) v1 ==== 107+0+4042 (secure 0 0 0) 0x7fae280593d0 con 0x7fae2c101d80 2024-02-22T16:17:11.941 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:11.941 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:11.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae1806cc20 msgr2=0x7fae1806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae1806cc20 0x7fae1806f0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009fd0 tx=0x7fae1c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 msgr2=0x7fae2c0fda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 0x7fae2c0fda00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fae28002a50 tx=0x7fae280046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 shutdown_connections 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fae1806cc20 0x7fae1806f0d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae2c101d80 0x7fae2c0fda00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 --2- 172.21.15.19:0/3509218102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2c102720 0x7fae2c0fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 >> 172.21.15.19:0/3509218102 conn(0x7fae2c072930 msgr2=0x7fae2c064f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 shutdown_connections 2024-02-22T16:17:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:11.970+0000 7fae3407d700 1 -- 172.21.15.19:0/3509218102 wait complete. 2024-02-22T16:17:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-02-22T16:17:11.955 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:11.956 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-02-22T16:17:11.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 -- 172.21.15.19:0/396734430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 msgr2=0x7f88b4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/396734430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b4102150 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f889c004940 tx=0x7f889c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 -- 172.21.15.19:0/396734430 shutdown_connections 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/396734430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88b4102720 0x7f88b40fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/396734430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b4102150 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 -- 172.21.15.19:0/396734430 >> 172.21.15.19:0/396734430 conn(0x7f88b4072930 msgr2=0x7f88b40f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:12.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 -- 172.21.15.19:0/396734430 shutdown_connections 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.106+0000 7f88b8ed3700 1 -- 172.21.15.19:0/396734430 wait complete. 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b8ed3700 1 Processor -- start 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b8ed3700 1 -- start start 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b8ed3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b410c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b8ed3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88b4102720 0x7f88b410cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b8ed3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b4065e20 con 0x7f88b4101d80 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b8ed3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b410cff0 con 0x7f88b4102720 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b410c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b259c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b410c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46166/0 (socket says 172.21.15.19:46166) 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b259c700 1 -- 172.21.15.19:0/488634883 learned_addr learned my addr 172.21.15.19:0/488634883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:12.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.107+0000 7f88b1d9b700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88b4102720 0x7f88b410cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88b259c700 1 -- 172.21.15.19:0/488634883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88b4102720 msgr2=0x7f88b410cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88b259c700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88b4102720 0x7f88b410cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88b259c700 1 -- 172.21.15.19:0/488634883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c005040 con 0x7f88b4101d80 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88b259c700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b410c570 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f889c009fd0 tx=0x7f889c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889c00e040 con 0x7f88b4101d80 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f889c013420 con 0x7f88b4101d80 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889c01c630 con 0x7f88b4101d80 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b410d220 con 0x7f88b4101d80 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.108+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b4119e20 con 0x7f88b4101d80 2024-02-22T16:17:12.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.109+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b40ff090 con 0x7f88b4101d80 2024-02-22T16:17:12.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.109+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f889c013590 con 0x7f88b4101d80 2024-02-22T16:17:12.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.110+0000 7f88aaffd700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88a006cc30 0x7f88a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.110+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f889c08d230 con 0x7f88b4101d80 2024-02-22T16:17:12.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.110+0000 7f88b1d9b700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88a006cc30 0x7f88a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.110+0000 7f88b1d9b700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88a006cc30 0x7f88a006f0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f88b4065780 tx=0x7f88a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:12.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.114+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f889c05ad90 con 0x7f88b4101d80 2024-02-22T16:17:12.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:12 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3610842485' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-22T16:17:12.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:12 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3509218102' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-22T16:17:12.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:12 smithi019 ceph-mon[173579]: pgmap v719: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:12.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.365+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f88b4065560 con 0x7f88b4101d80 2024-02-22T16:17:12.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.365+0000 7f88aaffd700 1 -- 172.21.15.19:0/488634883 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v43) v1 ==== 107+0+4821 (secure 0 0 0) 0x7f889c05a920 con 0x7f88b4101d80 2024-02-22T16:17:12.338 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:12.338 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":44291,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/2221505494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2221505494},{"type":"v1","addr":"172.21.15.62:6827","nonce":2221505494}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88a006cc30 msgr2=0x7f88a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88a006cc30 0x7f88a006f0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f88b4065780 tx=0x7f88a400b040 comp rx=0 tx=0).stop 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 msgr2=0x7f88b410c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b410c570 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f889c009fd0 tx=0x7f889c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 shutdown_connections 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f88a006cc30 0x7f88a006f0e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b4101d80 0x7f88b410c570 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 --2- 172.21.15.19:0/488634883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88b4102720 0x7f88b410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 >> 172.21.15.19:0/488634883 conn(0x7f88b4072930 msgr2=0x7f88b4073dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 shutdown_connections 2024-02-22T16:17:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.368+0000 7f88b8ed3700 1 -- 172.21.15.19:0/488634883 wait complete. 2024-02-22T16:17:12.342 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 33 2024-02-22T16:17:12.353 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:12.353 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-02-22T16:17:12.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-22T16:17:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.503+0000 7ff0069e1700 1 -- 172.21.15.19:0/3029590668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000101fa0 msgr2=0x7ff000102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3029590668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000101fa0 0x7ff000102370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7feff0004940 tx=0x7feff0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 -- 172.21.15.19:0/3029590668 shutdown_connections 2024-02-22T16:17:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3029590668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff000102940 0x7ff0000fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3029590668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000101fa0 0x7ff000102370 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 -- 172.21.15.19:0/3029590668 >> 172.21.15.19:0/3029590668 conn(0x7ff0000f7b70 msgr2=0x7ff0000f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 -- 172.21.15.19:0/3029590668 shutdown_connections 2024-02-22T16:17:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 -- 172.21.15.19:0/3029590668 wait complete. 2024-02-22T16:17:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 Processor -- start 2024-02-22T16:17:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.504+0000 7ff0069e1700 1 -- start start 2024-02-22T16:17:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7ff0069e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff000101fa0 0x7ff0001138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7ff0069e1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 0x7ff000113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7ff0069e1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000114460 con 0x7ff000102940 2024-02-22T16:17:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7ff0069e1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00010d9a0 con 0x7ff000101fa0 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 0x7ff000113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 0x7ff000113e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46194/0 (socket says 172.21.15.19:46194) 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 -- 172.21.15.19:0/3224548487 learned_addr learned my addr 172.21.15.19:0/3224548487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7feffffff700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff000101fa0 0x7ff0001138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 -- 172.21.15.19:0/3224548487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff000101fa0 msgr2=0x7ff0001138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff000101fa0 0x7ff0001138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 -- 172.21.15.19:0/3224548487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0005040 con 0x7ff000102940 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7fefff7fe700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 0x7ff000113e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feff400c800 tx=0x7feff4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feff40106f0 con 0x7ff000102940 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feff4010d00 con 0x7ff000102940 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00010dc80 con 0x7ff000102940 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feff40113a0 con 0x7ff000102940 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.505+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00010e240 con 0x7ff000102940 2024-02-22T16:17:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.506+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0000641f0 con 0x7ff000102940 2024-02-22T16:17:12.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.507+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feff4012400 con 0x7ff000102940 2024-02-22T16:17:12.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.507+0000 7feffcff9700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fefe806cc20 0x7fefe806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.507+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7feff408bff0 con 0x7ff000102940 2024-02-22T16:17:12.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.507+0000 7feffffff700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fefe806cc20 0x7fefe806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.508+0000 7feffffff700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fefe806cc20 0x7fefe806f0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feff0004f90 tx=0x7feff0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:12.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.512+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7feff4059ab0 con 0x7ff000102940 2024-02-22T16:17:12.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:12 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3610842485' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-22T16:17:12.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:12 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3509218102' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-22T16:17:12.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:12 smithi062 ceph-mon[106861]: pgmap v719: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:12.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.763+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7ff000064cb0 con 0x7ff000102940 2024-02-22T16:17:12.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.763+0000 7feffcff9700 1 -- 172.21.15.19:0/3224548487 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v43) v1 ==== 107+0+4042 (secure 0 0 0) 0x7feff4059640 con 0x7ff000102940 2024-02-22T16:17:12.737 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:12.737 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fefe806cc20 msgr2=0x7fefe806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fefe806cc20 0x7fefe806f0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feff0004f90 tx=0x7feff0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 msgr2=0x7ff000113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 0x7ff000113e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feff400c800 tx=0x7feff4004470 comp rx=0 tx=0).stop 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 shutdown_connections 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fefe806cc20 0x7fefe806f0d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff000101fa0 0x7ff0001138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 --2- 172.21.15.19:0/3224548487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff000102940 0x7ff000113e10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 >> 172.21.15.19:0/3224548487 conn(0x7ff0000f7b70 msgr2=0x7ff000065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:12.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 shutdown_connections 2024-02-22T16:17:12.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.766+0000 7ff0069e1700 1 -- 172.21.15.19:0/3224548487 wait complete. 2024-02-22T16:17:12.740 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-02-22T16:17:12.751 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:12.751 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-02-22T16:17:12.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-02-22T16:17:12.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 -- 172.21.15.19:0/906549467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc102720 msgr2=0x7f89fc0fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 --2- 172.21.15.19:0/906549467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc102720 0x7f89fc0fbd90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f89f8004940 tx=0x7f89f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 -- 172.21.15.19:0/906549467 shutdown_connections 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 --2- 172.21.15.19:0/906549467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc102720 0x7f89fc0fbd90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 --2- 172.21.15.19:0/906549467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc101d80 0x7f89fc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 -- 172.21.15.19:0/906549467 >> 172.21.15.19:0/906549467 conn(0x7f89fc072930 msgr2=0x7f89fc0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 -- 172.21.15.19:0/906549467 shutdown_connections 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 -- 172.21.15.19:0/906549467 wait complete. 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 Processor -- start 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.902+0000 7f8a0436d700 1 -- start start 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a0436d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 0x7f89fc0fda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a0436d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc102720 0x7f89fc0fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a0436d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0fe480 con 0x7f89fc101d80 2024-02-22T16:17:12.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a0436d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0fe5c0 con 0x7f89fc102720 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 0x7f89fc0fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 0x7f89fc0fda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46224/0 (socket says 172.21.15.19:46224) 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 -- 172.21.15.19:0/1832302637 learned_addr learned my addr 172.21.15.19:0/1832302637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a01908700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc102720 0x7f89fc0fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 -- 172.21.15.19:0/1832302637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc102720 msgr2=0x7f89fc0fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc102720 0x7f89fc0fdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 -- 172.21.15.19:0/1832302637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8005040 con 0x7f89fc101d80 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a02109700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 0x7f89fc0fda00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f89ec002a50 tx=0x7f89ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89ec007cc0 con 0x7f89fc101d80 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89ec00f040 con 0x7f89fc101d80 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc119b00 con 0x7f89fc101d80 2024-02-22T16:17:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89ec009540 con 0x7f89fc101d80 2024-02-22T16:17:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.903+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc119e50 con 0x7f89fc101d80 2024-02-22T16:17:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.904+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc064a80 con 0x7f89fc101d80 2024-02-22T16:17:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.905+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f89ec0096a0 con 0x7f89fc101d80 2024-02-22T16:17:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.905+0000 7f89f2ffd700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f89e806cc20 0x7f89e806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:12.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.905+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f89ec08bd80 con 0x7f89fc101d80 2024-02-22T16:17:12.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.905+0000 7f8a01908700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f89e806cc20 0x7f89e806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:12.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.906+0000 7f8a01908700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f89e806cc20 0x7f89e806f0d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b3b0 tx=0x7f89f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:12.910+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f89ec055db0 con 0x7f89fc101d80 2024-02-22T16:17:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.159+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f89fc11a040 con 0x7f89fc101d80 2024-02-22T16:17:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.160+0000 7f89f2ffd700 1 -- 172.21.15.19:0/1832302637 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v43) v1 ==== 107+0+4821 (secure 0 0 0) 0x7f89ec0593d0 con 0x7f89fc101d80 2024-02-22T16:17:13.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:13.133 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T15:58:54.154609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f89e806cc20 msgr2=0x7f89e806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f89e806cc20 0x7f89e806f0d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b3b0 tx=0x7f89f8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 msgr2=0x7f89fc0fda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 0x7f89fc0fda00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f89ec002a50 tx=0x7f89ec0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 shutdown_connections 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f89e806cc20 0x7f89e806f0d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89fc101d80 0x7f89fc0fda00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 --2- 172.21.15.19:0/1832302637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc102720 0x7f89fc0fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 >> 172.21.15.19:0/1832302637 conn(0x7f89fc072930 msgr2=0x7f89fc064f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.162+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 shutdown_connections 2024-02-22T16:17:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.163+0000 7f8a0436d700 1 -- 172.21.15.19:0/1832302637 wait complete. 2024-02-22T16:17:13.137 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-02-22T16:17:13.148 DEBUG:tasks.fs:max_mds not reduced in epoch 35 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-22T16:17:13.148 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-02-22T16:17:13.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-02-22T16:17:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- 172.21.15.19:0/3443910169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580102730 msgr2=0x7f75800fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 --2- 172.21.15.19:0/3443910169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580102730 0x7f75800fbda0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7570004940 tx=0x7f7570009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- 172.21.15.19:0/3443910169 shutdown_connections 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 --2- 172.21.15.19:0/3443910169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580102730 0x7f75800fbda0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 --2- 172.21.15.19:0/3443910169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7580101d90 0x7f7580102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- 172.21.15.19:0/3443910169 >> 172.21.15.19:0/3443910169 conn(0x7f75800f7940 msgr2=0x7f75800f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- 172.21.15.19:0/3443910169 shutdown_connections 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- 172.21.15.19:0/3443910169 wait complete. 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 Processor -- start 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- start start 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 0x7f75800fda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7580102730 0x7f75800fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:13.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800fe490 con 0x7f7580101d90 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.299+0000 7f75883c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75800fe600 con 0x7f7580102730 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 0x7f75800fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 0x7f75800fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46232/0 (socket says 172.21.15.19:46232) 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 -- 172.21.15.19:0/1649935696 learned_addr learned my addr 172.21.15.19:0/1649935696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758595b700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7580102730 0x7f75800fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 -- 172.21.15.19:0/1649935696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7580102730 msgr2=0x7f75800fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7580102730 0x7f75800fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 -- 172.21.15.19:0/1649935696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570005040 con 0x7f7580101d90 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f758615c700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 0x7f75800fda10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f757c002a80 tx=0x7f757c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757c007cc0 con 0x7f7580101d90 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f757c00f040 con 0x7f7580101d90 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757c00da60 con 0x7f7580101d90 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7580119a80 con 0x7f7580101d90 2024-02-22T16:17:13.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.300+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7580119dd0 con 0x7f7580101d90 2024-02-22T16:17:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.301+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75800641f0 con 0x7f7580101d90 2024-02-22T16:17:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.302+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f757c00dbc0 con 0x7f7580101d90 2024-02-22T16:17:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.302+0000 7f7576ffd700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f756c06cc20 0x7f756c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:13.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.302+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f757c08bc10 con 0x7f7580101d90 2024-02-22T16:17:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.302+0000 7f758595b700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f756c06cc20 0x7f756c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:13.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.303+0000 7f758595b700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f756c06cc20 0x7f756c06f0d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f757000b3b0 tx=0x7f7570009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:13.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.307+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f757c055c40 con 0x7f7580101d90 2024-02-22T16:17:13.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:13 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/488634883' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-22T16:17:13.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:13 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3224548487' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-22T16:17:13.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:13 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1832302637' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-22T16:17:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.558+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f758011a010 con 0x7f7580101d90 2024-02-22T16:17:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.559+0000 7f7576ffd700 1 -- 172.21.15.19:0/1649935696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v43) v1 ==== 107+0+4820 (secure 0 0 0) 0x7f757c059260 con 0x7f7580101d90 2024-02-22T16:17:13.532 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:13.532 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:14:56.377998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f756c06cc20 msgr2=0x7f756c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f756c06cc20 0x7f756c06f0d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f757000b3b0 tx=0x7f7570009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 msgr2=0x7f75800fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 0x7f75800fda10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f757c002a80 tx=0x7f757c0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 shutdown_connections 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f756c06cc20 0x7f756c06f0d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7580101d90 0x7f75800fda10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 --2- 172.21.15.19:0/1649935696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7580102730 0x7f75800fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 >> 172.21.15.19:0/1649935696 conn(0x7f75800f7940 msgr2=0x7f7580065c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 shutdown_connections 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.561+0000 7f75883c0700 1 -- 172.21.15.19:0/1649935696 wait complete. 2024-02-22T16:17:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-02-22T16:17:13.547 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-02-22T16:17:13.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-02-22T16:17:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 -- 172.21.15.19:0/1299166491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4102720 msgr2=0x7f1fe40fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 --2- 172.21.15.19:0/1299166491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4102720 0x7f1fe40fbd90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0004940 tx=0x7f1fe0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 -- 172.21.15.19:0/1299166491 shutdown_connections 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 --2- 172.21.15.19:0/1299166491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4102720 0x7f1fe40fbd90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 --2- 172.21.15.19:0/1299166491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4101d80 0x7f1fe4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 -- 172.21.15.19:0/1299166491 >> 172.21.15.19:0/1299166491 conn(0x7f1fe4072930 msgr2=0x7f1fe40f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 -- 172.21.15.19:0/1299166491 shutdown_connections 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.698+0000 7f1feb306700 1 -- 172.21.15.19:0/1299166491 wait complete. 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1feb306700 1 Processor -- start 2024-02-22T16:17:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1feb306700 1 -- start start 2024-02-22T16:17:13.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1feb306700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 0x7f1fe4112590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1feb306700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102720 0x7f1fe4112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1feb306700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4113120 con 0x7f1fe4101d80 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1feb306700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe410c660 con 0x7f1fe4102720 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe90a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 0x7f1fe4112590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe90a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 0x7f1fe4112590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46262/0 (socket says 172.21.15.19:46262) 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe90a2700 1 -- 172.21.15.19:0/1371175469 learned_addr learned my addr 172.21.15.19:0/1371175469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe88a1700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102720 0x7f1fe4112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe90a2700 1 -- 172.21.15.19:0/1371175469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102720 msgr2=0x7f1fe4112ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe90a2700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102720 0x7f1fe4112ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe90a2700 1 -- 172.21.15.19:0/1371175469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe0005040 con 0x7f1fe4101d80 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.699+0000 7f1fe88a1700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102720 0x7f1fe4112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:13.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.700+0000 7f1fe90a2700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 0x7f1fe4112590 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4002a50 tx=0x7f1fd40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.700+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd4007cc0 con 0x7f1fe4101d80 2024-02-22T16:17:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.700+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe410c940 con 0x7f1fe4101d80 2024-02-22T16:17:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.700+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1fd400f040 con 0x7f1fe4101d80 2024-02-22T16:17:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.700+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd400da70 con 0x7f1fe4101d80 2024-02-22T16:17:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.700+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe410cf00 con 0x7f1fe4101d80 2024-02-22T16:17:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.701+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe4104250 con 0x7f1fe4101d80 2024-02-22T16:17:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.701+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1fd400dc10 con 0x7f1fe4101d80 2024-02-22T16:17:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.702+0000 7f1fd9ffb700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1fd006cc30 0x7f1fd006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.702+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1fd408b7b0 con 0x7f1fe4101d80 2024-02-22T16:17:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.702+0000 7f1fe88a1700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1fd006cc30 0x7f1fd006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.702+0000 7f1fe88a1700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1fd006cc30 0x7f1fd006f0e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000b3b0 tx=0x7f1fe0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.707+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f1fd40591f0 con 0x7f1fe4101d80 2024-02-22T16:17:13.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:13 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/488634883' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-22T16:17:13.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:13 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3224548487' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-22T16:17:13.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:13 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1832302637' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-22T16:17:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.954+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f1fe410db70 con 0x7f1fe4101d80 2024-02-22T16:17:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.954+0000 7f1fd9ffb700 1 -- 172.21.15.19:0/1371175469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v43) v1 ==== 107+0+4843 (secure 0 0 0) 0x7f1fd405f030 con 0x7f1fe4101d80 2024-02-22T16:17:13.928 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:13.928 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:14:56.381946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1fd006cc30 msgr2=0x7f1fd006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1fd006cc30 0x7f1fd006f0e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000b3b0 tx=0x7f1fe0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 msgr2=0x7f1fe4112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 0x7f1fe4112590 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4002a50 tx=0x7f1fd40046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 shutdown_connections 2024-02-22T16:17:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f1fd006cc30 0x7f1fd006f0e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe4101d80 0x7f1fe4112590 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 --2- 172.21.15.19:0/1371175469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102720 0x7f1fe4112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 >> 172.21.15.19:0/1371175469 conn(0x7f1fe4072930 msgr2=0x7f1fe4073d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 shutdown_connections 2024-02-22T16:17:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:13.957+0000 7f1feb306700 1 -- 172.21.15.19:0/1371175469 wait complete. 2024-02-22T16:17:13.933 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 37 2024-02-22T16:17:13.942 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-02-22T16:17:13.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 -- 172.21.15.19:0/718247817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 msgr2=0x7f60f8102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/718247817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f8102380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60e8004940 tx=0x7f60e8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 -- 172.21.15.19:0/718247817 shutdown_connections 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/718247817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60f8102950 0x7f60f80fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/718247817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f8102380 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 -- 172.21.15.19:0/718247817 >> 172.21.15.19:0/718247817 conn(0x7f60f80f7b20 msgr2=0x7f60f80f9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 -- 172.21.15.19:0/718247817 shutdown_connections 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.092+0000 7f60fd7ba700 1 -- 172.21.15.19:0/718247817 wait complete. 2024-02-22T16:17:14.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60fd7ba700 1 Processor -- start 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60fd7ba700 1 -- start start 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60fd7ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f81138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60fd7ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60f8102950 0x7f60f8113e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60fd7ba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8114450 con 0x7f60f8101fb0 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60fd7ba700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f810d990 con 0x7f60f8102950 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f81138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60f6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f81138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46276/0 (socket says 172.21.15.19:46276) 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.093+0000 7f60f6ffd700 1 -- 172.21.15.19:0/3584050620 learned_addr learned my addr 172.21.15.19:0/3584050620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60f67fc700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60f8102950 0x7f60f8113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60f6ffd700 1 -- 172.21.15.19:0/3584050620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60f8102950 msgr2=0x7f60f8113e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60f6ffd700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60f8102950 0x7f60f8113e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60f6ffd700 1 -- 172.21.15.19:0/3584050620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8005040 con 0x7f60f8101fb0 2024-02-22T16:17:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60f6ffd700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f81138c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009fd0 tx=0x7f60e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e800e040 con 0x7f60f8101fb0 2024-02-22T16:17:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60e800bc50 con 0x7f60f8101fb0 2024-02-22T16:17:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f810dc10 con 0x7f60f8101fb0 2024-02-22T16:17:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e8013ea0 con 0x7f60f8101fb0 2024-02-22T16:17:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.094+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f810e1d0 con 0x7f60f8101fb0 2024-02-22T16:17:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.095+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f8064ed0 con 0x7f60f8101fb0 2024-02-22T16:17:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.096+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f60e8013420 con 0x7f60f8101fb0 2024-02-22T16:17:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.096+0000 7f60df7fe700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f60e006cc30 0x7f60e006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.096+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f60e808d5c0 con 0x7f60f8101fb0 2024-02-22T16:17:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.096+0000 7f60f67fc700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f60e006cc30 0x7f60e006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.096+0000 7f60f67fc700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f60e006cc30 0x7f60e006f0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f60f810f0f0 tx=0x7f60ec009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.100+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f60e805b000 con 0x7f60f8101fb0 2024-02-22T16:17:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.352+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f60f810eea0 con 0x7f60f8101fb0 2024-02-22T16:17:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.352+0000 7f60df7fe700 1 -- 172.21.15.19:0/3584050620 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v43) v1 ==== 107+0+4842 (secure 0 0 0) 0x7f60e805ab90 con 0x7f60f8101fb0 2024-02-22T16:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:14:57.383267+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f60e006cc30 msgr2=0x7f60e006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f60e006cc30 0x7f60e006f0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f60f810f0f0 tx=0x7f60ec009ee0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 msgr2=0x7f60f81138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f81138c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009fd0 tx=0x7f60e8009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 shutdown_connections 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f60e006cc30 0x7f60e006f0e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8101fb0 0x7f60f81138c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 --2- 172.21.15.19:0/3584050620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60f8102950 0x7f60f8113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 >> 172.21.15.19:0/3584050620 conn(0x7f60f80f7b20 msgr2=0x7f60f8105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:14.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 shutdown_connections 2024-02-22T16:17:14.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.355+0000 7f60fd7ba700 1 -- 172.21.15.19:0/3584050620 wait complete. 2024-02-22T16:17:14.329 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 38 2024-02-22T16:17:14.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-02-22T16:17:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 -- 172.21.15.19:0/3011943677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 msgr2=0x7f08100fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 --2- 172.21.15.19:0/3011943677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fe280 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0800004940 tx=0x7f0800009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 -- 172.21.15.19:0/3011943677 shutdown_connections 2024-02-22T16:17:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 --2- 172.21.15.19:0/3011943677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08100fe850 0x7f0810106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 --2- 172.21.15.19:0/3011943677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fe280 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 -- 172.21.15.19:0/3011943677 >> 172.21.15.19:0/3011943677 conn(0x7f0810072930 msgr2=0x7f08100f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 -- 172.21.15.19:0/3011943677 shutdown_connections 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.490+0000 7f0816530700 1 -- 172.21.15.19:0/3011943677 wait complete. 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0816530700 1 Processor -- start 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0816530700 1 -- start start 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0816530700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0816530700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08100fe850 0x7f08100fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0816530700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100fc7f0 con 0x7f08100fdeb0 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0816530700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100fc930 con 0x7f08100fe850 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f080ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f080ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46292/0 (socket says 172.21.15.19:46292) 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f080ffff700 1 -- 172.21.15.19:0/2584799023 learned_addr learned my addr 172.21.15.19:0/2584799023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f0807fff700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08100fe850 0x7f08100fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f080ffff700 1 -- 172.21.15.19:0/2584799023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08100fe850 msgr2=0x7f08100fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f080ffff700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08100fe850 0x7f08100fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.491+0000 7f080ffff700 1 -- 172.21.15.19:0/2584799023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800005040 con 0x7f08100fdeb0 2024-02-22T16:17:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.492+0000 7f080ffff700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fbd70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0800009150 tx=0x7f0800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.492+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080000e040 con 0x7f08100fdeb0 2024-02-22T16:17:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.492+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0800013420 con 0x7f08100fdeb0 2024-02-22T16:17:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.492+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08100fcbb0 con 0x7f08100fdeb0 2024-02-22T16:17:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.492+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080001c630 con 0x7f08100fdeb0 2024-02-22T16:17:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.492+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0810119d70 con 0x7f08100fdeb0 2024-02-22T16:17:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.493+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08101045e0 con 0x7f08100fdeb0 2024-02-22T16:17:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.493+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0800013970 con 0x7f08100fdeb0 2024-02-22T16:17:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.494+0000 7f080d7fa700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f07fc06cc20 0x7f07fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.494+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f080008d350 con 0x7f08100fdeb0 2024-02-22T16:17:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.494+0000 7f0807fff700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f07fc06cc20 0x7f07fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.494+0000 7f0807fff700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f07fc06cc20 0x7f07fc06f0d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f08100fd560 tx=0x7f07f8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:14.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.498+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f0800057380 con 0x7f08100fdeb0 2024-02-22T16:17:14.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:14 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1649935696' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-22T16:17:14.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:14 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/1371175469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-22T16:17:14.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:14 smithi019 ceph-mon[173579]: pgmap v720: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:14.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:14 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1649935696' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-22T16:17:14.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:14 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/1371175469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-22T16:17:14.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:14 smithi062 ceph-mon[106861]: pgmap v720: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.750+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f08100fd340 con 0x7f08100fdeb0 2024-02-22T16:17:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.750+0000 7f080d7fa700 1 -- 172.21.15.19:0/2584799023 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v43) v1 ==== 107+0+4845 (secure 0 0 0) 0x7f080005a9a0 con 0x7f08100fdeb0 2024-02-22T16:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:15:03.261995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:resolve","state_seq":236,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f07fc06cc20 msgr2=0x7f07fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f07fc06cc20 0x7f07fc06f0d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f08100fd560 tx=0x7f07f8009ee0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 msgr2=0x7f08100fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fbd70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0800009150 tx=0x7f0800009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 shutdown_connections 2024-02-22T16:17:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f07fc06cc20 0x7f07fc06f0d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08100fdeb0 0x7f08100fbd70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 --2- 172.21.15.19:0/2584799023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08100fe850 0x7f08100fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 >> 172.21.15.19:0/2584799023 conn(0x7f0810072930 msgr2=0x7f0810101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 shutdown_connections 2024-02-22T16:17:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.753+0000 7f0816530700 1 -- 172.21.15.19:0/2584799023 wait complete. 2024-02-22T16:17:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-02-22T16:17:14.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-02-22T16:17:14.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/3495235511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fde70 msgr2=0x7fb2d40fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/3495235511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fde70 0x7fb2d40fe240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4004940 tx=0x7fb2c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:14.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/3495235511 shutdown_connections 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/3495235511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fe810 0x7fb2d4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/3495235511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fde70 0x7fb2d40fe240 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/3495235511 >> 172.21.15.19:0/3495235511 conn(0x7fb2d4072930 msgr2=0x7fb2d40f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/3495235511 shutdown_connections 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/3495235511 wait complete. 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 Processor -- start 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.889+0000 7fb2d9f9d700 1 -- start start 2024-02-22T16:17:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d9f9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fde70 0x7fb2d4113670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d9f9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 0x7fb2d4113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d9f9d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4114240 con 0x7fb2d40fe810 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d9f9d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d410d740 con 0x7fb2d40fde70 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 0x7fb2d4113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 0x7fb2d4113bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46316/0 (socket says 172.21.15.19:46316) 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 -- 172.21.15.19:0/364772953 learned_addr learned my addr 172.21.15.19:0/364772953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d37fe700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fde70 0x7fb2d4113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 -- 172.21.15.19:0/364772953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fde70 msgr2=0x7fb2d4113670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:14.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fde70 0x7fb2d4113670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 -- 172.21.15.19:0/364772953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4005040 con 0x7fb2d40fe810 2024-02-22T16:17:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d37fe700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fde70 0x7fb2d4113670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-22T16:17:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2cbfff700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 0x7fb2d4113bf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc002a50 tx=0x7fb2bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2bc007cc0 con 0x7fb2d40fe810 2024-02-22T16:17:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2bc00f040 con 0x7fb2d40fe810 2024-02-22T16:17:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d410d9d0 con 0x7fb2d40fe810 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2bc009540 con 0x7fb2d40fe810 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.890+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d410df10 con 0x7fb2d40fe810 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.891+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d4104540 con 0x7fb2d40fe810 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.892+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb2bc0096a0 con 0x7fb2d40fe810 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.892+0000 7fb2d0ff9700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb2c006cc30 0x7fb2c006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.892+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb2bc08be80 con 0x7fb2d40fe810 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.892+0000 7fb2d37fe700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb2c006cc30 0x7fb2c006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.893+0000 7fb2d37fe700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb2c006cc30 0x7fb2c006f0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400b3b0 tx=0x7fb2c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:14.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:14.897+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fb2bc0598c0 con 0x7fb2d40fe810 2024-02-22T16:17:15.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.149+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7fb2d4064a80 con 0x7fb2d40fe810 2024-02-22T16:17:15.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.149+0000 7fb2d0ff9700 1 -- 172.21.15.19:0/364772953 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v43) v1 ==== 107+0+4848 (secure 0 0 0) 0x7fb2bc059450 con 0x7fb2d40fe810 2024-02-22T16:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:15:05.299889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:resolve","state_seq":242,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:resolve","state_seq":236,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.151+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb2c006cc30 msgr2=0x7fb2c006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.151+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb2c006cc30 0x7fb2c006f0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400b3b0 tx=0x7fb2c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.151+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 msgr2=0x7fb2d4113bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.151+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 0x7fb2d4113bf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc002a50 tx=0x7fb2bc0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 shutdown_connections 2024-02-22T16:17:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fb2c006cc30 0x7fb2c006f0e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d40fde70 0x7fb2d4113670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 --2- 172.21.15.19:0/364772953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2d40fe810 0x7fb2d4113bf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 >> 172.21.15.19:0/364772953 conn(0x7fb2d4072930 msgr2=0x7fb2d41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 shutdown_connections 2024-02-22T16:17:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.152+0000 7fb2d9f9d700 1 -- 172.21.15.19:0/364772953 wait complete. 2024-02-22T16:17:15.128 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 40 2024-02-22T16:17:15.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-02-22T16:17:15.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 -- 172.21.15.19:0/1962008536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 msgr2=0x7f865c0fc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 --2- 172.21.15.19:0/1962008536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c0fc020 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f864c004940 tx=0x7f864c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 -- 172.21.15.19:0/1962008536 shutdown_connections 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 --2- 172.21.15.19:0/1962008536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 0x7f865c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 --2- 172.21.15.19:0/1962008536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c0fc020 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 -- 172.21.15.19:0/1962008536 >> 172.21.15.19:0/1962008536 conn(0x7f865c0f7940 msgr2=0x7f865c0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 -- 172.21.15.19:0/1962008536 shutdown_connections 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 -- 172.21.15.19:0/1962008536 wait complete. 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.289+0000 7f8662988700 1 Processor -- start 2024-02-22T16:17:15.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f8662988700 1 -- start start 2024-02-22T16:17:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f8662988700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c115730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:15.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f8662988700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 0x7f865c115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f8662988700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1162c0 con 0x7f865c0fbc50 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f8662988700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c10f800 con 0x7f865c0fc560 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c115730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c115730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46336/0 (socket says 172.21.15.19:46336) 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 -- 172.21.15.19:0/2562458130 learned_addr learned my addr 172.21.15.19:0/2562458130 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865b7fe700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 0x7f865c115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 -- 172.21.15.19:0/2562458130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 msgr2=0x7f865c115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 0x7f865c115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 -- 172.21.15.19:0/2562458130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c005040 con 0x7f865c0fbc50 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865b7fe700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 0x7f865c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.290+0000 7f865bfff700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c115730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f864c000c00 tx=0x7f864c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.291+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864c021070 con 0x7f865c0fbc50 2024-02-22T16:17:15.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.291+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f864c01c400 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.291+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c10fa80 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.291+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864c00e070 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.291+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c110040 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.291+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c1069f0 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.292+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f864c01c570 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.292+0000 7f8658ff9700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f864806cc30 0x7f864806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.293+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f864c08d2a0 con 0x7f865c0fbc50 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.293+0000 7f865b7fe700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f864806cc30 0x7f864806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.293+0000 7f865b7fe700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f864806cc30 0x7f864806f0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8644004600 tx=0x7f864400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:15.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.297+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f864c05ad60 con 0x7f865c0fbc50 2024-02-22T16:17:15.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:15 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3584050620' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-22T16:17:15.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:15 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2584799023' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-22T16:17:15.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:15 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/364772953' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-02-22T16:17:15.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.546+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f865c110d10 con 0x7f865c0fbc50 2024-02-22T16:17:15.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.547+0000 7f8658ff9700 1 -- 172.21.15.19:0/2562458130 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v43) v1 ==== 107+0+4852 (secure 0 0 0) 0x7f864c01c820 con 0x7f865c0fbc50 2024-02-22T16:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:15:06.303605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":243,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:reconnect","state_seq":237,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:15.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f864806cc30 msgr2=0x7f864806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f864806cc30 0x7f864806f0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8644004600 tx=0x7f864400b040 comp rx=0 tx=0).stop 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 msgr2=0x7f865c115730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c115730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f864c000c00 tx=0x7f864c009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 shutdown_connections 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f864806cc30 0x7f864806f0e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c0fbc50 0x7f865c115730 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 --2- 172.21.15.19:0/2562458130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f865c0fc560 0x7f865c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.549+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 >> 172.21.15.19:0/2562458130 conn(0x7f865c0f7940 msgr2=0x7f865c0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.550+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 shutdown_connections 2024-02-22T16:17:15.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.550+0000 7f8662988700 1 -- 172.21.15.19:0/2562458130 wait complete. 2024-02-22T16:17:15.524 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 41 2024-02-22T16:17:15.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 42 2024-02-22T16:17:15.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 -- 172.21.15.19:0/1826247610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 msgr2=0x7f9acc102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/1826247610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc102150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0004940 tx=0x7f9ac0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 -- 172.21.15.19:0/1826247610 shutdown_connections 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/1826247610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc102720 0x7f9acc0fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/1826247610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc102150 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 -- 172.21.15.19:0/1826247610 >> 172.21.15.19:0/1826247610 conn(0x7f9acc072930 msgr2=0x7f9acc0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 -- 172.21.15.19:0/1826247610 shutdown_connections 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 -- 172.21.15.19:0/1826247610 wait complete. 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 Processor -- start 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.685+0000 7f9ad3b63700 1 -- start start 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad3b63700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc10e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad3b63700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc102720 0x7f9acc10ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:15.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad3b63700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc10f180 con 0x7f9acc101d80 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad3b63700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc10f2c0 con 0x7f9acc102720 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc10e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc10e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46358/0 (socket says 172.21.15.19:46358) 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 -- 172.21.15.19:0/918864350 learned_addr learned my addr 172.21.15.19:0/918864350 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad10fe700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc102720 0x7f9acc10ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 -- 172.21.15.19:0/918864350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc102720 msgr2=0x7f9acc10ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc102720 0x7f9acc10ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 -- 172.21.15.19:0/918864350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0005040 con 0x7f9acc101d80 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad18ff700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc10e700 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0008d80 tx=0x7f9ac0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ac000e030 con 0x7f9acc101d80 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ac0004290 con 0x7f9acc101d80 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc10f540 con 0x7f9acc101d80 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ac001ce60 con 0x7f9acc101d80 2024-02-22T16:17:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.686+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc117b70 con 0x7f9acc101d80 2024-02-22T16:17:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.687+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc064a80 con 0x7f9acc101d80 2024-02-22T16:17:15.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.688+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9ac001c690 con 0x7f9acc101d80 2024-02-22T16:17:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.688+0000 7f9abe7fc700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ab806cbc0 0x7f9ab806f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.688+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9ac008da00 con 0x7f9acc101d80 2024-02-22T16:17:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.688+0000 7f9ad10fe700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ab806cbc0 0x7f9ab806f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:15.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.689+0000 7f9ad10fe700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ab806cbc0 0x7f9ab806f070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8004600 tx=0x7f9ac800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:15.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.693+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f9ac0057a30 con 0x7f9acc101d80 2024-02-22T16:17:15.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:15 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3584050620' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-22T16:17:15.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:15 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2584799023' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-22T16:17:15.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:15 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/364772953' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-02-22T16:17:15.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.944+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7f9acc108d40 con 0x7f9acc101d80 2024-02-22T16:17:15.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.945+0000 7f9abe7fc700 1 -- 172.21.15.19:0/918864350 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v43) v1 ==== 107+0+4846 (secure 0 0 0) 0x7f9ac005b050 con 0x7f9acc101d80 2024-02-22T16:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44333,"name":"cephfs.smithi062.oyfagg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6825/2401270415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2401270415},{"type":"v1","addr":"172.21.15.62:6825","nonce":2401270415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":44365,"name":"cephfs.smithi062.eyhruo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6827/1657799059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1657799059},{"type":"v1","addr":"172.21.15.62:6827","nonce":1657799059}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-22T15:47:22.185528+0000","modified":"2024-02-22T16:15:07.310424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34392,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34392":{"gid":34392,"name":"cephfs.smithi019.lkzblb","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":244,"addr":"172.21.15.19:6827/2225716389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2225716389},{"type":"v1","addr":"172.21.15.19:6827","nonce":2225716389}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi019.fesnwx","rank":1,"incarnation":37,"state":"up:rejoin","state_seq":238,"addr":"172.21.15.19:6829/3128614024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3128614024},{"type":"v1","addr":"172.21.15.19:6829","nonce":3128614024}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-22T16:17:15.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ab806cbc0 msgr2=0x7f9ab806f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ab806cbc0 0x7f9ab806f070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8004600 tx=0x7f9ac800b040 comp rx=0 tx=0).stop 2024-02-22T16:17:15.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 msgr2=0x7f9acc10e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc10e700 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0008d80 tx=0x7f9ac0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 shutdown_connections 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f9ab806cbc0 0x7f9ab806f070 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9acc101d80 0x7f9acc10e700 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 --2- 172.21.15.19:0/918864350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc102720 0x7f9acc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 >> 172.21.15.19:0/918864350 conn(0x7f9acc072930 msgr2=0x7f9acc073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 shutdown_connections 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:15.947+0000 7f9ad3b63700 1 -- 172.21.15.19:0/918864350 wait complete. 2024-02-22T16:17:15.921 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-02-22T16:17:15.933 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-22T16:17:15.943 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-22T16:17:15.943 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T16:17:15.944 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-22T16:17:15.963 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T16:17:15.963 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-22T16:17:16.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-22T16:17:16.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 -- 172.21.15.19:0/3215107050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fe760 msgr2=0x7f072c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 --2- 172.21.15.19:0/3215107050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fe760 0x7f072c106e30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0720004940 tx=0x7f0720009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 -- 172.21.15.19:0/3215107050 shutdown_connections 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 --2- 172.21.15.19:0/3215107050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fe760 0x7f072c106e30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 --2- 172.21.15.19:0/3215107050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c0fddc0 0x7f072c0fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 -- 172.21.15.19:0/3215107050 >> 172.21.15.19:0/3215107050 conn(0x7f072c0f7880 msgr2=0x7f072c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 -- 172.21.15.19:0/3215107050 shutdown_connections 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.210+0000 7f0731a9e700 1 -- 172.21.15.19:0/3215107050 wait complete. 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f0731a9e700 1 Processor -- start 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f0731a9e700 1 -- start start 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f0731a9e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 0x7f072c1157b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f0731a9e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c0fe760 0x7f072c115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f0731a9e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c116340 con 0x7f072c0fddc0 2024-02-22T16:17:16.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f0731a9e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c10f880 con 0x7f072c0fe760 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.211+0000 7f072b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 0x7f072c1157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 0x7f072c1157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46378/0 (socket says 172.21.15.19:46378) 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072b7fe700 1 -- 172.21.15.19:0/4070688755 learned_addr learned my addr 172.21.15.19:0/4070688755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072affd700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c0fe760 0x7f072c115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072b7fe700 1 -- 172.21.15.19:0/4070688755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c0fe760 msgr2=0x7f072c115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072b7fe700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c0fe760 0x7f072c115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072b7fe700 1 -- 172.21.15.19:0/4070688755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720005040 con 0x7f072c0fddc0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f072b7fe700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 0x7f072c1157b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f071c002a50 tx=0x7f071c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071c007cc0 con 0x7f072c0fddc0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c10fb60 con 0x7f072c0fddc0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f071c00f040 con 0x7f072c0fddc0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071c009540 con 0x7f072c0fddc0 2024-02-22T16:17:16.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.212+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c1100a0 con 0x7f072c0fddc0 2024-02-22T16:17:16.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.213+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c1068f0 con 0x7f072c0fddc0 2024-02-22T16:17:16.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.214+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f071c0096a0 con 0x7f072c0fddc0 2024-02-22T16:17:16.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.214+0000 7f0713fff700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f071406cc20 0x7f071406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.214+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f071c08be80 con 0x7f072c0fddc0 2024-02-22T16:17:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.214+0000 7f072affd700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f071406cc20 0x7f071406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.215+0000 7f072affd700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f071406cc20 0x7f071406f0d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f072c110fe0 tx=0x7f0720009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:16.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.219+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f071c0598c0 con 0x7f072c0fddc0 2024-02-22T16:17:16.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.426+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f072c110d90 con 0x7f072c0fddc0 2024-02-22T16:17:16.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.427+0000 7f0713fff700 1 -- 172.21.15.19:0/4070688755 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16490 (secure 0 0 0) 0x7f071c059450 con 0x7f072c0fddc0 2024-02-22T16:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:59:50.275353+0000","last_up_change":"2024-02-22T15:58:51.952345+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-22T15:47:20.868526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-22T15:47:21.608692+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6803","nonce":2998470349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6805","nonce":2998470349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6809","nonce":2998470349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6807","nonce":2998470349}]},"public_addr":"172.21.15.19:6803/2998470349","cluster_addr":"172.21.15.19:6805/2998470349","heartbeat_back_addr":"172.21.15.19:6809/2998470349","heartbeat_front_addr":"172.21.15.19:6807/2998470349","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6811","nonce":1876741852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6813","nonce":1876741852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6817","nonce":1876741852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6815","nonce":1876741852}]},"public_addr":"172.21.15.19:6811/1876741852","cluster_addr":"172.21.15.19:6813/1876741852","heartbeat_back_addr":"172.21.15.19:6817/1876741852","heartbeat_front_addr":"172.21.15.19:6815/1876741852","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6819","nonce":947220918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6821","nonce":947220918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6825","nonce":947220918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6823","nonce":947220918}]},"public_addr":"172.21.15.19:6819/947220918","cluster_addr":"172.21.15.19:6821/947220918","heartbeat_back_addr":"172.21.15.19:6825/947220918","heartbeat_front_addr":"172.21.15.19:6823/947220918","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6801","nonce":1420983624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6803","nonce":1420983624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6807","nonce":1420983624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6805","nonce":1420983624}]},"public_addr":"172.21.15.62:6801/1420983624","cluster_addr":"172.21.15.62:6803/1420983624","heartbeat_back_addr":"172.21.15.62:6807/1420983624","heartbeat_front_addr":"172.21.15.62:6805/1420983624","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6809","nonce":227974398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6811","nonce":227974398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6815","nonce":227974398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6813","nonce":227974398}]},"public_addr":"172.21.15.62:6809/227974398","cluster_addr":"172.21.15.62:6811/227974398","heartbeat_back_addr":"172.21.15.62:6815/227974398","heartbeat_front_addr":"172.21.15.62:6813/227974398","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6817","nonce":823340437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6819","nonce":823340437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6823","nonce":823340437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6821","nonce":823340437}]},"public_addr":"172.21.15.62:6817/823340437","cluster_addr":"172.21.15.62:6819/823340437","heartbeat_back_addr":"172.21.15.62:6823/823340437","heartbeat_front_addr":"172.21.15.62:6821/823340437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6829/3100392983":"2024-02-23T15:58:54.154098+0000","172.21.15.19:6828/3100392983":"2024-02-23T15:58:54.154098+0000","172.21.15.62:0/3742623644":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6800/236956964":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6800/3932892700":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.62:0/2157627582":"2024-02-23T15:51:10.906421+0000","172.21.15.62:6825/1888029196":"2024-02-23T15:58:54.154118+0000","172.21.15.19:6801/3932892700":"2024-02-23T15:54:33.072657+0000","172.21.15.19:6827/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.19:6801/236956964":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6826/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.62:6829/2665490935":"2024-02-23T15:51:10.906421+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/576195047":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/163225331":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.62:0/1007979954":"2024-02-23T15:51:10.906421+0000","172.21.15.62:0/1428056285":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/2452913904":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1542879988":"2024-02-23T15:49:33.669856+0000","172.21.15.62:6828/2665490935":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/1375214472":"2024-02-23T15:49:33.669856+0000","172.21.15.19:0/3432676624":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/1503676495":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/809216990":"2024-02-23T15:54:33.072657+0000","172.21.15.62:6824/1888029196":"2024-02-23T15:58:54.154118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f071406cc20 msgr2=0x7f071406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f071406cc20 0x7f071406f0d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f072c110fe0 tx=0x7f0720009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 msgr2=0x7f072c1157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 0x7f072c1157b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f071c002a50 tx=0x7f071c0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 shutdown_connections 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f071406cc20 0x7f071406f0d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f072c0fddc0 0x7f072c1157b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 --2- 172.21.15.19:0/4070688755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f072c0fe760 0x7f072c115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.429+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 >> 172.21.15.19:0/4070688755 conn(0x7f072c0f7880 msgr2=0x7f072c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.430+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 shutdown_connections 2024-02-22T16:17:16.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.430+0000 7f0731a9e700 1 -- 172.21.15.19:0/4070688755 wait complete. 2024-02-22T16:17:16.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-22T15:44:43.417715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-22T15:47:20.868526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-22T15:47:21.608692+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-22T16:17:16.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-22T16:17:16.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:16 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2562458130' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-02-22T16:17:16.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:16 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/918864350' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-02-22T16:17:16.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:16 smithi019 ceph-mon[173579]: pgmap v721: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 -- 172.21.15.19:0/3489707288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 msgr2=0x7f358c0fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 --2- 172.21.15.19:0/3489707288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c0fbd90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3588004940 tx=0x7f3588009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 -- 172.21.15.19:0/3489707288 shutdown_connections 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 --2- 172.21.15.19:0/3489707288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c0fbd90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 --2- 172.21.15.19:0/3489707288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f358c101d80 0x7f358c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 -- 172.21.15.19:0/3489707288 >> 172.21.15.19:0/3489707288 conn(0x7f358c0f7950 msgr2=0x7f358c0f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:16.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 -- 172.21.15.19:0/3489707288 shutdown_connections 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.568+0000 7f3594610700 1 -- 172.21.15.19:0/3489707288 wait complete. 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3594610700 1 Processor -- start 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3594610700 1 -- start start 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3594610700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f358c101d80 0x7f358c112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3594610700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c112ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3594610700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1131f0 con 0x7f358c102720 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3594610700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c10c6f0 con 0x7f358c101d80 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c112ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c112ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46394/0 (socket says 172.21.15.19:46394) 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 -- 172.21.15.19:0/2305977059 learned_addr learned my addr 172.21.15.19:0/2305977059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f35923ac700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f358c101d80 0x7f358c112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 -- 172.21.15.19:0/2305977059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f358c101d80 msgr2=0x7f358c112620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f358c101d80 0x7f358c112620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 -- 172.21.15.19:0/2305977059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588005040 con 0x7f358c102720 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.569+0000 7f3591bab700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c112ba0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3588009fd0 tx=0x7f3588009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.570+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358800e040 con 0x7f358c102720 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.570+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c10c920 con 0x7f358c102720 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.570+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f358800bc50 con 0x7f358c102720 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.570+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c10ce60 con 0x7f358c102720 2024-02-22T16:17:16.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.570+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358801c3f0 con 0x7f358c102720 2024-02-22T16:17:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.571+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c104250 con 0x7f358c102720 2024-02-22T16:17:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.571+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3588013420 con 0x7f358c102720 2024-02-22T16:17:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.572+0000 7f3582ffd700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f357806cc20 0x7f357806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.572+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f358808d4f0 con 0x7f358c102720 2024-02-22T16:17:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.572+0000 7f35923ac700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f357806cc20 0x7f357806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.572+0000 7f35923ac700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f357806cc20 0x7f357806f0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f357c0041d0 tx=0x7f357c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:16.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.576+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f358805af10 con 0x7f358c102720 2024-02-22T16:17:16.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:16 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2562458130' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-02-22T16:17:16.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:16 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/918864350' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-02-22T16:17:16.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:16 smithi062 ceph-mon[106861]: pgmap v721: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:16.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.778+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f358c064ed0 con 0x7f358c102720 2024-02-22T16:17:16.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.779+0000 7f3582ffd700 1 -- 172.21.15.19:0/2305977059 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f358805aaa0 con 0x7f358c102720 2024-02-22T16:17:16.752 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-02-22T16:17:16.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f357806cc20 msgr2=0x7f357806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f357806cc20 0x7f357806f0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f357c0041d0 tx=0x7f357c009040 comp rx=0 tx=0).stop 2024-02-22T16:17:16.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 msgr2=0x7f358c112ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c112ba0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3588009fd0 tx=0x7f3588009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 shutdown_connections 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f357806cc20 0x7f357806f0d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f358c101d80 0x7f358c112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 --2- 172.21.15.19:0/2305977059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f358c102720 0x7f358c112ba0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 >> 172.21.15.19:0/2305977059 conn(0x7f358c0f7950 msgr2=0x7f358c0f95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 shutdown_connections 2024-02-22T16:17:16.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.781+0000 7f3594610700 1 -- 172.21.15.19:0/2305977059 wait complete. 2024-02-22T16:17:16.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-22T16:17:16.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/1938004375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec102e80 msgr2=0x7ff1ec103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/1938004375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec102e80 0x7ff1ec103250 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0004940 tx=0x7ff1e0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/1938004375 shutdown_connections 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/1938004375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec103820 0x7ff1ec0fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/1938004375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec102e80 0x7ff1ec103250 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/1938004375 >> 172.21.15.19:0/1938004375 conn(0x7ff1ec0f89d0 msgr2=0x7ff1ec0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/1938004375 shutdown_connections 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.921+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/1938004375 wait complete. 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1f0e4e700 1 Processor -- start 2024-02-22T16:17:16.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1f0e4e700 1 -- start start 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1f0e4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec102e80 0x7ff1ec113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1f0e4e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 0x7ff1ec113be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1f0e4e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec114230 con 0x7ff1ec103820 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1f0e4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec10d730 con 0x7ff1ec102e80 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 0x7ff1ec113be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1e9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 0x7ff1ec113be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46416/0 (socket says 172.21.15.19:46416) 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1e9d9b700 1 -- 172.21.15.19:0/3502987739 learned_addr learned my addr 172.21.15.19:0/3502987739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:16.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1ea59c700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec102e80 0x7ff1ec113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1e9d9b700 1 -- 172.21.15.19:0/3502987739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec102e80 msgr2=0x7ff1ec113660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1e9d9b700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec102e80 0x7ff1ec113660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.922+0000 7ff1e9d9b700 1 -- 172.21.15.19:0/3502987739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0005040 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.923+0000 7ff1e9d9b700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 0x7ff1ec113be0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc002a50 tx=0x7ff1dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.923+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1dc007cc0 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.923+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec10d9c0 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.923+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1dc00f040 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.923+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1dc00d920 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.923+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec10df00 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.924+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec100190 con 0x7ff1ec103820 2024-02-22T16:17:16.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.924+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff1dc009680 con 0x7ff1ec103820 2024-02-22T16:17:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.925+0000 7ff1daffd700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff1d406cc30 0x7ff1d406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.925+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff1dc08bdf0 con 0x7ff1ec103820 2024-02-22T16:17:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.925+0000 7ff1ea59c700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff1d406cc30 0x7ff1d406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:16.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.925+0000 7ff1ea59c700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff1d406cc30 0x7ff1d406f0e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0fc460 tx=0x7ff1e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:16.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:16.929+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff1dc059830 con 0x7ff1ec103820 2024-02-22T16:17:17.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.133+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff1ec0ffbb0 con 0x7ff1ec103820 2024-02-22T16:17:17.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.134+0000 7ff1daffd700 1 -- 172.21.15.19:0/3502987739 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff1dc0593c0 con 0x7ff1ec103820 2024-02-22T16:17:17.107 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T16:17:17.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff1d406cc30 msgr2=0x7ff1d406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff1d406cc30 0x7ff1d406f0e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0fc460 tx=0x7ff1e0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:17.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 msgr2=0x7ff1ec113be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 0x7ff1ec113be0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc002a50 tx=0x7ff1dc004600 comp rx=0 tx=0).stop 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 shutdown_connections 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff1d406cc30 0x7ff1d406f0e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec102e80 0x7ff1ec113660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 --2- 172.21.15.19:0/3502987739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec103820 0x7ff1ec113be0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 >> 172.21.15.19:0/3502987739 conn(0x7ff1ec0f89d0 msgr2=0x7ff1ec0fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 shutdown_connections 2024-02-22T16:17:17.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.136+0000 7ff1f0e4e700 1 -- 172.21.15.19:0/3502987739 wait complete. 2024-02-22T16:17:17.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 -- 172.21.15.19:0/3671036427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4103820 msgr2=0x7f19d40fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 --2- 172.21.15.19:0/3671036427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4103820 0x7f19d40fce90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19c4004940 tx=0x7f19c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 -- 172.21.15.19:0/3671036427 shutdown_connections 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 --2- 172.21.15.19:0/3671036427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4103820 0x7f19d40fce90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 --2- 172.21.15.19:0/3671036427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d4102e80 0x7f19d4103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 -- 172.21.15.19:0/3671036427 >> 172.21.15.19:0/3671036427 conn(0x7f19d40f89d0 msgr2=0x7f19d40fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 -- 172.21.15.19:0/3671036427 shutdown_connections 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 -- 172.21.15.19:0/3671036427 wait complete. 2024-02-22T16:17:17.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.275+0000 7f19da24c700 1 Processor -- start 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19da24c700 1 -- start start 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19da24c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 0x7f19d41135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19da24c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d4103820 0x7f19d4113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19da24c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4114160 con 0x7f19d4102e80 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19da24c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d410d6a0 con 0x7f19d4103820 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 0x7f19d41135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 0x7f19d41135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46432/0 (socket says 172.21.15.19:46432) 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 -- 172.21.15.19:0/3510199606 learned_addr learned my addr 172.21.15.19:0/3510199606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d2ffd700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d4103820 0x7f19d4113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 -- 172.21.15.19:0/3510199606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d4103820 msgr2=0x7f19d4113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d4103820 0x7f19d4113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 -- 172.21.15.19:0/3510199606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4005040 con 0x7f19d4102e80 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19d37fe700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 0x7f19d41135d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f19c800eb40 tx=0x7f19c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:17.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.276+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c8007cc0 con 0x7f19d4102e80 2024-02-22T16:17:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.277+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19c800f040 con 0x7f19d4102e80 2024-02-22T16:17:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.277+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d410d920 con 0x7f19d4102e80 2024-02-22T16:17:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.277+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c8009570 con 0x7f19d4102e80 2024-02-22T16:17:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.277+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d410dee0 con 0x7f19d4102e80 2024-02-22T16:17:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.278+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4105350 con 0x7f19d4102e80 2024-02-22T16:17:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.278+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f19c80096d0 con 0x7f19d4102e80 2024-02-22T16:17:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.278+0000 7f19bbfff700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19bc06cc20 0x7f19bc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.278+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19c808bd40 con 0x7f19d4102e80 2024-02-22T16:17:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.279+0000 7f19d2ffd700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19bc06cc20 0x7f19bc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.279+0000 7f19d2ffd700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19bc06cc20 0x7f19bc06f0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19d410ee70 tx=0x7f19c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:17.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.283+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f19c8059830 con 0x7f19d4102e80 2024-02-22T16:17:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.485+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f19d410ebb0 con 0x7f19d4102e80 2024-02-22T16:17:17.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.485+0000 7f19bbfff700 1 -- 172.21.15.19:0/3510199606 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f19c80593c0 con 0x7f19d4102e80 2024-02-22T16:17:17.459 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19bc06cc20 msgr2=0x7f19bc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19bc06cc20 0x7f19bc06f0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19d410ee70 tx=0x7f19c4009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 msgr2=0x7f19d41135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 0x7f19d41135d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f19c800eb40 tx=0x7f19c800b040 comp rx=0 tx=0).stop 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 shutdown_connections 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f19bc06cc20 0x7f19bc06f0d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19d4102e80 0x7f19d41135d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 --2- 172.21.15.19:0/3510199606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19d4103820 0x7f19d4113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 >> 172.21.15.19:0/3510199606 conn(0x7f19d40f89d0 msgr2=0x7f19d40fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 shutdown_connections 2024-02-22T16:17:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.488+0000 7f19da24c700 1 -- 172.21.15.19:0/3510199606 wait complete. 2024-02-22T16:17:17.472 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:17 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/4070688755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T16:17:17.472 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:17 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2305977059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T16:17:17.472 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:17 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3502987739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T16:17:17.473 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-22T16:17:17.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 -- 172.21.15.19:0/85980630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 msgr2=0x7f7ecc1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/85980630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc1021e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc004a90 tx=0x7f7ebc009dd0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 -- 172.21.15.19:0/85980630 shutdown_connections 2024-02-22T16:17:17.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/85980630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ecc0fc5c0 0x7f7ecc0fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/85980630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc1021e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 -- 172.21.15.19:0/85980630 >> 172.21.15.19:0/85980630 conn(0x7f7ecc072350 msgr2=0x7f7ecc072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 -- 172.21.15.19:0/85980630 shutdown_connections 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.623+0000 7f7ed22ba700 1 -- 172.21.15.19:0/85980630 wait complete. 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ed22ba700 1 Processor -- start 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ed22ba700 1 -- start start 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ed22ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ecc0fc5c0 0x7f7ecc1158a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ed22ba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ed22ba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc116430 con 0x7f7ecc101e10 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ed22ba700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc10f970 con 0x7f7ecc0fc5c0 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecb7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60360/0 (socket says 172.21.15.19:60360) 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecb7fe700 1 -- 172.21.15.19:0/2362339414 learned_addr learned my addr 172.21.15.19:0/2362339414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecbfff700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ecc0fc5c0 0x7f7ecc1158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:17.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecb7fe700 1 -- 172.21.15.19:0/2362339414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ecc0fc5c0 msgr2=0x7f7ecc1158a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecb7fe700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ecc0fc5c0 0x7f7ecc1158a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.624+0000 7f7ecb7fe700 1 -- 172.21.15.19:0/2362339414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc004650 con 0x7f7ecc101e10 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.625+0000 7f7ecb7fe700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc115de0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4002a50 tx=0x7f7eb40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.625+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb4007cc0 con 0x7f7ecc101e10 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.625+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7eb400f040 con 0x7f7ecc101e10 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.625+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc10fc50 con 0x7f7ecc101e10 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.625+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb400d940 con 0x7f7ecc101e10 2024-02-22T16:17:17.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.625+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc110190 con 0x7f7ecc101e10 2024-02-22T16:17:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.626+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc0641f0 con 0x7f7ecc101e10 2024-02-22T16:17:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.626+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7eb400daa0 con 0x7f7ecc101e10 2024-02-22T16:17:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.627+0000 7f7ec8ff9700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eb806cc30 0x7f7eb806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.627+0000 7f7ecbfff700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eb806cc30 0x7f7eb806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.627+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7eb408bdd0 con 0x7f7ecc101e10 2024-02-22T16:17:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.627+0000 7f7ecbfff700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eb806cc30 0x7f7eb806f0e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc000c00 tx=0x7f7ebc016040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:17.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.631+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f7eb4059770 con 0x7f7ecc101e10 2024-02-22T16:17:17.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:17 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/4070688755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T16:17:17.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:17 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2305977059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T16:17:17.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:17 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3502987739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T16:17:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.852+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ecc1103b0 con 0x7f7ecc101e10 2024-02-22T16:17:17.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.852+0000 7f7ec8ff9700 1 -- 172.21.15.19:0/2362339414 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2306 (secure 0 0 0) 0x7f7eb4059300 con 0x7f7ecc101e10 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6829/3100392983 2024-02-23T15:58:54.154098+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6828/3100392983 2024-02-23T15:58:54.154098+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/3742623644 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2141011198 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/189187168 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/236956964 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3932892700 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:17.826 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/947631679 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/2157627582 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6825/1888029196 2024-02-23T15:58:54.154118+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3932892700 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6827/2235854865 2024-02-23T15:47:37.015439+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/236956964 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6826/2235854865 2024-02-23T15:47:37.015439+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6829/2665490935 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2711018613 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2794582578 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:17.827 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1312244757 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3545757717 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/576195047 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3545757717 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/163225331 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/863136881 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2711018613 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1196920319 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1788498096 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/1007979954 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/1428056285 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/680289579 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/142659012 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2452913904 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:17.828 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1788498096 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1542879988 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6828/2665490935 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1375214472 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3432676624 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1503676495 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/809216990 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:17.829 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6824/1888029196 2024-02-23T15:58:54.154118+0000 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eb806cc30 msgr2=0x7f7eb806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eb806cc30 0x7f7eb806f0e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc000c00 tx=0x7f7ebc016040 comp rx=0 tx=0).stop 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 msgr2=0x7f7ecc115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc115de0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4002a50 tx=0x7f7eb40046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 shutdown_connections 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f7eb806cc30 0x7f7eb806f0e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ecc0fc5c0 0x7f7ecc1158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 --2- 172.21.15.19:0/2362339414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ecc101e10 0x7f7ecc115de0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 >> 172.21.15.19:0/2362339414 conn(0x7f7ecc072350 msgr2=0x7f7ecc0fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 shutdown_connections 2024-02-22T16:17:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:17.855+0000 7f7ed22ba700 1 -- 172.21.15.19:0/2362339414 wait complete. 2024-02-22T16:17:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:listed 39 entries 2024-02-22T16:17:17.840 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-02-22T16:17:17.840 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-22T16:17:17.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 -- 172.21.15.19:0/3241215713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390102dd0 msgr2=0x7fe3901031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 --2- 172.21.15.19:0/3241215713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390102dd0 0x7fe3901031a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe380009ce0 tx=0x7fe380004d10 comp rx=0 tx=0).stop 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 -- 172.21.15.19:0/3241215713 shutdown_connections 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 --2- 172.21.15.19:0/3241215713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390103770 0x7fe390064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 --2- 172.21.15.19:0/3241215713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390102dd0 0x7fe3901031a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 -- 172.21.15.19:0/3241215713 >> 172.21.15.19:0/3241215713 conn(0x7fe390072ba0 msgr2=0x7fe390072fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 -- 172.21.15.19:0/3241215713 shutdown_connections 2024-02-22T16:17:18.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 -- 172.21.15.19:0/3241215713 wait complete. 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.050+0000 7fe3961d4700 1 Processor -- start 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe3961d4700 1 -- start start 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe3961d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390102dd0 0x7fe390113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe3961d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 0x7fe390113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe3961d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3901142f0 con 0x7fe390103770 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe3961d4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39010d830 con 0x7fe390102dd0 2024-02-22T16:17:18.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 0x7fe390113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 0x7fe390113ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60378/0 (socket says 172.21.15.19:60378) 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 -- 172.21.15.19:0/172070936 learned_addr learned my addr 172.21.15.19:0/172070936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38f7fe700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390102dd0 0x7fe390113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 -- 172.21.15.19:0/172070936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390102dd0 msgr2=0x7fe390113760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390102dd0 0x7fe390113760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 -- 172.21.15.19:0/172070936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009940 con 0x7fe390103770 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe38effd700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 0x7fe390113ca0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe37c002a50 tx=0x7fe37c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.051+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37c007cc0 con 0x7fe390103770 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.052+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39010db10 con 0x7fe390103770 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.052+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe37c00f040 con 0x7fe390103770 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.052+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37c00d920 con 0x7fe390103770 2024-02-22T16:17:18.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.052+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39010e0d0 con 0x7fe390103770 2024-02-22T16:17:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.052+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3901000e0 con 0x7fe390103770 2024-02-22T16:17:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.053+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe37c009680 con 0x7fe390103770 2024-02-22T16:17:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.053+0000 7fe387fff700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe37406cc20 0x7fe37406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.053+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe37c08bdf0 con 0x7fe390103770 2024-02-22T16:17:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.053+0000 7fe38f7fe700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe37406cc20 0x7fe37406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.054+0000 7fe38f7fe700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe37406cc20 0x7fe37406f0d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe380003400 tx=0x7fe380004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:18.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.058+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe37c059830 con 0x7fe390103770 2024-02-22T16:17:18.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.265+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe39010ed70 con 0x7fe390103770 2024-02-22T16:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.266+0000 7fe387fff700 1 -- 172.21.15.19:0/172070936 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16490 (secure 0 0 0) 0x7fe37c0593c0 con 0x7fe390103770 2024-02-22T16:17:18.239 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-02-22T16:17:18.240 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":86,"fsid":"8db8e6fa-d198-11ee-95bf-87774f69a715","created":"2024-02-22T15:40:02.431182+0000","modified":"2024-02-22T15:59:50.275353+0000","last_up_change":"2024-02-22T15:58:51.952345+0000","last_in_change":"2024-02-22T15:45:40.011612+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-22T15:44:43.417715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-22T15:47:20.868526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-22T15:47:21.608692+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8dce22ce-c824-43c8-b665-33df9393d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6803","nonce":2998470349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6805","nonce":2998470349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6809","nonce":2998470349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2998470349},{"type":"v1","addr":"172.21.15.19:6807","nonce":2998470349}]},"public_addr":"172.21.15.19:6803/2998470349","cluster_addr":"172.21.15.19:6805/2998470349","heartbeat_back_addr":"172.21.15.19:6809/2998470349","heartbeat_front_addr":"172.21.15.19:6807/2998470349","state":["exists","up"]},{"osd":1,"uuid":"7fc2b0a0-987e-48ca-895f-f773652ceb17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6811","nonce":1876741852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6813","nonce":1876741852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6817","nonce":1876741852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1876741852},{"type":"v1","addr":"172.21.15.19:6815","nonce":1876741852}]},"public_addr":"172.21.15.19:6811/1876741852","cluster_addr":"172.21.15.19:6813/1876741852","heartbeat_back_addr":"172.21.15.19:6817/1876741852","heartbeat_front_addr":"172.21.15.19:6815/1876741852","state":["exists","up"]},{"osd":2,"uuid":"ae746a74-93bc-4d46-a925-f5824de285ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6819","nonce":947220918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6821","nonce":947220918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6825","nonce":947220918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":947220918},{"type":"v1","addr":"172.21.15.19:6823","nonce":947220918}]},"public_addr":"172.21.15.19:6819/947220918","cluster_addr":"172.21.15.19:6821/947220918","heartbeat_back_addr":"172.21.15.19:6825/947220918","heartbeat_front_addr":"172.21.15.19:6823/947220918","state":["exists","up"]},{"osd":3,"uuid":"e8e553ca-1a18-42a9-b3d2-fef6669ef7d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6801","nonce":1420983624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6803","nonce":1420983624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6807","nonce":1420983624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1420983624},{"type":"v1","addr":"172.21.15.62:6805","nonce":1420983624}]},"public_addr":"172.21.15.62:6801/1420983624","cluster_addr":"172.21.15.62:6803/1420983624","heartbeat_back_addr":"172.21.15.62:6807/1420983624","heartbeat_front_addr":"172.21.15.62:6805/1420983624","state":["exists","up"]},{"osd":4,"uuid":"08c28eb8-43da-423f-a330-dc773450fc4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6809","nonce":227974398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6811","nonce":227974398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6815","nonce":227974398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":227974398},{"type":"v1","addr":"172.21.15.62:6813","nonce":227974398}]},"public_addr":"172.21.15.62:6809/227974398","cluster_addr":"172.21.15.62:6811/227974398","heartbeat_back_addr":"172.21.15.62:6815/227974398","heartbeat_front_addr":"172.21.15.62:6813/227974398","state":["exists","up"]},{"osd":5,"uuid":"4dafa93f-c3bb-4b9c-a5fb-00aa3a7988e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6817","nonce":823340437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6819","nonce":823340437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6823","nonce":823340437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":823340437},{"type":"v1","addr":"172.21.15.62:6821","nonce":823340437}]},"public_addr":"172.21.15.62:6817/823340437","cluster_addr":"172.21.15.62:6819/823340437","heartbeat_back_addr":"172.21.15.62:6823/823340437","heartbeat_front_addr":"172.21.15.62:6821/823340437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:43:40.706733+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:09.974865+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:44:39.854243+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:04.614702+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:31.117424+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-22T15:45:57.434473+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6829/3100392983":"2024-02-23T15:58:54.154098+0000","172.21.15.19:6828/3100392983":"2024-02-23T15:58:54.154098+0000","172.21.15.62:0/3742623644":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/2141011198":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/189187168":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6800/236956964":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6800/3932892700":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/947631679":"2024-02-23T15:41:10.803793+0000","172.21.15.62:0/2157627582":"2024-02-23T15:51:10.906421+0000","172.21.15.62:6825/1888029196":"2024-02-23T15:58:54.154118+0000","172.21.15.19:6801/3932892700":"2024-02-23T15:54:33.072657+0000","172.21.15.19:6827/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.19:6801/236956964":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6826/2235854865":"2024-02-23T15:47:37.015439+0000","172.21.15.62:6829/2665490935":"2024-02-23T15:51:10.906421+0000","172.21.15.19:6800/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/2794582578":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1312244757":"2024-02-23T15:42:20.890862+0000","172.21.15.19:6800/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/576195047":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6801/3545757717":"2024-02-23T15:41:10.803793+0000","172.21.15.19:0/163225331":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/863136881":"2024-02-23T15:41:10.803793+0000","172.21.15.19:6801/2711018613":"2024-02-23T15:40:36.639808+0000","172.21.15.19:0/1196920319":"2024-02-23T15:40:36.639808+0000","172.21.15.19:6801/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.62:0/1007979954":"2024-02-23T15:51:10.906421+0000","172.21.15.62:0/1428056285":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/680289579":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/142659012":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/2452913904":"2024-02-23T15:49:33.669856+0000","172.21.15.19:6800/1788498096":"2024-02-23T15:42:20.890862+0000","172.21.15.19:0/1542879988":"2024-02-23T15:49:33.669856+0000","172.21.15.62:6828/2665490935":"2024-02-23T15:51:10.906421+0000","172.21.15.19:0/1375214472":"2024-02-23T15:49:33.669856+0000","172.21.15.19:0/3432676624":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/1503676495":"2024-02-23T15:54:33.072657+0000","172.21.15.19:0/809216990":"2024-02-23T15:54:33.072657+0000","172.21.15.62:6824/1888029196":"2024-02-23T15:58:54.154118+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-22T16:17:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe37406cc20 msgr2=0x7fe37406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe37406cc20 0x7fe37406f0d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe380003400 tx=0x7fe380004d10 comp rx=0 tx=0).stop 2024-02-22T16:17:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 msgr2=0x7fe390113ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 0x7fe390113ca0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe37c002a50 tx=0x7fe37c004600 comp rx=0 tx=0).stop 2024-02-22T16:17:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 shutdown_connections 2024-02-22T16:17:18.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe37406cc20 0x7fe37406f0d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe390102dd0 0x7fe390113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 --2- 172.21.15.19:0/172070936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe390103770 0x7fe390113ca0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 >> 172.21.15.19:0/172070936 conn(0x7fe390072ba0 msgr2=0x7fe3900faca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:18.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 shutdown_connections 2024-02-22T16:17:18.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.268+0000 7fe3961d4700 1 -- 172.21.15.19:0/172070936 wait complete. 2024-02-22T16:17:18.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-22T15:44:43.417715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-22T15:47:20.868526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-22T15:47:21.608692+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-22T16:17:18.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-22T16:17:18.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.406+0000 7f2411e5b700 1 -- 172.21.15.19:0/2474595562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c103830 msgr2=0x7f240c0fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.406+0000 7f2411e5b700 1 --2- 172.21.15.19:0/2474595562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c103830 0x7f240c0fcea0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2400004940 tx=0x7f2400009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:18.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.406+0000 7f2411e5b700 1 -- 172.21.15.19:0/2474595562 shutdown_connections 2024-02-22T16:17:18.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.406+0000 7f2411e5b700 1 --2- 172.21.15.19:0/2474595562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c103830 0x7f240c0fcea0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.406+0000 7f2411e5b700 1 --2- 172.21.15.19:0/2474595562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f240c102e90 0x7f240c103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.406+0000 7f2411e5b700 1 -- 172.21.15.19:0/2474595562 >> 172.21.15.19:0/2474595562 conn(0x7f240c0f8a20 msgr2=0x7f240c0fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 -- 172.21.15.19:0/2474595562 shutdown_connections 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 -- 172.21.15.19:0/2474595562 wait complete. 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 Processor -- start 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 -- start start 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 0x7f240c1135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f240c103830 0x7f240c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c1141c0 con 0x7f240c102e90 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.407+0000 7f2411e5b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c10d6c0 con 0x7f240c103830 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 0x7f240c1135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 0x7f240c1135f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60390/0 (socket says 172.21.15.19:60390) 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 -- 172.21.15.19:0/3469798382 learned_addr learned my addr 172.21.15.19:0/3469798382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 -- 172.21.15.19:0/3469798382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f240c103830 msgr2=0x7f240c113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f240c103830 0x7f240c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 -- 172.21.15.19:0/3469798382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400005040 con 0x7f240c102e90 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f240b7fe700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 0x7f240c1135f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f23fc002a80 tx=0x7f23fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23fc007cc0 con 0x7f240c102e90 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f23fc00f040 con 0x7f240c102e90 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c10d950 con 0x7f240c102e90 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23fc009640 con 0x7f240c102e90 2024-02-22T16:17:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.408+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c10de90 con 0x7f240c102e90 2024-02-22T16:17:18.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.409+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c105360 con 0x7f240c102e90 2024-02-22T16:17:18.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.410+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f23fc0097a0 con 0x7f240c102e90 2024-02-22T16:17:18.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.410+0000 7f23f3fff700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f23f406cc20 0x7f23f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.410+0000 7f240affd700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f23f406cc20 0x7f23f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.410+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f23fc08b890 con 0x7f240c102e90 2024-02-22T16:17:18.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.411+0000 7f240affd700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f23f406cc20 0x7f23f406f0d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2400004f90 tx=0x7f2400009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:18.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.415+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7f23fc0592d0 con 0x7f240c102e90 2024-02-22T16:17:18.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:18 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3510199606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T16:17:18.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:18 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2362339414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T16:17:18.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:18.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:18 smithi019 ceph-mon[173579]: pgmap v722: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.622+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f240c109e60 con 0x7f240c102e90 2024-02-22T16:17:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.622+0000 7f23f3fff700 1 -- 172.21.15.19:0/3469798382 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23fc058e60 con 0x7f240c102e90 2024-02-22T16:17:18.596 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-02-22T16:17:18.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f23f406cc20 msgr2=0x7f23f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f23f406cc20 0x7f23f406f0d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2400004f90 tx=0x7f2400009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 msgr2=0x7f240c1135f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 0x7f240c1135f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f23fc002a80 tx=0x7f23fc0046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 shutdown_connections 2024-02-22T16:17:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7f23f406cc20 0x7f23f406f0d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c102e90 0x7f240c1135f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.624+0000 7f2411e5b700 1 --2- 172.21.15.19:0/3469798382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f240c103830 0x7f240c113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.625+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 >> 172.21.15.19:0/3469798382 conn(0x7f240c0f8a20 msgr2=0x7f240c0fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.625+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 shutdown_connections 2024-02-22T16:17:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.625+0000 7f2411e5b700 1 -- 172.21.15.19:0/3469798382 wait complete. 2024-02-22T16:17:18.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-22T16:17:18.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:18 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3510199606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T16:17:18.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:18 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2362339414' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T16:17:18.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:18.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:18 smithi062 ceph-mon[106861]: pgmap v722: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 -- 172.21.15.19:0/1229710262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0103820 msgr2=0x7ff6b00fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 --2- 172.21.15.19:0/1229710262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0103820 0x7ff6b00fce90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0004940 tx=0x7ff6a0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 -- 172.21.15.19:0/1229710262 shutdown_connections 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 --2- 172.21.15.19:0/1229710262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0103820 0x7ff6b00fce90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 --2- 172.21.15.19:0/1229710262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6b0102e80 0x7ff6b0103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 -- 172.21.15.19:0/1229710262 >> 172.21.15.19:0/1229710262 conn(0x7ff6b00f89d0 msgr2=0x7ff6b00fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 -- 172.21.15.19:0/1229710262 shutdown_connections 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 -- 172.21.15.19:0/1229710262 wait complete. 2024-02-22T16:17:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.762+0000 7ff6b6809700 1 Processor -- start 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6b6809700 1 -- start start 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6b6809700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 0x7ff6b0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6b6809700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6b0103820 0x7ff6b00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6b6809700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0070d10 con 0x7ff6b0102e80 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6b6809700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0070e50 con 0x7ff6b0103820 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 0x7ff6b0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 0x7ff6b0070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60402/0 (socket says 172.21.15.19:60402) 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 -- 172.21.15.19:0/887748663 learned_addr learned my addr 172.21.15.19:0/887748663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6af7fe700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6b0103820 0x7ff6b00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 -- 172.21.15.19:0/887748663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6b0103820 msgr2=0x7ff6b00707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6b0103820 0x7ff6b00707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 -- 172.21.15.19:0/887748663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0005040 con 0x7ff6b0102e80 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6affff700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 0x7ff6b0070290 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4002a50 tx=0x7ff6a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.763+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a4007cc0 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.764+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6a400f040 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.764+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a400d9c0 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.764+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b0119b10 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.764+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b0119e60 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.765+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b0105350 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.765+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff6a400db20 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.766+0000 7ff6acff9700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff69806cc20 0x7ff69806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.766+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff6a408bd30 con 0x7ff6b0102e80 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.766+0000 7ff6af7fe700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff69806cc20 0x7ff69806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:18.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.766+0000 7ff6af7fe700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff69806cc20 0x7ff69806f0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b3b0 tx=0x7ff6a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.770+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7ff6a4055cc0 con 0x7ff6b0102e80 2024-02-22T16:17:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.970+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff6b00641f0 con 0x7ff6b0102e80 2024-02-22T16:17:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.971+0000 7ff6acff9700 1 -- 172.21.15.19:0/887748663 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff6a40592e0 con 0x7ff6b0102e80 2024-02-22T16:17:18.944 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T16:17:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff69806cc20 msgr2=0x7ff69806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff69806cc20 0x7ff69806f0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b3b0 tx=0x7ff6a0009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 msgr2=0x7ff6b0070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 0x7ff6b0070290 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4002a50 tx=0x7ff6a40046b0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 shutdown_connections 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7ff69806cc20 0x7ff69806f0d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6b0102e80 0x7ff6b0070290 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 --2- 172.21.15.19:0/887748663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6b0103820 0x7ff6b00707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 >> 172.21.15.19:0/887748663 conn(0x7ff6b00f89d0 msgr2=0x7ff6b00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 shutdown_connections 2024-02-22T16:17:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:18.973+0000 7ff6b6809700 1 -- 172.21.15.19:0/887748663 wait complete. 2024-02-22T16:17:18.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-22T16:17:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 -- 172.21.15.19:0/302836023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 msgr2=0x7fedd80fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 --2- 172.21.15.19:0/302836023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd80fbda0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fedc4004940 tx=0x7fedc4009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 -- 172.21.15.19:0/302836023 shutdown_connections 2024-02-22T16:17:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 --2- 172.21.15.19:0/302836023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd80fbda0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 --2- 172.21.15.19:0/302836023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedd8101d90 0x7fedd8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 -- 172.21.15.19:0/302836023 >> 172.21.15.19:0/302836023 conn(0x7fedd80f7940 msgr2=0x7fedd80f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 -- 172.21.15.19:0/302836023 shutdown_connections 2024-02-22T16:17:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.110+0000 7feddec4e700 1 -- 172.21.15.19:0/302836023 wait complete. 2024-02-22T16:17:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.111+0000 7feddec4e700 1 Processor -- start 2024-02-22T16:17:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.111+0000 7feddec4e700 1 -- start start 2024-02-22T16:17:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7feddec4e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedd8101d90 0x7fedd81124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7feddec4e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd8112a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7feddec4e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd8113080 con 0x7fedd8102730 2024-02-22T16:17:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7feddec4e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd810c5c0 con 0x7fedd8101d90 2024-02-22T16:17:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd8112a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd8112a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60422/0 (socket says 172.21.15.19:60422) 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 -- 172.21.15.19:0/2523560170 learned_addr learned my addr 172.21.15.19:0/2523560170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7feddc9ea700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedd8101d90 0x7fedd81124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 -- 172.21.15.19:0/2523560170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedd8101d90 msgr2=0x7fedd81124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedd8101d90 0x7fedd81124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 -- 172.21.15.19:0/2523560170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4005040 con 0x7fedd8102730 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd7fff700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd8112a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009fd0 tx=0x7fedc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedc400e040 con 0x7fedd8102730 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd810c840 con 0x7fedd8102730 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fedc400bc50 con 0x7fedd8102730 2024-02-22T16:17:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.112+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedc401c3f0 con 0x7fedd8102730 2024-02-22T16:17:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.113+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd810ce00 con 0x7fedd8102730 2024-02-22T16:17:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.113+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd8104260 con 0x7fedd8102730 2024-02-22T16:17:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.114+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fedc4013420 con 0x7fedd8102730 2024-02-22T16:17:19.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.114+0000 7fedd57fa700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fedc806cc20 0x7fedc806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.114+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fedc408d470 con 0x7fedd8102730 2024-02-22T16:17:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.115+0000 7feddc9ea700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fedc806cc20 0x7fedc806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.115+0000 7feddc9ea700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fedc806cc20 0x7fedc806f0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009fd0 tx=0x7fedcc009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:19.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.119+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fedc405af30 con 0x7fedd8102730 2024-02-22T16:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.320+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fedd80641f0 con 0x7fedd8102730 2024-02-22T16:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.321+0000 7fedd57fa700 1 -- 172.21.15.19:0/2523560170 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fedc405aac0 con 0x7fedd8102730 2024-02-22T16:17:19.294 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-02-22T16:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fedc806cc20 msgr2=0x7fedc806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fedc806cc20 0x7fedc806f0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009fd0 tx=0x7fedcc009f60 comp rx=0 tx=0).stop 2024-02-22T16:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 msgr2=0x7fedd8112a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd8112a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009fd0 tx=0x7fedc4009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 shutdown_connections 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fedc806cc20 0x7fedc806f0d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedd8101d90 0x7fedd81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 --2- 172.21.15.19:0/2523560170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedd8102730 0x7fedd8112a30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 >> 172.21.15.19:0/2523560170 conn(0x7fedd80f7940 msgr2=0x7fedd80f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 shutdown_connections 2024-02-22T16:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.323+0000 7feddec4e700 1 -- 172.21.15.19:0/2523560170 wait complete. 2024-02-22T16:17:19.309 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-22T16:17:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 -- 172.21.15.19:0/1879570300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348101da0 msgr2=0x7fe348102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/1879570300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348101da0 0x7fe348102170 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe338004940 tx=0x7fe338009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 -- 172.21.15.19:0/1879570300 shutdown_connections 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/1879570300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe348102740 0x7fe3480fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/1879570300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348101da0 0x7fe348102170 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 -- 172.21.15.19:0/1879570300 >> 172.21.15.19:0/1879570300 conn(0x7fe3480f7950 msgr2=0x7fe3480f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 -- 172.21.15.19:0/1879570300 shutdown_connections 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.459+0000 7fe34ca7a700 1 -- 172.21.15.19:0/1879570300 wait complete. 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34ca7a700 1 Processor -- start 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34ca7a700 1 -- start start 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34ca7a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe348101da0 0x7fe34810e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34ca7a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 0x7fe34810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34ca7a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34810f1e0 con 0x7fe348102740 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34ca7a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34810f320 con 0x7fe348101da0 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 0x7fe34810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:19.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe345d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 0x7fe34810eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60428/0 (socket says 172.21.15.19:60428) 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe345d9b700 1 -- 172.21.15.19:0/3930921592 learned_addr learned my addr 172.21.15.19:0/3930921592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe34659c700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe348101da0 0x7fe34810e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe345d9b700 1 -- 172.21.15.19:0/3930921592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe348101da0 msgr2=0x7fe34810e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe345d9b700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe348101da0 0x7fe34810e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.460+0000 7fe345d9b700 1 -- 172.21.15.19:0/3930921592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338005040 con 0x7fe348102740 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.461+0000 7fe345d9b700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 0x7fe34810eca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe33000eb40 tx=0x7fe33000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.461+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe330007cc0 con 0x7fe348102740 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.461+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe33000f040 con 0x7fe348102740 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.461+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34810f5a0 con 0x7fe348102740 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.461+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33000c8f0 con 0x7fe348102740 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.461+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe348117b50 con 0x7fe348102740 2024-02-22T16:17:19.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.462+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3480ff0b0 con 0x7fe348102740 2024-02-22T16:17:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.462+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe3300090d0 con 0x7fe348102740 2024-02-22T16:17:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.463+0000 7fe33effd700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe33406cc30 0x7fe33406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-22T16:17:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.463+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe33008b9d0 con 0x7fe348102740 2024-02-22T16:17:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.463+0000 7fe34659c700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe33406cc30 0x7fe33406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-22T16:17:19.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.463+0000 7fe34659c700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe33406cc30 0x7fe33406f0e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe338009fd0 tx=0x7fe338009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-22T16:17:19.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.467+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182595 (secure 0 0 0) 0x7fe330055ab0 con 0x7fe348102740 2024-02-22T16:17:19.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:19 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/172070936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T16:17:19.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:19 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3469798382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T16:17:19.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:19 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/887748663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T16:17:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.688+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3480641f0 con 0x7fe348102740 2024-02-22T16:17:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.688+0000 7fe33effd700 1 -- 172.21.15.19:0/3930921592 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2306 (secure 0 0 0) 0x7fe330019920 con 0x7fe348102740 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6829/3100392983 2024-02-23T15:58:54.154098+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6828/3100392983 2024-02-23T15:58:54.154098+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/3742623644 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2141011198 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/189187168 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/236956964 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3932892700 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/947631679 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/2157627582 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6825/1888029196 2024-02-23T15:58:54.154118+0000 2024-02-22T16:17:19.662 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3932892700 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6827/2235854865 2024-02-23T15:47:37.015439+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/236956964 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6826/2235854865 2024-02-23T15:47:37.015439+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6829/2665490935 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2711018613 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2794582578 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1312244757 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3545757717 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/576195047 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3545757717 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/163225331 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/863136881 2024-02-23T15:41:10.803793+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2711018613 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1196920319 2024-02-23T15:40:36.639808+0000 2024-02-22T16:17:19.663 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1788498096 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/1007979954 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:0/1428056285 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/680289579 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/142659012 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2452913904 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1788498096 2024-02-23T15:42:20.890862+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1542879988 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6828/2665490935 2024-02-23T15:51:10.906421+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1375214472 2024-02-23T15:49:33.669856+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3432676624 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1503676495 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/809216990 2024-02-23T15:54:33.072657+0000 2024-02-22T16:17:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.62:6824/1888029196 2024-02-23T15:58:54.154118+0000 2024-02-22T16:17:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe33406cc30 msgr2=0x7fe33406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe33406cc30 0x7fe33406f0e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe338009fd0 tx=0x7fe338009e30 comp rx=0 tx=0).stop 2024-02-22T16:17:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 msgr2=0x7fe34810eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 0x7fe34810eca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe33000eb40 tx=0x7fe33000b040 comp rx=0 tx=0).stop 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 shutdown_connections 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:6800/1006240378,v1:172.21.15.19:6801/1006240378] conn(0x7fe33406cc30 0x7fe33406f0e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe348101da0 0x7fe34810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 --2- 172.21.15.19:0/3930921592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe348102740 0x7fe34810eca0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 >> 172.21.15.19:0/3930921592 conn(0x7fe3480f7950 msgr2=0x7fe3480f9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 shutdown_connections 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-02-22T16:17:19.691+0000 7fe34ca7a700 1 -- 172.21.15.19:0/3930921592 wait complete. 2024-02-22T16:17:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:listed 39 entries 2024-02-22T16:17:19.676 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi019.front.sepia.ceph.com... 2024-02-22T16:17:19.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-22T16:17:19.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-22T16:17:19.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:19 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/172070936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-22T16:17:19.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:19 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3469798382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-22T16:17:19.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:19 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/887748663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-22T16:17:19.713 INFO:teuthology.orchestra.run:waiting for 300 2024-02-22T16:17:20.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:20 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/2523560170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T16:17:20.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:20 smithi019 ceph-mon[173579]: from='client.? 172.21.15.19:0/3930921592' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T16:17:20.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:20 smithi019 ceph-mon[173579]: pgmap v723: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:20.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:20 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/2523560170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-22T16:17:20.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:20 smithi062 ceph-mon[106861]: from='client.? 172.21.15.19:0/3930921592' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-22T16:17:20.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:20 smithi062 ceph-mon[106861]: pgmap v723: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:22.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:22 smithi019 ceph-mon[173579]: pgmap v724: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:22.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:22 smithi062 ceph-mon[106861]: pgmap v724: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:25.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:25 smithi062 ceph-mon[106861]: pgmap v725: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:25.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:25 smithi019 ceph-mon[173579]: pgmap v725: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:27.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:27 smithi019 ceph-mon[173579]: pgmap v726: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:29.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:27 smithi062 ceph-mon[106861]: pgmap v726: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:29.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:29 smithi062 ceph-mon[106861]: pgmap v727: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:29.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:29 smithi019 ceph-mon[173579]: pgmap v727: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:31.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:31 smithi019 ceph-mon[173579]: pgmap v728: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:31.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:31 smithi062 ceph-mon[106861]: pgmap v728: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:32.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:32 smithi019 ceph-mon[173579]: pgmap v729: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:32.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:32 smithi062 ceph-mon[106861]: pgmap v729: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:33.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:33.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:34.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:34 smithi019 ceph-mon[173579]: pgmap v730: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:34.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:34 smithi062 ceph-mon[106861]: pgmap v730: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:36.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:36 smithi019 ceph-mon[173579]: pgmap v731: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:36.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:36 smithi062 ceph-mon[106861]: pgmap v731: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:39.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:39 smithi062 ceph-mon[106861]: pgmap v732: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:39.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:39 smithi019 ceph-mon[173579]: pgmap v732: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:41.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:41 smithi019 ceph-mon[173579]: pgmap v733: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:41.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:41 smithi062 ceph-mon[106861]: pgmap v733: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:43.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:43 smithi062 ceph-mon[106861]: pgmap v734: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:43.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:43 smithi019 ceph-mon[173579]: pgmap v734: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:45.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:45 smithi062 ceph-mon[106861]: pgmap v735: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:45.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:45 smithi019 ceph-mon[173579]: pgmap v735: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:46.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:46 smithi019 ceph-mon[173579]: pgmap v736: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:46.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:46 smithi062 ceph-mon[106861]: pgmap v736: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:48.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:48.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:48 smithi019 ceph-mon[173579]: pgmap v737: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:48.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:17:48.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:48 smithi062 ceph-mon[106861]: pgmap v737: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:50.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:50 smithi019 ceph-mon[173579]: pgmap v738: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:50.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:50 smithi062 ceph-mon[106861]: pgmap v738: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:52.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:52 smithi019 ceph-mon[173579]: pgmap v739: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:52.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:52 smithi062 ceph-mon[106861]: pgmap v739: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:54.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:54 smithi019 ceph-mon[173579]: pgmap v740: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:54.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:54 smithi062 ceph-mon[106861]: pgmap v740: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:56.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:56 smithi019 ceph-mon[173579]: pgmap v741: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:56.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:56 smithi062 ceph-mon[106861]: pgmap v741: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:17:59.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:17:59 smithi062 ceph-mon[106861]: pgmap v742: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:17:59.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:17:59 smithi019 ceph-mon[173579]: pgmap v742: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:01.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:01 smithi019 ceph-mon[173579]: pgmap v743: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:01.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:01 smithi062 ceph-mon[106861]: pgmap v743: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:03.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:03 smithi062 ceph-mon[106861]: pgmap v744: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:03.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:03.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:03 smithi019 ceph-mon[173579]: pgmap v744: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:03.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:05.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:05 smithi062 ceph-mon[106861]: pgmap v745: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:05.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:05 smithi019 ceph-mon[173579]: pgmap v745: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:06.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:18:06.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:18:06.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:18:06.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:06 smithi062 ceph-mon[106861]: pgmap v746: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:06.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:06 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:18:06.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:18:06.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:18:06.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:18:06.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:06 smithi019 ceph-mon[173579]: pgmap v746: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:06.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:06 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:18:09.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:09 smithi062 ceph-mon[106861]: pgmap v747: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:09.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:09 smithi019 ceph-mon[173579]: pgmap v747: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:11.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:11 smithi019 ceph-mon[173579]: pgmap v748: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:11.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:11 smithi062 ceph-mon[106861]: pgmap v748: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:13.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:13 smithi062 ceph-mon[106861]: pgmap v749: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:13.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:13 smithi019 ceph-mon[173579]: pgmap v749: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:14.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:14 smithi062 ceph-mon[106861]: pgmap v750: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:14.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:14 smithi019 ceph-mon[173579]: pgmap v750: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:16.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:16 smithi019 ceph-mon[173579]: pgmap v751: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:16.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:16 smithi062 ceph-mon[106861]: pgmap v751: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:18.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:18.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:18 smithi019 ceph-mon[173579]: pgmap v752: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:18.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:18.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:18 smithi062 ceph-mon[106861]: pgmap v752: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:21.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:21 smithi062 ceph-mon[106861]: pgmap v753: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:21.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:21 smithi019 ceph-mon[173579]: pgmap v753: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:23.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:23 smithi062 ceph-mon[106861]: pgmap v754: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:23.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:23 smithi019 ceph-mon[173579]: pgmap v754: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:24.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:24 smithi019 ceph-mon[173579]: pgmap v755: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:24.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:24 smithi062 ceph-mon[106861]: pgmap v755: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:26.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:26 smithi019 ceph-mon[173579]: pgmap v756: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:26.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:26 smithi062 ceph-mon[106861]: pgmap v756: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:29.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:29 smithi062 ceph-mon[106861]: pgmap v757: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:29.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:29 smithi019 ceph-mon[173579]: pgmap v757: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:30.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:30 smithi019 ceph-mon[173579]: pgmap v758: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:30.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:30 smithi062 ceph-mon[106861]: pgmap v758: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:32.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:32 smithi019 ceph-mon[173579]: pgmap v759: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:32.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:32 smithi062 ceph-mon[106861]: pgmap v759: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:33.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:33.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:34.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:34 smithi062 ceph-mon[106861]: pgmap v760: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:34.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:34 smithi019 ceph-mon[173579]: pgmap v760: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:36.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:36 smithi019 ceph-mon[173579]: pgmap v761: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:36.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:36 smithi062 ceph-mon[106861]: pgmap v761: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:39.449 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:39 smithi062 ceph-mon[106861]: pgmap v762: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:39.489 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:39 smithi019 ceph-mon[173579]: pgmap v762: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:40.487 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:40 smithi019 ceph-mon[173579]: pgmap v763: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:40.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:40 smithi062 ceph-mon[106861]: pgmap v763: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:42.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:42 smithi019 ceph-mon[173579]: pgmap v764: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:42.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:42 smithi062 ceph-mon[106861]: pgmap v764: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:44.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:44 smithi019 ceph-mon[173579]: pgmap v765: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:44.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:44 smithi062 ceph-mon[106861]: pgmap v765: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:46.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:46 smithi019 ceph-mon[173579]: pgmap v766: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:46.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:46 smithi062 ceph-mon[106861]: pgmap v766: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:48.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:48.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:48 smithi019 ceph-mon[173579]: pgmap v767: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:48.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:18:48.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:48 smithi062 ceph-mon[106861]: pgmap v767: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:50.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:50 smithi019 ceph-mon[173579]: pgmap v768: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:50.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:50 smithi062 ceph-mon[106861]: pgmap v768: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:52.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:52 smithi019 ceph-mon[173579]: pgmap v769: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:52.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:52 smithi062 ceph-mon[106861]: pgmap v769: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:54.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:54 smithi019 ceph-mon[173579]: pgmap v770: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:54.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:54 smithi062 ceph-mon[106861]: pgmap v770: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:56.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:56 smithi062 ceph-mon[106861]: pgmap v771: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:56.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:56 smithi019 ceph-mon[173579]: pgmap v771: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:18:59.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:18:59 smithi062 ceph-mon[106861]: pgmap v772: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:18:59.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:18:59 smithi019 ceph-mon[173579]: pgmap v772: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:00.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:00 smithi019 ceph-mon[173579]: pgmap v773: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:00.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:00 smithi062 ceph-mon[106861]: pgmap v773: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:02.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:02 smithi062 ceph-mon[106861]: pgmap v774: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:02.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:02 smithi019 ceph-mon[173579]: pgmap v774: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:03.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:03.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:04.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:04 smithi062 ceph-mon[106861]: pgmap v775: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:04.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:04 smithi019 ceph-mon[173579]: pgmap v775: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:06.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:06 smithi062 ceph-mon[106861]: pgmap v776: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:06.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:06 smithi019 ceph-mon[173579]: pgmap v776: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:07.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:07 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:19:07.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:07 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:19:07.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:07 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:19:07.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:07 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:19:07.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:07 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:19:07.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:07 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:19:07.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:07 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:19:07.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:07 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:19:09.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:09 smithi062 ceph-mon[106861]: pgmap v777: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:09.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:09 smithi019 ceph-mon[173579]: pgmap v777: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:10.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:10 smithi062 ceph-mon[106861]: pgmap v778: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:10.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:10 smithi019 ceph-mon[173579]: pgmap v778: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:14.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:14 smithi062 ceph-mon[106861]: pgmap v779: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:14.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:14 smithi019 ceph-mon[173579]: pgmap v779: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:16.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:16 smithi062 ceph-mon[106861]: pgmap v780: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:16.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:16 smithi019 ceph-mon[173579]: pgmap v780: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:17.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:17 smithi062 ceph-mon[106861]: pgmap v781: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:17.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:17 smithi019 ceph-mon[173579]: pgmap v781: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:18.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:18.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:20.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:20 smithi062 ceph-mon[106861]: pgmap v782: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:20.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:20 smithi019 ceph-mon[173579]: pgmap v782: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:21.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:21 smithi062 ceph-mon[106861]: pgmap v783: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:21.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:21 smithi019 ceph-mon[173579]: pgmap v783: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:24.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:24 smithi062 ceph-mon[106861]: pgmap v784: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:24.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:24 smithi019 ceph-mon[173579]: pgmap v784: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:25.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:25 smithi062 ceph-mon[106861]: pgmap v785: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:25.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:25 smithi019 ceph-mon[173579]: pgmap v785: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:27.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:27 smithi062 ceph-mon[106861]: pgmap v786: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:27.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:27 smithi019 ceph-mon[173579]: pgmap v786: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:30.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:30 smithi062 ceph-mon[106861]: pgmap v787: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:30.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:30 smithi019 ceph-mon[173579]: pgmap v787: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:31.450 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:31 smithi062 ceph-mon[106861]: pgmap v788: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:31.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:31 smithi019 ceph-mon[173579]: pgmap v788: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:33.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:33 smithi062 ceph-mon[106861]: pgmap v789: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:33.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:33.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:33 smithi019 ceph-mon[173579]: pgmap v789: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:33.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:35.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:35 smithi019 ceph-mon[173579]: pgmap v790: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:35.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:35 smithi062 ceph-mon[106861]: pgmap v790: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:37.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:37 smithi019 ceph-mon[173579]: pgmap v791: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:37.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:37 smithi062 ceph-mon[106861]: pgmap v791: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:40.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:40 smithi062 ceph-mon[106861]: pgmap v792: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:40.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:40 smithi019 ceph-mon[173579]: pgmap v792: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:41.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:41 smithi062 ceph-mon[106861]: pgmap v793: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:41.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:41 smithi019 ceph-mon[173579]: pgmap v793: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:44.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:44 smithi062 ceph-mon[106861]: pgmap v794: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:44.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:44 smithi019 ceph-mon[173579]: pgmap v794: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:45.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:45 smithi062 ceph-mon[106861]: pgmap v795: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:45.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:45 smithi019 ceph-mon[173579]: pgmap v795: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:47.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:47 smithi062 ceph-mon[106861]: pgmap v796: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:47.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:47 smithi019 ceph-mon[173579]: pgmap v796: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:48.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:48.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:19:49.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:49 smithi062 ceph-mon[106861]: pgmap v797: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:49.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:49 smithi019 ceph-mon[173579]: pgmap v797: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:51.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:51 smithi062 ceph-mon[106861]: pgmap v798: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:51.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:51 smithi019 ceph-mon[173579]: pgmap v798: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:54.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:54 smithi062 ceph-mon[106861]: pgmap v799: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:54.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:54 smithi019 ceph-mon[173579]: pgmap v799: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:55.448 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:55 smithi062 ceph-mon[106861]: pgmap v800: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:55.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:55 smithi019 ceph-mon[173579]: pgmap v800: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:57.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:57 smithi019 ceph-mon[173579]: pgmap v801: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:57.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:57 smithi062 ceph-mon[106861]: pgmap v801: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:19:59.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:19:59 smithi062 ceph-mon[106861]: pgmap v802: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:19:59.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:19:59 smithi019 ceph-mon[173579]: pgmap v802: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:00.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:00 smithi062 ceph-mon[106861]: overall HEALTH_OK 2024-02-22T16:20:00.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:00 smithi019 ceph-mon[173579]: overall HEALTH_OK 2024-02-22T16:20:01.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:01 smithi062 ceph-mon[106861]: pgmap v803: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:01.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:01 smithi019 ceph-mon[173579]: pgmap v803: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:03.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:03 smithi062 ceph-mon[106861]: pgmap v804: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:03.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:03.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:03 smithi019 ceph-mon[173579]: pgmap v804: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:03.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:05.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:05 smithi062 ceph-mon[106861]: pgmap v805: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:05.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:05 smithi019 ceph-mon[173579]: pgmap v805: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:07.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:07 smithi062 ceph-mon[106861]: pgmap v806: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:07.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:07 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:20:07.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:07 smithi019 ceph-mon[173579]: pgmap v806: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:07.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:07 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:20:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:20:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:20:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:20:08.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:20:08.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:20:08.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:20:09.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:09 smithi062 ceph-mon[106861]: pgmap v807: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:09.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:09 smithi019 ceph-mon[173579]: pgmap v807: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:11.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:11 smithi062 ceph-mon[106861]: pgmap v808: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:11.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:11 smithi019 ceph-mon[173579]: pgmap v808: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:13.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:13 smithi062 ceph-mon[106861]: pgmap v809: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:13.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:13 smithi019 ceph-mon[173579]: pgmap v809: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:15.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:15 smithi062 ceph-mon[106861]: pgmap v810: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:15.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:15 smithi019 ceph-mon[173579]: pgmap v810: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:17.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:17 smithi062 ceph-mon[106861]: pgmap v811: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:17.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:17 smithi019 ceph-mon[173579]: pgmap v811: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:18.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:18.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:19.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:19 smithi062 ceph-mon[106861]: pgmap v812: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:19.736 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:19 smithi019 ceph-mon[173579]: pgmap v812: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:21.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:21 smithi062 ceph-mon[106861]: pgmap v813: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:21.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:21 smithi019 ceph-mon[173579]: pgmap v813: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:23.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:23 smithi062 ceph-mon[106861]: pgmap v814: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:23.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:23 smithi019 ceph-mon[173579]: pgmap v814: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:25.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:25 smithi062 ceph-mon[106861]: pgmap v815: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:25.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:25 smithi019 ceph-mon[173579]: pgmap v815: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:27.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:27 smithi062 ceph-mon[106861]: pgmap v816: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:27.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:27 smithi019 ceph-mon[173579]: pgmap v816: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:29.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:29 smithi019 ceph-mon[173579]: pgmap v817: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:29.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:29 smithi062 ceph-mon[106861]: pgmap v817: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:31.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:31 smithi019 ceph-mon[173579]: pgmap v818: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:31.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:31 smithi062 ceph-mon[106861]: pgmap v818: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:33.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:33 smithi019 ceph-mon[173579]: pgmap v819: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:33.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:33.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:33 smithi062 ceph-mon[106861]: pgmap v819: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:33.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:35.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:35 smithi019 ceph-mon[173579]: pgmap v820: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:35.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:35 smithi062 ceph-mon[106861]: pgmap v820: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:37.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:37 smithi019 ceph-mon[173579]: pgmap v821: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:37.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:37 smithi062 ceph-mon[106861]: pgmap v821: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:39.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:39 smithi019 ceph-mon[173579]: pgmap v822: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:39.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:39 smithi062 ceph-mon[106861]: pgmap v822: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:41.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:41 smithi019 ceph-mon[173579]: pgmap v823: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:41.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:41 smithi062 ceph-mon[106861]: pgmap v823: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:43.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:43 smithi019 ceph-mon[173579]: pgmap v824: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:43.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:43 smithi062 ceph-mon[106861]: pgmap v824: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:45.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:45 smithi019 ceph-mon[173579]: pgmap v825: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:45.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:45 smithi062 ceph-mon[106861]: pgmap v825: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:47.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:47 smithi019 ceph-mon[173579]: pgmap v826: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:47.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:47 smithi062 ceph-mon[106861]: pgmap v826: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:48.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:48.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:20:49.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:49 smithi019 ceph-mon[173579]: pgmap v827: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:49.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:49 smithi062 ceph-mon[106861]: pgmap v827: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:51.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:51 smithi019 ceph-mon[173579]: pgmap v828: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:51.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:51 smithi062 ceph-mon[106861]: pgmap v828: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:53.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:53 smithi019 ceph-mon[173579]: pgmap v829: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:53.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:53 smithi062 ceph-mon[106861]: pgmap v829: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:55.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:55 smithi019 ceph-mon[173579]: pgmap v830: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:55.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:55 smithi062 ceph-mon[106861]: pgmap v830: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:57.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:57 smithi062 ceph-mon[106861]: pgmap v831: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:57.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:57 smithi019 ceph-mon[173579]: pgmap v831: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:20:59.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:20:59 smithi019 ceph-mon[173579]: pgmap v832: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:20:59.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:20:59 smithi062 ceph-mon[106861]: pgmap v832: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:01.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:01 smithi019 ceph-mon[173579]: pgmap v833: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:01.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:01 smithi062 ceph-mon[106861]: pgmap v833: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:03.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:03 smithi019 ceph-mon[173579]: pgmap v834: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:03.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:03.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:03 smithi062 ceph-mon[106861]: pgmap v834: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:03.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:05.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:05 smithi019 ceph-mon[173579]: pgmap v835: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:05.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:05 smithi062 ceph-mon[106861]: pgmap v835: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:07.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:07 smithi019 ceph-mon[173579]: pgmap v836: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:07.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:07 smithi062 ceph-mon[106861]: pgmap v836: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:08.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:21:08.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:21:08.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:21:08.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:21:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:21:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:21:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:21:08.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:21:09.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:09 smithi019 ceph-mon[173579]: pgmap v837: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:09.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:09 smithi062 ceph-mon[106861]: pgmap v837: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:11.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:11 smithi019 ceph-mon[173579]: pgmap v838: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:11.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:11 smithi062 ceph-mon[106861]: pgmap v838: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:13.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:13 smithi019 ceph-mon[173579]: pgmap v839: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:13.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:13 smithi062 ceph-mon[106861]: pgmap v839: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:15.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:15 smithi062 ceph-mon[106861]: pgmap v840: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:15.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:15 smithi019 ceph-mon[173579]: pgmap v840: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:17.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:17 smithi062 ceph-mon[106861]: pgmap v841: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:17.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:17 smithi019 ceph-mon[173579]: pgmap v841: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:18.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:18.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:19.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:19 smithi062 ceph-mon[106861]: pgmap v842: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:19.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:19 smithi019 ceph-mon[173579]: pgmap v842: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:21.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:21 smithi062 ceph-mon[106861]: pgmap v843: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:21.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:21 smithi019 ceph-mon[173579]: pgmap v843: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:23.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:23 smithi062 ceph-mon[106861]: pgmap v844: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:23.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:23 smithi019 ceph-mon[173579]: pgmap v844: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:25.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:25 smithi062 ceph-mon[106861]: pgmap v845: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:25.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:25 smithi019 ceph-mon[173579]: pgmap v845: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:27.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:27 smithi062 ceph-mon[106861]: pgmap v846: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:27.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:27 smithi019 ceph-mon[173579]: pgmap v846: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:29.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:29 smithi019 ceph-mon[173579]: pgmap v847: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:29.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:29 smithi062 ceph-mon[106861]: pgmap v847: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:31.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:31 smithi062 ceph-mon[106861]: pgmap v848: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:31.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:31 smithi019 ceph-mon[173579]: pgmap v848: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:33.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:33 smithi062 ceph-mon[106861]: pgmap v849: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:33.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:33 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:33.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:33 smithi019 ceph-mon[173579]: pgmap v849: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:33.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:33 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:35.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:35 smithi062 ceph-mon[106861]: pgmap v850: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:35.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:35 smithi019 ceph-mon[173579]: pgmap v850: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:37.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:37 smithi062 ceph-mon[106861]: pgmap v851: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:37.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:37 smithi019 ceph-mon[173579]: pgmap v851: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:39.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:39 smithi019 ceph-mon[173579]: pgmap v852: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:39.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:39 smithi062 ceph-mon[106861]: pgmap v852: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:41.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:41 smithi062 ceph-mon[106861]: pgmap v853: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:41.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:41 smithi019 ceph-mon[173579]: pgmap v853: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:43.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:43 smithi062 ceph-mon[106861]: pgmap v854: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:43.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:43 smithi019 ceph-mon[173579]: pgmap v854: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:45.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:45 smithi062 ceph-mon[106861]: pgmap v855: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:45.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:45 smithi019 ceph-mon[173579]: pgmap v855: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:47.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:47 smithi062 ceph-mon[106861]: pgmap v856: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:47.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:47 smithi019 ceph-mon[173579]: pgmap v856: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:48.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:48 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:48.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:48 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:21:49.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:49 smithi062 ceph-mon[106861]: pgmap v857: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:49.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:49 smithi019 ceph-mon[173579]: pgmap v857: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:51.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:51 smithi062 ceph-mon[106861]: pgmap v858: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:51.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:51 smithi019 ceph-mon[173579]: pgmap v858: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:53.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:53 smithi062 ceph-mon[106861]: pgmap v859: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:53.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:53 smithi019 ceph-mon[173579]: pgmap v859: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:55.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:55 smithi062 ceph-mon[106861]: pgmap v860: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:55.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:55 smithi019 ceph-mon[173579]: pgmap v860: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:57.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:57 smithi062 ceph-mon[106861]: pgmap v861: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:57.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:57 smithi019 ceph-mon[173579]: pgmap v861: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:21:59.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:21:59 smithi019 ceph-mon[173579]: pgmap v862: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:21:59.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:21:59 smithi062 ceph-mon[106861]: pgmap v862: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:01.486 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:01 smithi019 ceph-mon[173579]: pgmap v863: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:01.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:01 smithi062 ceph-mon[106861]: pgmap v863: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:03.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:03 smithi019 ceph-mon[173579]: pgmap v864: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:03.485 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:03 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:22:03.699 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:03 smithi062 ceph-mon[106861]: pgmap v864: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:03.700 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:03 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:22:05.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:05 smithi062 ceph-mon[106861]: pgmap v865: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:05.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:05 smithi019 ceph-mon[173579]: pgmap v865: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:07.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:07 smithi062 ceph-mon[106861]: pgmap v866: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:07.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:07 smithi019 ceph-mon[173579]: pgmap v866: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:08.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:08 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:22:08.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:08 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-22T16:22:09.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:09 smithi062 ceph-mon[106861]: pgmap v867: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:09.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:09 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:22:09.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:09 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:22:09.698 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:09 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:22:09.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:09 smithi019 ceph-mon[173579]: pgmap v867: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:09.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:09 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-22T16:22:09.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:09 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-22T16:22:09.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:09 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' 2024-02-22T16:22:11.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:11 smithi062 ceph-mon[106861]: pgmap v868: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:11.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:11 smithi019 ceph-mon[173579]: pgmap v868: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:13.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:13 smithi062 ceph-mon[106861]: pgmap v869: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:13.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:13 smithi019 ceph-mon[173579]: pgmap v869: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:15.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:15 smithi062 ceph-mon[106861]: pgmap v870: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:15.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:15 smithi019 ceph-mon[173579]: pgmap v870: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:17.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:17 smithi062 ceph-mon[106861]: pgmap v871: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:17.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:17 smithi019 ceph-mon[173579]: pgmap v871: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-02-22T16:22:18.697 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:18 smithi062 ceph-mon[106861]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:22:18.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:18 smithi019 ceph-mon[173579]: from='mgr.34104 172.21.15.19:0/289654855' entity='mgr.smithi019.yvjrkp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-22T16:22:18.785 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-02-22T16:22:18.786 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-22T16:22:18.903 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-22T16:22:18.915 INFO:tasks.cephadm:Teardown begin 2024-02-22T16:22:18.916 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-22T16:22:18.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T16:22:19.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T16:22:19.043 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-22T16:22:19.044 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-22T16:22:19.067 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-22T16:22:19.105 INFO:tasks.cephadm:Stopping all daemons... 2024-02-22T16:22:19.106 INFO:tasks.cephadm.mon.smithi019:Stopping mon.smithi019... 2024-02-22T16:22:19.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019 2024-02-22T16:22:19.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:19 smithi019 systemd[1]: Stopping Ceph mon.smithi019 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:19.261 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:19 smithi019 ceph-mon[173579]: pgmap v872: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:19.447 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:19 smithi062 ceph-mon[106861]: pgmap v872: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-02-22T16:22:19.534 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:19 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T16:22:19.437+0000 7f643b511700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi019 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:19.534 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:19 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019[173556]: 2024-02-22T16:22:19.437+0000 7f643b511700 -1 mon.smithi019@0(leader) e3 *** Got Signal Terminated *** 2024-02-22T16:22:20.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Feb 22 16:22:20 smithi019 bash[233440]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi019 2024-02-22T16:22:20.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi019.service' 2024-02-22T16:22:20.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:22:20.481 INFO:tasks.cephadm.mon.smithi019:Stopped mon.smithi019 2024-02-22T16:22:20.481 INFO:tasks.cephadm.mon.smithi062:Stopping mon.smithi062... 2024-02-22T16:22:20.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi062 2024-02-22T16:22:20.947 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:20 smithi062 systemd[1]: Stopping Ceph mon.smithi062 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:20.947 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:20 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062[106838]: 2024-02-22T16:22:20.781+0000 7fed65782700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi062 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:20.947 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:20 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062[106838]: 2024-02-22T16:22:20.781+0000 7fed65782700 -1 mon.smithi062@1(peon) e3 *** Got Signal Terminated *** 2024-02-22T16:22:21.447 INFO:journalctl@ceph.mon.smithi062.smithi062.stdout:Feb 22 16:22:21 smithi062 bash[149823]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-mon-smithi062 2024-02-22T16:22:21.684 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@mon.smithi062.service' 2024-02-22T16:22:21.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:22:21.731 INFO:tasks.cephadm.mon.smithi062:Stopped mon.smithi062 2024-02-22T16:22:21.732 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-22T16:22:21.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.0 2024-02-22T16:22:22.235 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 16:22:21 smithi019 systemd[1]: Stopping Ceph osd.0 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:22.235 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 16:22:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T16:22:22.040+0000 7fd66ec51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:22.235 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 16:22:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T16:22:22.040+0000 7fd66ec51700 -1 osd.0 86 *** Got signal Terminated *** 2024-02-22T16:22:22.235 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 16:22:22 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0[188858]: 2024-02-22T16:22:22.040+0000 7fd66ec51700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T16:22:28.235 INFO:journalctl@ceph.osd.0.smithi019.stdout:Feb 22 16:22:28 smithi019 bash[233599]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-0 2024-02-22T16:22:29.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.0.service' 2024-02-22T16:22:29.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:22:29.929 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-22T16:22:29.930 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-22T16:22:29.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.1 2024-02-22T16:22:30.235 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 16:22:30 smithi019 systemd[1]: Stopping Ceph osd.1 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:30.735 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 16:22:30 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T16:22:30.273+0000 7f1619621700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:30.735 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 16:22:30 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T16:22:30.273+0000 7f1619621700 -1 osd.1 86 *** Got signal Terminated *** 2024-02-22T16:22:30.735 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 16:22:30 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1[196285]: 2024-02-22T16:22:30.273+0000 7f1619621700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T16:22:36.235 INFO:journalctl@ceph.osd.1.smithi019.stdout:Feb 22 16:22:36 smithi019 bash[233868]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-1 2024-02-22T16:22:37.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.1.service' 2024-02-22T16:22:37.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:22:37.842 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-22T16:22:37.843 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-22T16:22:37.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.2 2024-02-22T16:22:38.099 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 16:22:37 smithi019 systemd[1]: Stopping Ceph osd.2 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:38.485 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 16:22:38 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T16:22:38.126+0000 7ff291f54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:38.485 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 16:22:38 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T16:22:38.126+0000 7ff291f54700 -1 osd.2 86 *** Got signal Terminated *** 2024-02-22T16:22:38.485 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 16:22:38 smithi019 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2[203709]: 2024-02-22T16:22:38.126+0000 7ff291f54700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T16:22:44.235 INFO:journalctl@ceph.osd.2.smithi019.stdout:Feb 22 16:22:44 smithi019 bash[234136]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-2 2024-02-22T16:22:45.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.2.service' 2024-02-22T16:22:45.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:22:45.720 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-22T16:22:45.720 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-22T16:22:45.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.3 2024-02-22T16:22:46.015 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 16:22:45 smithi062 systemd[1]: Stopping Ceph osd.3 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:46.447 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 16:22:46 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T16:22:46.043+0000 7fae02d8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:46.448 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 16:22:46 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T16:22:46.043+0000 7fae02d8a700 -1 osd.3 86 *** Got signal Terminated *** 2024-02-22T16:22:46.448 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 16:22:46 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T16:22:46.043+0000 7fae02d8a700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T16:22:48.699 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:48 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:48.337+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:49.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:49 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:49.384+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:50.345 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 16:22:50 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3[120054]: 2024-02-22T16:22:50.037+0000 7fadfc37a700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.056285+0000 front 2024-02-22T16:22:24.056309+0000 (oldest deadline 2024-02-22T16:22:49.956181+0000) 2024-02-22T16:22:50.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:50 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:50.374+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:51.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:51 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:51.359+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:52.197 INFO:journalctl@ceph.osd.3.smithi062.stdout:Feb 22 16:22:51 smithi062 bash[149981]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-3 2024-02-22T16:22:52.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:52 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:52.330+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:53.542 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:53 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:53.297+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:53.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.3.service' 2024-02-22T16:22:53.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:22:53.593 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-22T16:22:53.594 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-22T16:22:53.594 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.4 2024-02-22T16:22:53.819 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:53 smithi062 systemd[1]: Stopping Ceph osd.4 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:22:54.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:53 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:53.847+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.465974+0000 front 2024-02-22T16:22:24.466190+0000 (oldest deadline 2024-02-22T16:22:53.266410+0000) 2024-02-22T16:22:54.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:53 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:53.945+0000 7f5ba2844700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:22:54.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:53 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:53.945+0000 7f5ba2844700 -1 osd.4 86 *** Got signal Terminated *** 2024-02-22T16:22:54.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:53 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:53.945+0000 7f5ba2844700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T16:22:54.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:54 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:54.307+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:55.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:54 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:54.814+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.465974+0000 front 2024-02-22T16:22:24.466190+0000 (oldest deadline 2024-02-22T16:22:53.266410+0000) 2024-02-22T16:22:55.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:55 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:55.265+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:56.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:55 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:55.839+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.465974+0000 front 2024-02-22T16:22:24.466190+0000 (oldest deadline 2024-02-22T16:22:53.266410+0000) 2024-02-22T16:22:56.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:56 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:56.240+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:57.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:56 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:56.806+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.465974+0000 front 2024-02-22T16:22:24.466190+0000 (oldest deadline 2024-02-22T16:22:53.266410+0000) 2024-02-22T16:22:57.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:57 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:57.289+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:58.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:57 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:57.772+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.465974+0000 front 2024-02-22T16:22:24.466190+0000 (oldest deadline 2024-02-22T16:22:53.266410+0000) 2024-02-22T16:22:58.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:58 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:58.246+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:59.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:58 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:58.785+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:24.465974+0000 front 2024-02-22T16:22:24.466190+0000 (oldest deadline 2024-02-22T16:22:53.266410+0000) 2024-02-22T16:22:59.197 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:58 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4[126173]: 2024-02-22T16:22:58.785+0000 7f5b9de38700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.266787+0000 front 2024-02-22T16:22:33.266878+0000 (oldest deadline 2024-02-22T16:22:58.566832+0000) 2024-02-22T16:22:59.694 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:59 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:59.284+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:22:59.695 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:22:59 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:22:59.284+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:22:59.947 INFO:journalctl@ceph.osd.4.smithi062.stdout:Feb 22 16:22:59 smithi062 bash[150247]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-4 2024-02-22T16:23:00.498 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:00 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:00.270+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:00.498 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:00 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:00.271+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:01.447 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:01 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:01.282+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:01.447 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:01 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:01.282+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:01.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.4.service' 2024-02-22T16:23:01.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:23:01.522 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-22T16:23:01.522 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-22T16:23:01.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.5 2024-02-22T16:23:01.827 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:01 smithi062 systemd[1]: Stopping Ceph osd.5 for 8db8e6fa-d198-11ee-95bf-87774f69a715... 2024-02-22T16:23:01.828 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:01 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:01.855+0000 7f33c71f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-22T16:23:02.197 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:01 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:01.855+0000 7f33c71f6700 -1 osd.5 86 *** Got signal Terminated *** 2024-02-22T16:23:02.197 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:01 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:01.855+0000 7f33c71f6700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-22T16:23:02.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:02 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:02.330+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:02.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:02 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:02.330+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:03.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:03 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:03.293+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:03.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:03 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:03.293+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:04.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:04 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:04.324+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:04.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:04 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:04.324+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:05.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:05 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:05.326+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:05.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:05 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:05.326+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:05.697 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:05 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:05.326+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.2 since back 2024-02-22T16:22:39.359077+0000 front 2024-02-22T16:22:39.359136+0000 (oldest deadline 2024-02-22T16:23:05.258915+0000) 2024-02-22T16:23:06.698 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:06 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:06.278+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-02-22T16:22:25.357756+0000 front 2024-02-22T16:22:25.357729+0000 (oldest deadline 2024-02-22T16:22:47.657488+0000) 2024-02-22T16:23:06.988 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:06 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:06.278+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6814 osd.1 since back 2024-02-22T16:22:33.558439+0000 front 2024-02-22T16:22:33.558391+0000 (oldest deadline 2024-02-22T16:22:58.858226+0000) 2024-02-22T16:23:06.988 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:06 smithi062 ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5[132438]: 2024-02-22T16:23:06.278+0000 7f33c17e8700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.19:6822 osd.2 since back 2024-02-22T16:22:39.359077+0000 front 2024-02-22T16:22:39.359136+0000 (oldest deadline 2024-02-22T16:23:05.258915+0000) 2024-02-22T16:23:07.947 INFO:journalctl@ceph.osd.5.smithi062.stdout:Feb 22 16:23:07 smithi062 bash[150513]: ceph-8db8e6fa-d198-11ee-95bf-87774f69a715-osd-5 2024-02-22T16:23:09.322 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8db8e6fa-d198-11ee-95bf-87774f69a715@osd.5.service' 2024-02-22T16:23:09.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-22T16:23:09.367 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-22T16:23:09.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 --force --keep-logs 2024-02-22T16:23:12.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi019.stderr:ceph-fuse[127608]: fuse finished with error 0 and tester_r 0 2024-02-22T16:23:29.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 --force --keep-logs 2024-02-22T16:23:41.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T16:23:41.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-22T16:23:41.130 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-22T16:23:41.132 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/remote/smithi019/crash 2024-02-22T16:23:41.133 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash -- . 2024-02-22T16:23:41.171 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash: Cannot open: No such file or directory 2024-02-22T16:23:41.171 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-02-22T16:23:41.172 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/remote/smithi062/crash 2024-02-22T16:23:41.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash -- . 2024-02-22T16:23:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/crash: Cannot open: No such file or directory 2024-02-22T16:23:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-02-22T16:23:41.201 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-22T16:23:41.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-22T16:23:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:grep: /var/log/ceph/8db8e6fa-d198-11ee-95bf-87774f69a715/ceph.log: No such file or directory 2024-02-22T16:23:41.248 INFO:tasks.cephadm:Compressing logs... 2024-02-22T16:23:41.249 DEBUG:teuthology.orchestra.run.smithi019:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T16:23:41.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T16:23:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-22T16:23:41.320 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-22T16:25:21.443 INFO:teuthology.orchestra.run.smithi062.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-02-22T16:25:32.081 INFO:tasks.cephadm:Archiving logs... 2024-02-22T16:25:32.082 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/remote/smithi019/log 2024-02-22T16:25:32.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-22T16:25:44.535 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/remote/smithi062/log 2024-02-22T16:25:44.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-22T16:25:55.393 INFO:tasks.cephadm:Removing cluster... 2024-02-22T16:25:55.394 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 --force 2024-02-22T16:25:55.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8db8e6fa-d198-11ee-95bf-87774f69a715 --force 2024-02-22T16:25:56.514 INFO:tasks.cephadm:Removing cephadm ... 2024-02-22T16:25:56.514 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-22T16:25:56.535 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-22T16:25:56.554 INFO:tasks.cephadm:Teardown complete 2024-02-22T16:25:56.554 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-22T16:25:56.566 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-22T16:25:56.567 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-22T16:25:56.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-22T16:25:56.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-22T16:25:56.670 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-22T16:25:56.681 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-22T16:25:56.681 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T16:25:56.684 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-22T16:25:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:bash: ntpq: command not found 2024-02-22T16:25:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2024-02-22T16:25:56.706 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T16:25:56.707 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-02-22T16:25:56.707 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 8 +1307us[+1307us] +/- 120ms 2024-02-22T16:25:56.707 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -1988us[-1976us] +/- 76ms 2024-02-22T16:25:56.707 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +1114us[+1114us] +/- 76ms 2024-02-22T16:25:56.708 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T16:25:56.709 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-22T16:25:56.710 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-02-22T16:25:56.710 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 +1291us[+1271us] +/- 120ms 2024-02-22T16:25:56.711 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -1989us[-2009us] +/- 77ms 2024-02-22T16:25:56.711 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +1115us[+1115us] +/- 77ms 2024-02-22T16:25:56.711 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-22T16:25:56.711 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-22T16:25:56.730 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-22T16:25:56.732 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-22T16:25:56.745 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-22T16:25:56.829 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-22T16:25:56.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T16:25:56.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T16:25:56.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-22T16:25:56.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T16:25:56.967 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-22T16:25:56.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-22T16:25:56.994 INFO:teuthology.task.internal:Duration was 5139.612487 seconds 2024-02-22T16:25:56.994 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-22T16:25:57.006 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-22T16:25:57.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-22T16:25:57.008 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-22T16:25:57.056 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T16:25:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-22T16:25:57.248 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-22T16:25:57.248 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-02-22T16:25:57.248 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-22T16:25:57.272 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-02-22T16:25:57.273 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-22T16:25:57.298 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-22T16:25:57.298 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T16:25:57.315 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-22T16:25:57.397 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-22T16:25:57.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-22T16:25:57.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-22T16:25:57.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-22T16:25:57.551 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-22T16:25:57.551 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-22T16:25:57.579 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-22T16:25:57.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-22T16:25:57.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-22T16:25:57.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-22T16:25:57.651 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-02-22T16:25:57.675 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-02-22T16:25:57.698 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-22T16:25:57.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T16:25:57.725 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-22T16:25:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T16:25:57.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-22T16:25:57.771 INFO:teuthology.task.internal:Transferring archived files... 2024-02-22T16:25:57.772 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/remote/smithi019 2024-02-22T16:25:57.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-22T16:25:57.823 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354/remote/smithi062 2024-02-22T16:25:57.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-22T16:25:57.869 INFO:teuthology.task.internal:Removing archive directory... 2024-02-22T16:25:57.870 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-22T16:25:57.872 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-22T16:25:57.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-22T16:25:57.931 INFO:teuthology.task.internal:Not uploading archives. 2024-02-22T16:25:57.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-22T16:25:57.943 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-22T16:25:57.943 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-22T16:25:57.945 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-22T16:25:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: 525356 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 22 16:25 /home/ubuntu/cephtest 2024-02-22T16:25:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: 526831 4 d--------- 2 ubuntu ubuntu 4096 Feb 22 15:47 /home/ubuntu/cephtest/mnt.0 2024-02-22T16:25:57.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-22T16:25:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-02-22T16:25:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-22T16:25:57.963 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_c1dc18f444603397768fcb7c34a8488b10284df6/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-22T16:25:57.963 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-22T16:25:57.976 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-22T16:25:57.990 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-22T16:25:57.990 INFO:teuthology.nuke:Checking targets against current locks 2024-02-22T16:25:58.018 DEBUG:teuthology.nuke:shortname: smithi019 2024-02-22T16:25:58.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T16:25:58.040 DEBUG:teuthology.nuke:shortname: smithi062 2024-02-22T16:25:58.040 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-22T16:25:58.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 14:38:42.272301', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC04T1B4YhQkhNaZ1T/tOtUSBIyA2hQNTTXwityN5MkgzaVGXmL4bFkGiD6D5eDzbSD5MC/oN2jPxoE8zLyVAfU='} 2024-02-22T16:25:58.064 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-21_23:06:32-fs-wip-yuri10-testing-2024-02-21-1249-reef-distro-default-smithi/7570354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-22 14:38:42.274065', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInULIZZDrJ/cMctnfpAMqX7AiyMkWr3Csj7NmbMNHgHv0l3vqskz4fj/QycLxwFwvSaU+vr5zdIUwBpg3F8TsQ='} 2024-02-22T16:25:58.103 INFO:teuthology.orchestra.console.smithi019:Power off 2024-02-22T16:25:58.103 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-22T16:25:58.128 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-02-22T16:25:58.128 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T16:25:58.147 INFO:teuthology.orchestra.console.smithi062:Power off 2024-02-22T16:25:58.147 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-22T16:25:58.160 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-02-22T16:25:58.178 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-02-22T16:25:58.178 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T16:25:58.208 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-02-22T16:26:02.162 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T16:26:02.210 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T16:26:02.271 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-02-22T16:26:02.320 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-02-22T16:26:06.276 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T16:26:06.321 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-22T16:26:06.387 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-02-22T16:26:06.434 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-02-22T16:26:06.488 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-02-22T16:26:06.535 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-02-22T16:26:06.639 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 5139.612486600876 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-02-22T16:26:06.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-22T16:26:06.727 INFO:teuthology.run:FAIL